144 results about "Test suite" patented technology

The invention discloses a software test case generation method and system based on a large language model, and relates to the technical field of software testing, and the system comprises a to-be-tested module detection subsystem which is used for constructing a to-be-tested module set; the test case generation subsystem is used for constructing a prompt template according to the to-be-tested module and generating a test case by utilizing a program generation model based on an adaptive focus context generation mechanism and the prompt template; the correction verification subsystem is used for performing multi-stage correction processing on the test case, eliminating errors and generating an initial test suite; and the test case supplement subsystem is used for carrying out coverage rate analysis and variation test on the initial test suite, constructing a model input text according to a result, and generating a supplement test case by utilizing a program generation model so as to update the initial test suite and obtain a final test suite. By combining the functional annotation of the to-be-tested module with the logical reasoning ability of the LLM, misjudgment caused by assertion errors can be reduced, and the accuracy of test feedback is ensured.

The invention discloses a method for automatically generating a dynamic language test case based on a large language model, which comprises the following steps of: firstly, in a preprocessing stage, arranging related information of a to-be-tested program into prompts, analyzing a parameter type of the to-be-tested program by the large language model according to the prompts, and generating code information required by a test through an abstract syntax tree; in the test case generation stage, a dynamic multi-objective optimization algorithm is adopted, test cases are continuously crossed and mutated through branch coverage feedback, in the partial evolution process, test data which cannot be generated by conventional variation are generated by utilizing a large language model, and finally, the screened excellent test cases are organized into a complete test suite. In a program repair phase, the test suite is run to detect potential errors, the errors are analyzed and iteratively repaired using a large language model, and an executable and error-free test suite is generated. According to the method, the test coverage rate of the dynamic language is remarkably improved, and the achievement of a coverage target is accelerated.

Systems, apparatus, articles of manufacture, and methods are disclosed to test an automated query language by interfacing with a query endpoint, generating a first test suite based on a plurality of combinations of a plurality of attributes corresponding to an API specification, transmitting the first test suite to be executed on the query endpoint, the first test suite to include instructions to cause the query endpoint to access data stored in a database accessible to a plurality of secondary endpoints, obtaining a response time corresponding to the access of the data, comparing the response time corresponding to the access of the data with a service level agreement required response time, and in response to the response time corresponding to the access of the data being longer than the service level agreement required response time, notifying a developer of the plurality of secondary endpoints.

A system and a method for creating an optimal test suite. The system may receive code coverage data from a set of testing channels. Further, a combined dataset is created by merging the code coverage data received from the set of testing channels. The combined dataset is analyzed to identify a line of code being executed by two or more test cases corresponding to two or more testing channels of the set of testing channels. Further, at least one test case from the two or more test cases having less efficiency is eliminated to create an optimal test suite. The efficiency is determined based on at least one of execution time, execution cost, and resources required to execute the line of code.

The invention discloses an LLM-based controller CAN communication test method and system, and the method comprises the steps: converting a DBC file into a configuration file, reading a VSS and CAN signal corresponding relation file, and inputting the VSS and CAN signal corresponding relation file to an LLM; calling an LLM model, and generating a test case according to the cue word; constructing an automatic framework to implement an execution function based on a behavior-driven development BDD mode; and associating the generated test case with an execution function in the automation framework, constructing a complete test suite, and operating the test suite to execute a CAN bus communication test. VSS path and CAN Signal verification is carried out on the basis of the controller which already realizes the VSS technology, the test case conforming to the BDD natural language is generated by using the LLM, the test case is directly used as the test script, the step of manually compiling the test case is omitted, the test efficiency is improved, and the test cost is reduced.

A remote software development infrastructure can include cross-continent datacenters with a variety of remote devices. The infrastructure can provide automated testing on the remote devices. A software development kit (SDK) can be used to integrate a test suite with the infrastructure. The SDK can modify the test suite to issue test commands to a remote test server of the infrastructure, as opposed to local test servers. A remote test server routes the test commands to a remote device in a datacenter. Both local testing and remote testing of a test script is possible. The SDK enables cross-browser, cross-device, cross-operating system and / or parallel testing of the test suite, without the user having to make code changes to the test suite. The test results and additional information are automatically captured and are aggregated in a dashboard in the form of aggregated reports and other graphical user interface elements.

A system for generating non-fungible token-based test suites from design diagrams comprises a processor associated with a server. The processor is configured to obtain image objects representing a workflow associated with a design diagram and a user profile of a user. The processor is configured to generate a design non-fungible token (NFT) associated with the design diagram to verify a design diagram ownership. The processor is configured to convert the image objects of the design diagram into corresponding text objects and to process the corresponding text objects of the design diagram to generate a plurality of text datasets corresponding to the workflow of the design diagram, analyze the plurality of the text datasets and generate a test suite which comprises a set of test scripts corresponding to the design diagram, and generate a test case NFT associated with the test suite to verify a test suite ownership.

The invention discloses an aero-engine compiler fuzzy test method and system based on a large language model. In a variation prompt generation stage, a seed program library is constructed by using embedded programs of various structures and compiler test suite programs, variation operators are selected through diversified guide strategies, and variation prompts are constructed based on a prompt template. In the test generation stage, a large language model is used for generating a variant program with complex data flow and control flow so as to enhance the test coverage rate of a compiler; global and local variables are monitored by inserting test output for the program, so that the program can effectively find out silent miscompilation errors with the maximum harm; in addition, a front-end error feedback strategy is designed, so that the effectiveness of a large language model generation test program is improved. In the differential testing stage, random difference testing and testing under different optimization levels are combined, so that the defects of the middle and rear ends of the compiler are detected. According to the method, the coverage rate of compiler testing can be increased, and silent miscompilation errors can be detected.

An information handling system executing a software test suite green data center policy generation system comprising a network interface device receiving data center hardware utilization analytics and a software test suite identifier for a selected software test suit and a processor to identify a reserved capacity of the data center processing system reserved for general execution of software test suites according to general quality of service requirements for input / output commands and to predict a future data center processor utilization rate and execution duration for the selected software test suite by forecasting actual use of data center resources during a future execution of the selected software test suite that is below the reserved capacity, and the network interface device to transmit an instruction to throttle power supplied to an over-allocated portion of the data center resources from the reserved capacity duration execution of the selected software test suite.

The invention provides a vector optimization algorithm translation method and device oriented to an RISC-V vector extension platform, and relates to the technical field of computer software cross-architecture translations. The method comprises the steps that source codes of vector optimization algorithms written by other platforms and translation cues are input into a large language model; outputting a translation code of a vector optimization algorithm oriented to the RISC-V vector extension platform; the translated codes are input into a compiler to be compiled; under the condition that compiling succeeds, a compiling product is linked to the test suite to obtain an executable program, and unit testing is conducted on the executable program on the RISC-V vector extension device; and under the condition that the unit test is passed, determining that the translation code is a first version of correct translation code of a vector optimization algorithm oriented to the RISC-V vector extension platform. According to the method, a vector optimization algorithm written by using a special vector language for other platforms is efficiently translated to the RISC-V vector extension platform.

Processing within a computing environment is facilitated by determining a set of microservices needed to execute a microservice test suite in a test environment, where the test environment is an active microservice test environment with a plurality of microservices. The process also includes determining that the test environment lacks sufficient resources to run the microservice test suite with the set of microservices, and dynamically removing at least one microservice of the plurality of microservices from the test suite not needed to execute the microservice test suite to free test environment resources for executing the microservice test suite with the set of microservices. The process also includes initiating executing, based on the dynamically removing, the microservice test suite in the test environment.

An initial corpus of source code is received. The initial corpus of source code is for training an Artificial Intelligence (AI) algorithm that generates source code. The initial corpus of source code is scanned, using a test suite, to identify one or more potential vulnerabilities in the initial corpus of the source code. The identified one or more potential vulnerabilities in the initial corpus of the source code are mitigated to produce a training corpus of source code. For example, the mitigation may comprise removing malware from the initial corpus. The mitigation is to remove the vulnerabilities so that the vulnerabilities do not show up in source code generated by the AI algorithm. The AI algorithm is then trained using the training corpus of source code. The trained AI algorithm is executed to produce generated source code.

There is provided a system and method of automatic software testing. The method includes obtaining an input including software code of a software program and metadata, and feeding the input to a machine learning model to generate a test suite usable for testing the program. The test suite comprises a set of tests meeting a predefined condition. The test suite is generated by generating at least one question related to at least one of: expected intents of one or more sections of the software code, or tests for testing the sections, and presenting the at least one question to a user; upon receiving feedback from the user, analyzing the feedback with respect to the predefined condition, and determining whether to generate at least one new question; and, in response to an affirmative determination, repeating the above process with respect to the new question, until the predefined condition is met.

Software and hardware for monitoring, testing, and developing communication and electronic systems on spacecrafts. Several systems are provided to monitor, test, and control a next generation SpaceCube, including a RadHard Monitor (RHM), a Mini ASTM Board, an FMC+ ASTM Board, a Mini Evaluation Board, a MEZZ and an Automated Test Suite. The RHM are FPGA IP and hardware configured to monitor COTS components. The Mini ASTM Board connects a Mini Processor Card to an ASTM for electrical testing. FMC+ ASTM Card connects FMC and FMC+ test cards to the ASTM for electrical testing. A Mini Evaluation Board supplies all necessary power to the Mini Processor card. The MEZZ is a multi-use GSE test board that is compatible with several different development platforms in the SpaceCube to allow developers to develop and test their software. The Automated Test Suite provides functional testing of an assembled SpaceCube board.

The invention discloses a data detection method and device, electronic equipment and a storage medium. The method comprises the steps of obtaining a source program file; performing first data extraction processing on the source program file to obtain a target test function set; performing data segmentation processing on a plurality of target test functions in the target test function set to obtain word vectors corresponding to the target test functions; calculating the text similarity between the word vectors corresponding to each target test function to obtain a similarity calculation result; generating a plurality of test suites based on the similarity calculation result, and executing each test suite to obtain a running result log; and detecting unstable test data in the source program file according to the running result log. The method can reduce the repeated operation frequency of the test, improve the test efficiency of the detection system and reduce the execution time cost, and can be widely applied to the technical field of data processing.

Systems and methods for testing sandboxed in-kernel programs are provided. A method of testing a program includes: obtaining a Berkeley Packet Filter (BPF) program to test; obtaining a test to run on the BPF program; performing the test on the BPF program for a plurality of Linux kernels and reporting a result of performing the test on the BPF program for a plurality of Linux kernels. This enables a generic testing solution for black box testing of BPF programs. The embodiments can integrate with existing testing suites and frameworks and can integrate into CI / CD pipelines. Some embodiments allow testing using any kernel from user space. Host kernel testing of BPF programs is enabled by a developer using the developer's own workstation and version of the Linux kernel. Guest kernel testing of BPF programs is enabled using specific kernel versions by providing the framework a specification of the target kernel environment.

Execution of software applications can be controlled based on application profiles to facilitate safety compliance. For example, a system can execute a test suite to identify a function call of a software application that is associated with a functional safety standard issued by a standard-setting organization. In response to identifying the function call, the system can generate a risk score for the function call. The risk score can indicate a likelihood of the function call causing non-compliance with the functional safety standard. The system can further generate an application profile including a permission for the software application based on the risk score. The permission may disable the function call of the software application. The system may then execute the software application based on the application profile. As a result, execution of the software application can comply with the functional safety standard.

The invention relates to the technical field of integrated circuits, and provides a verification platform, a verification method, electronic equipment and a medium. The verification platform comprises: an application layer bus function level model comprising a customized interface for a to-be-tested design for interacting with the to-be-tested design, the to-be-tested design being a PCIe protocol docking and protocol compliance test suite, and the to-be-tested design being a PCIe protocol docking and protocol compliance test suite; the PCIe protocol docking and protocol compliance test suite supports a verification demand of a physical layer intellectual property core and a verification demand of protocol consistency of a PCIe controller and a physical layer subsystem level; and the to-be-tested design interacts with the application layer bus function level model through the customized interface. Thus, the integration and through-flow debugging cost of subsystem protocol consistency verification is reduced, the verification period of a docking test of PCIe subsystem protocol standardization is shortened, and the completeness of docking verification of a full quantization protocol of a PCIe subsystem and the platform reuse efficiency are improved.

The invention discloses an intelligent test generation method, system and device for an H5 page and a medium, and the method specifically comprises the steps: collecting a user operation event sequence of the H5 page, and building an operation log stream with a timestamp through the user operation event sequence; collecting equipment fingerprint data and a network delay index, and performing space-time correlation coding on the equipment fingerprint data and the network delay index to generate an enhanced operation track data set; inputting the enhanced operation track data set into a pre-trained depth time sequence neural network to obtain an anomaly detection result; sending out safety early warning based on an abnormal detection result, extracting boundary condition features in the operation log stream, and automatically generating a boundary test case set; and fusing the boundary test case set with the user operation reproduction case, constructing a hybrid test suite, and triggering an automatic test assembly line through a CI / CD integrated interface. According to the method, the comprehensiveness, the accuracy and the efficiency of H5 page testing are remarkably improved, and the performance and the safety of the H5 page are effectively guaranteed.

The invention relates to the technical field of deep learning testing, in particular to a deep learning test sample generation method utilizing a reinforcement learning optimization genetic algorithm, which comprises the following steps: constructing different types of seed data sets based on an image data set, and determining and initializing an initial population; obtaining a deep learning model, executing a genetic algorithm, and optimizing the initialized initial population; combining reinforcement learning to dynamically optimize genetic algorithm parameters; and performing iterative optimization on the optimized initial population through the optimized genetic algorithm to generate a new population, screening a data sample with a prediction error, and obtaining a test suite. The deep learning model and the genetic algorithm are integrated, the genetic algorithm is executed, reinforcement learning is introduced to dynamically optimize operation parameters of the genetic algorithm, the problems of local optimum and parameter sensitivity are solved, generation of a test suite with smaller distribution difference is promoted, the convergence speed of fitness can be effectively improved, and the robustness of the deep learning model is improved.

Methods, system, and non-transitory processor-readable storage medium for a source code file association system are provided herein. An example method includes receiving, by a project management tool associated with a software test lifecycle system, a pull request to merge software code changes with a software project repository on an enterprise system. The software testing life cycle system executes a plurality of integration test suites associated with the pull request on at least one test system associated with the enterprise system. The integration test suites are executed to test the software code changes. The project management tool captures the integration test suite output. A source code file association system preprocesses the integration test suite output for input into a machine learning system. The integration test suite output is encoded. The output of the machine learning system identifies at least a portion of source code exercised by the integration test suites.

A remote software development infrastructure can include cross-continent datacenters with a variety of remote devices. The infrastructure can provide automated testing on the remote devices. A software development kit (SDK) can be used to integrate a test suite with the infrastructure. The SDK can modify the test suite to issue test commands to a remote test server of the infrastructure, as opposed to local test servers. A remote test server routes the test commands to a remote device in a datacenter. Both local testing and remote testing of a test script is possible. The SDK enables cross-browser, cross-device, cross-operating system and / or parallel testing of the test suite, without the user having to make code changes to the test suite. The test results and additional information are automatically captured and are aggregated in a dashboard in the form of aggregated reports and other graphical user interface elements.

The invention relates to the technical field of software engineering and automatic testing, and discloses a self-adaptive multi-dimensional risk testing strategy design method and system and a storage medium. The method comprises the following steps: triggering a risk assessment service through a CI / CD assembly line, preprocessing original data obtained by a plurality of heterogeneous data sources, and generating a risk feature vector; outputting a multi-dimensional risk index by using the risk assessment model, fusing the multi-dimensional risk index and the dynamic weight thereof based on an adaptive weighted fusion model, and calculating a comprehensive risk score; dynamically updating a risk score threshold value used for dividing the risk level, automatically generating a test strategy matched with the current risk level according to a preset mapping rule of the risk level and the test strategy, and calling the test suite to automatically execute the test strategy; and collecting a test execution result and online operation data as training data, and regularly optimizing parameters of the risk assessment model. The test strategy can be dynamically adjusted according to the real-time risk situation.

A computing platform may receive test input information comprising input information to validate code for an application. The computing platform may test the code by inputting the test input information into the application and executing the code to produce initial test results. The computing platform may store the test input information / initial test results in a table. The computing platform may identify that subsequent testing should be initiated. The computing platform may automatically identify the test input information and the corresponding initial test results using the stored table. The computing platform may automatically test the application by inputting the test input information into the application and executing the application to produce automated test results. The computing platform may compare the automated test results to the initial test results. Based on identifying that the automated test results match the initial test results, the computing platform may deploy the application.

The embodiment of the invention discloses a development data detection method and equipment based on a low-code platform and a medium, belongs to the technical field of software development, and solves the problem that the repair efficiency is reduced by manual coding when a software code is changed and a test fails. Based on the development data change amount, matching a test suite containing the corresponding number of test cases in a preset test suite set, and testing the change data through the test suite; under the condition that the test fails, performing frequent item set query in a historical database through an association rule mining algorithm, and constructing an item set association relationship corresponding to the modified data according to the frequent item set; inputting the change data and the item set association relationship corresponding to the change data into a preset failure data analysis model group, and outputting a test failure reason and a repair suggestion through the preset failure data analysis model group; and based on the test failure reason and the repair suggestion, filling the preset code update meta-language, and repairing the modified data through the filled update meta-language.

The invention discloses a multi-modal-based automatic test script generation method and system, and the method comprises the steps: a user describes a test demand through a natural language, uploads an interface screenshot to carry out multi-modal input, and calls a large language model to generate an AIRTEST script; performing mode matching on the generated script by using a regular expression, extracting all picture element names, and performing reordering processing to form an element name list; the original interface screenshot and the element name list are input into the visual large model, and the visual large model is required to return standardized coordinate data; image cutting operation is executed based on the coordinate data, and actual image resources are provided for the test script; and automatically generating a. Air item folder named by a timestamp, creating a Python script file with the same name in a directory, writing a test script code generated previously, storing picture resources obtained by cutting into the item folder, and forming a complete executable test suite meeting the requirements of an AIRTEST framework. According to the invention, the core pain points of high learning cost and difficult maintenance of a traditional automatic test tool can be solved.

The invention discloses a whole-process intelligent unit testing method driven by a large language model, belongs to the technical field of software testing, and aims to solve the problems of poor testing quality, difficulty in effective understanding of testing intentions and low coverage rate of testing codes in existing unit testing. The method comprises the following steps: intelligently analyzing a to-be-tested code driven by a large language model; automatically generating a test code driven by the large language model; testing code intelligent verification driven by the large language model; error codes driven by the large language model are automatically repaired; and automatic construction of a test suite and intelligent evaluation driven by a large language model. According to the method, the advantages of the large language model in the aspects of natural language understanding and code generation are fully played, so that the large language model can more accurately grasp the test intention and the function requirement, the generation quality and the coverage range of the test code are remarkably improved, and a brand new solution is provided for realizing efficient, intelligent and whole-process automatic unit test generation.

The invention discloses a database compatibility test method and system, which can structurally store data from different database systems, test scenes and compatibility indexes by defining a multi-dimensional array, cover key indexes such as data integrity, consistency, transaction processing, query performance and the like, and ensure that the test is comprehensive and systematic. A cluster containing a plurality of database instances is constructed, and the same test data set is configured, so that the consistency of a test environment is ensured, the accuracy and comparability of a test result are improved, a preset compatibility test suite is automatically executed, the test efficiency is greatly improved, manual intervention and errors are reduced, multi-dimensional array data are deeply analyzed, compatibility differences can be accurately identified, and the test efficiency is improved. According to the scheme, the performance bottleneck is positioned, potential problems are identified, the problems are found and solved in the development, upgrading or integration stage in time, risks are avoided, in addition, the scheme supports on-demand extension of array dimensions, test suites and data sets can be customized and adjusted, more requirements can be flexibly met, and high adaptability is achieved.

The invention relates to the technical field of automatic testing, in particular to a test case generation method and device, computer equipment, a computer readable storage medium and a computer program product. The method comprises the following steps: generating a test case population by adopting a heuristic search operator; according to the multi-target fitness of test case individuals in the test case population, the distance between the test case individuals and the execution time corresponding to each test case individual, target test case individuals are selected and serve as male parents to be subjected to crossover and mutation processing, and a new test case population is obtained; according to the event sequence corresponding to the test case set, a target test model is obtained through reasoning, and a target test case is generated. By adopting the method, the problem that the time cost and the test coverage rate are difficult to balance in the prior art can be solved, and the technical effects of furthest improving the test coverage range of the target test case and reducing the total execution time and size cost of the test case in the test suite are achieved.

A diversified validation test suite application (DVTSA) for a generative AI powered tool or large language model (LLM) tool includes at least first, second, and third control logics. The first control logic receives a seed test input or user input, analyzes the seed test input or user input, and extracts key elements for variations. The second control logic performs a coverage measurement of outputs of the first control logic. The third control logic causes a human validator to evaluate outputs of the second control logic relative to predefined coverage metrics and selectively and continuously iterate to cause outputs of the second control logic to increase LLM tool input robustness and output robustness from a first level to a second level greater than the first level, in both production and pre-production LLM tool processes.