5330 results about "Test case" patented technology

An example operation may include one or more of receiving a request to test a software program from a user device, the request comprising a description of requirements of the software program, generating a plurality of testing elements based on execution of a generative artificial intelligence (GenAI) model on the description of the requirements of the software program and a repository of test cases, generating a test case for testing the software program where the test case comprises the plurality of testing elements generated by the GenAI model, and storing the test case within a storage device.

An example operation may include one or more of generating a large language model via a user interface, executing the large language model on a repository of software test cases and requirements of the software test cases to train the large language model to understand connections between test case components and test case requirements, receiving a description of features of a software program to be tested, and in response to receiving the description of the features, generating a software test case based on execution of the large language model on the received descriptions, and displaying the software test case via the user interface.

The invention discloses a verification method and system of a chip simulation model, a medium and a program product, and relates to the technical field of chip verification, and the method comprises the steps: obtaining a simulation model and a performance parameter index of a to-be-verified chip; generating model training data based on the historical performance indexes, the corresponding historical use case set and the historical verification result, and performing model training based on the model training data to obtain a use case generation model; inputting the performance parameter indexes to a case generation model to obtain a test case set and a test verification expectation; executing a plurality of test cases in the test case set based on the simulation model, and recording an actual response result of the simulation model; based on the actual response result and the test verification expectation, generating a test log, and determining an abnormal mark in the test log; and marking the fault features corresponding to the exception, taking the test cases as newly added training data, and optimizing the case generation model based on the newly added training data. According to the invention, the chip verification efficiency can be improved.

The present invention discloses a system and method for performing artificial intelligence (AI) safety red-teaming with integrated policy fuzzing and adversarial prompting to systematically identify, characterize, and mitigate unsafe or non-compliant behaviors in AI models. The disclosed invention automates the process of generating, executing, and analyzing adversarial test cases through coordinated functional units comprising a policy fuzzing unit, an adversarial prompting unit, an execution sandbox, a telemetry processing unit, a scoring and triage processor, and a cryptographic provenance processor. The system applies grammar-driven and reinforcement-based fuzzing techniques to vary policy descriptors, model configuration parameters, and instruction hierarchies, while a learned adversarial prompt generator synthesizes contextually coherent adversarial prompts optimized for maximum policy violation likelihood. The generated prompts and policy vectors are executed in an isolated, instrumented sandbox that records input-output interactions, timing characteristics, and intermediate representations.

The present application provides a vulnerability remediation code identification method, comprising: acquiring commit information of open-source software, the commit information comprising a committed code snippet and text information associated with the code snippet; inputting the commit information into a first language model and performing identification to obtain function change information and test case information; inputting the function change information into a vulnerability remediation identification model to obtain a vulnerability remediation code snippet for a target vulnerability, and inputting the test case information into a vulnerability test case identification model to obtain a test case code snippet for the target vulnerability; and inputting the vulnerability remediation code snippet for the target vulnerability and the test case code snippet for the target vulnerability into a second language model and performing integration and identification to obtain an identification result. On the basis of notable characteristics in a test case for a vulnerability, the identification rate of vulnerability remediation code can be effectively increased. In this way, a user can discover potential vulnerabilities in advance, thus reducing security risks.

The invention discloses an unmanned aerial vehicle flight control system vulnerability detection method based on data flow analysis and LLM, and belongs to the technical field of intelligent software testing. Comprising the following steps: extracting a code function module associated with user operation in an unmanned aerial vehicle flight control system through a data flow analysis method, and establishing an operation-code mapping relation library; generating a structured natural language semantic description for each function module code by adopting a large language model LLM, and forming a multi-dimensional semantic feature vector; based on correlation analysis of multi-module semantic features, a combined test scene is constructed, and a natural language test case is generated; the natural language test case is converted into an executable test code through reverse semantic mapping, and coding reconstruction of test logic is completed; and executing a test code and capturing a runtime log in an unmanned aerial vehicle simulation environment, and performing vulnerability feature extraction and root cause positioning by using a large language model. According to the method, the efficiency is improved, and meanwhile, the deep coverage test of a complex interaction scene is supported.

A computing platform may generate, using a test case generation model, a plurality of large language model (LLM) test cases. The computing platform may input, into an LLM, the plurality of LLM test cases, which may produce a plurality of unverified LLM test results. The computing platform may input, into a validation model, the plurality of LLM test cases, which may produce a plurality of validated LLM test results. The computing platform may compare, using a falsified output evaluation model, the plurality of unverified LLM test results with the corresponding plurality of validated LLM test results, which may produce an LLM compliance score for the LLM. The computing platform may compare the LLM compliance score to a compliance threshold. Based on identifying that the LLM compliance score meets or exceeds the compliance threshold, the computing platform may automatically deploy the LLM for use in an enterprise environment.

The invention provides a test case generation method and system based on a large model and retrieval enhancement generation, and the method comprises the steps: obtaining a software description document, carrying out text fragment segmentation and vectorization, and constructing a software knowledge base; obtaining and vectorizing a demand document, and recalling a content optimization demand document in the reference software knowledge base based on the retrieval enhancement generation framework; constructing a first Prompt instruction, and splitting the optimized demand document into demand fragments of different test points by using a large language model; and finally, constructing a second Prompt instruction for the demand fragment of each test point, generating a test case by using a large language model, and establishing a mapping relationship between each test point and the test case for linkage output. According to the method, the understanding ability of the large model on the demand document is effectively enhanced; meanwhile, the requirements are decomposed into test points, mapping is carried out on the test points and related test cases, and completeness and accuracy of the requirements and the test cases can be conveniently sorted.

The invention relates to a fuzzy testing method, device and equipment based on a large language model, and the method comprises the steps: analyzing and analyzing RFC standard documents of various network protocols through the large language model, and extracting protocol rules and constraint conditions; according to the protocol rule and the constraint condition, a large language model driving method is adopted to generate a test case conforming to protocol characteristics; and performing a differential test through the test case, and verifying the consistency between the actual software of the network protocol and the RFC standard document. According to the invention, the accuracy and security of implementation of the complex network protocol can be improved, and the test efficiency and coverage rate are optimized.

The invention particularly relates to a unit test case rapid generation method based on an AI large model. According to the unit test case rapid generation method based on the AI large model, structure information is extracted through a static AST, an execution path is tracked through dynamic instrumentation, and code analysis is achieved; an analysis result is converted into cue words through AI, a multi-scene use case is generated through reasoning, and the multi-scene use case is matched with a language framework; counting row / branch / path coverage, identifying gaps and optimizing loop logic; hot spots and paths are displayed through a visual report, IDE one-key repair and manual correction of a feedback model are supported, intelligent retrieval and reuse of historical cases are realized by relying on FAISS, and the generation accuracy is continuously improved. According to the unit test case rapid generation method based on the AI large model, high-coverage-rate test case generation is achieved through the AI large model, the test case generation efficiency is remarkably improved, the test period is shortened, the software quality is improved, the workload of manually compiling and maintaining the test case is reduced, and the development cost is reduced.

The invention discloses a test case determination method and device, a computer readable storage medium and a processor. The method comprises the following steps: identifying a source code of software by utilizing a pre-trained large model to obtain a plurality of functional modules in the software and a dependency relationship among the plurality of functional modules; in response to updating of the source code of the software, determining a code updating part in the source code; determining a test case matched with the code updating part from a test case library of the software based on the code updating part and a dependency relationship among the plurality of functional modules, the test case being used for testing the code updating part; based on a test result of the test case, adjusting a selection strategy of the test case; and re-determining the test case for testing the code updating part based on the adjusted selection strategy of the test case. The technical problem that a traditional regression test strategy is low in test efficiency is solved.

The invention discloses a performance test and verification method, system and device for a multi-core heterogeneous chip and a medium, and the method comprises the steps: configuring a test parameter set according to a target application scene, the test parameter set comprising a processor core type recognition parameter, a communication topology weight parameter and a load balancing threshold parameter; generating a test case covering the heterogeneous computing core cooperative working mode; the method comprises the following steps: acquiring characteristic data during testing through a reconfigurable monitoring sub-module, wherein the characteristic data comprises inter-core communication delay information, shared cache hit rate information and power consumption distribution thermodynamic information; and performing quantitative analysis on the feature data through a multi-dimensional evaluation model to obtain a performance deviation index and an architecture optimization suggestion parameter. By optimizing the chip performance verification process under the multi-core heterogeneous architecture, the verification precision is effectively improved, and the chip performance verification method can be widely applied to the technical field of chip design.

The invention discloses a parameter feature-based test case generation method and a related product. In the scheme, a parameter document of target simulation software is obtained; and extracting parameter feature information of the target simulation software based on the parameter document. And on the basis of the parameter feature information, matching a test case template and a code generation rule in a pre-constructed code knowledge base, and generating an initial test case code. And dynamically optimizing the initial test case code according to feedback data generated in a simulation test process to generate a target test case code. According to the technical scheme, the parameter feature information is automatically extracted, the complex dependency relationship between the parameters is accurately analyzed, the test case is automatically generated based on the knowledge base matching template and rule, software changes are adapted in real time in combination with a dynamic optimization mechanism, dynamic and efficient generation of test case codes is achieved, and the test efficiency is improved. The coverage degree and the adaptability of the test case codes are improved, and then the accuracy, the comprehensiveness and the efficiency of the test are improved.

The invention discloses a variation test method and device based on an intelligent automation script. The method comprises the following steps: extracting a statement execution sequence and a branch dependency relationship in an original source code by using a context-aware syntax tree, and generating a control flow diagram and a data flow diagram; performing code semantic analysis on the multi-dimensional code features by using a large language model, identifying potential defect types and variation rules, predicting the test efficiency of variants in combination with a deep Q learning model, and generating to-be-processed variants; performing grammar check and equivalence analysis on the to-be-processed variants to obtain effective variants, and testing the effective variants; variation test indexes are calculated, code weak points are positioned, and test blind areas are identified according to survival variants, so that a visual analysis report is generated; and iteratively generating a new test case according to the current survival variant and the visual analysis report, and testing the current survival variant by using the new test case. According to the invention, the variation test efficiency can be improved.

The invention discloses a scene granularity interface test case generation method based on multiple agents, and the method comprises a plurality of large language model agents and related components, namely context memory, an observer, a decision maker, an executor, a supervisor, a recorder and a to-be-tested application. An observer identifies the interface control, detection and attribute identification of the control are effectively focused, and it is ensured that accurate data input is provided for subsequent testing steps. Based on the output of an observer, a decision maker generates a detailed plan of the next operation according to the current page state, and an operation strategy can be independently formulated, so that test generation is supported by business scene logic. And an executor analyzes and executes the generated plan operation, carries out test operation strictly according to the plan, and ensures the traceability and repeatability of the test process. Furthermore, the supervisor verifies whether the operation is executed according to the plan and meets the requirements of the target scene, and makes corresponding feedback according to the check result, thereby ensuring that the wrong operation is corrected in time, avoiding the continuous accumulation of errors, and ensuring the effectiveness of each test step. The recorder is responsible for storing the execution instruction and the context information of the test process to the context memory, and the operation history and result in the test process can be traced through the test context information, so that a better decision can be made in the subsequent iteration, and the result export after the test is completed can be effectively supported. And after the recording is finished, the test generation enters the next iteration until the coverage of the test scene is finished.

The invention relates to a method for obtaining a hint injection attack test case for a large language model, which combines a conditional variation auto-encoder cVAE and a Markov chain, and obtains a large language model test case through data generation and context modeling, gradual exposure of malicious instructions and simulation of multiple rounds of dialogue attacks in reality. Realizing multiple rounds of dialogue attacks on the large language model, and challenging the defense capability of the large language model; the design scheme introduces a concealment technology, role play attack, state transition and other technologies, improves the complexity and concealment of attacks, has the core advantages of automation, higher concealment, wide coverage, batch testing and the like, can evaluate the security defense capability of the large language model more truly and comprehensively, and improves the security defense capability of the large language model. The defects of an existing defense mechanism are found, and research on multi-round prompt injection attacks and improvement of a security defense mechanism are promoted.

The embodiment of the invention provides a test case generation method and device.The method comprises the steps that a test case generation task is determined, and the test case generation task carries a target document, a task type and a document type of a to-be-generated test case; analyzing the target document according to the document type by using an information extraction agent to generate an analysis result of the target document; generating an intelligent agent by using a test point, and generating an initial test point according to the task type and the analysis result; and generating a first test case corresponding to the test case generation task by using a test case generation agent according to the initial test point and the analysis result. The analysis logic of a qualification test expert is simulated through a plurality of agents, and the efficiency and integrity of test case output are greatly improved.

The invention discloses an automatic driving vehicle key signal testing method and system based on real data driving. According to the method, data are collected in a real road scene, test cases are automatically generated, a standard test case library is constructed, hierarchical or parallel testing is carried out in virtual simulation, hardware-in-the-loop and real vehicle environments, and result feedback and closed-loop iteration are achieved through automatic analysis. The system comprises a data collection module, a test case generation module, a standard test case library module, a test execution module and a result feedback module. According to the invention, the problems of insufficient scene coverage and low authenticity and efficiency of the existing test method are solved, the comprehensiveness, authenticity and test efficiency of the test scene are improved, and the safety and reliability of the key signal system of the automatic driving vehicle are guaranteed.

The invention provides an automatic software testing method and system based on artificial intelligence. The method comprises the following steps: analyzing a software demand document by using an NLP technology, converting the software demand document into structured data, extracting a key function point, and identifying a software function point; generating a test case covering all function points through a deep learning model in combination with historical test data training; automatically executing a test operation, analyzing a test result, analyzing a test log by using a machine learning algorithm, identifying and classifying potential defects by using a data mining and pattern recognition technology, and generating a repair suggestion report; and analyzing feedback data in the test process, dynamically adjusting a strategy according to a test result by using a reinforcement learning algorithm, and optimizing a test case and the strategy. According to the invention, intelligent analysis of test data, automatic generation of test cases and automatic execution of a test process are realized; flexible configuration and expansion can be carried out according to the actual condition of software; the test efficiency and quality are improved, and the test cost and manpower consumption are reduced.

The invention discloses a test case generation method and device, equipment and a medium, and relates to the technical field of software testing, and the test case generation method comprises the following steps: storing request data subjected to structured processing into a seed case library; the request data are various types of request data in the operation process of the service system; determining a cue word according to the target information based on the large model, and generating an initial test case through the cue word and a generation rule; the target information comprises characteristics of request data in the seed case library, an interface document of the tested software and a business logic specification; executing the initial test case according to the test request of the tested software, performing statistical analysis on the code coverage rate of the tested software in the execution process, and screening the initial test case according to an analysis result to obtain a screened test case; and after the execution is finished, filtering the screened test cases according to the first execution result of each test case to obtain a target test case. Therefore, the testing efficiency and quality can be improved, and automatic testing is achieved.

The invention discloses an integrated verification and test system and method for complex rules of a substation monitoring system, and the system comprises a rule editing module which provides a visual interface to support a user to edit the complex rules containing various types of variables and time sequence constraints; the analysis and verification module is used for performing grammar, variable conflict and business compliance multi-level verification on the rules; the intelligent test case generation module is used for automatically generating test cases covering normal, boundary and abnormal conditions based on the rule tree structure; the simulation test interface module converts the use case into a standard test script, submits the standard test script to a simulation tool for execution, and outputs the standard test script to a monitoring system through protocol conversion; the result analysis module is used for comparing an execution result with an expected result and generating a structured test report; according to the invention, full-process automation of rule verification and testing is realized, the verification efficiency is remarkably improved, and the test requirements of various transformer substation business scenes are met.

The invention relates to the technical field of chip verification, and discloses a chip verification method, test equipment and a chip verification system.The method comprises the steps that a template file is built, a verification environment and a test case are generated based on the template file, a configuration file is obtained, and parameters in the verification environment and the test case are updated through parameters in the configuration file; according to the method, the verification environment and the test case are updated, the updated verification environment and the updated test case are obtained, regression testing is carried out on the chip on the basis of the updated verification environment and the updated test case, the test result is obtained, the verification environment can be rapidly generated on the basis of the template file, parameters can be rapidly replaced, and the chip testing efficiency is improved.

The invention relates to the technical field of software test automation, in particular to a self-repairing UI automatic detection method based on multi-modal fusion, which mainly comprises the following steps: extracting visual features to obtain feature vectors, and constructing a virtual DOM tree with semantic tags to carry out similarity calculation to obtain structural similarity; and carrying out weight dynamic adjustment on the feature vector and the structural similarity through a feature fusion decision maker, outputting a fusion weight proportion under a current confidence threshold, fusing the feature vector and the structural similarity through the weight proportion to form a decision combination, and generating a new locator. Through the method, the problem of test case vulnerability caused by a dynamic user interface is solved, the detection speed of real-time sensing of interface change is improved, and meanwhile, the positioning success rate during use of a new positioner automatically generated by a positioning strategy is also remarkably improved.

The invention discloses an agent evaluation method and device, electronic equipment, a storage medium and a program, and the method comprises the steps: determining a test case set of a target agent based on an agent role; generating a test result of the target agent based on the test case set; and determining an evaluation result corresponding to each test result according to an intelligent agent evaluation model. According to the embodiment of the invention, automatic evaluation of the intelligent agent can be realized, quantitative evaluation is carried out on the intelligent agent evaluation process, and objectivity of intelligent agent evaluation can be improved.

An end-to-end approach for (i) identifying potential vulnerabilities in software applications, including security vulnerabilities, (ii) verifying / confirming the potential vulnerabilities as actual vulnerabilities and (iii) in response, identifying the necessary remedial actions necessary to eliminate or at least mitigate the vulnerabilities. An intelligent agent is implemented that is configured to detect a change to the application's code or computing environment and, as a result of detection of changes to the code or computing environment, identify potential vulnerabilities, verify / confirm the potential vulnerabilities as actual vulnerabilities through determination / creation of vulnerability test cases and automatically identify the necessary remedial actions necessary to eliminate or mitigate the vulnerabilities.

The invention discloses an artificial intelligence robot path planning method and system, and the method comprises the steps: 1, carrying out the comprehensive perception of an environment geometric structure, object semantics and dynamic obstacles, eliminating scene differences, and further constructing the topological graph representation of an environment; 2, pre-training a path planning model, designing a scene context encoder, encoding a scene specific rule into a low-dimensional vector, and outputting a scene context code and updated planning model parameters to provide a planning capability adapted to a new scene for the step 3; 3, dynamically adjusting a track according to real-time sensor data by adopting a hierarchical decision-making architecture; step 2, recording success / failure path segments in the new scene, regularly updating a local planning module, regularly feeding back empirical data in the new scene to the step 2, and triggering a conservative obstacle avoidance mode when the confidence coefficient is lower than a threshold value; and 4, constructing a quantitative evaluation system, building an automatic test platform, simulating diversified scenes and dynamic interference, and automatically generating a test case.

Self-tuning merged code testing is provided which includes testing merged code using a suite of test cases, where the merged code includes one or more code changes, and obtaining, based on the testing, a test case failure using the suite of test cases. Further, the process includes determining, using an artificial intelligence engine, a likely faulty code change of the one or more code changes resulting in the test case failure, and customizing, based on the likely faulty code change, the suite of test cases to facilitate verifying that the likely faulty code change is a faulty code change. In addition, the process includes continuing testing of the merged code using the customized suite of test cases to facilitate verifying that the likely faulty code change is the faulty code change.

The invention discloses a training sample data generation method and device, a readable storage medium and a program product, and relates to the technical field of artificial intelligence. The method comprises the following steps: inputting a seed topic and a cue word of a programming language processing task into a sample data generation model; and modifying the seed topic in the modification range boundary condition by using the cue word based on the same algorithm thinking, and generating a test case set covering the constraint condition of the new topic according to the type of the new topic and the case generation condition. Answering the new question passing the verification, and outputting an answer passing the verification of the test case set; taking the new question, the test case set and the answer as training sample data; and adjusting and modifying the range boundary condition and / or the cue word and / or the use case generation condition, and generating new training sample data by utilizing the new question. The problem that training data generated by related technologies cannot meet model training requirements can be solved, and high-quality training sample data can be efficiently generated for programming language processing tasks.

The invention discloses a regression test case determination method and device, a storage medium and electronic equipment, and relates to the field of artificial intelligence. The method comprises the steps of obtaining a target change code of a target program; the target change code is input into a vector machine model, code features of the target change code are extracted through the vector machine model, a potential defect score of the target change code is determined according to the code features, and the potential defect score is used for representing the probability that the target program is abnormal due to the target change code; and determining at least one regression test case for testing the target change code from the M test cases according to the potential defect score of the target change code, the code feature of the target change code and the case feature of each test case in the M test cases. The technical problem that in the prior art, the efficiency of compiling the regression test case is low is solved.

Embodiments of the present disclosure provide a fuzz testing method, an electronic device, and a computer readable storage medium. The fuzz testing method comprises: acquiring a data packet; generating a protocol rule tree model on the basis of the data packet; generating a test case on the basis of the protocol rule tree model; sending the test case to a device under test; and determining a test result on the basis of response information of the device under test.