4307 results about "Test case" patented technology

The present invention discloses a system and method for performing artificial intelligence (AI) safety red-teaming with integrated policy fuzzing and adversarial prompting to systematically identify, characterize, and mitigate unsafe or non-compliant behaviors in AI models. The disclosed invention automates the process of generating, executing, and analyzing adversarial test cases through coordinated functional units comprising a policy fuzzing unit, an adversarial prompting unit, an execution sandbox, a telemetry processing unit, a scoring and triage processor, and a cryptographic provenance processor. The system applies grammar-driven and reinforcement-based fuzzing techniques to vary policy descriptors, model configuration parameters, and instruction hierarchies, while a learned adversarial prompt generator synthesizes contextually coherent adversarial prompts optimized for maximum policy violation likelihood. The generated prompts and policy vectors are executed in an isolated, instrumented sandbox that records input-output interactions, timing characteristics, and intermediate representations.

The invention discloses an unmanned aerial vehicle flight control system vulnerability detection method based on data flow analysis and LLM, and belongs to the technical field of intelligent software testing. Comprising the following steps: extracting a code function module associated with user operation in an unmanned aerial vehicle flight control system through a data flow analysis method, and establishing an operation-code mapping relation library; generating a structured natural language semantic description for each function module code by adopting a large language model LLM, and forming a multi-dimensional semantic feature vector; based on correlation analysis of multi-module semantic features, a combined test scene is constructed, and a natural language test case is generated; the natural language test case is converted into an executable test code through reverse semantic mapping, and coding reconstruction of test logic is completed; and executing a test code and capturing a runtime log in an unmanned aerial vehicle simulation environment, and performing vulnerability feature extraction and root cause positioning by using a large language model. According to the method, the efficiency is improved, and meanwhile, the deep coverage test of a complex interaction scene is supported.

A computing platform may generate, using a test case generation model, a plurality of large language model (LLM) test cases. The computing platform may input, into an LLM, the plurality of LLM test cases, which may produce a plurality of unverified LLM test results. The computing platform may input, into a validation model, the plurality of LLM test cases, which may produce a plurality of validated LLM test results. The computing platform may compare, using a falsified output evaluation model, the plurality of unverified LLM test results with the corresponding plurality of validated LLM test results, which may produce an LLM compliance score for the LLM. The computing platform may compare the LLM compliance score to a compliance threshold. Based on identifying that the LLM compliance score meets or exceeds the compliance threshold, the computing platform may automatically deploy the LLM for use in an enterprise environment.

The invention relates to a fuzzy testing method, device and equipment based on a large language model, and the method comprises the steps: analyzing and analyzing RFC standard documents of various network protocols through the large language model, and extracting protocol rules and constraint conditions; according to the protocol rule and the constraint condition, a large language model driving method is adopted to generate a test case conforming to protocol characteristics; and performing a differential test through the test case, and verifying the consistency between the actual software of the network protocol and the RFC standard document. According to the invention, the accuracy and security of implementation of the complex network protocol can be improved, and the test efficiency and coverage rate are optimized.

The invention particularly relates to a unit test case rapid generation method based on an AI large model. According to the unit test case rapid generation method based on the AI large model, structure information is extracted through a static AST, an execution path is tracked through dynamic instrumentation, and code analysis is achieved; an analysis result is converted into cue words through AI, a multi-scene use case is generated through reasoning, and the multi-scene use case is matched with a language framework; counting row / branch / path coverage, identifying gaps and optimizing loop logic; hot spots and paths are displayed through a visual report, IDE one-key repair and manual correction of a feedback model are supported, intelligent retrieval and reuse of historical cases are realized by relying on FAISS, and the generation accuracy is continuously improved. According to the unit test case rapid generation method based on the AI large model, high-coverage-rate test case generation is achieved through the AI large model, the test case generation efficiency is remarkably improved, the test period is shortened, the software quality is improved, the workload of manually compiling and maintaining the test case is reduced, and the development cost is reduced.

The invention discloses a performance test and verification method, system and device for a multi-core heterogeneous chip and a medium, and the method comprises the steps: configuring a test parameter set according to a target application scene, the test parameter set comprising a processor core type recognition parameter, a communication topology weight parameter and a load balancing threshold parameter; generating a test case covering the heterogeneous computing core cooperative working mode; the method comprises the following steps: acquiring characteristic data during testing through a reconfigurable monitoring sub-module, wherein the characteristic data comprises inter-core communication delay information, shared cache hit rate information and power consumption distribution thermodynamic information; and performing quantitative analysis on the feature data through a multi-dimensional evaluation model to obtain a performance deviation index and an architecture optimization suggestion parameter. By optimizing the chip performance verification process under the multi-core heterogeneous architecture, the verification precision is effectively improved, and the chip performance verification method can be widely applied to the technical field of chip design.

The invention discloses a parameter feature-based test case generation method and a related product. In the scheme, a parameter document of target simulation software is obtained; and extracting parameter feature information of the target simulation software based on the parameter document. And on the basis of the parameter feature information, matching a test case template and a code generation rule in a pre-constructed code knowledge base, and generating an initial test case code. And dynamically optimizing the initial test case code according to feedback data generated in a simulation test process to generate a target test case code. According to the technical scheme, the parameter feature information is automatically extracted, the complex dependency relationship between the parameters is accurately analyzed, the test case is automatically generated based on the knowledge base matching template and rule, software changes are adapted in real time in combination with a dynamic optimization mechanism, dynamic and efficient generation of test case codes is achieved, and the test efficiency is improved. The coverage degree and the adaptability of the test case codes are improved, and then the accuracy, the comprehensiveness and the efficiency of the test are improved.

The invention discloses a variation test method and device based on an intelligent automation script. The method comprises the following steps: extracting a statement execution sequence and a branch dependency relationship in an original source code by using a context-aware syntax tree, and generating a control flow diagram and a data flow diagram; performing code semantic analysis on the multi-dimensional code features by using a large language model, identifying potential defect types and variation rules, predicting the test efficiency of variants in combination with a deep Q learning model, and generating to-be-processed variants; performing grammar check and equivalence analysis on the to-be-processed variants to obtain effective variants, and testing the effective variants; variation test indexes are calculated, code weak points are positioned, and test blind areas are identified according to survival variants, so that a visual analysis report is generated; and iteratively generating a new test case according to the current survival variant and the visual analysis report, and testing the current survival variant by using the new test case. According to the invention, the variation test efficiency can be improved.

The invention relates to a method for obtaining a hint injection attack test case for a large language model, which combines a conditional variation auto-encoder cVAE and a Markov chain, and obtains a large language model test case through data generation and context modeling, gradual exposure of malicious instructions and simulation of multiple rounds of dialogue attacks in reality. Realizing multiple rounds of dialogue attacks on the large language model, and challenging the defense capability of the large language model; the design scheme introduces a concealment technology, role play attack, state transition and other technologies, improves the complexity and concealment of attacks, has the core advantages of automation, higher concealment, wide coverage, batch testing and the like, can evaluate the security defense capability of the large language model more truly and comprehensively, and improves the security defense capability of the large language model. The defects of an existing defense mechanism are found, and research on multi-round prompt injection attacks and improvement of a security defense mechanism are promoted.

The embodiment of the invention provides a test case generation method and device.The method comprises the steps that a test case generation task is determined, and the test case generation task carries a target document, a task type and a document type of a to-be-generated test case; analyzing the target document according to the document type by using an information extraction agent to generate an analysis result of the target document; generating an intelligent agent by using a test point, and generating an initial test point according to the task type and the analysis result; and generating a first test case corresponding to the test case generation task by using a test case generation agent according to the initial test point and the analysis result. The analysis logic of a qualification test expert is simulated through a plurality of agents, and the efficiency and integrity of test case output are greatly improved.

The invention discloses an automatic driving vehicle key signal testing method and system based on real data driving. According to the method, data are collected in a real road scene, test cases are automatically generated, a standard test case library is constructed, hierarchical or parallel testing is carried out in virtual simulation, hardware-in-the-loop and real vehicle environments, and result feedback and closed-loop iteration are achieved through automatic analysis. The system comprises a data collection module, a test case generation module, a standard test case library module, a test execution module and a result feedback module. According to the invention, the problems of insufficient scene coverage and low authenticity and efficiency of the existing test method are solved, the comprehensiveness, authenticity and test efficiency of the test scene are improved, and the safety and reliability of the key signal system of the automatic driving vehicle are guaranteed.

The invention provides an automatic software testing method and system based on artificial intelligence. The method comprises the following steps: analyzing a software demand document by using an NLP technology, converting the software demand document into structured data, extracting a key function point, and identifying a software function point; generating a test case covering all function points through a deep learning model in combination with historical test data training; automatically executing a test operation, analyzing a test result, analyzing a test log by using a machine learning algorithm, identifying and classifying potential defects by using a data mining and pattern recognition technology, and generating a repair suggestion report; and analyzing feedback data in the test process, dynamically adjusting a strategy according to a test result by using a reinforcement learning algorithm, and optimizing a test case and the strategy. According to the invention, intelligent analysis of test data, automatic generation of test cases and automatic execution of a test process are realized; flexible configuration and expansion can be carried out according to the actual condition of software; the test efficiency and quality are improved, and the test cost and manpower consumption are reduced.

The invention discloses a test case generation method and device, equipment and a medium, and relates to the technical field of software testing, and the test case generation method comprises the following steps: storing request data subjected to structured processing into a seed case library; the request data are various types of request data in the operation process of the service system; determining a cue word according to the target information based on the large model, and generating an initial test case through the cue word and a generation rule; the target information comprises characteristics of request data in the seed case library, an interface document of the tested software and a business logic specification; executing the initial test case according to the test request of the tested software, performing statistical analysis on the code coverage rate of the tested software in the execution process, and screening the initial test case according to an analysis result to obtain a screened test case; and after the execution is finished, filtering the screened test cases according to the first execution result of each test case to obtain a target test case. Therefore, the testing efficiency and quality can be improved, and automatic testing is achieved.

The invention discloses an integrated verification and test system and method for complex rules of a substation monitoring system, and the system comprises a rule editing module which provides a visual interface to support a user to edit the complex rules containing various types of variables and time sequence constraints; the analysis and verification module is used for performing grammar, variable conflict and business compliance multi-level verification on the rules; the intelligent test case generation module is used for automatically generating test cases covering normal, boundary and abnormal conditions based on the rule tree structure; the simulation test interface module converts the use case into a standard test script, submits the standard test script to a simulation tool for execution, and outputs the standard test script to a monitoring system through protocol conversion; the result analysis module is used for comparing an execution result with an expected result and generating a structured test report; according to the invention, full-process automation of rule verification and testing is realized, the verification efficiency is remarkably improved, and the test requirements of various transformer substation business scenes are met.

The invention relates to the technical field of chip verification, and discloses a chip verification method, test equipment and a chip verification system.The method comprises the steps that a template file is built, a verification environment and a test case are generated based on the template file, a configuration file is obtained, and parameters in the verification environment and the test case are updated through parameters in the configuration file; according to the method, the verification environment and the test case are updated, the updated verification environment and the updated test case are obtained, regression testing is carried out on the chip on the basis of the updated verification environment and the updated test case, the test result is obtained, the verification environment can be rapidly generated on the basis of the template file, parameters can be rapidly replaced, and the chip testing efficiency is improved.

The invention relates to the technical field of software test automation, in particular to a self-repairing UI automatic detection method based on multi-modal fusion, which mainly comprises the following steps: extracting visual features to obtain feature vectors, and constructing a virtual DOM tree with semantic tags to carry out similarity calculation to obtain structural similarity; and carrying out weight dynamic adjustment on the feature vector and the structural similarity through a feature fusion decision maker, outputting a fusion weight proportion under a current confidence threshold, fusing the feature vector and the structural similarity through the weight proportion to form a decision combination, and generating a new locator. Through the method, the problem of test case vulnerability caused by a dynamic user interface is solved, the detection speed of real-time sensing of interface change is improved, and meanwhile, the positioning success rate during use of a new positioner automatically generated by a positioning strategy is also remarkably improved.

The invention discloses an artificial intelligence robot path planning method and system, and the method comprises the steps: 1, carrying out the comprehensive perception of an environment geometric structure, object semantics and dynamic obstacles, eliminating scene differences, and further constructing the topological graph representation of an environment; 2, pre-training a path planning model, designing a scene context encoder, encoding a scene specific rule into a low-dimensional vector, and outputting a scene context code and updated planning model parameters to provide a planning capability adapted to a new scene for the step 3; 3, dynamically adjusting a track according to real-time sensor data by adopting a hierarchical decision-making architecture; step 2, recording success / failure path segments in the new scene, regularly updating a local planning module, regularly feeding back empirical data in the new scene to the step 2, and triggering a conservative obstacle avoidance mode when the confidence coefficient is lower than a threshold value; and 4, constructing a quantitative evaluation system, building an automatic test platform, simulating diversified scenes and dynamic interference, and automatically generating a test case.

The invention discloses a training sample data generation method and device, a readable storage medium and a program product, and relates to the technical field of artificial intelligence. The method comprises the following steps: inputting a seed topic and a cue word of a programming language processing task into a sample data generation model; and modifying the seed topic in the modification range boundary condition by using the cue word based on the same algorithm thinking, and generating a test case set covering the constraint condition of the new topic according to the type of the new topic and the case generation condition. Answering the new question passing the verification, and outputting an answer passing the verification of the test case set; taking the new question, the test case set and the answer as training sample data; and adjusting and modifying the range boundary condition and / or the cue word and / or the use case generation condition, and generating new training sample data by utilizing the new question. The problem that training data generated by related technologies cannot meet model training requirements can be solved, and high-quality training sample data can be efficiently generated for programming language processing tasks.

Embodiments of the present disclosure provide a fuzz testing method, an electronic device, and a computer readable storage medium. The fuzz testing method comprises: acquiring a data packet; generating a protocol rule tree model on the basis of the data packet; generating a test case on the basis of the protocol rule tree model; sending the test case to a device under test; and determining a test result on the basis of response information of the device under test.

The invention discloses a ship mobile energy storage system CAN communication protocol automatic test system and method, and the system comprises a protocol modeling and test configuration module, a simulation node behavior simulation module, a CAN bus interaction and power closed loop module, and a test analysis and result evaluation module. A CAN communication protocol specification of the ship mobile energy storage system and parameters of tested equipment can be obtained, a protocol analysis model and a test configuration file are constructed for format verification and logic consistency verification, meanwhile, a corresponding test case sequence is automatically generated, and the message sending time is controlled to generate a time sequence simulation message flow; and sending the time sequence simulation message flow to the tested equipment, simulating energy interaction behavior acquisition to generate a power feedback data set, testing to obtain a signal consistency deviation and a power execution error to construct a protocol conformity evaluation matrix, and generating a visual test report. According to the invention, the high coverage rate and automatic closed-loop test of the CAN communication protocol can be realized.

An automation testing system includes a processor and a memory storing historical data which at least comprising past test results including input parameters and testing outcomes for each test case included in the past test results. The processor is configured to identify input parameters for a first iteration of a first software application having first functionalities; execute an initial test on the first iteration to generate initial results; based on the input parameters and the initial results, collecting first historical data at least comprising first past test results for at least one second software application having second functionalities corresponding to the first functionalities; training a model employing AI or ML based on the initial results and the first historical data to generate a trained model; executing the trained model with the input parameters as input to generate a set of test cases for testing the first functionalities.

The present disclosure provides a verification system, a verification method, an electronic device and a storage medium. The verification system includes a simulation verification device, a first portion and a second portion. The first portion includes a first master module and a second slave module. The second slave module includes a memory unit connected to a memory access interface at a periphery of a design under test. The second portion includes a first direct programming interface, a second direct programming interface, a function library module, and a test case module. The first direct programming interface communicates with the first master module and is configured to call a first function to implement front door access to a register of the design under test. The second direct programming interface communicates with the memory unit and is configured to call a second function to implement back door access to the memory unit.

The invention discloses an automatic driving test case generation method based on a large model retrieval enhancement technology, and aims to improve the generation efficiency of an automatic driving test scene. The method comprises the following steps: firstly, cutting a preprocessed data text into blocks, and vectorizing the text through an embedded model to construct an efficient vector database; secondly, preliminarily retrieving related text blocks by using an improved hybrid retrieval algorithm, and reordering by using a BERT cross encoder to improve the relevance of retrieval results; aiming at the complex task requirement in the automatic driving test, the method is divided into a plurality of sub-tasks to be executed in parallel, so that the retrieval efficiency and accuracy are improved. And then, a LoRA strategy is adopted to finely adjust the large language model so as to adapt to an automatic driving scene generation task. And finally, jointly inputting the cue word template and the reordering result into a fine tuning model to generate a test case. According to the invention, a brand new technical scheme is provided for automatic driving test scene generation, and the method is of great significance in improving the test evaluation level of automatic driving and accelerating the landing of the automatic driving technology.

The invention discloses an automatic driving test case self-evolution repairing method based on a large language model, and belongs to the technical field of testing and performance optimization of an automatic driving system. According to the automatic driving test case self-evolution repairing method based on the large language model, deep semantic mining is conducted on a performance log of the model by means of semantic comprehension and recommendation capacity of the large language model, and a high-quality repairing scene is recommended from a structured test scene library. The method comprises the steps of performance log collection and pre-evaluation execution, failure mode semantic extraction, a scene semantic retrieval mechanism, initial recommendation construction, reflection type semantic optimization, few-sample fine adjustment adaptation and a closed-loop self-evolution mechanism. The method has obvious advantages in the aspects of automatic driving test case recommendation, failure mode recognition, rapid model capability adaptation and system closed-loop self-evolution, and the safety, generalization capability and test maintainability of the automatic driving system can be remarkably improved.

The invention relates to a test case generation method and system based on multi-agent efficient collaboration, belongs to the technical field of software testing, and solves the problems of incomplete scene coverage, logic disorder and the like when a single large model processes a complex task. The method comprises the following steps: constructing a multi-modal professional field knowledge base; the task planning agent module generates a test case generation task based on a project development document and a software source code file of a to-be-tested project and distributes the test case generation task to the test demand analysis agent module; based on the test case generation task, extracting a test demand point related to the tested software configuration item, describing the test demand point, and labeling a corresponding tracking relationship between a test demand point ID and a code snippet or a function in the software source code file to construct a test demand-code snippet / function set; and generating a test case and a test description document based on the test demand-code snippet / function set and the multi-modal professional domain knowledge base. And the ability of generating the test case of the large model is enhanced through organic and efficient cooperation of multiple agents.

The invention discloses an intelligent agent network access evaluation method and system based on automatic label generation and multi-dimensional evaluation, and the method comprises the steps: automatically generating an intelligent agent multi-dimensional label through a large-scale language model, constructing an intelligent agent portrait, and combining automatic test case generation and multi-dimensional evaluation index collection. The problems that in the prior art, agent evaluation standardization is lacked, evaluation efficiency is low, and'black box 'diagnosis is difficult are solved, standardized and automatic quality evaluation is carried out on various heterogeneous agents connected to a platform, and an agent portrait capable of being used for accurate retrieval and scheduling is generated. The method and the system can be widely applied to scenes such as a multi-agent collaboration platform, an agent store and an enterprise-level agent management system, and have the remarkable advantages of remarkably improving the ecological quality of the agent, greatly reducing the manual evaluation cost, accelerating the introduction and commercialization of a new agent and the like.

The invention discloses an AI-driven automatic testing and document generation method and system, and the method comprises the steps: obtaining document data of test software, carrying out the preprocessing and information extraction, and forming structured data; identifying key test points from the structured data by using an agent, and generating a test case; executing the test case by utilizing a preset automatic test framework to obtain a test result; retrieving data information associated with the structured data in the knowledge base by utilizing RAG to form a retrieval result set; fusing the retrieval result set with the structured data and the test result according to a preset weight strategy to construct a complete context; and taking the context as input, calling the large language model by utilizing an agent, and generating a technical document. By means of the technical scheme, accurate and real-time technical document generation is achieved, the document generation quality can be continuously improved, and the real-time updating capacity and the data driving decision support capacity of the document are improved.

Disclosed herein are system, method, and computer program product embodiments for detecting anomalies during software testing. The methods include generating a plurality of test reports for the software program by executing one or more test cases on a plurality of versions of the software program, generating a control chart based on the plurality of test reports, generating an alert when at least one testing characteristic includes an anomaly over the plurality of versions of the software program as determined based on the control chart. The control chart includes a plot associated with at least one testing characteristic of the software program, and a historical context associated with execution of the one or more test cases on the plurality of versions of the software program.

The invention relates to the technical field of SRAM (Static Random Access Memory) memory anti-radiation testing, and discloses an SRAM memory anti-radiation testing method and system based on running state injection, and the method comprises the following steps: system initialization, mapping preparation and time sequence baseline establishment; carrying out model quantitative compiling, sensitivity analysis and physical bit candidate generation; generating a running state injection plan and loading a test case; performing operation state execution, disturbance injection and synchronous monitoring acquisition; multi-source triggering, weight read-back and fault decoupling judgment are carried out; and performing quantitative evaluation on anti-radiation performance, constructing a degradation curve and outputting a report. The system corresponds to the method. According to the invention, a hardware-in-the-loop operation state injection and online evaluation technology is constructed, and the technical problems of guidance and position limited disturbance / equivalent irradiation injection based on weight bit level sensitivity and logic-physical mapping are solved.

The invention discloses an unmanned aerial vehicle automatic test method, which comprises the steps of building a high-fidelity semi-physical simulation platform which comprises an unmanned aerial vehicle motion simulator, a three-dimensional environment simulator, a flight control module and a ground control station, and loading a complete machine fault model and a dynamic environment model; designing an intelligent multi-fault injection scheme, and defining a single fault scene, a combined fault scene and a fault evolution scheme; configuring a dynamic environment simulation scene including static environment parameters, dynamic environment elements and environment, and a fault interaction scene; executing a self-adaptive test process, automatically generating a test case, intelligently scheduling a test sequence and dynamically adjusting test parameters; and performing multi-source data intelligent analysis and evaluation, collecting test data, performing fusion analysis on fault influence, and generating an evaluation report. By constructing a whole machine fault and dynamic environment model, the scene authenticity is improved by 40%, and for example, the motor fault yaw deviation under strong wind is closer to a real machine compared with a traditional method.