2661results about "Software testing/debugging" patented technology

A system, method, and computer-program product includes identifying a plurality of code synthesis items for a target programming language, generating a code synthesis prompt based on a first sampling of the plurality of code synthesis items, synthesizing, via a large language model, a plurality of raw code segments using the code synthesis prompt, executing the plurality of raw code segments with a code interpreter associated with the target programming language, determining one or more valid code segments of the plurality of raw code segments that the code interpreter successfully executed, aggregating, via a second sampling, the one or more valid code segments into one or more validated code synthesis training samples, and training a code generation model using the one or more validated code synthesis training samples. User interfaces may be provided to allow target coding tasks to be specified via text or speech.

The techniques disclosed herein enhance software testing through automated test case generation and implementation. This is accomplished through utilizing a natural language interface and a generative probabilistic model, such as a large language model. A first user input can provide descriptions of a software feature to be tested and various use cases for the software feature. Accordingly, the probabilistic model generates a plurality of test cases for evaluating various aspects of the software feature in accordance with the first user input. The natural language descriptions can be accompanied by a contextual input to constrain the probabilistic model and ensure consistent outputs. Subsequently, a second user input selects a test case from the generated test cases to be implemented. In response, the probabilistic model generates a software implementation of the selected test case. The software implementation is then executed with log data extracted from the testing environment for processing results.

In one embodiment, a computer-implemented method executed using one or more processors of an incident management system comprises receiving a notification of an incident associated with a computer system, and in response to receiving the notification: extracting an error message from the notification; reading a set of computer program code changes that have been implemented in the computer system; matching the error message to the set of computer program code changes and outputting a set of one or more candidate code changes that may correspond to the error message; based on the set of candidate code changes, generating an automatic remediation for the incident; and using the one or more processors of the incident management system, executing the automatic remediation on the computer system.

Technical solutions are directed to creation and deployment of multi-stage test problems. A processor can provide, to a client, a first data structure for a first stage of a multi-stage test problem including a description for generation of a client solution for the first stage and a test case to evaluate the client solution. The processor can receive, from the client, a client data structure comprising the client solution including a client computer code generated at the client for the first stage. The processor can determine, by evaluating the client solution using an input value and an output value of the test case, that the output of the client solution satisfies a validity condition of the output value. The processor can provide, to the client, based on satisfying the validity condition, a second data structures for a second stage of the multi-stage test problem.

Systems and methods described herein relate to automated troubleshooting for application development tools. First data comprise a plurality of delta objects and a user query. The plurality of delta objects identify design-time changes made to an application using a development tool. The user query is received via a user interface and relates to at least one of the design-time changes. The first data is preprocessed to obtain second data. The preprocessing of the first data includes modifying a subset of the first data and adding one or more predetermined instructions to the first data. The second data is provided to a machine learning model to obtain a response to the user query. Output indicative of the response is caused to be presented in the user interface.

An intelligent technical procedure to merge two or more application programming interfaces (APIs) leveraging generative AI is disclosed. API metadata is extracted from two or more APIs (e.g., name of API, endpoints, request / response formats (JSON or XLM), authentication mechanism (API keys, OAuth, or JWT tokens), rate limits, error codes, error messages, version scheme and API documentation etc.) to formulate and generate new API based on user prompt. Using Abstract Syntax tree (AST) advance deep learning generative AI algorithms like Generative pre-trained transformer (GPT) or the like, the method can merge and construct new API based on user's requirement prompt. Method leverages smart contracts to test new generated API and deploy API in desired environment based on blockchain consent protocol to manage API.

The present disclosure relates to a method for generating code for a mobile communication tester. The method comprises the steps of: receiving information on a test scenario at an inputting environment; and automatically generating a test script code for the mobile communication tester based on the received information; wherein the test script code is configured in such a way that, when it is executed by the mobile communication tester, it causes the mobile communication tester to generate a test routine according to the test scenario, wherein the test routine adheres to a mobile communication standard.

Systems and methods for generating synthetic test data for testing a software solution. Systems and methods include receiving a testing task from a user, identifying test properties of the testing task, gathering initial information based on the test properties of the testing task, forming a training dataset based on the initial information and the test properties, pretraining a generative AI model based on a large language model (LLM) using the training dataset, configuring synthetic test data based on the test properties, and generating synthetic test data according to the testing task using the generative AI model.

The invention discloses a method and a system for realizing automatic code generation by using multiple AI Agents, and relates to the field of electric digital data processing, a test driving agent receives a task transmitted by a previous node, and generates a task number, a task description, test data, a unit test script and a code statement; the code generation agent generates or updates codes according to the received task description and code declaration; the test driving agent returns an integrated test script and test data according to the task number; the continuous integration agent performs integration verification and regression test and feeds back information; the test driving agent returns a function test script, a performance test script and test data according to the task number; and the continuous deployment agent carries out continuous delivery and deployment and feeds back information. By means of mutual conversation and cooperative work of multiple AI Agents and in combination with a feedback mechanism, the problem that a single large model cannot complete complex software development tasks is solved, full-process operation automation can be achieved, and the code generation quality and efficiency can be improved.

Systems and methods for artificial intelligence (Al) assisted workflow integration for software development in a digital model platform are provided. In one embodiment, the methods receive a user selection of a target digital tool from a plurality of digital tools that are not directly interoperable with each other. Receive a user request comprising a description of a function script executable by the digital model platform on a target digital model type associated with the target digital tool. Fine-tune a scripting Al agent on prior user actions involving the target digital tool and on a resource-capability mapping of the digital model platform. Generate the function script and generate a unit test script, where the unit test script when interpreted tests the function script against the description. Finally, interpret the unit test script to generate a verification result for the function script; and, generate a test report based on the verification result.

The present application provides a code testing method, applied to a code testing system. When a test code is generated, context information of a code under test is extracted, and the test code is generated on the basis of the context information of the code under test. The context information often has a service meaning and can embody a test scenario, so that the prior knowledge of the service and the test scenario is achieved in the generation process of test codes, thereby generating test codes having higher quality.

In one implementation, a device obtains code generated by a language model-based agent to perform an action of a particular type with respect to a computer network. The device determines one or more parameters to execute the code in a testing environment. The device performs a validation assessment of the code to assess whether it is able to perform actions of the particular type by executing it with the one or more parameters in the testing environment. The device makes, based on the validation assessment, the code available to the language model-based agent to perform a subsequent action of the particular type.

Embodiments of the present disclosure provide a fuzz testing method, an electronic device, and a computer readable storage medium. The fuzz testing method comprises: acquiring a data packet; generating a protocol rule tree model on the basis of the data packet; generating a test case on the basis of the protocol rule tree model; sending the test case to a device under test; and determining a test result on the basis of response information of the device under test.

A system and method for automatic generation and automatic conduct of a chaos experiment in distributed applications includes a software application having a layer and a chaos configuration, a chaos engine having a chaos orchestrator, a chaos prompt generator having a prompt template composer, and a chaos engineering large language model. The chaos engineering large language model is configured to recommend chaos experiments for the chaos engine to process and to inject into the software application. The chaos engineering large language model is also configured to patch the software application in accordance with objectives and parameters developed in the system and method. The system and method may be automated through a trigger. The trigger may be the chaos engineering large language model recommending one or more chaos experiments to the chaos engine. The trigger may also be the implementation of one or more patches to the software application.

A method, computer system, and a computer program product for task assistance is provided. The present invention may include acquiring a request expression input by a user. The present invention may include identifying a request intent associated with a task based on the request expression. The present invention may include determining a response script corresponding to the request intent. The present invention may include executing the response script to complete the task and presenting the process of running the task in a user-interface (UI).

An automation testing system includes a processor and a memory storing historical data which at least comprising past test results including input parameters and testing outcomes for each test case included in the past test results. The processor is configured to identify input parameters for a first iteration of a first software application having first functionalities; execute an initial test on the first iteration to generate initial results; based on the input parameters and the initial results, collecting first historical data at least comprising first past test results for at least one second software application having second functionalities corresponding to the first functionalities; training a model employing AI or ML based on the initial results and the first historical data to generate a trained model; executing the trained model with the input parameters as input to generate a set of test cases for testing the first functionalities.

Systems are provided for implementing methods for generating and processing fix suggestions for alerts produced by code analysis tools. The methods involve receiving an alert, parsing the alert to identify relevant information, and assembling a prompt for a Large Language Model (LLM). The prompt includes a general description of the problem, a description of the alert message, a source-code location, and a request for a proposed fix to the alert. The prompt is sent to the LLM, and a response is obtained. The response includes a proposed fix comprising a set of edits to be applied to the source code and / or configuration files. The response is processed and validated, and a fix suggestion is assembled and provided based on the response. The fix suggestion includes a natural language explanation of the proposed fix, suggested source-code changes, and selectable options for accepting, rejecting, and editing the fix suggestion.

The invention discloses an APP automatic testing method based on multi-modal perception and an Agent system, and relates to the field of image recognition or understanding, and the method comprises the following steps: a real-time perception layer carries out the real-time screenshot of a screen interface of an APP; when the interface changes, processing the screenshot in three paths: detecting a control bounding box; performing visual semantic coding on the interface elements; synchronously extracting the text content, and generating a function label. And fusing the three paths of output results to construct a UI tree. When a task instruction is input, the dynamic reasoning layer generates an action sequence according to the task instruction and the UI tree through a nerve-symbol collaboration mechanism, updates a dynamic document and stores the dynamic document in a real-time knowledge base. And the execution optimization layer converts the action of the action sequence into an operation code exclusive to the platform, executes the operation code, automatically marks an abnormal path at the same time, and updates an avoidance strategy to a real-time knowledge base. According to the method, real-time sensing, dynamic reasoning and closed loop execution are realized through a three-level architecture, and the generalization ability, the exception coverage and the execution efficiency are remarkably improved.

The disclosed system for testing of fleet management software generates virtual assets (e.g., simulations of machinery) that are associated with simulated sensor values. For at least one particular asset in the set, the system generates a simulated application programming interface (API) message definition structured to capture the simulated sensor values. The system binds the set of virtual assets to a simulation scenario record that includes a trigger condition. The system causes a scheduler to detect that the trigger condition is met, fetch the simulation scenario record, and generate simulated API messages for the assets according to the API message definitions.

Provided in the embodiments of the present application are an automated software testing method, system and apparatus, and a storage medium. The method comprises: in response to acquiring a function requirement of software or a target function requirement selected by a user, processing the function requirement or the target function requirement on the basis of a test case generation model, so as to generate at least one test case expressed by a natural language (S310); on the basis of a preset condition, obtaining test information of the test case from the test case (S320); on the basis of the test information, constructing a test script corresponding to the test case, and executing the test script to obtain an execution result (S330); and processing the execution result on the basis of an execution result determination model, and determining an execution state of the test case corresponding to the execution result (S340).

A system and method include reception of code files of a program, the program including code regions changed by applying a patch to the program, generation of an interprocedural data flow graph from the code files, identification of first nodes of the graph representing the code regions changed by applying the patch to the program, identification of a plurality of paths of the graph, each of the identified paths originating at a source node of the graph, including at least one of the identified first nodes, and terminating at a sink node of the graph, where each of the identified first nodes is included in at least one of the identified plurality of paths, combination of the identified plurality of paths based on their common first nodes to generate a reduced graph, and input of the reduced graph to a trained classification model to generate a classification.

The present disclosure provides techniques and solutions for automatically correcting software tests. When a test failure is detected, it is determined whether a screenshot or code associated with a second version of a software artifact includes a user interface element that has a semantically equivalent identifier to a user interface element in a screenshot or code associated with a first version of the software artifact. Identifying a semantically equivalent identifier can include determining a section of the user interface, in the screenshot of the code, of the first version of the software artifact where the user interface is located and searching the corresponding section of the user interface of the second version of the software artifact. A definition of the software test can be updated to reference the semantically equivalent user interface element.

Disclosed herein are system, method, and computer program product embodiments for detecting anomalies during software testing. The methods include generating a plurality of test reports for the software program by executing one or more test cases on a plurality of versions of the software program, generating a control chart based on the plurality of test reports, generating an alert when at least one testing characteristic includes an anomaly over the plurality of versions of the software program as determined based on the control chart. The control chart includes a plot associated with at least one testing characteristic of the software program, and a historical context associated with execution of the one or more test cases on the plurality of versions of the software program.

A method includes disassembling a reference binary of a library to generate a control flow graph of the referenced binary, normalizing the control flow graph to generate a normalized graph, traversing the normalized graph to generate execution traces from the normalized graph, and generating library vector embeddings. Generating library vector embeddings includes, for each execution trace of at least a subset of the execution traces, processing the execution trace by a vector embedding model to generate a library vector embedding of the execution trace. The method further includes relating, in storage, a library identifier of the library to the plurality of library vector embeddings as a fingerprint of the library.

An apparatus comprises at least one processing device configured to select a test case associated with a test script comprising functional code for testing an information technology asset and a test case description. The at least one processing device is also configured to generate, utilizing a first machine learning model, first pseudocode for the test script based at least in part on the functional code of the test script, and to generate, utilizing a second machine learning model, second pseudocode for the test script based at least in part on the test case description. The at least one processing device is further configured to determine a stability of the test script based at least in part on a similarity between the first and second pseudocode, and to automatically control one or more characteristics of the functional code of the test script based at least in part on the determined stability.

Disclosed in the present invention is a test case generation method, comprising the steps of: in a graphical tool, editing to form a test model, wherein a scenario, an interaction, and service content are edited, and creating a logic process relationship among the scenario, the interaction, and the service content; in the test model, respectively creating corresponding assertions for the scenario, the interaction, and the service content, and adding the corresponding assertions into the logic process relationship in series; performing test resource scheduling, and configuring a matched use case execution environment for the test model; and compiling the test model into a script which can be automatically executed and executing the script, and after all the assertions are successful, adding the test model into a test case library. Further disclosed in the present invention are a corresponding test management method and system, an apparatus, and a storage medium. By implementing the present invention, the development and automatic test of vehicle-mounted software in an intelligent scenario can be realized, improving the efficiency of a test process and vehicle-mounted software design.

During a software development lifecycle of a software application, application library versions are modified and multiple versions are built and packaged to be installed on different computing systems. An intelligent prediction and capture of runtime errors that may be encountered during a runtime of a software application provides a system-agnostic intelligent software code compiler / build system that intelligently identifies runtime errors at a build stage using artificial intelligence (AI) and automatically encapsulates specific software code to auto-heal runtime exceptions caused by updates to library files. The system may optionally also output a solution to the runtime error to the graphical user interface (GUI) of a software developer capable of performing another software application build.

Various aspects of the disclosure relate to automated testing for application programming interfaces (APIs). A dynamic API validation computing system leverages a generative AI model to dynamically generate a multitude of data sets and rules for use during API validation activities. A federated byzantine agreement mechanism performs the validation of each API using the generated test cases and test data. A generator engine incorporates a generative AI model that may be trained on a large corpus of API metadata and / or data characteristics to predict data patterns and / or validation rules for each of the APIs under test. The generator engine may also predict a structure and format of requests based on the training model inputs. Multiple Test cases for an API created through use of the generative AI model may be distributed and executed on different testing nodes that reach a consensus about whether each API has passed or failed.

Methods, systems, and apparatus, including medium-encoded computer program products for executing a test strategy, include: receiving a request to generate a test script for a new test strategy defined for a software solution, wherein the received request includes a specification of the new test strategy; generating the test script based on executing a large language model that receives as input the specification of the new test strategy, wherein the large language model is trained to automatically generate test scripts based on technical documentation for a testing framework defined for the software solution and integration tests generated for the software solution; and executing the test script for the new test strategy to obtain output data, the output data including performance data for the software solution.