8071results about "Software testing/debugging" patented technology

An example operation may include one or more of receiving a request to test a software program from a user device, the request comprising a description of requirements of the software program, generating a plurality of testing elements based on execution of a generative artificial intelligence (GenAI) model on the description of the requirements of the software program and a repository of test cases, generating a test case for testing the software program where the test case comprises the plurality of testing elements generated by the GenAI model, and storing the test case within a storage device.

An example operation may include one or more of receiving, via a user interface, a request to test a software program, reading source code of the software program and identifying a vulnerability in the source code based on the reading, generating repair code for fixing the identified vulnerability based on execution of a generative artificial intelligence (GenAI) model on the source code and a repository of repair code used to repair previous vulnerabilities, and displaying information about the repair code via the user interface.

An example operation may include one or more of generating a large language model via a user interface, executing the large language model on a repository of software test cases and requirements of the software test cases to train the large language model to understand connections between test case components and test case requirements, receiving a description of features of a software program to be tested, and in response to receiving the description of the features, generating a software test case based on execution of the large language model on the received descriptions, and displaying the software test case via the user interface.

An example operation may include one or more of executing tests on a software application via a test environment of a test platform and logging results of the tests in a log file, identifying a new feature to be added to the software application based on execution of a machine learning model on logged results of the tests stored in the log file, generating source code for the new feature to be added to the software application based on execution of a generative artificial intelligence (GenAI) model on the new feature and a repository of source code, and displaying the generated source code via a user interface of the software application.

An example operation may include one or more of receiving a description of a plurality of testing elements for testing a software program and storing the software test within a storage device, generating an automation script for automating execution of the software test based on execution of a generative artificial intelligence model (GenAI) model on the plurality of testing elements and a repository of automation scripts, attaching the automation script to the software test within the storage device, and in response to a request to execute the software test, executing the plurality of testing elements within the software test based on the attached automation script.

A generative artificial intelligence (AI) driven code fixing pipeline has been created that uses a transformer-based large language model (LLM) to patch flawed program code. A pre-trained LLM is fine-tuned to generate a response that is a modified version of a code fragment in a prompt to the pre-trained model. After fine-tuning, the pre-trained LLM (hereinafter “code fix model”) is integrated into a pipeline that includes a program code cybersecurity scanner and a prompt generator. The scanner generates indications of flaws in program code and weakness types for those flaws. These indications flow into the prompt generator. The prompt generator retrieves reference code pairs based on weakness type and programming language to generate a batch of prompts to run inference on with the code fix model. The responses generated by the code fix model are presented as patching alternatives.

The invention provides a unit test case generation system based on a large language model, and the system comprises a function dependency analysis engine module which is used for preprocessing a target project, extracting a function dependency relationship, and generating an abstract syntax tree and a source code file; the large model module is used for automatically generating a test case through the code context information and the optimized cue word; and the verification and optimization module is used for verifying whether the test cases can be successfully executed or not by compiling and running the test cases, marking part of the test cases needing to be further processed, automatically analyzing error reasons, optimizing cue words and realizing automatic repair of the part of the test cases. According to the unit test case generation system based on the large language model, the function dependency analysis and the large language model are combined, the high-quality test case can be automatically generated, dependence on manpower or existing test cases is not needed, manual intervention is reduced, and the test efficiency and comprehensiveness are improved.

An example operation may include one or more of storing a repository of test data, receiving a request for new test data to be generated for a software program, where the request comprises a text-based description of the new test data, identifying one or more code modules included on the software program based on source code of the software program stored in the memory, executing a generative artificial intelligence (GenAI) model on the text-based description of the new test data and the one or more code modules to generate test data for testing the one or more code modules, and storing the test data in the repository of test data.

Disclosed in the present invention are a danger test case generation method for a visual perception algorithm, and a related device. The method comprises: acquiring images captured by a visual sensor of a self-driving vehicle, cleaning and marking the collected images, and obtaining an observation data set according to no-reference image quality evaluation features; according to the observation data set and by means of performing fusion with domain knowledge, constructing a causal structure diagram that indicates the influence of environmental factors on perception performance; separating key environmental factors from the causal structure diagram, and quantitatively calculating the degree of influence of each environmental factor on visual perception performance; and on the basis of qualitative analysis of a causal relationship between a key environment variable and the visual perception performance, and quantitative estimation of a causal effect, proposing a challenge index, so as to generate, in batches, danger test cases for perception. The present invention has significant advantages in multiple aspects such as causal relationship mining, challenge index measurement, dangerous-scenario recognition and search space modeling, such that the performance of a visual algorithm for self-driving in a complex environment is evaluated and verified more accurately.

Examples of the present disclosure describe systems and methods for automatically detecting and repairing reliability issues in operating systems and applications using a generative artificial intelligence (“AI”) system. In examples, a generative AI system receives a request to evaluate a detected issue in a software service or application. In response, the system analyzes error information associated with the detected issue to build an error context for the software code that caused the detected issue. The error context is used to identify the location of the software code file that comprises the software code. The error context and the software code file are used to identify a prompt. The prompt, the error context, and / or the identified software code are provided as input to a language model. The language model provides an output that is responsive to the user request and may perform actions to further evaluate or repair the detected issue.

One or more unit-test cases are generated from a monolingual code corpus and the generated unit-test cases are filtered to generate a corpus of unit-test cases which have acceptability scores exceeding one or more predefined thresholds. One or more of the code samples of the monolingual code corpus are translated from a source language to a target language using a pretrained Large Language Model and the generated unit-test cases are translated from the source language to the target language. The LLM-translated code samples are validated using the translated unit-test cases and a parallel-data training corpus comprising the LLM-translated code samples that pass the validation is created. The pretrained large language model (LLM) is fine-tuned using the parallel-data training corpus, a given code segment is translated using the fine-tuned large language model (LLM), the translated given code segment is tested and the tested given code segment is deployed.

A vulnerability detection and repair system utilize a classifier model to detect a software vulnerability in a source code snippet and the tokens in the source code snippet attributable to the vulnerability. A large language model is then given the vulnerable source code snippet, its vulnerability type, the vulnerability tokens, and a few-shot examples to determine whether or not the source code snippet includes the identified vulnerability. The few-shot examples include positive and negative samples of the type of vulnerability to guide the large language model towards the correct output.

Method and system for analyzing at least one computing system to determine attributes of software within computing systems, includes: loading input data representing or pertaining to the software of the computing system; loading and executing at least two individual analyses on the data format of the software of the input data; generating an individual analysis result for each of the at least two individual analyses; loading and executing multi-layer / multi-data format analysis on the individual analysis results; generating multi-layer and / or multi-data format analysis result indicating an additional attribute being different from the attributes indicated by the first and second ones of the individual analysis results; generating an output data describing the individual analysis results and / or the multi-layer and / or multi-data format analysis result; storing the output data; and determining if the output data satisfies a predetermined condition, and if so, executing action corresponding to the output data on the computing system.

A network system to use machine learning systems to create chaos testing scenarios on cloud-based applications. The system uses inputs from applications that are implemented on user computing devices to allow users to interface with a network or other system. The system creates a model of the application based on input data received from a network of applications, the model representing a structure, method, and dependencies of the application. The system identifies points of failure of the application and generates one or more chaos testing simulation scenarios that target the identified points of failure. The system performs the chaos testing based on the received simulation scenarios and logs the results of the testing. The system generates recommendations to revise code of the application based on the outcome of the chaos testing. A large language model may be used to provide documentation and analysis of the chaos testing.

The invention relates to the technical field of software development, in particular to a zero-code software development and test system based on artificial intelligence, and provides the following scheme that the system comprises a natural language analysis module, a code generation module, a code optimization module, an intelligent test module and a cross-platform deployment module. The natural language analysis module analyzes the natural language description input by the user into a software development requirement document; the code generation module generates source codes based on the demand document and can select a programming language and a framework; the code optimization module is used for performing static analysis and reconstruction optimization on the generated codes; the intelligent test module automatically generates a test script through interface model extraction, case generation and case analysis logic, executes a test and feeds back a test result; and the cross-platform deployment module realizes automatic packaging, compatibility check, deployment configuration and release management of software. The invention aims to simplify the software development process and improve the development efficiency and quality.

Systems and methods for software development using machine learning models are disclosed. In one embodiment, a method for programmatically generating source code using a machine learning model includes obtaining a set of project configuration information that defines a programming task, indexing original source code in a code repository, advancing the programming task by constructing a prompt to a machine learning model to obtain new source code as an output, writing the new source code to the code repository, identifying an intervention request that requires human input to complete the programming task, obtaining human input for the intervention request from a human using digital communications as an outcome, and writing the intervention request and outcome to an intervention database.

A system, method, and computer-program product includes identifying a plurality of code synthesis items for a target programming language, generating a code synthesis prompt based on a first sampling of the plurality of code synthesis items, synthesizing, via a large language model, a plurality of raw code segments using the code synthesis prompt, executing the plurality of raw code segments with a code interpreter associated with the target programming language, determining one or more valid code segments of the plurality of raw code segments that the code interpreter successfully executed, aggregating, via a second sampling, the one or more valid code segments into one or more validated code synthesis training samples, and training a code generation model using the one or more validated code synthesis training samples. User interfaces may be provided to allow target coding tasks to be specified via text or speech.

The invention relates to the technical field of software engineering, in particular to a source code detection method, which comprises the steps of source code preprocessing, code dependency analysis, semantic understanding and analysis, code quality evaluation, dynamic behavior simulation, intelligent vulnerability detection and the like to realize comprehensive detection of potential vulnerabilities and quality problems in source codes. A support vector machine and a K-means clustering algorithm are combined, known and unknown vulnerabilities can be effectively recognized, in addition, a continuous optimization feedback mechanism is introduced, a mixed vulnerability detection model can be continuously optimized according to repair operation and feedback information of a developer, the detection accuracy is improved, and the coverage range is expanded. The generated comprehensive detection report covers code structures, vulnerability positioning, quality problems and optimization suggestions, supports integration of various output formats and development tool chains, helps developers to efficiently repair problems, and improves code quality and project progress; according to the method, the comprehensiveness, the self-adaptability and the user experience of code detection are effectively improved.

Method for prediction of test failures are disclosed. A method may include retrieving, by a test failure prediction computer program, test results for a test; training, by the test failure prediction computer program, a machine learning engine based on one or more hypotheses; receiving, by the test failure prediction computer program, metrics for a code file; providing, by the test failure prediction computer program, the metrics for the code file to the trained machine learning engine; receiving, by the test failure prediction computer program, an output from the trained machine learning engine that provides a probability of failure; and outputting, by the test failure prediction computer program, the output.

The invention relates to the field of data processing, in particular to a security evaluation method based on a large language model and a related device. The method comprises the following steps: identifying a security evaluation dimension matched with a large language model; generating an autonomous test case of the large language model based on the security evaluation dimension; the autonomous test case comprises a normal user and an abnormal case; the information content in the abnormal use case corresponds to the type of the large language model; the autonomous test case is input into the large language model to execute a model function, and a response result of the large language model to the autonomous test case is obtained; and generating a security evaluation report for the large language model based on the response result. According to the method, the autonomous test case is adopted to actively evaluate the safety of the model, the automatic comprehensive evaluation of the model is realized, the model safety evaluation efficiency and accuracy are improved, the adaptability to models in different fields and scales is enhanced, the automation and continuous optimization of safety evaluation are realized, and the operation safety of the model is ensured.

The present application provides a vulnerability detection method, comprising: acquiring a target branch of a target API in a detected code; generating a test case of the target API on the basis of proof information of a vulnerability API, wherein the test case of the target API is used for testing whether the target API has a vulnerability; determining an original seed for performing fuzz testing on the target branch of the target API; inputting the original seed into the test case of the target API; testing the target branch by means of the test case into which the original seed is inputted; screening for a seed on the basis of a test result; mutating the screened seed, and then inputting the mutated seed into the test case of the target API; and performing the next round of testing on the target branch by means of the test case into which the mutated seed is inputted so as to detect the vulnerability in the detected code. According to the method, saturation fuzz testing is performed on a specific branch (i.e., the target branch) of a specific API in a detected code, thereby achieving the objective of triggering the vulnerability quickly, realizing efficient fuzz testing for a vulnerability discovery scenario, and improving the vulnerability discovery efficiency.

The invention discloses a PLC code generation and verification method based on a large language model, and the method comprises the steps: generating multi-stage multi-mode input data which is used for representing user demands and formulating a system design scheme according to a demand analysis result; the multiple stages comprise a demand analysis stage and a design stage of a complete engineering cycle; the multi-mode input comprises a Pamp (Passive Amplitude Point); an ID graph and a natural language; then, according to the multi-stage multi-mode input data, a PLC code is generated; then, verification testing is carried out on the generated PLC code, and the verification testing comprises code verification and compiler verification; and finally, after the verification test on the PLC code is completed, deploying the PLC code. The PLC code deployment method is high in accuracy and automation degree, an efficient and accurate PLC code deployment method can be provided for network security researchers, and the PLC code deployment method can be widely applied to the technical field of computers.

A method for automating a software development lifecycle by using artificial intelligence is disclosed. The method includes receiving a request to generate software components, the request including requirements in a natural language format; determining, by using a first model, tasks to generate the requested software components based on the request, each of the tasks including a sequence of subtasks, an acceptance criterion, and a predicted effort level; generating, by using a second model, sets of software code based on the tasks, the sets of software code corresponding to the requested software components; generating, by using a third model, unit tests for each of the sets of software code; initiating the unit tests for each of the sets of software code; and releasing, by using a fourth model, the sets of software code based on a result of the corresponding unit tests.

Methods, systems, and computer programs are presented for implementing a tool that generates and executes the code in a controlled environment to satisfy user requests entered as text prompts. One method includes receiving a user prompt with a user request received in a user interface (UI), generating a first Large Language Model (LLM) prompt to create a plan, and receiving the plan from an LLM, the plan comprising text describing a sequence of operations. The method further includes generating a second LLM prompt, to create code, specifying which code instructions are permitted in the created code. Further, the method receives the code from the LLM that received the second LLM prompt, and executes the code in a controlled environment. Further, the results generated by executing the code are presented in the UI.

An issue source scanning system scans a source of information about an issue to identify characteristics of code which indicate that the issue will manifest in code. A code scanning system scans code to match the identified characteristics against the code to determine whether the source code includes the identified characteristics. The code scanning system automatically generates a suggested operation to perform based upon a match between the identified characteristic and the code.

The techniques disclosed herein enhance software testing through automated test case generation and implementation. This is accomplished through utilizing a natural language interface and a generative probabilistic model, such as a large language model. A first user input can provide descriptions of a software feature to be tested and various use cases for the software feature. Accordingly, the probabilistic model generates a plurality of test cases for evaluating various aspects of the software feature in accordance with the first user input. The natural language descriptions can be accompanied by a contextual input to constrain the probabilistic model and ensure consistent outputs. Subsequently, a second user input selects a test case from the generated test cases to be implemented. In response, the probabilistic model generates a software implementation of the selected test case. The software implementation is then executed with log data extracted from the testing environment for processing results.

In one embodiment, a computer-implemented method executed using one or more processors of an incident management system comprises receiving a notification of an incident associated with a computer system, and in response to receiving the notification: extracting an error message from the notification; reading a set of computer program code changes that have been implemented in the computer system; matching the error message to the set of computer program code changes and outputting a set of one or more candidate code changes that may correspond to the error message; based on the set of candidate code changes, generating an automatic remediation for the incident; and using the one or more processors of the incident management system, executing the automatic remediation on the computer system.

Technical solutions are directed to creation and deployment of multi-stage test problems. A processor can provide, to a client, a first data structure for a first stage of a multi-stage test problem including a description for generation of a client solution for the first stage and a test case to evaluate the client solution. The processor can receive, from the client, a client data structure comprising the client solution including a client computer code generated at the client for the first stage. The processor can determine, by evaluating the client solution using an input value and an output value of the test case, that the output of the client solution satisfies a validity condition of the output value. The processor can provide, to the client, based on satisfying the validity condition, a second data structures for a second stage of the multi-stage test problem.

Systems and methods described herein relate to automated troubleshooting for application development tools. First data comprise a plurality of delta objects and a user query. The plurality of delta objects identify design-time changes made to an application using a development tool. The user query is received via a user interface and relates to at least one of the design-time changes. The first data is preprocessed to obtain second data. The preprocessing of the first data includes modifying a subset of the first data and adding one or more predetermined instructions to the first data. The second data is provided to a machine learning model to obtain a response to the user query. Output indicative of the response is caused to be presented in the user interface.

An intelligent technical procedure to merge two or more application programming interfaces (APIs) leveraging generative AI is disclosed. API metadata is extracted from two or more APIs (e.g., name of API, endpoints, request / response formats (JSON or XLM), authentication mechanism (API keys, OAuth, or JWT tokens), rate limits, error codes, error messages, version scheme and API documentation etc.) to formulate and generate new API based on user prompt. Using Abstract Syntax tree (AST) advance deep learning generative AI algorithms like Generative pre-trained transformer (GPT) or the like, the method can merge and construct new API based on user's requirement prompt. Method leverages smart contracts to test new generated API and deploy API in desired environment based on blockchain consent protocol to manage API.