Patents
Literature
Patsnap Eureka AI that helps you search prior art, draft patents, and assess FTO risks, powered by patent and scientific literature data.

205 results about "Validation testing" patented technology

Software Testing - Validation Testing. The process of evaluating software during the development process or at the end of the development process to determine whether it satisfies specified business requirements. Validation Testing ensures that the product actually meets the client's needs.

Saving production runs of a function as unit test and automatic output regeneration

An artificial intelligence system can be used to respond to natural language inputs. The AI System may, for example, receive a first user input for a LLM, generate a first prompt based on the first user input, transmit the first prompt to an LLM, receive an output from the LLM, and evaluate the output from the LLM with reference to one or more validation tests. Responsive to determining that the output from the LLM is not validated, generate a second prompt for the LLM, where the second prompt indicates at least an aspect of the output that caused the output to not be evaluated (e.g., a portion of the output that may need to be updated or corrected), transmit the second prompt to the LLM, and receive an updated output from the LLM. The AI system can include an application for testing functions that utilize interactions with language models.
Owner:PALANTIR TECHNOLOGIES INC

Industrial MBSE model generation and evaluation method based on large language model

The invention discloses an industrial MBSE model generation and evaluation method based on a large language model, and the method comprises the steps: collecting and sorting an aerodynamic model, assembly design rules and related basic design parameters, and carrying out the formatted expression of the rules applied in MBSE, so as to form a special enhanced knowledge base in the industrial field; the method comprises the following steps: converting a demand described by a natural language of a user into a standard MBSE model code based on a large language model through a cue word engineering technology, selecting an optimization algorithm and a simulation algorithm through a function call technology, performing real-time optimization on the model according to a design target, and applying the generated MBSE model to propulsion device design and verification test. And determining availability and model consistency. Reconstruction and acceleration of digital design are realized through automatic configuration based on requirements and components. The method has remarkable advantages in the aspects of reliability, effectiveness and adaptability.
Owner:SHANGHAI JIAOTONG UNIV

PLC code generation and verification method based on large language model

The invention discloses a PLC code generation and verification method based on a large language model, and the method comprises the steps: generating multi-stage multi-mode input data which is used for representing user demands and formulating a system design scheme according to a demand analysis result; the multiple stages comprise a demand analysis stage and a design stage of a complete engineering cycle; the multi-mode input comprises a Pamp (Passive Amplitude Point); an ID graph and a natural language; then, according to the multi-stage multi-mode input data, a PLC code is generated; then, verification testing is carried out on the generated PLC code, and the verification testing comprises code verification and compiler verification; and finally, after the verification test on the PLC code is completed, deploying the PLC code. The PLC code deployment method is high in accuracy and automation degree, an efficient and accurate PLC code deployment method can be provided for network security researchers, and the PLC code deployment method can be widely applied to the technical field of computers.
Owner:GUANGZHOU UNIVERSITY

UVM verification platform and verification test execution method

The invention discloses a UVM verification platform and a verification test execution method, and relates to the technical field of integrated circuits, the verification platform comprises a file management module used for sending a test case file to a script processing module; the script processing module is used for acquiring a test case name and a hardware parameter according to the test case file, and sending the test case name and the hardware parameter to the command generation module; the command generation module is used for generating a target simulation command according to the test case name and the hardware parameters and sending the target simulation command to the compiling library; and the compiling library is used for verifying and testing the design unit according to the basic use case information and the target simulation command. According to the technical scheme, on the premise that the number of the test cases is not changed, the scale of the compiling library of the UVM verification platform is reduced, the compiling time consumption and the dynamic simulation time consumption of the compiling library are reduced, and the verification test execution efficiency of the UVM verification platform is improved.
Owner:BEIJING TONGFANG MICROELECTRONICS

Android application GUI test case automatic generation method based on large language model

PendingCN119088717AImprove efficiencyEnsure mobile application reliabilitySoftware testing/debuggingProgramming languageLinguistic model
The invention discloses an Android application GUI test case automatic generation method based on a large language model, in the test process, an automatic test tool automatically runs Android Apk, current interface layout information is obtained, and then the layout information is provided for a large model for auxiliary analysis. Test requirements of testers and current layout information of the application are understood, the reasoning ability of the large language model is optimized through a cue word engineering method, a test path is automatically generated on the current test application to meet the test requirements, and finally a test assertion is generated to verify the correctness and integrity of the test. According to the method, the efficiency of automatically generating the test case by the Android is improved, and the method for helping developers to guarantee the reliability of the mobile application is provided.
Owner:NANJING UNIV

Automatic interface signal verification control method, device, equipment and medium

The invention provides an automatic interface signal verification management and control method, device and equipment and a medium, and belongs to the technical field of chip verification. Configuring a template file and a configuration file, and creating a to-be-tested design interface and a verification environment interface which interact with each other; creating a verification environment module; the to-be-tested design interface and the verification environment interface are bound to a verification platform, a verification environment module is called, and an interface handle is transmitted to the verification environment module through an inter-module parameter information transmission mechanism; based on the verification platform, tasks of sending an interface signal to the DUT interface, receiving the interface signal from the DUT interface, generating a verification test sequence and the like are realized, and the interface signal of the DUT interface is verified. The verification mode can greatly shorten the test period and reduce manual intervention and errors. And various test scenes and boundary conditions are simulated to ensure that the DUT interface signal can work normally under different conditions, so that the test requirements are met.
Owner:SHANDONG YUNHAI GUOCHUANG CLOUD COMPUTING EQUIP IND INNOVATION CENT CO LTD

Point-to-point access verification method and device, electronic equipment and medium

ActiveCN119990002ACAD circuit designSpecial data processing applicationsUniversal Verification MethodologyVerification
The invention relates to a point-to-point access verification method and device, electronic equipment and a medium. The method comprises the steps of obtaining configuration information of a to-be-verified module; writing the configuration information of the to-be-verified module into a verification document, and storing the verification document as a JSON (JavaScript Object Notation) file; the verification document comprises at least one verification test item; each verification test item comprises a plurality of verification test paths; the JSON file is analyzed, and configuration information of the module to be verified is extracted; generating a plurality of universal verification methodology UVM components and UVM excitation files according to the configuration information based on a preset configuration script; establishing a script based on a UVM environment, and integrating the UVM component and the UVM excitation file to a UVM verification platform; in response to a target verification test command, performing test verification on a plurality of to-be-tested paths of a target verification test item on the UVM verification platform; the number of the target verification test items is greater than or equal to 1.
Owner:厦门国科安芯科技有限公司

Generative ai application artifact management enabling safeguarding by key users

A computer-implemented method includes customizing artifacts of a large language model (LLM). A generative artificial intelligence (AI) (genAI) function of a software application that interfaces with the LLM and is instrumented for customer-side management is activated, as a genAI application. A verification test is activated to determine whether interactions between the LLM and genAI application generate expected results. Based on a result of the verification test, either adjusting the artifacts of the LLM or updating the genAI function of the software application. A new verification test is defined to determine whether interactions between the LLM and genAI application generate expected results. The new verification test is activated.
Owner:SAP SE

Seamlessly verifying upgrades using mirrored dataplanes

A system and method are provided for implementing a network component and verifying an update of the network component. The network component can be, e.g., a software-defined wide area network, a firewall, a router, or a load balancer. The network component can be an embedded network edge device that is implemented, e.g., in software, in circuitry, or using hardware acceleration (e.g., a data processing unit (DPU), a smart network interface card (SmartNIC), etc.). The updated version of the network component is verified by implementing it on a shadow dataplane concurrently with the current version operating on a primary dataplane, and comparing the performances of these two versions. Based on this comparison satisfying various verification criteria, the updated version passes a verification test and can be promoted to the primary dataplane.
Owner:CISCO TECHNOLOGY INC

AI auxiliary test case generation and verification method and terminal

The invention discloses an AI auxiliary test case generation and verification method and a terminal, and the method comprises the steps: obtaining and analyzing game commodity data, filling a script template with the analyzed game commodity data, verifying and adjusting the script template through a trained AI model, recognizing a test scene according to the adjusted script template, and carrying out the verification of the test scene. A corresponding test case is generated; and executing the test case, verifying the execution result of the test case through database query and log analysis, and generating a test report, so that the method can adapt to tests of different scenes and requirements, and the reliability and execution efficiency of the test are improved.
Owner:FUJIAN TQ DIGITAL

AI-based weblog analysis processing method

The invention discloses an AI-based network log analysis processing method, which comprises the following steps of: firstly, collecting network equipment log information through a collection script supporting multiple protocol modes, and writing the collection script and deploying and starting after determining equipment and protocol types; thirdly, carrying out preliminary classification, data supplementation and formatting processing on the original logs; and then, sending the processed log data to an AI large model for semantic analysis, including deep understanding, basic classification, secondary analysis and providing of processing suggestions. An analysis result is stored in a data think tank for historical comparison, and an inconsistent result is manually confirmed and corrected; when the network equipment triggers a fault alarm, calling the data think tank to supplement alarm content; and meanwhile, equipment related data handbooks provided by manufacturers are uploaded regularly, the analysis accuracy of the AI model is corrected, verification and testing are carried out, it is ensured that the scheme can comprehensively and efficiently collect and process logs, accurately analyze faults and continuously optimize the model, and effective support is provided for network fault processing.
Owner:QIDONG JIASHENG INFORMATION TECHNOLOGY CO LTD

Random excitation generation method, system and equipment for chip verification test

The invention discloses a random excitation generation method, system and device for a chip verification test, and relates to the technical field of chip verification, and the method comprises the steps: determining the priority of all variables in excitation according to the constraint condition of excitation needed by a to-be-tested chip verification test case; whether inequality constraints exist in the constraint conditions or not is detected, if yes, the inequality constraints are equivalently replaced with a unified representation form, then subsequent steps are executed, and if not, the subsequent steps are directly executed; on the basis of the priorities of the variables, intersection of all the related constraints of each variable in the constraint conditions is solved, all the related constraints of each variable are converted into a range constraint, and thus an optimized constraint condition is generated; and on the basis of the optimized constraint conditions, according to the priorities of the variables, the random excitation required by the verification test case of the to-be-tested chip is obtained through successive stochastic operation from high to low.
Owner:JINAN XINYU SOFTWARE TECH CO LTD

Multi-data plane architecture for continuous integration, continuous deployment (CI / CD)

A system and method are provided for continuous integration, continuous deployment of a network component, such as a software-defined wide area network, a firewall, a router, or a load balancer. The software development lifecycle is achieved without interrupting the data flow of the network by using a multi-dataplane architecture, including a primary dataplane and a shadow dataplane. A packet dispatcher relays ingress data packets to the primary dataplane executing a current version of the network component and the shadow dataplane executing an upgrade to the network component. A control plane agent analyzes / compares the performances of the respective dataplanes for verification testing, and the control plane agent upgrades the network component to the new version upon passing the verification testing. The upgrades is achieved without interruption to the data flow of the network component by gradually transitioning to outputting egress data packets generated using the upgraded version.
Owner:CISCO TECHNOLOGY INC

Enterprise-level application-oriented security deployment method, equipment and medium

The embodiment of the invention discloses a security deployment method and device for enterprise-level applications and a medium, and relates to the technical field of security deployment, and the method comprises the steps: obtaining application topological graph data generated by a preset visual development platform, so as to split the application topological graph data into an atomized micro-service unit set through a quantization deconstruction engine, generating a micro-service dependency tree; scanning the target deployment environment, extracting an environment feature fingerprint of the target deployment environment, and generating an environment adaptation layer code according to the environment feature fingerprint; injecting an environment adaptation layer code into each atomized micro-service unit to generate an environment perception type micro-service package, and constructing a cross-cloud communication link matrix based on the micro-service dependency tree; and based on the environment-aware micro-service packet and the cross-cloud communication link matrix, executing a deployment verification test in the sandbox environment, generating a deployment verification report, and according to the deployment verification report, distributing the environment-aware micro-service packet to a target deployment environment node.
Owner:INSPUR YUNZHOU (SHANDONG) IND INTERNET CO LTD

Automated artificial intelligence teaching and learning environment system and method

A method for providing an automated AI teaching and learning environment includes selecting an AI agent from an AI agent pool as a candidate AI agent according to performance of the AI agent after previous training and further according to capability requirements for capability-based training, and providing the capability-based training for the candidate AI agent. Providing the capability-based training includes acquiring first training data, generating a trained artificial intelligence (AI) agent by training the candidate AI agent to have at least one capability associated with using the first training data, generating synthetic validation data different from the synthetic training data, validating the trained AI agent by performing validation testing on the trained AI agent using first validation data that includes the synthetic validation data, and providing the trained AI agent as a validated AI agent after the validating the AI agent.
Owner:TEXTRON INNOVATIONS INC

Component die validation built-in self-test (VBIST) engine

A component die validation built-in self-test (VBIST) engine is presented. In an aspect, a component die includes component circuitry for performing a component function, interface circuitry for communicating with another die, and a VBIST circuit. The VBIST circuit includes a traffic generator that generates test data streams, a tracker that receives and validates test data streams, and a configurable switching matrix for coupling the traffic generator to at least one of the component circuitry, the interface circuitry, or the tracker, and for coupling at least one of the component circuitry, the interface circuitry, or the traffic generator to the tracker. The VBIST circuit can send traffic to and from the component circuitry directly, or indirectly via the interface circuitry in loopback mode, and can be used for memory initialization and test.
Owner:AMPERE COMPUTING LLC

Automatic testing method and device based on front-end UI and terminal

The invention discloses an automatic testing method and device based on a front-end UI and a terminal, and belongs to the technical field of front-end development testing. Comprising the following steps: detecting a first test instruction of each set of front-end UI program, controlling to carry out demand analysis, determining a function module which needs to be subjected to UI automatic test according to a demand analysis result, and determining a test target; performing automatic technology type selection, and selecting a specified test tool and a continuous integration tool; generating a compiled test case according to the determined function module needing to be tested and the test target; running the test case, and generating a test script according to the test case and an automation technology type selection result; when the test script is verified to be correct, integrating the test script into a continuous integration tool, and automatically running a test through the test script when the code of the subsequent version of the front-end UI program is submitted; and generating a test report, positioning problems and giving repair suggestions. According to the invention, the testing efficiency is improved, and the labor cost of testing is reduced.
Owner:SHENZHEN COOCAA NETWORK TECH CO LTD

Artificial intelligence algorithm model accuracy verification test method

The invention relates to the technical field of artificial intelligence, and discloses an artificial intelligence algorithm model accuracy verification test method comprising the following steps: S1, data set preparation: collecting diversified data sets to cover more scenes and conditions; s2, whether cross validation is carried out or not is judged, if yes, a cross validation method is adopted, a data set is divided into a plurality of mutually exclusive subsets, a part of the subsets are cyclically used as a test set, the rest of the subsets are used as a training set for multiple times of verification and testing so as to evaluate the performance and stability of the model, and if not, the next step is carried out; and S3, selecting a proper evaluation index, and if cross validation is not selected, selecting a proper evaluation index. According to the method, the accuracy of the artificial intelligence algorithm model can be comprehensively and systematically evaluated, developers are helped to accurately understand the performance of the model in different data sets and scenes, and the model with better performance can be screened out by comparing the performance of different models in the same test set.
Owner:JIANGSU SUYUN INFORMATION TECH CO LTD

Accessibility verification testing

A computer-implemented method includes identifying a webpage comprising a set of user interface (UI) elements, analyzing the set of user UI elements to identify a set of interactable elements, classifying the elements of the set interactable elements as either focusable or not focusable, extracting features from source code corresponding to interactable elements of the set of interactable elements classified as focusable, and building an accessibility issue detection model using the extracted features from source code corresponding to focusable interactable elements as training data. The method may further include extracting features from source code corresponding to interactable elements classified as not focusable and updating the accessibility issue detection model using the extracted features from source code corresponding to interactable elements which are not focusable as training data. The method may further include storing one or more passed results and one or more failed results in a results database.
Owner:INTERNATIONAL BUSINESS MACHINE CORPORATION

Stress evaluation method and system for TSV (Through Silicon Via) packaging structure

The invention provides a stress evaluation method and system for a TSV packaging structure, and relates to the technical field of semiconductor packaging, and the method comprises the steps: constructing a global coarse mesh model, cutting to generate a local TSV sub-model, and generating a failure feature mapping table; based on the failure feature mapping table and a preset process temperature curve, coupling solution is carried out on the local TSV sub-model, and a multi-physical field data set is output; training a failure probability prediction model by combining the failure feature mapping table based on the multi-physics field data set, and generating a digital failure atlas template; performing dynamic risk assessment based on the digital failure atlas template, the real-time process parameters and the dynamically calculated electrothermal coupling parameters, and generating a failure risk report; analyzing the failure risk report, and generating a mode-based optimization guide; and performing spatial deviation comparison based on the verification test data executed by the mode-based optimization guide, and when the spatial deviation exceeds a preset threshold value, outputting a failure map version updating mark and triggering the failure feature mapping table to iteratively update.
Owner:CHINA ELECTRONICS RELIABILITY AND ENVIRONMENTAL TESTING INSTITUTE ((THE FIFTH INSTITUTE OF ELECTRONICS MINISTRY OF INDUSTRY AND INFORMATION TECHNOLOGY) (CHINA SAIBAO LABORATORY)

Software testing system and method, electronic equipment and storage medium

The invention relates to a software testing system and method, electronic equipment and a storage medium, and the software testing system comprises an operation parameter analysis module which obtains and analyzes test operation parameters of to-be-tested software, and determines a test case configuration file of at least one test case according to the test operation parameters; the case initialization module is used for generating at least one test case according to the test case configuration file and a first preset template; the case operation module is used for executing the test case to obtain an execution result of the test case; and the data verification module verifies the execution result of the test case to obtain a function test result of the to-be-tested software. The invention provides a highly modularized and automatic software testing system, the generation process of the test case is simplified by introducing the predefined template, a tester only needs to maintain the test operation parameters and the test case configuration file, repeated labor is reduced, the test requirements of different software and requirements can be met, and the test efficiency is improved. The universality of the system is enhanced, and the test efficiency is improved.
Owner:PHLEXING TECH CO LTD

Systems and methods for hybrid integration and development pipelines

Systems and methods provide a HybridOps model for the identification, capture, isolation, feature engineering and adjudication of source signal data signatures for inclusion in calibration quality standard reference signal data signature libraries that improve machine learning and validation, reduces model bias and reduces model drift. The HybridOps model may include an “unlocked” AI / ML (machine learning enabled) public facing deployment pipeline in parallel with a clone AI / ML deployed in an internal development environment using a ML-Ops pipeline and in parallel with a clone“locked” AI / ML (machine learning disabled) as a standard reference. The three deployed models enables monitoring and measuring model drift, context drift and product progression for improved verification and validation of model reliability. The parallel environment AI / ML testing using randomized learning, validation, testing sets drawn from calibration quality adjudicated standard reference signal data signature libraries provides for validation, verification and test to failure procedures.
Owner:COVID COUGH INC

Dynamic input-sensitive validation of machine learning model outputs and methods and systems of the same

The systems and methods disclosed herein enable evaluation of machine learning model outputs within a virtual environment. The disclosed model validation platform enables testing of code generated for detection of malicious or anomalous outputs. For example, the model validation platform can construct a virtual machine isolated from the system and test model-generated code for validation of LLM-generated outputs. In some implementations, the model validation platform determines parameters of the virtual machine and / or associated validation test based on an evaluation of the machine learning model's output and / or the associated underlying prompt. For example, the parameters of the validation test depend on an evaluation of the user or the provided input (e.g., depending on the presence of sensitive data within the prompt). By doing so, the system enables dynamic evaluation of machine learning model outputs to improve the security and robustness of associated generated code.
Owner:CITIBANK N A

Micro-grid closed-loop cross validation simulation system, method, equipment and medium

The invention relates to the technical field of micro-grid simulation, in particular to a micro-grid closed-loop cross validation simulation system, method, device and medium, and the system comprises a bottom layer data generation module, a topology automatic mapping module, a communication protocol module, a master station control module, an index evaluation module, a human-computer interface module and a historical section playback module. The system can simulate steady-state operation characteristics of different types of micro-grid elements, interaction with an external micro-grid energy management system can be realized based on the system, and a closed-loop cross validation test is realized through a master station simulation function of the system; based on the system, simulation of different characteristic resources can be realized, verification of different investment subjects and different user operation targets, micro-grid dynamic aggregation capability verification, distribution and micro cooperation verification and the like can be realized, meanwhile, based on the system, micro-grid group verification can be realized, and a test environment is provided for master station micro-grid group cooperation control.
Owner:STATE GRID JIANGSU ELECTRIC POWER CO LTD RESEARCH INSTITUTE +1

Probe processor-oriented verification and test integrated platform

The invention relates to a verification and test integrated platform for a probe processor, and belongs to the technical field of processor verification and test. A probe processor adopts a self-defined probe instruction and a special probe operator unit, and a traditional verification and test mode is difficult to consider the requirements of module-level verification and system-level test. The UVM verification system, the test system and the management module are integrated, and unified management of verification and test is achieved. The UVM verification system is based on a UVM architecture, a virtual interface is associated with a to-be-tested unit (such as a probe operator, GPIO, UART and the like) of a probe processor through a forced binding mechanism, and module-level verification is achieved. The test system is responsible for compiling test codes into machine codes and loading the machine codes to the probe processor by starting the read-only memory, and system-level test is executed by the probe processor. And the management module adopts an automatic script to realize intelligent switching between verification and test modes, so that the probe processor is suitable for different test scenes, the test efficiency is improved, and the environmental redundancy is reduced. According to the method, the reusability of the test case is improved, the test consistency is ensured, the resource utilization rate is optimized, the automation degree and efficiency of verification are improved, and the method is suitable for efficient verification and test of probe processors and similar architecture chips.
Owner:BEIJING TECH & BUSINESS UNIV

Random function selection and insertion during compilation for post-silicon validation

A computer system, computer readable storage medium, and computer-implemented method for generating a test program for a device-under-test (DUT). The method includes transmitting a plurality of uncompiled snippets and a plurality of uncompiled micro-functions into a compiler. The method also includes selecting, randomly, a portion of uncompiled micro-functions from the plurality of uncompiled micro-functions. The method further includes compiling the uncompiled snippets and the portion of uncompiled micro-functions, thereby generating a plurality of compiled snippets and a compiled portion of micro-functions. The method also includes interweaving, randomly, the compiled portion of micro-functions with the plurality of compiled snippets, thereby at least partially generating a test program for the DUT. The method further includes executing one or more post-silicon validation tests for the DUT with the test program.
Owner:INTERNATIONAL BUSINESS MACHINE CORPORATION

AI-Driven Defect Remediation System Based on Bias Detection

Systems and methods for bias testing and remediation are disclosed. Decentralized Web Application Testing Systems use a Holochain framework to distribute testing workloads across Full Nodes and Lightning Nodes. A Holochain Node Management Application for configuring nodes, a UI Application creates test cases, and a Version Management System tracks changes. Test results are stored and analyzed in a Test Result Store, with a Bias Intelligence module detecting biases and generating additional test cases. A Consensus Algorithm validates test cases through decentralized nomination. An AI-Driven Defect Remediation System automates defect detection, root cause analysis, and remediation using AI modules. Machine learning algorithms identify patterns and anomalies, while NLP techniques generate code fixes. Predictive maintenance monitors application performance to preemptively address issues. A feedback loop mechanism continuously improves AI models through reinforcement learning. Together, these systems provide a holistic approach to web application testing and maintenance.
Owner:BANK OF AMERICA CORP

Chip parameter verification method and device, equipment and storage medium

The invention discloses a chip parameter verification method and device, equipment and a storage medium. The chip parameter verification method comprises the steps of sampling a to-be-tested parameter space of a chip to obtain a parameter point file; performing a regression test on the parameter point file to obtain verification test results, and associating each verification test result with the corresponding parameter point to obtain a data pair set; performing parameter point iteration prediction on the set through a model according to the data; and when it is determined that the predicted parameter points meet the optimization target, taking the predicted parameter points as to-be-applied target parameter points of the chip. The parameter point file is obtained by sampling from the to-be-tested parameter space, and the sampling result is used for the regression verification test, so that the number of tasks needing to be executed is reduced, the overall verification investment is reduced, fitting and prediction are carried out through the model according to the verification test result, the coupling effect and hidden defects among parameters are conveniently found, and the verification efficiency is improved. And the accuracy of the determined target parameter is improved.
Owner:ZHONGHAO XINYING (HANGZHOU) TECH CO LTD

Generation of diversified validation test suite for generative artificial intelligence powered tools

A diversified validation test suite application (DVTSA) for a generative AI powered tool or large language model (LLM) tool includes at least first, second, and third control logics. The first control logic receives a seed test input or user input, analyzes the seed test input or user input, and extracts key elements for variations. The second control logic performs a coverage measurement of outputs of the first control logic. The third control logic causes a human validator to evaluate outputs of the second control logic relative to predefined coverage metrics and selectively and continuously iterate to cause outputs of the second control logic to increase LLM tool input robustness and output robustness from a first level to a second level greater than the first level, in both production and pre-production LLM tool processes.
Owner:GM GLOBAL TECHNOLOGY OPERATIONS LLC

Interactive document structured information extraction method and device, electronic equipment and storage medium

The invention discloses an interactive document structured information extraction method and device, electronic equipment and a storage medium, and particularly relates to the field of document processing. The interactive document structured information extraction method comprises the steps that a document is recognized, and a document format is obtained; matching the document format with a sample format stored in a knowledge base, selecting a matched format, and executing a corresponding task instruction to extract document structured information; the user performs error correction on the extraction result; outputting an accurate result and verifying the consistency of the extraction result and the accurate result; querying a similar task instruction in a knowledge base according to the accurate result, and generating a new task instruction; and executing the new task instruction on the document format to obtain a test result, and verifying the consistency of the test result and the accurate result. According to the method, dynamic learning and optimization can be realized, the accuracy and robustness of information extraction are remarkably improved, the method has strong instant learning ability and adaptivity, and the learning and training cost is reduced.
Owner:SHANGHAI WENTONG CLOUD SOFTWARE DEVELOPMENT CO LTD