System and method for protecting proprietary data when using third party AI / ML services

By verifying the answers of third-party AI/ML services within proprietary systems, the problem that baseline AI/ML services may output illusions and data exposure is solved, and a secure and accurate fine-tuning of commercial AI/ML models is achieved, reducing costs and dependencies Risks of third-party services.

CN119066670APending Publication Date: 2024-12-03LNRS DATA SERVICES INC
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202410705842.0
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Priority Date
2024-04-17
Filing Date
2024-06-03
Publication Date
2024-12-03

AI Technical Summary

Technical Problem

Existing baseline AI/ML services may output illusions when processing commercial data, especially in digital and analytics, and passing proprietary data to third-party services has security, privacy, and competition issues, and requires each time the model is updated. Repeated training leads to high costs.

Method used

Using a multi-stage approach, identify potential hallucinations through internal operations and automation, use third-party AI/ML services to answer questions, and verify answers within proprietary systems to ensure the security and accuracy of answers and avoid exposing proprietary data to Third party.

Benefits of technology

It enables the creation and fine-tuning of commercially accurate AI/ML models with desired characteristics without exposing proprietary data, reducing costs and reducing reliance on third-party services, and improving data security and privacy protection.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119066670A_ABST
    Figure CN119066670A_ABST
Patent Text Reader

Abstract

Systems and methods are provided that utilize AI / ML services to address commercial problems while protecting proprietary information. Third party AI / ML services are used in a multi-stage method to answer questions and / or solve puzzles, create code to validate answers, execute the code within a proprietary system, and then use the answers to create better answers without exposing proprietary data to the third party.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The disclosed technology generally relates to protecting proprietary data, and specifically to systems and methods for protecting proprietary data when using third-party artificial intelligence and / or machine language services. Background Art

[0002] The rise of ChatGPT (Chat Generative Pre-trained Transformer) and other artificial intelligence and / or machine language (AI / ML) services offers businesses a huge opportunity to streamline internal operations, build new products, and / or provide efficient services to customers. However, there is a risk that unimproved baseline AI / ML services used to answer specific business questions may output hallucinations, which can include (very confident) wrong answers, especially when numbers and analytics are involved.

[0003] Figure 1 A general process 100 for fine-tuning an AI / ML model so that hallucinations can be avoided and so that the (refined) model can be suitable for being hosted on a commercial server is depicted. Boxes 102, 106, and 108 may include proprietary data or sensitive customer data. Boxes 104 and 110 represent processes performed using a third-party service. This process 100 may include a "pre-processing" phase, where, in box 102, proprietary data and business logic may be formulated into a format for fine-tuning. In box 102, a third-party AI / ML service may be used to fine-tune a baseline model. In some instances, proprietary data may be used in such a way that the resulting model 105 may be used for a specific business purpose. In typical use, and as shown in box 106, a problem, question, or change in the proprietary data may be input into a proprietary system. In box 108, the proprietary system may formulate a prompt (including any seed parameters). In box 110, the prompt may be sent to a third-party service, which may answer the question using the fine-tuned model 105. However, this process 100 may risk passing proprietary data to a third party that is no longer in the control of the business that created it, raising security, privacy, and competition issues. In addition, the training process may need to be repeated at significant expense each time there is a new service or updated model.

[0004] An example alternative to the above process 100 for fine-tuning a model may include creating a "secure model" for hosting without interacting with a third-party service. However, such a model may be incomplete in content, or it may not have the same desired features as the third-party service. A method and system is needed that allows the creation and / or fine-tuning of commercially accurate AI / ML models with desired third-party features without exposing proprietary data. Summary of the invention

[0005] Some or all of the above needs may be addressed by certain embodiments of the disclosed technology. Certain embodiments of the disclosed technology may include: using a multi-stage approach with a third-party AI / ML model to answer a query, creating code to verify the answer, executing the code within a proprietary system, and then using the answer to create a better answer.

[0006] In some implementations, one or more hallucinations for certain queries may be determined, for example, by internal operations, by asking a customer a question, or by other automated means. In some implementations, questions may be formulated and certain pre-seed parameters may be created. The process may utilize a third-party AI / ML service to answer the questions, and the results may be parsed for possible hallucinations to be verified by data present in the proprietary system. The process may reformulate the AI / ML answers into coded questions. Pre-seed parameters using proprietary system patterns / baselines may then be generated. The third-party service may be used again to answer the coded questions. The process may verify the coded responses for security / privacy / performance, and the coded responses may be executed in the proprietary system. In some implementations, the original question may be reformulated to include the results of the proprietary execution, and a third-party service may be used to answer the original question using data from the results. In some implementations, the results may be verified to ensure that they meet an acceptable level of quality.

[0007] According to certain exemplary implementations of the disclosed technology, a method for using AI / ML services to solve business problems while protecting proprietary information is provided. The method may include one or more of the following steps: identifying a question to be answered; formulating a prompt based on the question; submitting the prompt to a third-party AI / ML service; receiving an initial answer to the question based on the submitted prompt; verifying the initial answer based on data in a proprietary system; reformulating the verified initial answer into a coded question; submitting the coded question to a third-party AI / ML; receiving a coded response to the coded question from a third-party AI / ML service; executing the coded response in a proprietary system to generate a proprietary response; verifying the proprietary response for one or more of security, privacy, performance, and accuracy based on data in the proprietary system; creating a reformulated prompt based on the question and the verified proprietary response; submitting the reformulated prompt to a third-party AI / ML service; receiving a final result from a third-party AI / ML service; and verifying the final result based on predetermined criteria.

[0008] Certain implementations of the disclosed technology may include a system having a memory for storing instructions and a processor that can execute the instructions to perform one or more steps of the above-described method.

[0009] Certain implementations may include a non-volatile computer-readable memory storing instructions that, when executed by a processor, perform one or more steps of the above-described method.

[0010] Other embodiments, features, and aspects of the disclosed technology are described in detail herein and are considered to be part of the disclosed technology claimed for protection. Other embodiments, features, and aspects can be understood with reference to the following detailed description, drawings, and claims.

[0011] CROSS-REFERENCE TO RELATED APPLICATIONS

[0012] This application claims priority to U.S. Provisional Patent Application No. 63 / 505.524, filed on June 1, 2023, the contents of which are incorporated herein by reference as if fully presented. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] Reference will now be made to the accompanying drawings and flow charts, which are not necessarily drawn to scale, in which:

[0014] Figure 1 is a block diagram of a standard fine-tuning process 100 for a model.

[0015] Figure 2 is an illustrative multi-stage process 200 for fine-tuning a model in accordance with certain exemplary implementations of the disclosed technology.

[0016] Figure 3An example process 300 for a use case is depicted in accordance with certain example implementations of the disclosed technology.

[0017] Figure 4 is a block diagram of an illustrative computing system 400 according to an exemplary implementation of the disclosed technology.

[0018] Figure 5 is a flow chart of a method 500 according to an exemplary embodiment of the disclosed technology.

[0019] Figure 6 is a flow chart of a method 600 according to an exemplary embodiment of the disclosed technology. DETAILED DESCRIPTION

[0020] Embodiments of the disclosed technology will be described more fully below with reference to the accompanying drawings, in which certain example embodiments are shown. However, the disclosed technology may be embodied in many different forms and should not be construed as limited to the embodiments set forth herein; rather, these embodiments are provided so that this disclosure will be thorough and complete and will fully convey the scope of the disclosed technology to those skilled in the art.

[0021] The disclosed techniques may provide an alternative and / or improvement to the less secure process of model fine-tuning, as described above and in Figure 1 For example, Figure 2 Exemplary embodiments of the disclosed technology are described that are capable of leveraging third-party AI / ML services in a multi-stage approach to answer questions and / or solve problems, create code to verify the answers, execute the code within a proprietary system, and then use the answers to create better answers without exposing proprietary data to third parties.

[0022] According to an exemplary embodiment, and with reference to Figure 2 , some implementations of the disclosed technology can use a baseline AI / ML service to solve puzzles and / or answer questions. In some implementations, the results (including hallucinations) can be analyzed, reformulated, and then fed back to the AI / ML service to identify the actual code that needs to be run to obtain the correct answer or solution. In some implementations, the business can run such actual code in its own protected proprietary environment. Once the answer is received, the results can be fed back to the AI / ML service to complete the answer / solution without hallucinations and without sharing proprietary data and / or algorithms with third parties.

[0023] According to some implementations, and with reference to Figure 2In the process 200 shown, blocks 206, 212, and 220 may be processed using a third-party AI / ML service, while the remaining blocks 202, 204, 208, 210, 214, 216, 218, and 222 may be performed using a proprietary system, so that sensitive or proprietary information or algorithms are not exposed to the third-party AI / ML service. In block 202, problems, issues, and / or changes in proprietary data may be input into the proprietary system. In this step, associated topics related to the problems, issues, and / or changes may be found in a number of ways, including automated means. For example, a customer may ask a question of a business operating a private / proprietary system. In some implementations, the question may be in the form of a chat, email, voice, etc. In some implementations, the question may be automatically extracted for further use in process 200. In the event that a business needs to change or update proprietary data on its system, the corresponding changes in the proprietary data may be input into the proprietary system.

[0024] In box 204, the puzzles, questions, and / or changes in the proprietary data can be formulated as prompts. In some implementations, the prompts can include seed parameters (i.e., parameters of questions based on the dynamic results of (one or more) initial questions) and / or pre-seed parameters (i.e., parameters of questions based on proprietary data structures, proprietary definitions, or other proprietary information known before asking the questions), for example, the seed parameters and / or pre-seed parameters can help guide the AI / ML towards generating content that is more closely aligned with the desired topic without being overly prescriptive. In some implementations, box 204 can utilize prompt engineering, where seed words (or numbers) can be strategically included to guide the AI / ML in generating content that is more closely aligned with the desired topic. In some implementations, prompt engineering can enable fine control of the AI / ML model via clearly defined prompts, which can guide the AI / ML system to produce outputs consistent with specific goals, while the seed words can provide flexibility and creative breadth. Certain implementations of the disclosed technology may utilize seeds and / or pre-seeds to steer the AI / ML in a particular direction, thereby encouraging a variety of potential responses, which may be beneficial in scenarios that require creativity and exploration. In certain implementations, any proprietary or sensitive information in the formulated prompts may be cleared.

[0025] In box 206, the model on the third-party AI / ML service can be fine-tuned, for example, by training the model on the target data set. In some implementations, the original capabilities of the pre-trained model can be maintained while it is adapted to accommodate more specialized use cases. In some implementations, training / fine-tuning the model can help improve performance and accuracy on a wide range of tasks. In some implementations, the training data can be structured, where each line represents a prompt completion pair corresponding to a training example. In some implementations, a command line interface (CLI) data preparation tool can be used to verify, provide suggestions, and reformat the data into the required format for fine-tuning. According to certain exemplary implementations of the disclosed technology, prompts (as formulated in box 204) can be submitted to the (fine-tuned) AI / ML model, and the resulting answers can be captured.

[0026] In box 208, the result answer from box 206 can be parsed for possible hallucinations. In some implementations, data present in the proprietary system can be used to verify the answer and / or determine whether the result answer includes any hallucinations. One method that can be used to reduce hallucinations is a retrieval-augmented generation (RAG) model combined with a vector database. The method can achieve effective utilization of a large language model (LLM) with proprietary data. According to some exemplary implementations of the disclosed technology, a trusted knowledge source (i.e., data from a proprietary system) can be searched for relevant data. The model can use those results to generate user-friendly responses and merge relevant details into a single concise answer. In some implementations, a vector database can be used to improve the performance of the RAG model. In some implementations, the vector database can store text as a digital vector that embeds or captures its meaning. Questions can also be converted into digital vectors. Relevant documents or paragraphs can then be found in the vector database, even when they do not share the same words. This approach can be used to mitigate the risk of hallucinations that can occur when relying on an AI / ML / LLM model to generate answers based solely on its training data.

[0027] In box 210, process 200 can reformulate the AI / ML answer into a coding question and create pre-seed parameters using proprietary system patterns / baselines. For example, without sharing actual proprietary data to a third party, this step can take the results of the AI / ML answer and change it into a coding question based on a proprietary pattern, baseline product, code, and / or process. In some implementations, the pattern can be a database definition captured at a specific time. In some implementations, the baseline version of the pattern can be an initial version of the database pattern and can be used to help create full visibility of the evolution of the pattern. In some implementations, the baseline can be a static representation of the project and can therefore be used as a benchmark to measure performance as the project progresses. In some implementations, multiple baselines can be created to establish metrics throughout the project life cycle.

[0028] In block 212, the third-party AI / ML service may be used to answer the coded question (as formulated in block 210). In this step, the process of asking the coded question based on previous responses combined with business information may be performed without sharing actual proprietary data. In this step, the third-party AI / ML service may output a coded response.

[0029] In block 214, the proprietary system may validate the encoded response for security, privacy, and / or performance. In some implementations, a process similar to that described above in block 208 may be employed to validate the encoded response.

[0030] The validated encoded response may be executed in the proprietary system in block 216. In this step, the results of the encoded response using AI / ML (based on the previous steps) may enable the AI / ML model to be run on the company's own proprietary system.

[0031] In block 218 , the original problem may be reformulated to include the results from the proprietary execution of block 216 .

[0032] In box 220, the third party service can be used to answer the reformulated original question using data from the results. In this step, the inquiry question can be based on the previous steps.

[0033] In block 222, the proprietary system may verify the results to ensure that they meet an acceptable quality level. In some implementations, the acceptable quality level may be based on certain metrics and / or predetermined thresholds. In some implementations, a process similar to that described above in block 208 may be employed to verify the results.

[0034] As one of ordinary skill in the art can appreciate, the disclosed techniques can provide improvements over traditional systems, allowing the use of third-party AI / ML services while reducing or eliminating exposure of proprietary data and / or proprietary processes to third parties.

[0035] Certain implementations of the disclosed technology can further reduce costs. For example, the disclosed technology does not require third-party fine-tuning, which can be an expensive step, and it can allow migration between AI / ML services (different versions or different companies) without having to reinvest in fine-tuning or other expensive integration steps.

[0036] Certain implementations of the disclosed technology can provide improvements in flexibility. For example, the code generation step can use SQL language to run in a proprietary database, or use code to execute within the system. Certain implementations can create surveys or other tools to collect information from customers in proprietary systems without sharing data. In certain implementations, the entire process can be automatically run in seconds or within a predetermined period of time (such as, months), with external user input collected as part of the step.

[0037] Figure 3 Depicted is a use case of an example process for determining the impact of a merger between two airlines. According to certain example implementations of the disclosed technology, the example use case may follow the above with respect to Figure 2 Some or all of the steps discussed. In step (1), a query (Q) may be formulated (e.g., using a proprietary system) and posed to a third-party AI / ML system (A1) to ask what the impact would be if two airlines merged. The query may include a request to segment results by domestic, international, and U.S. region.

[0038] In step (2), the results received from the third-party AI / ML system may be parsed, checked for hallucinations, and verified by the proprietary system.

[0039] In step (3), the results received from the third-party AI / ML system may be reformulated (by the proprietary system) into a coding problem for generation by the third-party AI / ML system (A2).

[0040] In step (4), the coding problem generated by the third-party AI / ML system (A2) can be verified and the generated coding problem can be executed on the proprietary system (A1).

[0041] In step (5), the response from step (4) may be parsed to include the generated coded questions. The proprietary system (A1) may be asked the initial questions, but broken down by region and generate corresponding coded questions.

[0042] In step (6), the results from step (5) may be parsed and a response may be created for the user.

[0043] Figure 4 A block diagram of an illustrative computing device 400 that can be used to implement certain aspects of the disclosed technology is depicted. Various implementations and methods herein can be embodied in a non-transitory computer-readable medium executed by a processor. It will be understood that computing device 400 is provided for example purposes only and does not limit the scope of different implementations of communication systems and methods.

[0044] Figure 4 The computing device 400 includes one or more processors in which computer instructions are processed. The computing device 400 may include a processor 402, or it may be connected to a processor 402. Figure 4 In some examples, the computing device may be a processor, a controller, or a central processing unit (CPU). In still other examples, the computing device may be a set of hardware components.

[0045] The computing device 400 may include a display interface 404 that serves as a communication interface and provides functionality for presenting video, graphics, images, and text on a display. In some example implementations of the disclosed technology, the display interface 404 may be directly connected to a local display. In another example implementation, the display interface 404 may be configured to provide data, images, and other information to an external / remote display. In some example implementations, the display interface 404 may communicate wirelessly with an external / remote display, for example, via a Wi-Fi channel or other available network connection interface 412.

[0046] In an example implementation, the network connection interface 412 can be configured as a communication interface and can provide functionality for presenting video, graphics, images, text, other information, or any combination thereof on a display. In one example, the communication interface may include a serial port, a parallel port, a general-purpose input and output (GPIO) port, a game port, a universal serial bus (USB), a micro USB port, a high-definition multimedia (HDMI) port, a video port, an audio port, a Bluetooth port, a near-field communication (NFC) port, another similar communication interface, or any combination thereof. In one example, the display interface 404 is operably coupled to a local display. In another example, the display interface 404 can communicate wirelessly with an external / remote display, for example, via a network connection interface 412 (such as a Wi-Fi transceiver).

[0047] The computing device 400 may include a keyboard interface 406 that provides a communication interface for a keyboard. According to some example implementations of the disclosed technology, a presence-sensitive display interface 408 may provide a communication interface to various devices such as a pointing device, a touch screen, and the like.

[0048] The computing device 400 may be configured to use input devices via one or more of the input / output interfaces (e.g., keyboard interface 406, display interface 404, presence-sensitive display interface 408, network connection interface 412, camera interface 414, sound interface 416, etc.) to allow a user to capture information into the computing device 400. The input devices may include a mouse, a trackball, a directional pad, a trackpad, a touch-authentication trackpad, a presence-sensitive trackpad, a presence-sensitive display, a scroll wheel, a digital camera, a digital video camera, a webcam, a microphone, a sensor, a smart card, etc. In addition, the input devices may be integrated with the computing device 400 or may be separate devices. For example, the input devices may be accelerometers, magnetometers, digital cameras, microphones, and optical sensors.

[0049] Example implementations of computing device 400 may include: an antenna interface 410 that provides a communication interface to an antenna; and a network connection interface 412 that provides a communication interface to a network. According to some example implementations, antenna interface 410 may be used to communicate with a Bluetooth transceiver.

[0050] In some implementations, a camera interface 414 may be provided that serves as a communication interface and provides functionality for capturing digital images from a camera. In some implementations, a sound interface 416 is provided as a communication interface for converting sound into electrical signals using a microphone and for converting electrical signals into sound using a speaker. According to an example implementation, a random-access memory (RAM) 418 is provided, where computer instructions and data may be stored in a volatile memory device for processing by the CPU 402.

[0051] According to an example implementation, the computing device 400 includes a read-only memory (ROM) 420, where unchanging low-level system code or data for basic system functions (such as basic input and output (I / O), booting, or receiving keystrokes from a keyboard) is stored in a non-volatile memory device. According to an example implementation, the computing device 400 includes a storage medium 422 or other suitable type of memory (e.g., RAM, ROM, programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), magnetic disk, optical disk, floppy disk, hard disk, removable cartridge, flash drive), where files are stored, including an operating system 424, application programs 426 (e.g., including a web browser application, a gadget engine or a gadget engine and / or other applications as necessary) and data files 428. According to an example implementation, the computing device 400 includes a power supply 430 that provides appropriate alternating current (AC) or direct current (DC) to the power components. According to an example implementation, the computing device 400 includes a telephone subsystem 432 that allows the device 400 to send and receive voice over a telephone network. The component devices and the CPU 402 communicate with each other via a bus 434.

[0052] According to an example implementation, the CPU 402 has a suitable structure as a computer processor. In one arrangement, the computer CPU 402 may include more than one processing unit. The RAM 418 is coupled to the computer bus 434 to provide fast RAM storage to the CPU 402 during the execution of software programs (such as operating system applications and device drivers). More specifically, the CPU 402 loads computer executable processing steps from the storage medium 422 or other media into the fields of the RAM 418 to execute the software program. Data can be stored in the RAM 418, where the data can be accessed by the computer CPU 402 during execution. In an example configuration, the device 400 includes at least 128MB of RAM and 256MB of flash memory.

[0053] The storage medium 422 itself may include multiple physical drive units, such as a redundant array of independent disks (RAID), a floppy disk drive, a flash memory, a USB flash drive, an external hard drive, a thumb drive, a pen drive, a key drive, a High-Density Digital Versatile Disc (HD-DVD) optical drive, an internal hard drive, a Blu-ray optical drive or a Holographic Digital Data Storage (HDDS) optical drive, an external micro dual in-line memory module (DIMM) synchronous dynamic random access memory (SDRAM), or an external micro DIMM SDRAM. Such computer-readable storage media allow the device 400 to access computer-executable processing steps, applications, etc. stored on removable and non-removable storage media to download data from the device 400 or upload data to the device 400. A computer program product (such as a computer program product utilizing a communication system) may be tangibly embodied in the storage medium 422, which may include a machine-readable storage medium.

[0054] According to an example implementation, the term computing device as used herein may be a CPU, or may be conceptualized as a CPU (e.g., Figure 4 In this example implementation, the computing device (CPU) may be coupled, connected, and / or in communication with one or more peripheral devices.

[0055] Various implementations of the communication systems and methods herein may be embodied in a non-transitory computer-readable medium for execution by a processor. Example implementations may be used in applications of mobile computing devices (e.g., smart phones or tablet computers), but other computing devices may also be used such as portable computers, tablet PCs, Internet tablets, PDAs (Personal Digital Assistants), ultra-mobile PCs (UMPCs (Ultra Mobile Personal Computers)), etc.

[0056] Now refer to Figure 5 Flowchart of describes an exemplary method 500 for utilizing AI / ML services to solve business challenges while protecting proprietary information. Method 500 begins at box 502 and includes identifying a topic or question to be answered according to an exemplary embodiment of the disclosed technology. In box 504, method 500 includes formulating the question and creating pre-seed parameters for prompt engineering. In box 506, method 500 includes utilizing a third-party AI / ML service to provide an initial answer to the question. In box 508, method 500 includes analyzing the results for possible hallucinations and validating them against data in a proprietary system. In box 510, method 500 includes reformulating the AI / ML answer into a coded question, creating pre-seed parameters using a proprietary system model / baseline. In box 512, method 500 includes utilizing a third-party AI / ML service to answer the coded question. In box 514, method 500 includes validating the coded response for security / privacy / performance and executing the coded response in the proprietary system. In block 516, method 500 includes reformulating the original question to include the results of the proprietary execution, and answering the original question using the third-party AI / ML service with data from the results. In block 518, method 500 includes validating the results to ensure that they meet an acceptable level of quality. Method 500 ends after block 518.

[0057] In certain implementations, proprietary systems may include sensitive data and / or algorithms that are not shared with third-party AI / ML services.

[0058] In certain implementations, the initial answer provided by the third-party AI / ML service can be analyzed to detect possible hallucinations or errors.

[0059] In some implementations, coded questions can be formulated based on the results of the initial answers. In some implementations, coded questions can be designed to obtain more accurate answers while protecting proprietary information.

[0060] In some implementations, the encoded response can be validated for security, privacy, and performance before being executed in a proprietary system.

[0061] In certain implementations, the final answer obtained from the third-party AI / ML service may be validated to ensure that it meets acceptable standards.

[0062] Now refer to Figure 6 600 for solving business problems using AI / ML services while protecting proprietary information. The method 600 begins at box 602 and includes identifying a question to be answered according to an exemplary embodiment of the disclosed technology. In box 604, the method 600 includes formulating a prompt based on the question. In box 606, the method 600 includes submitting the prompt to a third-party AI / ML service. In box 608, the method 600 includes receiving an initial answer to the question based on the submitted prompt. In box 610, the method 600 includes verifying the initial answer based on data in a proprietary system. In box 612, the method 600 includes reformulating the verified initial answer into a coded question. In box 614, the method 600 includes submitting the coded question to the third-party AI / ML. In box 616, the method 600 includes receiving a coded response to the coded question from the third-party AI / ML service. In box 618, the method 600 includes executing the coded response in the proprietary system to generate a proprietary response. In block 620, the method 600 includes validating the proprietary response for one or more of security, privacy, performance, and accuracy based on data in the proprietary system. In block 622, the method 600 includes creating a reformulated prompt based on the question and the validated proprietary response. In block 624, the method 600 includes submitting the reformulated prompt to a third-party AI / ML service. In block 626, the method 600 includes receiving a final result from the third-party AI / ML service. In block 628, the method 600 includes validating the final result based on predetermined criteria.

[0063] In some implementations, verifying the initial response can include analyzing the initial response for possible illusions or errors.

[0064] According to certain exemplary implementations of the disclosed technology, reformulating the validated initial answer may include creating prompt seed parameters based on a pattern or baseline of a proprietary system.

[0065] In certain implementations, proprietary systems may include sensitive data and / or algorithms that are not shared with third-party AI / ML services.

[0066] According to certain example implementations of the disclosed technology, validating the proprietary response may include determining that the proprietary response does not include a hallucination or an error.

[0067] In some implementations, coded questions can be formulated based on the results of the initial answers. In some implementations, coded questions can be designed to obtain more accurate answers while protecting proprietary information.

[0068] In some implementations, the reformulated hints can be validated for security, privacy, and performance before being executed in a proprietary system.

[0069] In certain implementations, the final results obtained from the third-party AI / ML service may be validated to ensure that they meet acceptable standards.

[0070] In some implementations, the question to be answered can include identifying an issue to be resolved.

[0071] Some embodiments of the disclosed technology may include hardware and / or software applications that are executed to assist any number of operations. In an exemplary embodiment, one or more I / O interfaces may assist the communication between input / output devices. For example, a universal serial bus port, a serial port, a disk drive, a CD-ROM (Compact Disc Read-Only Memory) driver, and / or one or more user interface devices (such as, display, keyboard, keypad, mouse, control panel, touch screen display, microphone, etc.) may assist user interaction. One or more I / O interfaces may be used to receive or collect data and / or user instructions from a variety of input devices. The received data may be processed and / or stored in one or more memory devices by one or more computer processors as required in various embodiments of the disclosed technology.

[0072] The one or more network interfaces can facilitate connecting the input and output to one or more suitable networks and / or connections; for example, facilitating connections for communicating with any number of sensors associated with the system. The one or more network interfaces can also facilitate connecting to one or more suitable networks (e.g., local area networks, wide area networks, the Internet, cellular networks, radio frequency networks, Bluetooth TM (Owned by Telefonaktiebolaget LM Ericsson) Enabled Networks, Wi-Fi TM (owned by the Wi-Fi Alliance) enabled network, satellite-based network, any wired network, any wireless network, etc.) for communicating with external devices and / or systems.

[0073] As required, embodiments of the disclosed technology may include Figure 4 More or fewer components than shown.

[0074] The above reference to the block diagrams and flow charts of the system and method and / or computer program product according to the exemplary embodiments of the disclosed technology describes certain embodiments of the disclosed technology. It should be understood that one or more blocks in the block diagrams and flow charts and the combination of blocks in the block diagrams and flow charts can be implemented by computer executable program instructions respectively. Similarly, according to some embodiments of the disclosed technology, some blocks of the block diagrams and flow charts may not necessarily be executed in the order presented or may not necessarily be executed at all.

[0075] These computer executable program instructions can be loaded onto a general purpose computer, a special purpose computer, a processor, or other programmable data processing device to produce a specific machine, so that the instructions executed on the computer, processor, or other programmable data processing device create a device for implementing one or more functions specified in one or more boxes of the flowchart. These computer program instructions can also be stored in a computer readable memory so that the instructions stored in the computer readable memory produce a manufactured product including an instruction device that implements one or more functions specified in one or more boxes of the flowchart, and these computer program instructions make the computer or other programmable data processing device work in a specific way. As an example, an embodiment of the disclosed technology can provide a computer program product, which includes a computer-usable medium having a computer-readable program code or program instructions embodied therein, and the computer-readable program code is suitable for being executed to implement one or more functions specified in one or more boxes of the flowchart. Computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating elements or steps are performed on the computer or other programmable device to produce a computer-implemented process so that the instructions executed on the computer or other programmable device provide elements or steps for implementing the functions specified in one or more boxes of the flowchart.

[0076] Therefore, the blocks of the block diagrams and flow charts support a combination of means for performing the specified functions, a combination of elements or steps for performing the specified functions, and a program instruction device for performing the specified functions. It should also be understood that each block in the block diagrams and flow charts and the combination of blocks in the block diagrams and flow charts can be implemented by a dedicated, hardware-based computer system or a combination of dedicated hardware and computer instructions that performs the specified functions, elements or steps.

[0077] Although certain embodiments of the disclosed technology have been described in connection with what are presently considered to be the most practical and various embodiments, it should be understood that the disclosed technology is not limited to the disclosed embodiments, but on the contrary is intended to cover various modifications and equivalent arrangements included within the scope of the appended claims. Although specific terms are employed herein, they are used in a generic and descriptive sense only and not for purposes of limitation.

[0078] In the foregoing description, many specific details are set forth. However, it should be understood that the embodiments can be implemented without these specific details. In other instances, well-known methods, structures, and techniques are not shown in detail so as not to obscure the understanding of this specification. The term "exemplary" is used synonymously with the term "example" in this article and is not meant to indicate excellent or optimal. References to "one embodiment," "embodiment," "exemplary embodiments," "various embodiments," etc. indicate that the embodiments of the disclosed technology as described may include specific features, structures, or characteristics, but not every embodiment must include specific features, structures, or characteristics. In addition, the repeated use of the phrase "in one embodiment" does not necessarily refer to the same embodiment, although it may refer to the same embodiment.

[0079] As used herein, unless otherwise specified, the use of ordinal adjectives "first," "second," "third," etc. to describe common objects merely indicates that different instances of similar objects are mentioned and is not intended to imply that the objects as described must be in a given sequence in time, space, order, or in any other manner.

[0080] This written description uses examples to disclose certain embodiments of the disclosed technology, including the best mode, and also discloses certain embodiments that enable any person skilled in the art to practice the disclosed technology, including making and using any devices or systems and performing any incorporated methods. The patentable scope of certain embodiments of the disclosed technology is defined in the claims and may include other examples that occur to one skilled in the art. Such other examples are intended to be within the scope of the claims if they have structural elements that do not differ from the literal language of the claims, or if they include equivalent structural elements with insubstantial differences from the literal language of the claims.

Claims

1. A method for using AI / ML services to solve business problems while protecting proprietary information, comprising: Identify questions to be answered; Develop prompts based on the stated questions; Submitting said prompt to a third-party AI / ML service; receiving an initial response to the question based on the submitted prompt; verifying said initial response against data in a proprietary system; reformulated validated initial responses into coded questions; Submitting the coding problem to the third-party AI / ML; receiving a coded response to the coded question from the third-party AI / ML service; executing the encoded response in the proprietary system to generate a proprietary response; validating the proprietary response against data in the proprietary system for one or more of security, privacy, performance, and accuracy; creating reformulated prompts based on said questions and validated proprietary responses; submitting the reformulated prompt to the third-party AI / ML service; Receive final results from the third-party AI / ML service; as well as The final result is verified based on predetermined criteria.

2. The method according to claim 1, wherein: Formulating an initial prompt includes creating pre-seed parameters for the prompt.

3. The method according to claim 1, wherein: Verifying the initial response includes analyzing the initial response for possible illusion or error.

4. The method according to claim 1, wherein: Reformulating the validated initial response includes creating prompt seed parameters based on a pattern or baseline of the proprietary system.

5. The method according to claim 1, wherein: The proprietary system includes sensitive data and / or algorithms that are not shared with the third-party AI / ML service.

6. The method according to claim 1, wherein: Verifying the proprietary response includes determining that the proprietary response does not include a hallucination or an error.

7. The method according to claim 1, wherein: The coded questions are formulated based on the results of the initial responses and are designed to obtain more accurate responses while protecting proprietary information.

8. The method according to claim 1, wherein: The reformulated prompts are validated for security, privacy, and performance before being executed in the proprietary system.

9. The method according to claim 1, wherein: Validate the final results obtained from the third-party AI / ML service to ensure that they meet acceptable standards.

10. The method according to claim 1, wherein: Identifying questions to be answered includes: Identifying issues to be addressed.

11. A system comprising: one or more processors; a memory in communication with the one or more processors; Instructions stored in the memory, which when executed by the one or more processors cause the system to perform a method comprising: Identify questions to be answered; Develop prompts based on the stated questions; Submitting said prompt to a third-party AI / ML service; receiving an initial response to the question based on the submitted prompt; verifying said initial response against data in a proprietary system; reformulated validated initial responses into coded questions; Submitting the coding problem to the third-party AI / ML; receiving a coded response to the coded question from the third-party AI / ML service; executing the encoded response in the proprietary system to generate a proprietary response; validating the proprietary response against data in the proprietary system for one or more of security, privacy, performance, and accuracy; creating reformulated prompts based on said questions and validated proprietary responses; submitting the reformulated prompt to the third-party AI / ML service; Receive final results from the third-party AI / ML service; and The final result is verified based on predetermined criteria.

12. The system according to claim 11, wherein: Formulating an initial prompt includes creating pre-seed parameters for the prompt.

13. The system according to claim 11, wherein: Verifying the initial response includes analyzing the initial response for possible illusion or error.

14. The system according to claim 11, wherein: Reformulating the validated initial response includes creating prompt seed parameters based on a pattern or baseline of the proprietary system.

15. The system according to claim 11, wherein: The proprietary system includes sensitive data and / or algorithms that are not shared with the third-party AI / ML service.

16. The system according to claim 11, wherein: Verifying the proprietary response includes determining that the proprietary response does not include a hallucination or an error.

17. The system of claim 11, wherein: The coded questions are formulated based on the results of the initial responses and are designed to obtain more accurate responses while protecting proprietary information.

18. The system of claim 11, wherein: The reformulated prompts are validated for security, privacy, and performance before being executed in the proprietary system.

19. The method according to claim 11, wherein: Identifying questions to be answered includes: Identifying issues to be addressed.

20. At least one non-transitory computer-readable medium comprising a set of instructions that, in response to being executed by a processor circuit, causes the processor circuit to perform the following method: Identify questions to be answered; Develop prompts based on the stated questions; Submitting said prompt to a third-party AI / ML service; receiving an initial response to the question based on the submitted prompt; verifying said initial response against data in a proprietary system; reformulated validated initial responses into coded questions; Submitting the coding problem to the third-party AI / ML; receiving a coded response to the coded question from the third-party AI / ML service; executing the encoded response in the proprietary system to generate a proprietary response; validating the proprietary response against data in the proprietary system for one or more of security, privacy, performance, and accuracy; creating reformulated prompts based on said questions and validated proprietary responses; submitting the reformulated prompt to the third-party AI / ML service; Receive final results from the third-party AI / ML service; and The final result is verified based on predetermined criteria.