Data processing system and method for masking sensitive data

Abstract

To provide a method, data processing system and computer-readable storage medium for masking sensitive data for fetching information from foundation models.SOLUTION: The method comprises: receiving from a user a query pertaining to a request for information; generating prompts based on the query from the user, by masking sensitive information in the query; receiving responses from foundation models in response to inputting the prompts; generating a common result set based on the responses; generating a response by validating the common result set with the sensitive information and the query; generating a user response by supplementing the response with the sensitive information; and providing the user response to the user in response to the query.SELECTED DRAWING: Figure 6

Description

[Technical field]

[0001] FIELD OF THE DISCLOSURE The present disclosure relates generally to the field of data processing, and more particularly to a data processing system and method for masking sensitive data. [Background technology]

[0002] In modern data management systems, ensuring the security and confidentiality of sensitive information during interactions with third-party platforms such as the underlying model is of paramount importance. There is an increased emphasis on protecting sensitive data from unauthorized access or exposure while facilitating the process of seamlessly retrieving and presenting the data. Addressing such challenges highlights the ongoing demand for robust frameworks that can protect sensitive information and ensure that its integrity and confidentiality are maintained across various interactions between users and the platform. Summary of the Invention [Means for solving the problem]

[0003] The present disclosure relates generally to the field of data processing, and more particularly to a data processing system and method for masking sensitive data to fetch information from an underlying model.

[0004] Implementations of the present disclosure are generally directed to masking sensitive data during information retrieval. Sensitive information is identified and masked before a request for information reaches a third-party tool, ensuring that enterprise data remains safe within the enterprise environment. Advantageously, sensitive data is masked in a manner that preserves contextual and relational relevance within the sensitive data. This ensures that external tools provide accurate and appropriate responses without gaining access to the sensitive information. The external tool may be implemented as an external foundational model. Leveraging masked sensitive data for information retrieval reduces the required computation time without compromising accuracy. As a result, the proposed system and method revolutionize the field of foundational models by keeping sensitive data away from external / third-party tools implementing said foundational models without compromising the accuracy of the model output.

[0005] In general, an innovative aspect of the subject matter described herein provides a data processing system including at least one processor and at least one non-transitory processor-readable medium storing instructions executed by the at least one processor, the at least one processor configured to receive a query from a user for a request for information, generate a plurality of prompts based on the query from the user, the plurality of prompts being generated by masking sensitive information in the query, receive a plurality of responses from a plurality of foundational models in response to inputting the plurality of prompts, generate a common result set using the sensitive information and the query, generate a user response by supplementing the response with the sensitive information, and provide the user response to the user in response to the query.

[0006] The present disclosure further describes a method for masking sensitive data during information retrieval.The present disclosure further describes a non-transitory processor-readable storage medium coupled to one or more processors and having instructions stored thereon, the instructions, when executed by the one or more processors, causing the one or more processors to perform operations in accordance with the methods described herein.

[0007] It will be understood that methods according to the present disclosure can include any combination of the aspects and features described herein, i.e., methods according to the present disclosure are not limited to the combinations of aspects and features specifically described herein, but include any combination of the aspects and features set forth.

[0008] The details of one or more implementations of the disclosure are set forth in the accompanying drawings and the description below. Other features and advantages of the disclosure will become apparent from the description and drawings, and from the claims.

[0009] Various embodiments in accordance with the present disclosure will now be described with reference to the following drawings.

[0010] Like reference numbers and designations in the various drawings indicate like elements. [Brief description of the drawings]

[0011] [Figure 1] 1 illustrates an example environment that can be used to perform implementations of the present disclosure. [Diagram 2] 1 illustrates an example architecture of a data processing system for masking sensitive data during information acquisition in accordance with an implementation of the present disclosure. [Diagram 3] FIG. 1 shows a block diagram illustrating a process flow for masking sensitive data from an underlying model in accordance with an implementation of the present disclosure. [Figure 4] FIG. 2 shows a block diagram illustrating an example process flow for generating a user response from a user query in accordance with an implementation of the present disclosure. [Diagram 5]1 illustrates a detailed process flow for masking sensitive data during information capture according to an implementation of the present disclosure. [Figure 6] FIG. 1 is a flow diagram presenting an exemplary method according to an implementation of the present disclosure. [Figure 7] 1 illustrates a computer system that can be used to implement the knowledge system. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0012] In the following description, various embodiments are shown in the accompanying drawing figures by way of example and not by way of limitation. References to various embodiments in this disclosure do not necessarily refer to the same embodiment, and such references imply at least one. Although specific implementations and other details are mentioned, it should be understood that this is done for illustrative purposes only. Those skilled in the art will recognize that other components and configurations may be used without departing from the scope of the claimed subject matter.

[0013] Any reference to an "example" herein (such as "for example," "an example of," "by way of example," or the like) shall be considered a non-limiting example, whether expressly stated or not.

[0014] The terms used herein generally have their ordinary meaning in the art, within the context of this disclosure and in the specific context in which each term is used. Alternative terms and synonyms may be used for any one or more of the terms mentioned herein, and no particular emphasis should be placed on whether a term is detailed or mentioned in this specification. Synonyms are provided for certain terms. The description of one or more synonyms does not preclude the use of other synonyms. The use of examples anywhere in this specification, including examples of any terms mentioned herein, is for illustrative purposes only and is not intended to further limit the scope and meaning of the disclosure or any term exemplified. Similarly, the disclosure is not limited to the various embodiments shown herein.

[0015] Without intending to limit the scope of the present disclosure, examples of instruments, devices, systems, methods, and their related results according to the embodiments of the present disclosure are given below. Note that for the convenience of the reader, headings or subheadings may be used in the examples, but this does not limit the scope of the present disclosure in any way. Unless otherwise defined, technical and scientific terms used in this specification have the meanings that are commonly understood by those skilled in the art to which the present disclosure pertains. In case of conflict, the present document, including definitions, will take precedence.

[0016] When the term "comprising" is used, it means "including, but not necessarily limited to" and specifically indicates a non-exclusive inclusion or belongingness in the combination, group, series, and the like so listed.

[0017] The term "a" means "one or more" unless the context clearly indicates a single component.

[0018] "First," "second," etc. are labels intended to distinguish between otherwise similarly named components or blocks, but do not imply any ordering or numerical limitation.

[0019] "And / or" with two candidates means either or both of the listed candidates ("A and / or B" refers to A only, B only, or both A and B together), and when more than two candidates are presented it means either one of the candidates individually, all of the candidates together, or some combination of fewer than all of the candidates. When A through N are candidates, phrases in the form "at least one of A...and N" mean "and / or" with respect to the listed candidates (e.g., at least one A, at least one N, at least one A and at least one N, etc.).

[0020] It should also be noted that in some alternative implementations, the functions / acts shown may occur in an order different from that shown in the figures. For example, two steps disclosed or shown as successive may in fact be performed virtually simultaneously, or the steps may sometimes be performed in the reverse order, depending on the functionality / acts involved.

[0021] In the following description, specific details are given to thoroughly understand the embodiments. However, it will be apparent to those skilled in the art that the embodiments may be practiced without such specific details. For example, systems may be shown in block diagrams to avoid obscuring the embodiments in unnecessary detail. In other instances, well-known processes, structures, and techniques may be shown without unnecessary detail to avoid obscuring the example embodiments.

[0022] The specification and drawings are to be regarded in an illustrative rather than restrictive sense, although it will be apparent that various modifications and changes may be made thereto without departing from the broader spirit and scope of the invention as set forth in the appended claims.

[0023] The emergence of new-age third-party tools and platforms, such as foundational models, brings both opportunities and challenges in safeguarding sensitive information. Traditional tools and platforms leverage advanced algorithms and data processing methods to increase the accuracy and relevance of the data obtained. Despite such advancements, the integration of sensitive data with such tools brings inherent risks, including the possibility of unauthorized access and data leakage, thereby necessitating robust security measures.

[0024] Traditional methods of securing sensitive information during interactions with any tool or platform, such as the underlying model, have typically relied on encryption and access control mechanisms. However, such approaches often face challenges in ensuring comprehensive protection against unauthorized access or inadvertent exposure. As the digital ecosystem evolves and data interactions become increasingly complex, there is a need for a robust security framework that can effectively mask or conceal sensitive information from external entities.

[0025] Recent advances in cybersecurity have highlighted the inherent vulnerabilities of traditional security measures when interfacing with external systems. Incorporating advanced masking methods and secure communication protocols has become essential to mitigate risks and protect sensitive data during the information capture and presentation process. Until now, masking sensitive information has often involved methods such as deletion or substitution with generic variables, which can result in erroneous responses and unreliable data output, commonly referred to as hallucination, which can lead to a lack of confidence in the data captured.

[0026] Moreover, the proliferation of interconnected digital platforms and the rise of distributed computing paradigms have exacerbated the complexities surrounding data security and privacy. Seamless integration of secure masking methods must be ensured while maintaining the efficiency and accuracy of information retrieval operations. Addressing such challenges requires innovative approaches that enhance the security posture of data interactions while adhering to applicable regulatory requirements and industry standards for data protection.

[0027] An implementation of the disclosed subject matter involves generating a response to a user query and generating multiple prompts by masking sensitive information in the query. The generated multiple prompts are then used to aggregate responses from various underlying models. A common result set is then generated from the aggregated responses, and the common result set is validated against the user query (the original query received from the user) and the sensitive information. The validated information is then integrated to create a user response, which is provided to the user as a consistent response to the user query. Such an approach maintains data security and privacy without compromising the accuracy and reliability of the model output.

[0028] 1 illustrates an example environment 100 that may be used to perform implementations of the present disclosure. In some examples, the example environment 100 enables masking of sensitive information when obtaining information about a user query.

[0029] As shown in FIG. 1, the example environment 100 includes computing devices 102 and 104, a back-end system 106, and a network 108. In some examples, the computing devices 102 and 104 are used by individual users 110 and 112 to log in and interact with a computing platform that executes applications according to implementations of the present disclosure. Examples of computing devices 102 and 104 may include desktop computing devices, smartphones, laptops, tablets, voice-enabled devices, and / or the like. It is contemplated that implementations of the present disclosure may be implemented using any suitable type of computing device. In some examples, the computing devices 102 and 104 may each include a web browser application executing on the device, which may be used to display one or more web pages of the computing platform that executes the applications. In some examples, the computing devices 102 and 104 may each display one or more graphical user interfaces (GUIs) that enable the individual users 110 and 112 to interact with the computing platform.

[0030] In some examples, the network 108 includes a local area network (LAN), a wide area network (WAN), the Internet, or a combination thereof, connecting the websites, the computing devices 102 and 104, and the backend system 106. In some examples, the network 108 may be accessed over wired and / or wireless communication links. For example, a computing device such as a smartphone may access the network 108 using a cellular network.

[0031] In some examples, one or more of the backend systems 106 may be implemented as on-premise systems operated by an enterprise or a third party engaged in cross-platform interaction and data management. In some examples, the backend systems 106 may be implemented as off-premise systems (e.g., cloud or on-demand) operated by an enterprise or a third party acting on behalf of the enterprise. In some examples, one or more of the backend systems 106 may be implemented in a cloud environment. For simplicity, the backend systems 106 shown in FIG. 1 may be a cloud environment intended to represent various forms of servers including web servers, application servers, proxy servers, network servers, server pools, and / or the like.

[0032] In some examples, the backend systems 106 each include one or more data processing systems 114 hosting components for information acquisition (e.g., knowledge embedding). Additionally, the data processing systems 114 receive requests from the users 110 and 112 through their respective computing devices 102 and 104 for services provided by the data processing systems 114. In response to the received requests, the data processing systems 114 provide the requested services to the computing devices 102 and 104 over the network 108. The requests received from the users 110 and 112 through their respective computing devices 102 and 104 may be user queries. The data processing systems 114 may be able to utilize multiple foundational models while masking sensitive data from the multiple foundational models to ensure that accurate responses to queries by the users 110 and 112 are generated without compromising data security. The interactions between the data processing systems 114 and the users 110, 112 may be conversational in nature, including conversational queries and conversational responses to queries.

[0033] According to implementations of the present disclosure, the data processing system 114 may be configured to mask sensitive data while a user interacts with the underlying model, while ensuring accuracy in information retrieval. In conjunction with the following figures, a number of examples illustrating masking sensitive data to ensure data security during information retrieval are described in detail.

[0034] 2 illustrates an example architecture 202 of a data processing system 114 for masking sensitive data during information retrieval according to an implementation of the present disclosure. In an example, as illustrated in FIG. 2, the data processing system 114 receives one or more queries and generates content / responses to the one or more queries. The responses may include, but are not limited to, text, images, audio, video, and / or the like in response to the queries. The one or more queries may include prompts to generate a response to the query about the world, or a particular system, or enterprise, or business, etc. It is noted that the generated response may include additional information, feedback / input, and / or the like required to answer the query.

[0035] The data processing system 114 includes a knowledge base 204, a user interface (UI) / user experience (UX) module 206, and a processing engine 208. The knowledge base 204 may be described as a structured repository or database associated with the data processing system 114. The knowledge base 204 may incorporate various knowledge representation schemes, such as ontologies, taxonomies, or semantic networks, to encode and organize information in a machine-understandable format, thereby enabling advanced search, reasoning, and argumentation capabilities. Additionally, the knowledge base 204 may leverage advanced techniques, including natural language processing, machine learning, and knowledge engineering methods, to enhance the process of knowledge acquisition, updating, and improvement, ensuring continued relevance and adaptability to evolving needs and circumstances.

[0036] In one embodiment of the present disclosure, the knowledge base 204 is stored with user queries 210, masking guidelines 212, enterprise data 214, persona rules 216, responses 218, metadata 220, information about the enterprise or business, and additional information (not shown) about the data processing system 114. User queries 210 refer to one or more queries received from a user. User queries 210 may relate to a request to obtain an answer or response by processing information related to the enterprise. Additionally, user queries 210 may include sensitive information.

[0037] The masking guidelines 212 may be described as guidelines or techniques regarding the sequential steps and decision-making process involved in masking sensitive data in the user query 210. Examples of masking guidelines / techniques 212 include anonymization techniques, tokenization techniques, data scrambling techniques, Dynamic Data Masking (DDM), format-preserving encryption techniques, static data masking techniques, and the like.

[0038] Enterprise data 214 may include information encompassing data, facts, and insights obtained from various data sources related to the enterprise. Such information may be organized in a coherent manner to support decision-making, problem solving, and system operations within data processing system 114. Enterprise data 214 may be temporal, numerical, and / or textual.

[0039] Enterprise data 214 includes enterprise information that is represented across multiple hierarchies and incorporates domain terminology. Hierarchy refers to structured levels or layers in organizational data that define relationships and classifications from broader categories to specific details. Examples of hierarchies may include organizational structure levels (departments, teams, roles, and the like), product classifications (categories, subcategories, products, and the like), and geographic divisions (regions, countries, cities, and the like). Domain terminology refers to specialized language, vocabulary, or jargon that is specific to a particular field or industry and that facilitates proper communication and understanding within that context. Examples of domain terminology may include medical terminology (diagnosis, treatment, patient care, and the like), legal terminology (e.g., jurisdiction, litigation, contracts, and the like), and financial terminology (assets, liabilities, dividends, and the like).

[0040] In this regard, the enterprise data 214 may be stored across multiple sub-databases in the knowledge base 204, such that each of the multiple sub-databases relates to a domain or solution related to the enterprise, such as an HR database for an HR domain, a fraud database for an anti-fraud domain, a collections database, a contract financial management database, and the like. Each of the multiple sub-databases may have specific data tables with relational interactions based on the specific data stored in each of the sub-databases.

[0041] Persona rules 216 refer to rules regarding the accessibility of information about user personas. For example, developers may not necessarily need to have access to information related to the employment of other employees, but administrators or human resources employees may. Responses 218 refer to responses to queries generated by one of the foundation models or data processing systems 114. Responses 218 may include answers to user queries. Metadata 220 may be described as descriptive information about data stored in knowledge base 204, including user queries 210, masking guidelines 212, enterprise data 214, persona rules 216, and responses 218. Additionally, each query and response may have associated metadata that improves the accuracy of responses 218 by providing context to the query or response.

[0042] Further, the knowledge base 204 includes additional data, such as raw data about the enterprise. The raw data may be utilized for training purposes. Typically, the raw data includes hierarchies, labels, categorical variables, and the like. Such components of the raw data provide insights about the data based on which a specific masking methodology may be implemented based on each scenario to mask sensitive data.

[0043] The UI / UX module 206 may be defined as a module that designs and manages the user interface (UI) through which a user interacts with the data processing system 114 and the user experience (UX) during the interaction. The UI / UX module 206 may integrate various technologies and frameworks to optimize visual layouts, interactive components, and overall usability, often utilizing principles of human-computer interaction (HCI) and graphic design.

[0044] In some examples, the UI / UX module 206 may represent one or more front-end components / interfaces 222a-222n of a chatbot that may execute on one or more of the computing devices 102 and 104 to enable receiving a user query and providing one or more user responses to the user query. In some examples, a user query may be received through various modalities, including, but not limited to, a question input to a chatbot, a request provided through a graphical user interface (GUI), an email, and / or the like.

[0045] The processing engine 208 is configured to process queries received through the UI / UX module 206 using multiple foundational models. The foundational models may be described as general purpose generative artificial intelligence (GAI) models, such as large-scale deep learning neural networks. The large-scale deep learning neural networks may be trained using a wide range of generalized and unlabeled training data, which may perform a number of general tasks. Example tasks may include generating text, generating images, conversing in natural language, generating video, generating audio, and / or the like. In some examples, applications may be built that interact with the foundational models. In some examples, multiple foundational models may be used to perform various functionalities of the application.

[0046] The underlying model may include, for example, a large language model (LLM), which is a form of GAI that may be used to generate text for various use cases. In some examples, the LLM may be integrated with a digital assistant (e.g., a chatbot) to provide text responses to user input / queries instead of traditional rule-based systems. The LLM may be described as an advanced language model that is trained using deep learning methods on vast amounts of text data. The LLM typically generates human-like text and performs various Natural Language Processing (NLP) tasks (e.g., translation, question answering, and / or the like). In some examples, the LLM refers to a model that uses deep learning methods and has multiple parameters. The LLM typically captures complex patterns of language and produces text that is often indistinguishable from that written by a human. The produced text may be processed through deep learning architectures such as recurrent neural networks (RNNs), transformer models, and / or the like.

[0047] Although implementations of the present disclosure are described in further detail with non-limiting reference to LLM as an example underlying model, it is contemplated that implementations of the present disclosure may be realized using any suitable underlying or Machine Learning (ML) or Artificial Intelligence (AI) models. Such models may generate content / responses based on any suitable modality (e.g., text, audio, images, video, and / or the like). In some examples, the responses may correspond to one or more tasks expressed by the conversational query.

[0048] In some examples, the foundation model may be provided by one or more third parties. In some examples, the foundation model may be provided via data processing system 114. The foundation model receives requests / queries and provides responses to the queries. For example, questions / requests for information may be received as queries through an application programming interface (API).

[0049] The processing engine 208 includes one or more processors 224 , identification modules 226 , 228 , a prompt generation module 230 , an information acquisition module 232 , a persona module 234 , and a rules engine 236 .

[0050] The processor 224 may include, for example, a microprocessor, a microcomputer, a microcontroller, a digital signal processor, a central processing unit, a state machine, a logic circuit, and / or any device that manipulates data or signals based on operational instructions. The processor 224 is configured to fetch and execute computer readable instructions in a memory operatively coupled to the data processing system 114 to mask sensitive information, among other capabilities.

[0051] The identification module 226 identifies sensitive information in and about the user query 210. The identification module 226 is trained using raw data before being deployed. The raw data includes details such as hierarchies, labels, categorical variables, and the like, and the identification module 226 inherently understands the relationships between different data points in the raw data. The identification module 226 understands at least one of linear relationships between different data points, non-linear relationships between different data points. Advantageously, by understanding the non-linear relationships, the identification module 226 can identify the appropriate sensitive information in a fast and accurate manner. Here, the identification module 226 may utilize multiple relationship identification methods to identify the sensitive information, including statistical correlation, Pearson correlation, ontology correlation, and the like. This allows the identification module 226 to understand the enterprise data 214 in detail. Once deployed, the identification module 226 uses fuzzy matching to identify the sensitive information.

[0052] Fuzzy matching refers to a technique used in the information retrieval and data validation process that allows for approximate matching without exact matches, accepting variations such as misspellings, synonyms, or similar patterns. For example, if the enterprise data 214 contains a customer name "Smith & Co.", the identification module 226 may match this with similar entries such as "Smith and Company" or "Smith & Company" to effectively consolidate and manage customer information utilizing fuzzy matching techniques. The identification module 226 then provides the identified sensitive information from the enterprise data 214 to the masking module 228 along with details indicating the relationship of the sensitive information.

[0053] The masking module 228 masks the sensitive information identified by the identification module 226 in the user query 210. The masking module 228 masks the sensitive information using one or more masking techniques or masking guidelines 212. The one or more masking techniques / guidelines 212 are selected based on the identified sensitive information and its correlation and context with respect to the enterprise data 214.

[0054] The masking module 228 masks the sensitive information using various techniques, such as, but not limited to, anonymizing the sensitive data, substituting contextual and correlated variables for sensitive information, and obfuscating multiple scopes, to ensure data privacy while leveraging accurate responses from multiple underlying models. For example, if the identified sensitive information in the user query 210 is the term "North America," the masking module 228 may substitute the term "first American region" to preserve the region's location in the United States without the specificity that impacts privacy. In this manner, the masking module 228 generates a masked query by masking the sensitive information in the query. The variables substituted for the sensitive information preserve contextual and relational relevance to ensure accuracy of the response generated in response to the masked query.

[0055] Upon masking the sensitive information in the user query, the prompt generation module 230 generates multiple prompts to obtain information from the multiple foundational models. The multiple prompts are generated based on the masked query generated by the masking module 228. Advantageously, the prompt generation module 230 generates multiple prompts that repeat the user query in natural language of different writing styles to verify accuracy and completeness of responses received from the multiple foundational models.

[0056] Typically, the foundation model often hallucinates and provides contradictory answers, which results in an inaccurate response and is unusable. By providing multiple prompts, the functionality of the prompt generation module 230 provides a solution to such hallucinations, since multiple foundation models ask themselves questions and the answers they provide are checked against each other. For example, if the user query is "What is the sales projection for California team," the masked query may be "What is the sales projection for first US team," and the prompt generation module 230 may generate related prompts such as "please provide sales projection for first US team," "first US team sales projection 2025 Q1," "sales projection US teams for FY25," and the like.

[0057] Upon receiving the multiple prompts, the information acquisition module 232 retrieves appropriate information from the knowledge base 204 and the multiple foundational models to generate a user response. In one embodiment, the information acquisition module 232 utilizes the multiple foundational models to generate a response to the multiple prompts. In some cases, all of the multiple prompts are provided to each of the multiple foundational models. In other cases, one or more of the multiple prompts may be provided to each of the multiple foundational models. In operation, any permutation or combination of the multiple prompts and multiple foundational models may be implemented by the information acquisition module 232.

[0058] Further, the information acquisition module 232 acquires corresponding appropriate information from the enterprise data 214 to supplement the multiple responses. The enterprise data 214 includes confidential information related to the query. Furthermore, the enterprise data 214 may include confidential information related to any enterprise, operation, service, facility, or the like. In one example, if the data processing system 114 is used by a government, the enterprise data 214 (and by extension, the confidential information) may relate to voter databases, demographic information, citizen details, and the like. In another example, if the data processing system 114 is used by a hospital, the enterprise data 214 (and by extension, the confidential information) may relate to medical records, health care information, identification information, and the like.

[0059] Since the enterprise data 214 includes hierarchies and domain terminology, when unmasking the responses to generate user responses, the information acquisition module 232 maps the domain terminology to multiple foundational models. During preprocessing, the identification module 226 trains / tunes multiple foundational models based on the multiple domain terminology. The identification module 226 may train a separate foundational model for each domain. For example, the identification module 226 may train / tune a first foundational model for the medical domain and a second foundational model for the aerospace domain. The identification module 226 may utilize one or more training methods to train / tune the foundational models. The one or more training methods include supervised fine-tuning, feedback, reinforcement learning, reinforcement learning with human feedback, and the like.

[0060] Further, the identification module 226 generates an index including a mapping between the multiple foundational models and the domain terminology / domain on which each of the multiple foundational models is trained / tuned. For the previous example, the index may include a mapping between a first foundational model and a medical domain and a mapping between a second foundational model and an aerospace domain. The index may be implemented as at least one of a vector index, a one-to-many index, a hash map index, and a one-to-one index. During response unmasking, the information acquisition module 232 maps the domain terminology to the multiple foundational models based on the index.

[0061] Further, the information retrieval module 232 receives the multiple responses and generates a common result set. The common result set refers to a set of common responses among the multiple responses received from the multiple underlying models. For example, for a query "what would be the annual revenue growth for this fiscal year?", the multiple responses received are: +7%, +5%, +2%, -2%, +1%, +5%, +7%, +1%, +5%, where positive numbers represent expected annual revenue growth and negative numbers represent decline. Further, the information retrieval module 232 may identify a common result set based on the occurrence of a particular response. For this example, the response "+5%" is selected as the common result set since it is repeated the maximum number of times. A simple example is provided to illustrate how the common result set is generated. Notably, the actual implementation may vary in complexity based on the user query 210.

[0062] The information acquisition module 232 then validates the common results with the confidential information. Once the confidential information is supplemented, the information acquisition module 232 validates the common results by determining whether the common result set is true against the confidential information. For example, if the common result set is "the first US team showed a 40% increase in sales," the information acquisition module 232 validates the common result set with the confidential information (actual sales numbers and team / region names) to ensure correctness. Advantageously, the information acquisition module 232 validates the common results with the confidential information to ensure accuracy and prevent hallucination.

[0063] In some cases, the common result set may not correlate with the sensitive information. In such cases, the information acquisition module 232 regenerates the common result set. For example, if the common result set is "the first US team showed a 40% increase in sales" but during validation with sensitive information (actual sales numbers and team / region name), the information acquisition module 232 determines that North America (sensitive information corresponding to the first US team) actually showed a 10% increase in sales, the common result set may be regenerated by selecting from the multiple responses a different response set that accurately reflects the enterprise data 214 in light of the user query 210. Alternatively, in such cases, the information acquisition module 232 may locally modify the common result set. For the above example, the information acquisition module 232 may locally modify the common result set to "the first US team showed a 10% increase in sales."

[0064] Further, the information acquisition module 232 generates a response based on the verified common result set. The information acquisition module 232 generates the response by supplementing the (previously verified) common result set with the sensitive information. Advantageously, supplementing the multiple responses with the sensitive information provides more context and ensures accuracy to the complete (supplemented) response. Further, if the information acquisition module 232 determines that the response is not correlated with the sensitive information, the information acquisition module 232 may determine another response by verifying the common result set based on the sensitive information.

[0065] Upon generating the common result set, the information acquisition module 232 generates a user response by supplementing the response with sensitive information. Considering the above example, the user response may be "the North American sales team showed a 10% increase in sales, reaching $25,000 in sales for FY24." The user response is provided to the user via the UI / UX module 206.

[0066] The persona module 234 validates the user, and the user response is provided to the user only if the validation is successful. The persona module 234 is communicatively coupled to the information retrieval module 232 and provides information regarding the user validation to the information retrieval module 232. The persona module 234 fetches role information corresponding to the user from the enterprise data 214. The persona module 234 then determines whether the user should have access to information regarding the user query based on the persona rules 216. If it is determined that the user should not have access to the user response, the persona module 234 communicates a notification to the user informing him or her that the requested information is not accessible. For example, if a developer attempts to access employment information in a colleague's HR record, the persona module 234 may send a notification that "you do not have access to this information."

[0067] The rules engine 236 is utilized to generate rules for preventing fraudulent activity in the enterprise. The rules engine 236 automatically identifies fraudulent patterns in the enterprise data 214 and generates and executes individual processor-readable masking rules to prevent fraud. During pre-processing, the identification module 226 trains one or more classification models on the fraud-related data such that the one or more classification models may identify patterns for fraudulent activity in the data. The one or more classification models may utilize any classification or pattern recognition method to identify patterns in the fraud-related data. The fraud-related data may be historical data related to fraud, real-time data related to fraud, enterprise data related to fraud from known fraud cases, generic data related to fraud, and the like. Patterns for fraudulent activity in the data may be represented as anomalies in the data.

[0068] Alternatively, the identification module 226 may train one or more classification models in real time on fraud-related data, where training may be performed periodically, such as monthly, weekly, quarterly, yearly, etc. Advantageously, training one or more classification models in real time keeps the classification models up to date with known fraud events and new fraud patterns or techniques, such that the classification models are constantly informed of new fraud techniques and can identify patterns for which one or more anti-fraud masking rules should be generated.

[0069] Since the rules engine 236 has access to the knowledge base 204, the rules engine 236 continuously processes the enterprise data 214. When the rules engine 236 identifies a fraudulent activity pattern, it generates one or more masking rules (logic) to counter (and thereby prevent) the fraudulent activity pattern. Here, the rules engine 236 utilizes one or more classification models to identify fraudulent activity patterns in the enterprise data 214. When the rules engine 236 identifies a fraudulent activity pattern, it translates the fraudulent activity patterns identified in the data into fraud-related logic. Furthermore, the fraud-related logic may be mapped with past occurrences of fraudulent activity to identify similar fraudulent activity that has occurred in the past. Such countermeasure logic obtained from similar fraudulent activity that has occurred in the past may be utilized to generate countermeasure logic for current fraudulent activity.

[0070] The rule engine 236 may utilize one or more grid search techniques to generate countermeasure logic, and thus one or more masking rules. A grid search technique refers to processing data by intelligently applying a large amount of permutations and combinations to data in a hyperparameter search space. Examples of grid search techniques include utilizing at least one of a Bayesian optimization technique, a stochastic optimization technique, and a tree-structured Parzen estimator (TPE) technique. In some cases, the rule engine 236 may utilize a trained foundation model that has been previously trained for logic generation to generate countermeasure logic. Here, generating countermeasure logic includes generating rule conditions and actions by the trained foundation model. The rule engine 236 utilizes the rule conditions and actions to generate one or more masking rules.

[0071] The rules engine 236 then executes one or more masking rules in the data processing system 114 to prevent fraudulent activity. For example, if a fraudulent pattern is found in an international banking transaction, the rules engine 236 may generate countermeasure logic that flags future international banking transactions (after the fraudulent pattern is found in the international banking transaction). In this regard, the rules engine 236 may execute a masking rule that requires a user to provide additional verification of identity to transfer money in and out of the country. Advantageously, the timely detection of fraudulent activity and the automatic generation and execution of masking rules ensures that future fraudulent activity is prevented and that fraudsters cannot exploit the data processing system 114.

[0072] The data processing system 114 may be utilized to mask data utilizing underlying models in at least one domain including revenue and probability forecasting, medical claims management, medical fraud detection, talent acquisition and management, banking, financial portfolio management, intellectual property management, government operations, social services management, defense services management, and the like.

[0073] FIG. 3 shows a block diagram 300 illustrating a process flow for masking sensitive data from an underlying model in accordance with an implementation of the present disclosure.

[0074] Block diagram 300 illustrates a query 302 being provided to a secure environment 304. Query 302 refers to a user query provided by a user of the enterprise. Secure environment 304 refers to a controlled computational infrastructure configured to protect sensitive information from unauthorized access or disclosure, particularly in interactions involving third-party tools such as foundation model 312. In this manner, queries or prompts provided to foundation model 312 are masked for sensitive information, and further, responses or model outputs received from foundation model 312 do not include any sensitive information. Sensitive information is included in the model output to generate user response 310 before being provided to the user.

[0075] 3, a query 302 is input to a secure environment 304 via a data processing system 114. The processing engine 208 / 306 and the knowledge base 204 / 308 are communicatively coupled to each other and are contained within the secure environment 304. The secure environment 304 may refer to an internal environment of an enterprise. Advantageously, sensitive data regarding the enterprise is kept within the secure environment 304 and masked from the foundation model 312.

[0076] Sensitive information is masked by the processing engine 306. The processing engine 306 masks sensitive information by replacing it with variables that preserve the contextual and relational associations, so that the foundation model 312 provides accurate responses and prevents hallucination in the foundation model 312 by sharing the contextual and relational associations between the masked / guarded sensitive data.

[0077] In one example, if the query 302 is "What market units are projected to have negative growth next quarter?", the query is provided to the secure environment 304. In the secure environment 304, the processing engine 306 identifies sensitive information by pre-processing the query. The sensitive data in the query may be "negative growth," "market units," and "quarter / year information." The processing engine 306 then masks the sensitive information to generate multiple responses, which are sent to the foundation model 312. The foundation model 312 generates a natural language response to the query 302 and shares it with the processing engine 306. The processing engine 306 further extracts the appropriate enterprise data 214 from the knowledge base 308 and supplements it in the response to generate the user response 310. The appropriate enterprise data 214 may be related to the sensitive data. Thus, a user response 310 might be, "The Northeast, India, and ANZ market units are projected to have negative growth next quarter."

[0078] FIG. 4 illustrates a block diagram 400 illustrating an example process flow for generating a user response from a user query in accordance with an implementation of the present disclosure.

[0079] 4, the data processing system 114 receives a query 402 / 302 / 210 from a user and processes it to generate a response(s) 420 / 310 / 218. The user may be associated with the enterprise as an employee, owner, merchant, vendor, and the like. Furthermore, the query 402 may be for a request for information about the enterprise. Furthermore, the query may be constructed (i.e., expressed) in any natural language recognized by the data processing system 114.

[0080] The data processing system 114 then generates a number of prompts 404a, 404b, 404c, ... 404n based on the query 402 from the user. The number of prompts 404a, 404b, 404c, ... 404n are generated by masking the sensitive information in the query 402. In operation, the data processing system 114 replaces the sensitive information in the query 402 with variables based on one or more masking criteria. A variable refers to a symbol or placeholder that represents a value or an entity in at least one of mathematics, science, programming, and / or computing domains. Furthermore, the variable represents the contextual and relational relevance of the sensitive information. In this way, the foundation model 406a, 406b, 406c, ... 406n accurately understands the relational dependencies in the enterprise data 214, which leads to accurate responses that reduce hallucination.

[0081] During masking, the data processing system 114 identifies technical terms that belong to the inherent structure of the data and replaces them with explicit variable indicators that are consistent with the enterprise data 214. In this regard, the data processing system 114 also takes into account spelling errors and alternative names. In one example, if the query 402 is "For Nrth America what are the most dilutive market units to our revenue growth at fiscal year 2023?", the masked query may be expressed as "For MarketX, what are the most dilutive market units to our revenue growth at fiscal year ValueX?". In this way, multiple foundation models that lack knowledge of the hierarchical details and structures in the enterprise data 214 can correctly interpret the relationship of the sensitive information to the query 402. For example, sensitive information regarding the client group "South CMT" may be understood by multiple foundation models as a Client Group segment.

[0082] The plurality of prompts 404a, 404b, 404c, ... 404n refer to specific input queries / statements that are provided to the plurality of underlying models to generate responses 408a, 408b, 408c, ... 408n. The plurality of prompts 404a, 404b, 404c, ... 404n act as instructions that guide the models in processing and producing appropriate information based on the nature and structure of the prompt itself. In operation, the plurality of prompts 404a, 404b, 404c, ... 404n include multiple representations of the query 402.

[0083] When multiple foundational models 406a, 406b, 406c, ... 406n are prompted with different versions of the same query, the likelihood of hallucination is dramatically reduced. In one example, if the query 402 is "Which market has the highest revenue projection?", the first prompt 404a may be "The highest projected revenue was made for which market?" and the second prompt 404b may be "In which market would I expect the highest revenue?". In another example, if query 402 is "Which client groups are responsible for driving the revenue growth of our three most impactful services in the North American market?" a first prompt may include query 402 with an imprecise filter, a second prompt may include query 402 with an exact filter, and a third prompt may include query 402 with an imprecise aggregation.

[0084] In response to input of the plurality of prompts 404a, 404b, 404c, ... 404n, the plurality of foundation models 406a, 406b, 406c, ... 406n transmit the plurality of responses 408a, 408b, 408c, ... 408n to the data processing system 114. The data processing system 114 then generates a common result set based on the plurality of responses 408a, 408b, 408c, ... 408n. The common result set is a set of responses that are most common among the plurality of responses 408a, 408b, 408c, ... 408n received from the plurality of foundation models 406a, 406b, 406c, ... 406n. In some cases, the common result set may also be validated using the query 402.

[0085] Based on the common result set, enterprise information 410a, 410b, 410c, ... 410n for each of the multiple responses 408a, 408b, 408c, ... 408n of the common result set is extracted from the knowledge base 204 / 308 associated with the data processing system 114. The enterprise information may include confidential information. In one example, if the query 402 is related to revenue growth in various markets and a first prompt includes a query 402 with an imprecise filter, the first information may be related to revenue growth in each region of “Midwest RES, Northeast H&PS”, if a second prompt includes a precise query 402, the second information may be related to revenue growth in each region of “South RES, Midwest PRD, Canada RES”, and if a third prompt includes a query 402 with an imprecise aggregation, the third information may be related to revenue growth in each region of “Canada H&PS, Canada RES, South RES”.

[0086] The data processing system 114 generates a response 412 by validating the common result set with the sensitive information and the query 402. In this regard, the data processing system 114 executes each of the multiple responses 408a, 408b, 408c, ... 408n in the virtual environment by appending (i.e., including) the sensitive information within the responses 408a, 408b, 408c, ... 408n. In this manner, the responses 408a, 408b, 408c, ... 408n are checked against the query 402 and the enterprise information 410a, 410b, 410c, ... 410n to ensure accuracy.

[0087] A virtual environment refers to a simulated / software-based environment that mimics real-world functionality. A virtual environment allows for testing and deployment of actions that may occur in the real world without the need to implement them in the real world. This advantageously allows for increased accuracy without compromising on increased computational resource involvement. As this is performed in a virtual environment, learnings from the validation can be utilized to select and improve responses 412. In this way, responses 408a, 408b, 408c, ... 408n are performed by validating responses 412 against a data repository (knowledge base) and queries 402.

[0088] In some cases, each of the responses 408a, 408b, 408c, ... 408n is provided with a score based on its individual accuracy when performed. A response 412 is selected based on which response of the multiple responses 408a, 408b, 408c, ... 408n received the highest score relative to the other responses.

[0089] The data processing system 114 then generates a user response 414 by supplementing the response 412 with the sensitive information. The term "supplementing" refers to replacing masked or otherwise concealed data with the original sensitive data, or to including sensitive information in the response 412, or to adjusting the response 412 in light of the sensitive information. The data processing system 114 provides the user response 414 to the user based on the validation.

[0090] Before providing the user response 414 to the user, the data processing system 114 validates the user based on one or more predefined criteria. The predefined criteria or criteria refer to specific conditions or requirements set by the enterprise to validate the user's eligibility or authority. The predefined criteria or criteria relate to the user's persona. A persona refers to a profile or characterization of a user based on attributes such as role, preferences, behavior, or other distinctive elements. Persona information includes role, location, interests, ethnicity, team, group, and the like.

[0091] The data processing system 114 is communicatively coupled to the knowledge base 204 / 308 so that it has constant access to persona or role-based information for each user associated with the enterprise (and by extension, the data processing system 114). In this regard, the data processing system 114 validates queries 402 and user responses 414 against one or more persona rules. A persona rule refers to a guideline or standard established based on a user's role or profile within the enterprise to ensure that queries 402 and user responses 414 conform to specified standards or permissions. For example, a persona rule may dictate which departments or hierarchical roles can access certain information about the enterprise, such as employment records.

[0092] In this regard, the data processing system 114 shares the user response 414 with the user if the user is authorized to access the appropriate information related to the query 402 and the user response 422. Alternatively, if the user is not authorized to access the information, the data processing system 114 shares a notification with the user informing them that they are unable to access the information. The level of promotion of sensitive data in the user response 414 may depend on the persona information, such that the user's role / privileges are taken into account by creating the user response 414 from the selected response 412. Thus, different user responses corresponding to multiple personas may be generated from the same selected response 412.

[0093] Additionally, the user response 414 may be provided to the user based on one or more masking rules, which refer to specific rules that, when executed, add another level of security by masking sensitive data to protect it from unauthorized activity. In this regard, the data processing system 114 identifies patterns in the enterprise data 214 associated with the data processing system 114. A pattern refers to a recognizable sequence or behavior that is indicative of unauthorized activity aimed at illicitly obtaining sensitive information. These patterns typically involve repeated methods or means used to exploit vulnerabilities in security systems or protocols, allowing unauthorized access to sensitive data.

[0094] The data processing system 114 generates one or more masking rules based on the identified patterns. For example, if it is identified that a fraudster is attempting to log into and empty a bank customer's account via an international banking transaction, a masking rule may be implemented to verify the user's identity based on biometric information or security questions. Advantageously, the data processing system 114 automatically executes one or more masking rules to ensure improved security. As a result, this saves costs and time, since the process of generating and executing masking rules is a very time-consuming process that typically requires manual intervention by a domain-specific expert.

[0095] FIG. 5 illustrates a detailed process flow 500 for masking sensitive data during information capture in accordance with an implementation of the present disclosure.

[0096] As shown in Figure 5, the process flow 500 of masking sensitive data during information capture is implemented utilizing two or more domains simultaneously. These domains, namely, secure enterprise domain 502 (also referred to above as secure environment 304) and external domain 504, enable the solution presented in this disclosure to function properly. By utilizing two or more domains in accordance with the implementation of this disclosure, sensitive enterprise data is safely stored within the secure enterprise domain 502, leading to enhanced data protection measures and ensuring compliance with strict security protocols.

[0097] The data processing system 114 performs the iterative steps with respect to the secure enterprise domain 502, and the foundational models 312 / 408a-408n perform the iterative steps with respect to the external domain. In some cases, the foundational models 312 / 408a-408n are implemented as generative artificially intelligent (generative AI or Gen AI) models. In such cases, the foundational models 312 / 408a-408n may employ prompt engineering techniques for accuracy. The term "prompt engineering" refers to the process of designing / crafting effective prompts / queries that elicit desired responses from an artificial intelligence system. This involves structuring prompts to optimize response comprehension and accuracy, often tailored to a specific task or application. The prompt engineering technique may be implemented as a few-shot prompting technique.

[0098] 5 also depicts sub-processes within the process flow 500, where one or more steps of the process flow 500 are part of at least one sub-process that are repeated as an input generalization sub-process, a data model design and implementation sub-process, an information acquisition supervision sub-process, a query post-processing sub-process, and a natural language response generation sub-process.

[0099] At step 506, the user provides a query to the data processing system 114 via the UI / UX module 206. In some cases, the user may provide the query via a chatbot hosted by the UI / UX module 206. As a result, under the input generalization sub-process at step 508, the data processing system 114 masks the sensitive information in the query and prepares multiple prompts for multiple underlying models. During masking, numeric values ​​may be supplemented with context variables, text values ​​may be replaced with generalized terms, etc., and the like, to properly mask the sensitive information.

[0100] The plurality of foundational models receive a plurality of prompts from the data processing system 114. At step 510, under an information acquisition supervision sub-process, at least one foundational model of the plurality of foundational models generates a plurality of instances of an information acquisition query to acquire appropriate enterprise data (or enterprise information) from the knowledge base 204. The knowledge base 204 may be implemented as at least one of a vector database, a structured query language (SQL) database, a not only structured query language (no-SQL) database, and the like. The appropriate enterprise data may further be masked to protect sensitive information present in the enterprise data 214.

[0101] Further, under the data model design and implementation sub-process, the information retrieval query is then provided to the data processing system 114, which further utilizes the information retrieval query to extract enterprise data 214 from the knowledge base 204. At step 512, the data processing system 114 unmasks the information retrieval query to extract the enterprise data 214 from the knowledge base 204. In some cases, the extracted enterprise data 214 is stored locally in the knowledge base 204 for further processing.

[0102] In step 514, under the post-query processing sub-process, the data processing system 114 masks the sensitive information of the query to generate multiple prompts for the query. These prompts are written in a distinctive style and manner based on the query (the same query) to obtain multiple data points from multiple foundation models. This advantageously helps ensure accuracy of the responses. The data processing system 114 provides the multiple prompts to multiple foundation models.

[0103] Thereafter, under the natural language response generation sub-process, the foundational models generate responses in response to the prompts at step 516. The responses are generated based on, among other things, the query, the information retrieval query, and the masked enterprise data 214. The foundational models provide the responses to the data processing system 114.

[0104] In step 518, the data processing system 114 unmasks the response and shares it with the user via the UI / UX module 206.

[0105] Of course, the data processing system 114 will be configured to have multiple conversations with the user, often across various dialogs, and thus the process depicted in Figure 5 will continually repeat based on interactions with the user.

[0106] 6 is a flow diagram illustrating an example method 600 according to an implementation of the present disclosure. In some implementations, the method 600 may be performed in a data processing system 114, such as described in connection with FIG.

[0107] At step 602, a query is received from a user. The user may relate to a user of the enterprise. In some cases, the user may be employed by the enterprise. In other cases, the user may be a merchant, customer, vendor, and the like associated with the enterprise. The query relates to a request for information. The requested information may relate to the enterprise. Additionally, the query may include or relate to confidential information related to the enterprise.

[0108] At step 604, multiple prompts are generated based on a query from a user. The multiple prompts are generated to fetch appropriate responses from multiple underlying models. Furthermore, the multiple prompts are generated by masking sensitive information in the query. The sensitive information may relate to enterprise data 214 that, if disclosed or leaked, may harm an individual, organization, or system, leading to privacy violation, financial loss, or reputational damage. The data processing system 114 may generate multiple prompts corresponding to the query, each of which is variably configured. This process ensures comprehensive coverage of possible responses tailored to various contextual interpretations of the query.

[0109] At step 606, responses are received from the foundational models. The responses are received in response to the input of the prompts. The foundational models may use deep learning methods, such as transformer architectures, to generate the responses by processing the resulting prompts. The foundational models may utilize pre-trained weights and context embeddings to adapt their output based on semantic nuances embedded within each prompt to generate diverse, contextually appropriate responses.

[0110] At step 606, a common result set based on the multiple responses is generated, which involves aggregating the multiple responses and validating their mutual consistency and relevance to ensure that the final set comprehensively addresses diverse contextual interpretations of the query.

[0111] At step 610, a response is generated by validating the common result set with the sensitive information and the query. This validation involves augmenting the response with the sensitive information and validating it against the enterprise data 214 and the query (original). If inconsistencies or inaccuracies are detected, the response is regenerated to maintain consistency and accuracy.

[0112] At step 612, a user response is generated by supplementing the response with the sensitive information. In some cases, the sensitive information is added directly into the response. In other cases, the response may be adjusted to match the sensitive information. Advantageously, this ensures comprehensive and contextually appropriate interaction with the user while maintaining data integrity and confidentiality.

[0113] A user response is provided to the user in response to the query at step 614. Advantageously, method 600 enables secure, accurate, efficient and reliable information retrieval, thereby achieving increased accuracy.

[0114] 7 illustrates a computer system 700 that may be used to implement the data processing system 114. More specifically, computing machines such as desktops, laptops, smartphones, tablets, and wearables that may be used to process conversational interactions in the data processing system 114 may have the structure of the computer system 700. The computer system 700 may include additional components not shown, and some of the process components described may be removed and / or modified. In another example, the computer system 700 may be deployed on an external cloud platform such as the cloud, an in-house corporate cloud computing cluster, or an organization's computing resources, and / or the like.

[0115] The computer system 700 includes a processor(s) 702, such as a central processing unit, an ASIC, or another type of processing circuit, input / output devices 704, such as a display, mouse keyboard, a network interface 706, such as a local area network (LAN), a wireless 802.11x LAN, a 3G or 4G mobile WAN, or a WiMax WAN, and a processor-readable medium 708. Each of these components may be operatively coupled to a bus 710. The computer-readable medium 708 may be any suitable medium that participates in providing instructions to the processor(s) 702 for execution. For example, the computer-readable medium 708 may be a non-transitory or non-volatile medium, such as a magnetic disk or solid-state non-volatile memory, or a volatile medium, such as a RAM. The instructions or modules stored on the computer-readable medium 708 may include machine-readable instructions 712 that are executed by the processor(s) 702 to cause the processor(s) 702 to perform the methods and functions of the data processing system 114.

[0116] Data processing system 114 may be implemented as software stored on a non-transitory processor-readable medium and executed by processor(s) 702. For example, computer-readable medium 708 may store an operating system 714, such as MAC OS, MS WINDOWS, UNIX, or LINUX, and code for data processing system 114. Operating system 714 may be multi-user, multi-processing, multi-tasking, multi-threading, real-time, and the like. For example, during run-time, operating system 714 runs and code for data processing system 114 is executed by processor(s) 702.

[0117] The computer system 700 may also include data storage 716, which may include non-volatile data storage. The data storage 716 stores any data used or generated by the data processing system 114.

[0118] The network interface 706 connects the computer system 700 to internal systems, for example, via a LAN. The network interface 706 may further connect the computer system 700 to the Internet. For example, the computer system 700 may connect to a web browser and other external applications and systems via the network interface 706.

[0119] Some of the variations have been described and illustrated herein with examples. The terms, descriptions, and figures used herein are set forth by way of illustration only and are not intended as limitations. Numerous variations are possible within the spirit and scope of the subject matter as intended to be defined by the appended claims and equivalents thereof.

[0120] Implementations of the present disclosure provide multiple technical improvements and address shortcomings of conventional foundational models. For example, implementations of the present disclosure provide accurate retrieval of information using foundational models to respond to user queries. Such accuracy translates directly to improved performance of the foundational models, while ensuring data privacy by masking sensitive information from the foundational models. This accuracy translates to efficient, actionable, and correct responses from the foundational models, dramatically reducing the likelihood of hallucination.

[0121] Furthermore, the present invention increases the accuracy of responses generated by the underlying model by intelligently masking sensitive information while preserving the contextual and relational nuances in the data. This approach eliminates the need for in-house domain experts to manually monitor security measures, thus optimizing computational efficiency. Preserving the relational context of the masked data not only ensures more accurate responses, but also reduces the occurrence of misleading information capture, thereby mitigating the risks associated with data inaccuracies, such as hallucination. Furthermore, intelligently protecting sensitive data defends enterprises against potential fraudulent activities and enhances data security and compliance with regulatory standards.

[0122] The implementations and all functional operations described herein may be realized in digital electronic circuitry, or in computer software, firmware, or hardware, including the structures disclosed herein and their structural equivalents, or in a combination of one or more of them. The implementations may be realized as one or more computer program products (i.e., one or more modules of computer program instructions encoded on a computer-readable medium for execution by or control of the operation of a data processing device). The computer-readable medium may be a machine-readable storage device, a machine-readable storage substrate, a memory device, an article of matter that provides a machine-readable propagated signal, or a combination of one or more of them. The term "computing system" encompasses all apparatus, devices, and machines that process data, including, by way of example, a programmable processor, a computer, or multiple processors or computers. In addition to hardware, an apparatus may include code (e.g., code constituting a processor firmware, a protocol stack, a database management system, an operating system, or any suitable combination of one or more of them) that creates an execution environment for the subject computer program. A propagated signal is an artificially generated signal (eg, a machine-generated electrical, optical, or electromagnetic signal) that is generated to encode information that is transmitted to an appropriate receiver device.

[0123] A computer program (also known as a program, software, software application, script, or code) may be written in any suitable form of programming language, including compiled or interpreted languages, and may be deployed in any suitable form, including as a stand-alone program or as a module, component, subroutine, or other unit suitable for use in a computing environment. A computer program does not necessarily correspond to a file in a file system. A program may be stored as part of a file that holds other programs or data (e.g., one or more scripts stored in a markup language document), in a single file dedicated to the program in question, or in multiple associated files (e.g., multiple files that store one or more modules, subprograms, or portions of code). A computer program may be deployed to run on one computer or on multiple computers that are located at one site or distributed across multiple sites and interconnected by a communication network.

[0124] The processes and logic flows described herein may be performed by one or more programmable processors executing one or more computer programs that perform functions by operating on input data and generating output. The processes and logic flows may also be performed by, and apparatus may be implemented as, special purpose logic circuitry, such as a field programmable gate array (FPGA) or application specific integrated circuit (ASIC).

[0125] Processors suitable for executing computer programs include, by way of example, both general purpose and special purpose microprocessors, and any one or more processors of any suitable type of digital computer. Typically, a processor receives instructions and data from a read-only memory or a random access memory, or both. Components of a computer may include a processor for executing instructions and one or more memory devices for storing instructions and data. Typically, a computer also includes one or more mass storage devices (e.g., magnetic, magneto-optical, or optical disks) for storing data, or is operatively coupled to receive and / or transfer data thereto. However, a computer need not have such devices. Additionally, a computer may be embedded in another device (e.g., a mobile phone, a personal digital assistant (PDA), a mobile audio player, a Global Positioning System (GPS) receiver). Computer-readable media suitable for storing computer program instructions and data include all forms of non-volatile memory, media, and memory devices, including, by way of example, semiconductor memory devices (e.g., EPROM, EEPROM, and flash memory devices), magnetic disks (e.g., internal hard disks or removable disks), magneto-optical disks, and CD-ROM and DVD-ROM disks. The processor and memory may be supplemented by, or incorporated in, special purpose logic circuitry.

[0126] To provide interaction with a user, each implementation may be realized on a computer having a display device (e.g., a cathode ray tube (CRT), liquid crystal display (LCD) monitor) that displays information to the user, as well as a keyboard and pointing device (e.g., a mouse, trackball, touchpad) by which the user can provide input to the computer. Other types of devices may be used to provide interaction with the user as well. For example, feedback provided to the user may be any suitable form of sensory feedback (e.g., visual feedback, auditory feedback, haptic feedback), and input from the user may be received in any suitable form, including acoustic, speech, or tactile input.

[0127] The implementation may be realized in a computing system including a back-end component (e.g., as a data server), a computing system including a middleware component (e.g., an application server), and / or a computing system including a front-end component (e.g., a client computer having a graphical user interface or a web browser through which a user can interact with the implementation), or any suitable combination of one or more such back-end, middleware, or front-end components. The components of the system may be interconnected by any suitable form or medium of digital data communication (e.g., a communications network). Examples of communications networks include local area networks ("LANs") and wide area networks ("WANs"), such as the Internet.

[0128] A computing system may include clients and servers. Clients and servers are generally remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other.

[0129] Although this specification contains many details, these should not be construed as limitations on the scope of the disclosure or the claims, but rather as describing features specific to a particular implementation. Certain features described in this specification in the context of separate implementations can also be implemented in combination in a single implementation. Conversely, various features described in the context of a single implementation can also be implemented separately in multiple implementations, or in any suitable combination of components. Furthermore, although features may be described above as operating in a particular combination, and may even be initially claimed as such, in some cases one or more features of a claimed combination can be deleted from the combination, and the claimed combination may be directed to a component of the combination or a variation of the component of the combination.

[0130] Similarly, although operations are shown in a particular order in the figures, this should not be understood as requiring that the operations be performed in the particular order or sequence shown, or that all of the operations shown be performed, to achieve desirable results. In certain situations, multitasking and parallel processing may be advantageous. Furthermore, the separation of various system components in the implementations described above should not be understood as requiring such separation in all implementations, and it will be understood that the program components and systems described may generally be integrated into a single software product or packaged into multiple software products.

[0131] Several implementations have been described. However, it will be appreciated that various modifications may be made without departing from the spirit and scope of the present disclosure. For example, various forms of the flows shown above may be used, in which steps are rearranged, added, or removed. Accordingly, other implementations are within the scope of the following claims.

Claims

1. at least one hardware processor; The following: receiving a query from a user regarding a request for information; generating a plurality of prompts based on the query from the user, the plurality of prompts being generated by masking sensitive information in the query; receiving a plurality of responses from a plurality of foundation models in response to inputting the plurality of prompts; generating a common result set based on the plurality of responses; generating a response by verifying the common result set with the sensitive information and the query; generating a user response by supplementing the response with the sensitive information; and Providing the user response to the query to the user. at least one non-transitory processor-readable medium storing instructions for execution by the at least one processor for A data processing system comprising:

2. The data processing system of claim 1 , wherein the query is constructed in natural language.

3. The instructions for providing the user response may include causing the at least one processor to: validating the user based on one or more predetermined criteria related to the user's persona; providing the user response to the user based on the verification in response to the query; and 2. The data processing system of claim 1, further comprising:

4. The instructions for masking sensitive data in the query may include instructions to the at least one processor: The data processing system of claim 1 , further comprising: replacing the sensitive information in the query with variables based on one or more masking criteria, the variables expressing contextual and relational relevance for the sensitive information.

5. 2. The data processing system of claim 1, wherein the at least one non-transitory processor-readable medium includes a data repository associated with the data processing system, the information in the data repository being represented across multiple hierarchies and incorporating domain terminology.

6. The instructions for masking the sensitive data in the user response may include instructions to the at least one processor: The data processing system of claim 5 , further comprising: a data processing unit configured to map said domain terminology to said plurality of underlying models.

7. The instructions for generating the common result set may include causing the at least one processor to: executing each of the responses in a virtual environment by appending the sensitive information thereto, the responses being executed by validating the responses against the data repository and the queries; selecting the response based on a result of the validation; and 2. The data processing system of claim 1, further comprising:

8. 2. The data processing system of claim 1, wherein the instructions for providing the user response are based on one or more masking rules, the one or more masking rules causing the at least one processor to identify and mask the sensitive information based on the one or more masking rules, the one or more masking rules including rules for masking the sensitive data.

9. The instructions for providing the user response based on the one or more masking rules may further include instructions for the at least one processor to: identifying patterns in data associated with the data processing system, the patterns being related to fraudulent activity; generating the one or more masking rules based on the identified patterns; and executing processor-readable instructions relating to the one or more masking rules; 9. The data processing system of claim 8,

10. receiving, by a processor, a query from a user regarding a request for information; generating, by the processor, a number of prompts based on the query from the user, the number of prompts being generated by masking sensitive information in the query; receiving, by the processor, a plurality of responses from a plurality of foundation models in response to the plurality of prompts; generating, by the processor, a common result set based on the plurality of responses; generating, by the processor, a response by verifying the common result set with the sensitive information and the query; generating, by said processor, a user response by supplementing said response with said sensitive information; providing, by the processor, the user response to the user in response to the query; A processor-executable method comprising:

11. The processor executable method of claim 10 , wherein the query is in natural language.

12. The instructions for providing the user response include: validating, by the processor, the user based on one or more predetermined criteria related to the user's persona; providing, by the processor, the user response to the user in response to the query based on the verification; 11. The processor executable method of claim 10, comprising:

13. The instructions for masking sensitive data in the query include:

11. The processor executable method of claim 10, further comprising replacing the sensitive information in the query with variables based on one or more masking criteria, the variables expressing contextual and relational relevance for the sensitive information.

14. Information in a data repository associated with the processor is represented across multiple hierarchies, incorporating domain terminology, and the instructions for masking the sensitive data in the user responses include: mapping, by the processor, the domain vocabulary to the plurality of underlying models; 11. The processor executable method of claim 10, comprising:

15. The instructions for generating the common result set include: executing, by the processor, each of the responses in a virtual environment by appending the sensitive information thereto, the responses being executed by validating the responses against the data repository and the queries; selecting, by the processor, the response based on a result of the verification; 11. The processor executable method of claim 10, comprising:

16. 11. The processor executable method of claim 10, wherein the instructions for providing the user response are based on one or more masking rules, the one or more masking rules causing at least one of the processors to identify and mask the sensitive information based on the one or more masking rules, the one or more masking rules including rules for masking the sensitive data.

17. The instructions for providing the user response based on the one or more masking rules include: identifying, by the processor, a pattern in data associated with the data processing system, the pattern being related to fraudulent activity; generating, by the processor, the one or more masking rules based on the identified patterns; executing, by the processor, processor-readable instructions relating to the one or more masking rules; 11. The processor executable method of claim 10, comprising:

18. A non-transitory processor-readable storage medium containing machine-readable instructions, the instructions configured to cause a processor to: receiving a query from a user regarding a request for information; generating a plurality of prompts based on the query from the user, the plurality of prompts being generated by masking sensitive information in the query; receiving a plurality of responses from a plurality of foundation models in response to inputting the plurality of prompts; generating a common result set based on the plurality of responses; generating a response by verifying the common result set with the sensitive information and the query; generating a user response by supplementing the response with the sensitive information; providing said user response to said user in response to said query; A non-transitory processor-readable storage medium that causes

19. The instructions for providing the user response may include causing at least one of the processors to: validating the user based on one or more predetermined criteria related to the user's persona; providing the user response to the user based on the verification in response to the query; and 20. The non-transitory processor-readable storage medium of claim 18,

20. The instructions for masking sensitive data in the query include instructions for at least one of the processors:

20. The non-transitory processor-readable storage medium of claim 18, further comprising: replacing the sensitive information in the query with variables based on one or more masking criteria, the variables expressing contextual and relational relevance for the sensitive information.

Images