Method, computing device, and computer program for providing artificial intelligence response service through policy violation determination based on user intention

Abstract

Provided are a method, computing device, and computer program for providing an artificial intelligence response service through policy violation determination based on a user intention. According to various embodiments of the present disclosure, provided is a method by which a computing device provides an artificial intelligence response service through policy violation determination based on a user intention, the method comprising the steps of: acquiring a user input including one or more queries from a user; determining a user's intention for the one or more queries on the basis of the acquired user input; determining, on the basis of the determined intention, whether a policy is violated; and providing, to the user, a response corresponding to the acquired user input on the basis of whether the policy is violated.

Description

Method for providing artificial intelligence response service through user intent-based policy violation judgment, computing device and computer program

[0001] Various embodiments of the present disclosure relate to a method for providing an artificial intelligence response service through user intent-based policy violation determination, a computing device, and a computer program.

[0002] With the recent advancements in large-scale language models and multimodal AI technology, AI services that generate natural language responses by taking text, images, and voice as input are being applied across various industrial sectors.

[0003] While such generative AI services can provide a high level of response quality to user queries, they also carry the potential to generate inappropriate or policy-violating responses; therefore, various forms of safety control technologies have traditionally been proposed to ensure the safety of AI services.

[0004] For example, some conventional technologies embed policy violation judgment logic into the AI ​​model itself to restrict responses to specific types of queries. This approach relies on internal model judgment and rejects or restricts responses based on policies predefined during the model training phase.

[0005] However, this model-internal safety control method has limitations in that the judgment criteria are not clearly exposed externally, making it difficult to determine the basis on which a policy violation was determined. Additionally, there are inefficiencies in terms of operation and maintenance because the model must be retrained or fine-tuned to reflect changes when policy content is modified or new violation cases occur.

[0006] Meanwhile, training-free multimodal safety technologies have also been proposed. These technologies detect harmful content by applying predefined rules or patterns to input data and have a relatively simple structure. However, because these methods often rely on simple keyword or format-based detection and fail to consider complex contexts or the potential intent of user queries, there is a possibility of false positives or misses.

[0007] Furthermore, some conventional technologies apply classification guard models using LLM or VLM to classify input or output content into specific categories and block or allow responses based on the classification results. While this classification approach can provide a certain level of flexibility, it has the problem that it is difficult to adequately reflect the detailed context of the policy or various exceptional situations when the classification criteria are fixed.

[0008] In addition, methods utilizing Content Safety APIs provided in a cloud environment to determine the harmfulness of text or images are also being employed. While this approach has the advantage of being easy to implement as it relies on external services to determine policy violations, it has limitations, such as the difficulty in finely reflecting internal organizational policies or domain-specific criteria, and the fact that the judgment process is dependent on external systems.

[0009] Conventional technologies of this nature tend to ensure safety primarily based on the superficial characteristics of the input data itself or single-stage classification results, and have limitations in determining policy violations by comprehensively considering the context or potential intent of user queries. Furthermore, structures that determine policy violations based on consistent criteria across input and output and reflect this in response control have not been sufficiently presented.

[0010] The aforementioned background technology is one that the inventor possessed or acquired in the process of deriving the content of the present disclosure, and it cannot be considered as prior art disclosed to the general public prior to the filing of this application.

[0011] The problem that the present disclosure aims to solve is to provide a method for providing an AI response service through user intent-based policy violation judgment, a computing device, and a computer program, for the purpose of resolving the aforementioned conventional problems, which provides an AI-based response to a user's query, and effectively prevents inappropriate or dangerous information from being provided to the user by blocking the generation of the AI ​​response to the query in advance when the user's intent violates a policy, thereby fundamentally improving the safety and reliability of the AI ​​service.

[0012] The problems that this disclosure aims to solve are not limited to those mentioned above, and other unmentioned problems will be clearly understood by a person skilled in the art from the description below.

[0013] The present disclosure may be implemented in various ways, including a method, an apparatus (system), or a computer program stored on a readable storage medium.

[0014] A method for providing an artificial intelligence response service through a user intent-based policy violation determination according to an embodiment of the present disclosure for solving the above-described problem may include, in a method performed by a computing device, the steps of obtaining a user input including one or more queries from a user; determining the user's intent regarding the one or more queries based on the obtained user input; determining whether there is a policy violation based on the determined intent; and providing a response corresponding to the obtained user input to the user based on the determined policy violation.

[0015] In various embodiments, the acquired user input includes content related to one or more queries, and the step of determining the user's intent may include the step of converting the content into text and the step of determining the user's intent based on the converted text and the one or more queries.

[0016] In various embodiments, the step of converting the content into text may include, when the content is an image, the step of converting the image into text by performing image captioning on the image through a vision natural language processing-based image captioning model.

[0017] In various embodiments, the step of converting the content into text may include, when the acquired user input includes two or more different contents, the step of generating two or more texts corresponding to each of the two or more contents by individually converting each of the two or more contents.

[0018] In various embodiments, the step of determining the user's intent based on the converted text and the one or more queries may include the step of generating a first prompt containing the converted text, the one or more queries, and a command instructing the determination of intent, and the step of deriving a result of determining the user's intent regarding the one or more queries as result data by inputting the generated first prompt into a large-scale language model.

[0019] In various embodiments, the step of determining whether there is a policy violation may include the step of generating a second prompt containing the converted text, the one or more queries, the determined user's intent, and a command instructing the determination of whether there is a policy violation, and the step of deriving a result of determining whether there is a policy violation regarding the determined user's intent as result data by inputting the generated second prompt into a large-scale language model.

[0020] In various embodiments, the step of generating the second prompt may include selecting a policy violation case corresponding to the determined user's intent from among a plurality of policy violation cases stored in a database, and adding the selected policy violation case to the generated second prompt.

[0021] In various embodiments, the step of selecting the policy violation case may include: generating a first vector by vectorizing the determined user's intent; generating a plurality of second vectors by vectorizing each of the plurality of policy violation cases stored in the database; calculating a similarity between the generated first vector and the generated plurality of second vectors; and selecting a policy violation case among the plurality of policy violation cases stored in the database that corresponds to a second vector whose calculated similarity is greater than or equal to a threshold value.

[0022] In various embodiments, the step of providing the response may include, when it is determined that there is no policy violation based on the determined user's intent, generating an answer to one or more queries through a large-scale language model, and providing the generated answer to the user as a response corresponding to the acquired user input.

[0023] In various embodiments, the step of providing the generated answer may include evaluating the validity of the generated answer based on at least one of grounds, tone, profanity, and copyright, and providing the generated answer to the user only when the generated answer is determined to be valid.

[0024] In various embodiments, the step of providing the response may include, if it is determined that there is a policy violation based on the determined user's intent, blocking the generation of an answer to one or more queries through a large-scale language model and providing a message to the user notifying them of the policy violation.

[0025] In various embodiments, the method further comprises the steps of identifying sensitive information from at least one of the acquired user input and the response generated in response to the acquired user input, and preprocessing the identified sensitive information, wherein the preprocessing may be removing the identified sensitive information or masking the identified sensitive information.

[0026] In various embodiments, the step of identifying sensitive information may include: dividing at least one of the acquired user input and the response generated in response to the acquired user input into a plurality of text chunks in the form of sentences or contexts; calculating a similarity between the text included in a sensitive document database that is pre-built and includes a plurality of sensitive documents and the divided plurality of text chunks; and identifying a text chunk among the divided plurality of text chunks in which the calculated similarity is greater than or equal to a threshold value as sensitive information.

[0027] In various embodiments, the step of identifying the sensitive information may include identifying a string represented by a predefined regular expression pattern among the text included in at least one of the acquired user input and the response generated in response to the acquired user input as sensitive information.

[0028] In various embodiments, the step of identifying the sensitive information may include the step of extracting an object corresponding to the sensitive information from at least one of the acquired user input and the response generated in response to the acquired user input using a neural network-based object recognition model.

[0029] In various embodiments, the step of identifying sensitive information may include a step of deriving a plurality of sensitive information extraction results by extracting sensitive information from at least one of the acquired user input and the response generated in response to the acquired user input based on a plurality of different sensitive information identification methods, and a step of identifying sensitive information included in at least one of the acquired user input and the response generated in response to the acquired user input by inputting the derived plurality of sensitive information extraction results into a language model in a single context.

[0030] A computing device for performing a method of providing an artificial intelligence response service through a user intent-based policy violation determination according to another embodiment of the present disclosure for solving the above-described problem comprises a processor, a network interface, a memory, and a computer program loaded into said memory and executed by said processor, wherein the processor can perform the method of providing an artificial intelligence response service through a user intent-based policy violation determination by executing one or more instructions included in said computer program.

[0031] A computer program according to another embodiment of the present disclosure for solving the above-mentioned problem may be combined with a computing device and stored on a non-transient recording medium readable by the computing device to execute a method for providing an artificial intelligence response service through user intent-based policy violation determination.

[0032] Other specific details of the present disclosure are included in the detailed description and drawings.

[0033] According to various embodiments of the present disclosure, there is an advantage in that an AI-based response is provided to a user's query, and if the user's intent violates a policy, the generation of the AI ​​response to that query is blocked in advance, thereby effectively preventing inappropriate or dangerous information from being provided to the user and fundamentally improving the safety and reliability of the AI ​​service.

[0034] The effects of the present disclosure are not limited to those mentioned above, and other unmentioned effects will be clearly understood by a person skilled in the art from the description below.

[0035] The following drawings attached to this specification illustrate preferred embodiments of the present disclosure and serve to further enhance understanding of the technical concept of the present disclosure together with the detailed description of the invention; therefore, the present disclosure should not be interpreted as being limited only to the matters described in such drawings.

[0036] FIGS. 1 and FIGS. 2 are drawings illustrating an artificial intelligence response service provision system through user intent-based policy violation determination according to one embodiment of the present disclosure.

[0037] FIG. 3 is a diagram illustrating the hardware configuration of a computing device according to various embodiments of the present disclosure.

[0038] FIG. 4 is a flowchart of a method for providing an artificial intelligence response service through user intent-based policy violation determination according to various embodiments of the present disclosure.

[0039] FIG. 5 is a flowchart of a method for determining user intent according to various embodiments of the present disclosure.

[0040] FIG. 6 is a flowchart of a method for determining whether a policy has been violated according to various embodiments of the present disclosure.

[0041] FIG. 7 is a flowchart of a method for providing a response based on whether there is a policy violation according to various embodiments of the present disclosure.

[0042] FIG. 8 is a flowchart of a sensitive information preprocessing method applicable to various embodiments of the present disclosure.

[0043] FIGS. 9 and 10 are drawings illustrating the results of identifying sensitive information according to various embodiments of the present disclosure.

[0044] FIGS. 11 to 14 are drawings illustrating a user interface (UI) that provides an artificial intelligence response service through user intent-based policy violation determination according to various embodiments of the present disclosure.

[0045] FIGS. 15 to 17 are drawings illustrating examples in which an artificial intelligence response service provision method through user intent-based policy violation determination according to various embodiments of the present disclosure is applied.

[0046] The advantages and features of the present disclosure and the methods for achieving them will become clear by referring to the embodiments described below in detail together with the accompanying drawings. However, the present disclosure is not limited to the embodiments disclosed below but may be implemented in various different forms. These embodiments are provided merely to ensure that the disclosure is complete and to fully inform those skilled in the art of the scope of the present disclosure, and the present disclosure is defined only by the scope of the claims.

[0047] The terms used herein are for describing the embodiments and are not intended to limit the disclosure. In this specification, the singular form includes the plural form unless specifically stated otherwise in the text. As used herein, "comprises" and / or "comprising" do not exclude the presence or addition of one or more other components in addition to the components mentioned.

[0048] Throughout this specification, the same reference numerals refer to the same components, and "and / or" includes each of the mentioned components and all combinations of one or more thereof. Although terms such as "first," "second," etc., are used to describe various components, they are not limited by these terms. These terms are used merely to distinguish one component from another. Accordingly, the first component mentioned below may be the second component within the technical scope of this disclosure.

[0049] As used herein, the terms “part” or “module” refer to hardware components such as software, FPGAs, or ASICs, and the “part” or “module” performs certain roles. However, the “part” or “module” is not limited to software or hardware. The “part” or “module” may be configured to reside in an addressable storage medium or configured to run on one or more processors. Thus, by example, the “part” or “module” includes components such as software components, object-oriented software components, class components, and task components, as well as processes, functions, attributes, procedures, subroutines, segments of program code, drivers, firmware, microcode, circuits, data, databases, data structures, tables, arrays, and variables. The functions provided within the components and “parts” or “modules” may be combined into a smaller number of components and “parts” or “modules,” or further separated into additional components and “parts” or “modules.”

[0050] Spatially relative terms such as "below," "beneath," "lower," "above," and "upper" may be used to facilitate the description of the relationship between one component and other components as illustrated in the drawings. Spatially relative terms should be understood as encompassing different orientations of components during use or operation, in addition to the orientations depicted in the drawings. For example, if a component depicted in a drawing is inverted, a component described as "below" or "beneath" of another component may be placed "above" of that component. Therefore, the exemplary term "below" may encompass both the lower and upper directions. Components may also be oriented in other directions, and accordingly, spatially relative terms may be interpreted according to the orientation.

[0051] Expressions such as "first," "second," or "first," "second" as used in this specification are used to distinguish one object from another when referring to a plurality of objects of the same kind, unless otherwise indicated by the context, and do not limit the order or importance of said objects.

[0052] Expressions used herein such as “A, B, and C,” “A, B, or C,” “A, B, and / or C,” or “at least one of A, B, and C,” “at least one of A, B, or C,” “at least one of A, B, and / or C,” “at least one selected from A, B, and C,” “at least one selected from A, B, or C,” “at least one selected from A, B, and / or C,” etc., may mean each of the listed items or all possible combinations of the listed items. For example, “at least one selected from A and B” may refer to (1) A, (2) at least one of A, (3) B, (4) at least one of B, (5) at least one of A and at least one of B, (6) at least one of A and B, (7) at least one of B and A, and (8) all of A and B.

[0053] As used herein, the expression “based on” is used to describe one or more factors affecting an act or action of a decision or judgment described in the phrase or sentence containing such expression, and such expression does not exclude additional factors affecting said act or action of a decision or judgment.

[0054] As used in this specification, the expression that a certain component (e.g., a first component) is "connected" or "connected" to another component (e.g., a second component) may mean that the said certain component is not only directly connected or connected to the said other component, but is also connected or connected through a new other component (e.g., a third component).

[0055] As used herein, the expression "configured to" may have meanings such as "set to," "capable of," "modified to," "made to," or "capable of." Such expression is not limited to the meaning of "specifically designed in hardware," and, for example, a processor configured to perform a specific operation may mean a generic-purpose processor capable of performing that specific operation by executing software.

[0056] Unless otherwise defined, all terms used herein (including technical and scientific terms) may be used in a meaning commonly understood by those skilled in the art to which this disclosure pertains. Additionally, terms defined in commonly used dictionaries are not to be interpreted ideally or excessively unless explicitly and specifically defined otherwise.

[0057] In this specification, the term "computer" refers to any type of hardware device comprising at least one processor, and may be understood to include software configurations operating on said hardware device according to the embodiments. For example, the term "computer" may be understood to include smartphones, tablet PCs, desktops, laptops, and user clients and applications running on each of these devices, but is not limited thereto.

[0058] Hereinafter, embodiments of the present disclosure will be described in detail with reference to the attached drawings.

[0059] Each step described in this specification is described as being performed by a computer, but the subject of each step is not limited thereto, and depending on the embodiment, at least some of each step may be performed on different devices.

[0060]

[0061] FIGS. 1 and FIGS. 2 are drawings illustrating an artificial intelligence response service provision system through user intent-based policy violation determination according to one embodiment of the present disclosure.

[0062] Referring to FIGS. 1 and 2, an artificial intelligence response service providing system through user intent-based policy violation determination according to one embodiment of the present disclosure may include a computing device (100), a user terminal (200), an external server (300), and a network (400).

[0063] Here, the artificial intelligence response service providing system through user intent-based policy violation judgment illustrated in FIGS. 1 and 2 is according to one embodiment, and its components are not limited to the embodiment illustrated in FIGS. 1 and 2 and may be added, changed, or deleted as needed.

[0064] In one embodiment, the computing device (100) can provide an artificial intelligence response service through user intent-based policy violation determination.

[0065] Here, an AI response service based on user intent-based policy violation judgment may refer to a service that generates and provides an AI-based response in response to a user's query, determines whether the user's intent violates a policy, and provides an appropriate response accordingly (e.g., a normal response or a warning message indicating a policy violation).

[0066] Here, the service provided by the computing device (100) may be provided to the user through a dedicated application installed on the user terminal (200), a web page based on a web browser, or other network-accessible interface.

[0067] For example, a computing device (100) may receive user input, commands, and request data on an application or web page from a user terminal (200) and execute a service function corresponding to the request, or perform data processing, analysis, and calculation for providing the service.

[0068] Additionally, the computing device (100) can transmit processing results to a user terminal (200) to update a user interface (UI) (e.g., FIGS. 11 to 14) or induce subsequent actions, and may include procedures such as user authentication, authorization management, access to stored data, and inter-server interaction as needed. In this way, the service provided by the computing device (100) can be implemented in both web-based and app-based environments, and can provide consistent functions in various user terminals and network environments.

[0069] Here, the user terminal (200) may refer to any form of entity(s) in a system having a mechanism for communicating with a computing device (100). For example, such a user terminal (200) may include a PC (personal computer), a notebook, a mobile terminal, a smartphone, a tablet PC, and a wearable device, and may include any type of terminal capable of connecting to a wired or wireless network. Additionally, the user terminal (200) may include any computing device implemented by at least one of an agent, an API (Application Programming Interface), and a plug-in. Additionally, the user terminal (200) may include an application source and / or a client application.

[0070] Additionally, the network (400) may refer to a connection structure capable of exchanging information between each node, such as multiple terminals and servers. For example, the network (400) may include a Local Area Network (LAN), a Wide Area Network (WAN), the World Wide Web (WWW), a wired / wireless data network, a telephone network, a wired / wireless television network, a Controller Area Network (CAN), and Ethernet.

[0071] Wireless data communication networks may include, but are not limited to, 3G, 4G, 5G, 3GPP (3rd Generation Partnership Project), 5GPP (5th Generation Partnership Project), LTE (Long Term Evolution), WIMAX (World Interoperability for Microwave Access), Wi-Fi, Internet, LAN (Local Area Network), Wireless LAN (Wireless Local Area Network), WAN (Wide Area Network), PAN (Personal Area Network), RF (Radio Frequency), Bluetooth network, NFC (Near-Field Communication) network, satellite broadcasting network, analog broadcasting network, DMB (Digital Multimedia Broadcasting) network, etc.

[0072] In one embodiment, an external server (300) may be connected to a computing device (100) via a network (400) and may store and manage various information and data necessary for the computing device (100) to perform an artificial intelligence response service provision method based on user intent-based policy violation determination, or may collect, store, and manage various information and data generated as the computing device (100) performs the artificial intelligence response service provision method based on user intent-based policy violation determination. For example, the external server (300) may be provided outside the computing device (100) or may be a server logically separated from the computing device (100).

[0073] In various embodiments, the external server (300) may include a policy violation case database (310) and a policy database (320).

[0074] First, the policy violation case database (310) may be configured to store multiple policy violation cases for user inputs, user intents, generated responses, or combinations thereof that were previously determined to be policy violations.

[0075] Here, policy violation cases may include descriptive information about text and images, intent information, violation judgment results, and related policy identification information, and may be stored as data converted into a vector form for comparison of similarity with user input.

[0076] The policy violation case database (310) can be used as reference data to determine whether there is a policy violation through analysis of similarity with user input or user intent entered by the user.

[0077] Next, the policy database (320) can be configured to store policy information applied to the artificial intelligence response service.

[0078] Here, policy information may include basic policies regarding violence, hate, self-harm, etc., industry-specific policies such as finance / medical, organization-specific policies for internal regulations or brand protection, and region-specific policies including legal compliance or cultural considerations.

[0079] Policy information stored in the policy database (320) can be used as reference information for classifying, managing, or interpreting cases of policy violations.

[0080] According to various embodiments of the present disclosure, policy violation determination can be performed based on actual violation cases stored in a policy violation case database (310) without relying on simple rule-based policy application, and at the same time, the consistency and explainability of policy violation determination can be improved by referring to policy information stored in a policy database (320).

[0081] In addition, even if a change in policy or a change in policy violation cases occurs, the policy violation judgment logic can be flexibly maintained by updating the database without retraining the artificial intelligence model. Hereinafter, with reference to FIG. 3, the hardware configuration of a computing device (100) that provides an artificial intelligence response service through user intent-based policy violation judgment will be described.

[0082]

[0083] FIG. 3 is a diagram illustrating the hardware configuration of a computing device according to various embodiments of the present disclosure.

[0084] Referring to FIG. 3, a computing device (100) according to another embodiment of the present disclosure may include one or more processors (110), a memory (120) for loading a computer program (151) executed by the processor (110), a bus (130), a communication interface (140), and a storage (150) for storing the computer program (151). Here, FIG. 3 illustrates only the components related to the embodiments of the present disclosure. Accordingly, a person skilled in the art to which the present disclosure pertains will understand that other general-purpose components may be included in addition to the components illustrated in FIG. 3.

[0085] The processor (110) controls the overall operation of each component of the computing device (100). The processor (110) may be configured to include a CPU (Central Processing Unit), an MPU (Micro Processor Unit), an MCU (Micro Controller Unit), a GPU (Graphic Processing Unit), or any form of processor well known in the art of the present disclosure.

[0086] Additionally, the processor (110) may perform operations for at least one application or program for executing the method according to the embodiments of the present disclosure, and the computing device (100) may have one or more processors.

[0087] In various embodiments, the processor (110) may further include Random Access Memory (RAM) (not shown) and Read-Only Memory (ROM) (not shown) for temporarily and / or permanently storing signals (or data) processed within the processor (110). Additionally, the processor (110) may be implemented in the form of a System on Chip (SoC) comprising at least one of a graphics processing unit, RAM, and ROM.

[0088] Memory (120) stores various data, instructions and / or information. Memory (120) may load a computer program (151) from storage (150) to execute a method / operation according to various embodiments of the present disclosure. When a computer program (151) is loaded into memory (120), the processor (110) may perform the method / operation by executing one or more instructions constituting the computer program (151). Memory (120) may be implemented as a volatile memory such as RAM, but the technical scope of the present disclosure is not limited thereto.

[0089] The bus (130) provides communication functions between components of the computing device (100). The bus (130) can be implemented as various types of buses, such as an address bus, a data bus, and a control bus.

[0090] The communication interface (140) supports wired and wireless internet communication of the computing device (100). Additionally, the communication interface (140) may support various communication methods other than internet communication. To this end, the communication interface (140) may be configured to include a communication module well known in the art of the present disclosure. In some embodiments, the communication interface (140) may be omitted.

[0091] Storage (150) can store computer programs (151) non-temporarily. When performing an artificial intelligence response service provision process through user intent-based policy violation judgment via a computing device (100), storage (150) can store various information necessary to provide the artificial intelligence response service provision process through user intent-based policy violation judgment.

[0092] The storage (150) may be configured to include non-volatile memory such as ROM (Read Only Memory), EPROM (Erasable Programmable ROM), EEPROM (Electrically Erasable Programmable ROM), flash memory, a hard disk, a removable disk, or any form of computer-readable recording medium well known in the art to which the present disclosure belongs.

[0093] A computer program (151) may include one or more instructions that cause a processor (110) to perform a method / operation according to various embodiments of the present disclosure when loaded into memory (120). That is, the processor (110) may perform the method / operation according to various embodiments of the present disclosure by executing the one or more instructions.

[0094] In one embodiment, a computer program (151) may include one or more instructions for performing a method of providing an artificial intelligence response service through user intent-based policy violation determination, the step of obtaining user input including one or more queries from a user; the step of determining the user's intent regarding the one or more queries based on the obtained user input; the step of determining whether there is a policy violation based on the determined intent; and the step of providing a response corresponding to the obtained user input to the user based on the determined policy violation. Hereinafter, with reference to FIGS. 4 to 10, a method of providing an artificial intelligence response service through user intent-based policy violation determination performed by a computing device (100) will be described.

[0095]

[0096] FIG. 4 is a flowchart of a method for providing an artificial intelligence response service through user intent-based policy violation determination according to various embodiments of the present disclosure.

[0097] Referring to FIG. 4, in step S110, the computing device (100) can obtain user input from the user.

[0098] Here, user input is obtained through a user interface (UI) (e.g., FIGS. 11 to 14) provided to a user terminal (200), and may include one or more queries and content related to one or more queries (e.g., images, voice, video, text, etc.), but is not limited thereto.

[0099] In step S120, the computing device (100) can determine the intent of the user's query based on the user input obtained through step S110.

[0100] In various embodiments, the computing device (100) can determine the user's intent regarding a query by analyzing user input through a previously trained artificial intelligence model.

[0101] Here, an artificial intelligence model (e.g., a neural network) may be composed of one or more network functions, and each network function may be implemented as a set of interconnected computational units, which can generally be referred to as nodes or neurons. The nodes constituting one or more network functions are interconnected by links, and links may act as mediating elements that form input-output relationships between two nodes.

[0102] The distinction between input and output nodes is a relative concept; a node may be an output node in relation to a specific node, but an input node in relation to another. Links between nodes can have weights, and these weights can define the functionality and performance of an artificial intelligence model by being adjusted by the user or algorithm during the learning process. For example, if a single output node is linked to multiple input nodes, the output node can determine its output value based on the input values ​​of each input node and the weight values ​​assigned to the corresponding links.

[0103] The structure of an artificial intelligence model can be defined by the number of nodes and links, the weights set for each link, and the connection patterns between nodes; depending on these configurations, various AI models with different characteristics can be implemented. Some of the nodes may be organized into layers based on factors such as distance from the initial input node or relationship with the final output node. For example, a set of nodes with a specific distance (n) from the initial input node can be defined as the nth layer. The input layer can consist of nodes into which data is directly input from the outside, the output layer can consist of nodes from which the final result is produced, and the nodes in between can constitute a hidden layer. The hidden nodes of the hidden layer can perform operations by receiving the output of the previous layer or the outputs of neighboring nodes as input.

[0104] In various embodiments, the artificial intelligence model may be a deep learning model comprising an input layer, a plurality of hidden layers, and an output layer. For example, a deep neural network (DNN) may include a plurality of hidden layers and be utilized to learn latent structures within various data such as photos, text, videos, and voice. The deep learning model may include, but is not limited to, various forms such as a convolutional neural network (CNN), a recurrent neural network (RNN), an autoencoder, a Generative Adversarial Network (GAN), a restricted Boltzmann machine (RBM), a deep confidence network (DBN), a Q-network, a U-network, and a Siamese network.

[0105] Furthermore, an artificial intelligence model can be an autoencoder. An autoencoder can have a structure designed to generate an output similar to the input data. An autoencoder may include one or more hidden layers positioned between the input and output layers, and can have a symmetric or asymmetric structure where the number of nodes gradually decreases from the input layer to a bottleneck layer, and then expands again toward the output layer. Since the bottleneck layer may struggle to transmit sufficient information if it is too small, it can be set to a size greater than a certain threshold. Autoencoders can perform non-linear dimensionality reduction and can be utilized to compress and restore features of the input data.

[0106] In various embodiments, the artificial intelligence model of the present disclosure may be a Large Language Model (LLM). For example, the artificial intelligence model of the present disclosure may be a lightweight text LLM (e.g., proprietary 3B to 8B class models), an open text LLM (e.g., LLaMA, Qwen, Mistral, Gemma, etc.), a commercial text LLM (e.g., GPT-5-mini, Claude 4.0 Haiku, Gemini Flash, etc.), and / or a text LLM fine-tuned to a domain / policy / organizational language.

[0107] Large-scale language models can be a type of deep neural network configured to learn linguistic patterns, contextual relationships, and semantic structures based on massive text corpora. Generally, they can include billions of parameters, and through these numerous parameters, they can perform various natural language processing (NLP) functions such as sentence generation, context understanding, summarization, question answering, inference, and translation. For example, large-scale language models may include transformer-based structures, which can analyze the relationships between words within a sentence in a multi-layered manner by utilizing self-attention and multi-head attention mechanisms. Through these attention mechanisms, the model can effectively identify the contextual features of the input text and generate linguistic output appropriate to that context.

[0108] In addition, large-scale language models may have a two-stage training procedure including pre-training and fine-tuning. In the pre-training stage, general linguistic knowledge is learned using large-scale general-purpose text data (e.g., web documents, books, news, etc.), and subsequently, in the fine-tuning stage, performance can be optimized using datasets specialized for specific domains or tasks (e.g., medicine, law, education, customer support, etc.). In some embodiments, additional adapter layers or knowledge insertion modules may be combined with the pre-trained model to extend functionality for specific applications. Furthermore, large-scale language models can be trained to perform various language-related tasks, such as predicting the next word in a sentence or restoring masked words, by applying various pre-training techniques, such as causal language model methods or masked language model methods. Large-scale language models configured in this manner are capable of understanding complex sentence structures and generating natural text, making them useful for implementing language-based interfaces in various application services.

[0109] At this time, the large-scale language model can operate in a zero-shot prompting manner that analyzes user intent based on natural language instructions without separate additional training. That is, the computing device (100) can determine user intent even for new types of queries that are not predefined by providing instruction information to the large-scale language model to perform intent analysis along with user input.

[0110] In addition, large-scale language models can be replaced with models fine-tuned to suit specific industries, policies, or service environments. For example, a financial intent detection model to detect illegal activities or insider information leaks in the financial sector, or an education domain-specific intent detection model to determine fraud in an educational environment, can be used as the aforementioned artificial intelligence model.

[0111] Accordingly, the computing device (100) can selectively apply a general-purpose intent analysis based on zero-shot prompting and an intent analysis based on a fine-tuning model trained to meet specific policy requirements.

[0112] In step S130, the computing device (100) can determine whether there is a policy violation regarding the user's query based on the user's intent determined through step S120.

[0113] In various embodiments, the computing device (100) can determine whether a user's query violates a policy through an artificial intelligence model (e.g., a large-scale language model). At this time, the computing device (100) can determine whether a policy violates and the grounds therefor by utilizing not only the query and content included in the user input and the user's intent, but also policy violation cases stored in the policy violation case database (310) and policy data included in the policy database (320).

[0114] At this time, the computing device (100) can selectively apply a policy violation judgment based on zero-shot prompting and a policy violation judgment based on a fine-tuning model learned to match specific policy requirements, in the same way as the user's intention judgment process.

[0115] For example, a large-scale language model can operate in a zero-shot prompting manner that determines whether a policy has been violated based on natural language instructions without separate additional training. That is, the computing device (100) can determine whether a policy has been violated even for a new type of query that is not predefined by providing instruction information to the large-scale language model to perform a determination of whether a policy has been violated, along with user input and user intent.

[0116] As another example, large-scale language models can be replaced with models fine-tuned to suit specific industries, policies, or service environments. For instance, financial intent detection models to detect illegal activities or insider information leaks in the financial sector, or education domain-specific policy violation detection models to determine fraud in educational environments, can be used as artificial intelligence models.

[0117] In step S140, the computing device (100) can provide a response to the user based on whether there is a policy violation determined through step S130.

[0118] Here, the response provided to the user may be an answer to the query or a message notifying of a policy violation, which may be determined based on the result of the judgment on whether a policy violation has occurred.

[0119]

[0120] FIG. 5 is a flowchart of a method for determining user intent according to various embodiments of the present disclosure.

[0121] Referring to FIG. 5, in step S210, the computing device (100) can convert the content included in the user input into text.

[0122] For example, when the content included in the user input is an image, the computing device (100) can convert the image into text by performing image captioning on the image through a Vision-Language Model (VLM) based on vision natural language processing.

[0123] Here, image captioning may refer to generating natural language text that describes the content of an image by analyzing objects, scenes, actions, or relationships between them contained in the image.

[0124] As another example, when the content included in the user input is voice data containing voice, the computing device (100) can convert the voice included in the voice data into text through a speech-to-text (STT) model.

[0125] As another example, when the content included in the user input is video data including voice and multiple image frames, the computing device (100) can convert the voice included in the video data into text through a speech-to-text (STT) model and convert each of the multiple image frames into text through an image captioning model.

[0126] In various embodiments, when the user input includes two or more different contents, the computing device (100) can generate two or more texts corresponding to each of the two or more contents by converting each of the two or more contents individually and in parallel. That is, when the user input includes a plurality of different images, the computing device (100) can generate text for each image individually by performing parallel image captioning for the plurality of images.

[0127] In step S220, the computing device (100) may generate a first prompt for determining intent regarding a user's query. For example, the computing device (100) may generate a first prompt containing one or more queries included in the text and user input converted through step S210 and a command indicating intent determination, but is not limited thereto.

[0128] In step S230, the computing device (100) can derive a result of determining the user's intent for one or more queries as result data by inputting the first prompt generated through step S220 into a large language model.

[0129]

[0130] FIG. 6 is a flowchart of a method for determining whether a policy has been violated according to various embodiments of the present disclosure.

[0131] Referring to FIG. 6, in step S310, the computing device (100) can determine the user's intent regarding the query and then select a policy violation case corresponding to the user's intent.

[0132] In various embodiments, the computing device (100) can select a policy violation case corresponding to the user's intent from among a plurality of policy violation cases already stored in the policy violation case database (310).

[0133] More specifically, first, the computing device (100) can generate a first vector by vectorizing the user's intention through an embedding model.

[0134] Afterwards, the computing device (100) can generate multiple second vectors corresponding to each of the multiple policy violation cases by vectorizing each of the multiple policy violation cases stored in the policy violation case database (310) through an embedding model.

[0135] Subsequently, the computing device (100) can calculate the similarity between the first vector and each of the plurality of second vectors. For example, the computing device (100) can calculate cosine similarity as the similarity between the first vector and the plurality of second vectors, but is not limited thereto, and various types of similarity may be applied.

[0136] Afterwards, the computing device (100) can select a policy violation case corresponding to the user's intention based on the similarity between the first vector and a plurality of second vectors.

[0137] For example, the computing device (100) can select a policy violation case corresponding to a second vector among a plurality of second vectors whose similarity to a first vector is greater than or equal to a threshold value as a policy violation case corresponding to the user's intention.

[0138] As another example, the computing device (100) may select the second vector with the highest similarity to the first vector among a plurality of second vectors, and select a policy violation case corresponding to the selected second vector as a policy violation case corresponding to the user's intention.

[0139] In step S320, the computing device (100) may generate a second prompt to determine whether there is a policy violation based on the user's intent. For example, the computing device (100) may generate a second prompt containing text generated by converting content included in the user input, one or more queries, the user's intent, and a command instructing to determine whether there is a policy violation.

[0140] At this time, the computing device (100) can induce a large-scale language model to more accurately determine whether there is a policy violation by adding a policy violation case corresponding to the user's intention to the second prompt, thereby comparing and analyzing whether the user input is substantially similar to a past policy violation case.

[0141] In step S330, the computing device (100) inputs the second prompt generated through step S320 into a large language model, thereby deriving a result of determining whether there is a policy violation regarding the user's intent as result data.

[0142]

[0143] FIG. 7 is a flowchart of a method for providing a response based on whether there is a policy violation according to various embodiments of the present disclosure.

[0144] Referring to FIG. 7, in step S410, the computing device (100) can determine whether there is a policy violation based on the user's intent.

[0145] For example, a computing device (100) can determine whether there is a policy violation based on result data derived by inputting into a large language model text generated by converting content included in user input, one or more queries, user intent, policy violation cases, and a second prompt containing a command instructing to determine whether there is a policy violation.

[0146] In step S420, if the computing device (100) determines through step S410 that there is no policy violation based on the user's intent, it can generate an answer to one or more queries through a large language model, and the generated answer can be provided to the user as a response.

[0147] At this time, the computing device (100) can provide an answer to the user only when it determines that the answer is valid.

[0148] More specifically, first, the computing device (100) can evaluate the validity of the answer based on at least one of grounds, tone, vulgarity and copyright.

[0149] For example, a computing device (100) can input an answer into a large language model for evaluation to determine whether the answer was written based on user input or referenced information.

[0150] As another example, the computing device (100) can determine whether the answer contains an aggressive or inappropriate tone or profanity by performing a tone evaluation based on a text classification model or a language model.

[0151] As another example, the computing device (100) can verify whether the answer contains content that unauthorizedly reproduces or quotes another person's work by comparing the similarity between the answer and an external document database.

[0152] Afterward, the computing device (100) may provide the answer to the user if it is determined to be valid (e.g., written based on user input or referenced information, does not contain an aggressive or inappropriate tone or profanity, and does not contain content that unauthorizedly reproduces or quotes another person's work).

[0153] In step S430, if the computing device (100) determines through step S410 that there is a policy violation based on the user's intent, it may block the generation of answers to one or more queries through a large language model and provide a message to the user notifying them of the policy violation.

[0154] A computing device (100) according to various embodiments of the present disclosure can detect sensitive information (e.g., personal identification information, confidential information, etc.) from input obtained from a user or output to be provided to a user, and preprocess the information, taking into consideration that sensitive information (e.g., personal identification information, confidential information, etc.) may be leaked through user prompt input or file upload in a generative AI service. Hereinafter, the description will be made with reference to FIGS. 8 to 10.

[0155]

[0156] FIG. 8 is a flowchart of a sensitive information preprocessing method applicable to various embodiments of the present disclosure, and FIG. 9 to 10 are drawings illustrating the results of identifying sensitive information according to various embodiments of the present disclosure.

[0157] Referring to FIGS. 8 through 10, in step S510, the computing device (100) can identify sensitive information from user input and / or response.

[0158] Here, user input is obtained from the user and may mean, for example, content, text generated as a result of transforming the content, and / or one or more query texts.

[0159] Additionally, the response is generated to be provided to the user in response to user input, and may mean, for example, answer text to one or more queries and / or text of a message notifying of a policy violation.

[0160] In various embodiments, the computing device (100) can identify a string of text included in at least one of user input and / or response that is represented by a predefined regular expression pattern as sensitive information. For example, the computing device (100) can determine that personal identification information represented by a string of a specific format, such as a phone number, account number, credit card number, resident registration number, or email address, is sensitive information based on whether it matches the regular expression pattern.

[0161] In various embodiments, the computing device (100) may use a neural network-based Named Entity Recognition (NER) model to extract an entity corresponding to sensitive information from at least one of the texts included in at least one of the user inputs and / or responses. For example, the computing device (100) may recognize expressions referring to personal names, organization names, location information, personal identification numbers, financial identifiers, or internal assets included in the text as entities through the entity recognition model, and may identify the recognized entity as sensitive information if it corresponds to personal identification information or a predefined category of sensitive information.

[0162] In various embodiments, the computing device (100) can identify sensitive information from at least one of the texts included in at least one of the user input and / or response based on similarity to a sensitive document.

[0163] More specifically, first, the computing device (100) can divide at least one of the texts included in at least one of the user input and / or response into multiple text chunks of sentence or context unit.

[0164] Subsequently, the computing device (100) can calculate the similarity between text and multiple text chunks included in a sensitive document database that is pre-built and includes multiple sensitive documents.

[0165] Subsequently, the computing device (100) can identify text chunks among a plurality of text chunks that have a similarity to text included in a sensitive document database above a threshold value as sensitive information.

[0166] In various embodiments, the computing device (100) can derive a plurality of sensitive information extraction results by extracting sensitive information from at least one of the texts included in at least one of the user inputs and / or responses based on a plurality of different sensitive information identification methods (e.g., a regular expression-based identification method, an entity recognition model-based identification method, and a similarity-based identification method with a sensitive document), and can identify sensitive information included in at least one of the texts included in at least one of the user inputs and / or responses by inputting the derived plurality of sensitive information extraction results into a language model in a single context.

[0167] For example, when a string corresponding to a phone number is detected by a regular expression-based identification method, a person name entity is extracted by an entity recognition model-based identification method, and a sentence similar to an internal document is detected by a similarity-based identification method with a sensitive document, each identification result is combined into one context information and provided to a language model, thereby comprehensively determining whether the information contained in the text corresponds to personal identification information or internal information.

[0168] In step S520, the computing device (100) may preprocess the sensitive information identified through step S510. For example, the computing device (100) may remove or mask the sensitive information identified from user input and / or response, but is not limited thereto.

[0169]

[0170] The method for providing an AI response service through user intent-based policy violation determination described above has been explained with reference to the flowchart illustrated in the drawings. For the sake of simplicity, the method for providing an AI response service through user intent-based policy violation determination has been illustrated and described using a series of blocks; however, the present disclosure is not limited to the order of the blocks, and some blocks may be performed in a different order or simultaneously than those illustrated and described in this specification. Additionally, new blocks not described in this specification and drawings may be added, or some blocks may be deleted or modified. Hereinafter, with reference to FIGS. 15 to 17, exemplary cases in which the method for providing an AI response service through user intent-based policy violation determination according to various embodiments of the present disclosure is applied will be described.

[0171]

[0172] FIGS. 15 to 17 are drawings illustrating examples in which an artificial intelligence response service provision method through user intent-based policy violation determination according to various embodiments of the present disclosure is applied.

[0173] First, FIG. 15 is a diagram illustrating an example of a situation in which an employee of a chemical company asks an internal AI assistant a question regarding the manufacture of hazardous materials that violates internal policies, in a situation where the use of company assets to manufacture hazardous materials or to inquire about related information is prohibited.

[0174] Referring to FIG. 15, first, a computing device (100) may receive user input including an image and a query from a user. For example, the user input may include an image containing a chemical and a text query requesting a step-by-step method for manufacturing a specific chemical based on the image.

[0175] Subsequently, the computing device (100) can generate text describing objects, scenes, and situations contained in an image by performing image captioning on the image using a vision natural language processing-based image captioning model. For example, the computing device (100) can generate a text caption indicating that beakers containing various types of chemicals and heating equipment are placed on a table in a chemical laboratory environment.

[0176] Subsequently, the computing device (100) can determine the user intent embedded in the user query by taking the text generated as an image captioning result and the user query together as input. The computing device (100) can use a large-scale language model to analyze whether the user query corresponds to a policy-sensitive intent, such as a query about a chemical synthesis method, a query about a hazardous substance manufacturing method, or a request for a detailed step-by-step procedure.

[0177] Subsequently, the computing device (100) may search for cases similar to the user intent among multiple policy violation cases stored in a policy violation case database in order to determine whether there is a policy violation based on the determined user intent. To do this, the computing device (100) may convert the user intent into a vector through an embedding model and select the policy violation case with the highest similarity by comparing the similarity between the vector and the case vector stored in the policy violation case database. For example, the computing device (100) may select past policy violation cases related to the manufacture of chemical substances.

[0178] Subsequently, the computing device (100) can determine whether there is a policy violation by comprehensively considering the image captioning result text, user query, user intent, and selected policy violation cases. The computing device (100) can use a large-scale language model to comprehensively analyze whether the user query corresponds to a type of policy violation that is substantially the same or similar to the policy violation cases.

[0179] At this time, if the computing device (100) determines that user input corresponds to a policy violation, it may block the generation of an artificial intelligence response to the relevant query and provide a predefined rejection response to the user. For example, the computing device (100) may provide a notification message to the user informing them that an inquiry regarding the manufacturing method of hazardous materials may violate internal safety regulations.

[0180] On the other hand, if the computing device (100) determines that the user input does not constitute a policy violation, it can generate a normal response to the user query and provide the generated response to the user.

[0181]

[0182] Next, FIG. 16 is a diagram illustrating an example of a situation in which a securities firm's analyst is requested to prepare a report for an external proposal using internal confidential documents, while the regulation "Policy #F-73: Prohibition on investment advisory and creation of external documents using non-public material information (MNPI)" is registered in the policy database.

[0183] Referring to FIG. 16, first, a computing device (100) may receive user input including an image of a confidential document related to M&A and a text query requesting the creation of a draft report for an external proposal based on the confidential document. Here, the user input may include an image of a document containing undisclosed important information that is to be used only for internal review.

[0184] Subsequently, the computing device (100) can generate text describing the title, nature, and content of the document included in the image by performing image captioning for the confidential document image using an image captioning model. For example, the computing device (100) can generate a text caption indicating that it is an internal document image with the title “List of Companies Under Confidential M&A Review.”

[0185] Subsequently, the computing device (100) can determine the user intent embedded in the user query by taking the text generated as an image captioning result and the user query together as input. The computing device (100) can use a large-scale language model to analyze whether the user query corresponds to an attempt to analyze investment using internal confidential information or an attempt to generate undisclosed information as an external document.

[0186] Subsequently, the computing device (100) can search for cases similar to the user's intent among multiple policy violation cases stored in the policy violation case database in order to determine whether there is a policy violation based on the user's intent. To do this, the computing device (100) converts the user's intent into a vector through an embedding model and searches for policy violation cases related to Policy #F-73 regarding the “prohibition of investment advisory and creation of external documents using undisclosed material information (MNPI)” by comparing the similarity between the vector and the case vector stored in the policy violation case database.

[0187] Next, the computing device (100) can determine whether there is a policy violation by comprehensively considering the image captioning result text, user query, user intent, and the detected policy violation cases. The computing device (100) can use a large-scale language model to comprehensively analyze whether a user query constitutes an act violating Policy #F-73, such as requesting the generation of a report for external proposals based on confidential documents corresponding to undisclosed important information.

[0188] At this time, if the computing device (100) determines from the analysis results that the user input corresponds to a policy violation, it may block the generation of an artificial intelligence response to the query. For example, the computing device (100) may provide the user with a predefined rejection response indicating that the requested content cannot be processed because it violates an internal information protection policy.

[0189]

[0190] Next, FIG. 17 is a diagram illustrating an example in which an artificial intelligence response service provision method through user intent-based policy violation determination according to various embodiments of the present disclosure is applied in an online education platform environment.

[0191] Referring to FIG. 17, first, a computing device (100) can obtain user input including images and queries from a student user. Here, the user input may include, for example, images related to learning or images containing test / assignment situations, and text queries regarding them.

[0192] Subsequently, the computing device (100) can generate text describing the content of an image by performing image captioning on the image using a vision-natural language processing-based image captioning model. For example, it can generate text captions indicating the meaning of the scene from an experiment scene image, an exam paper image, or a learning material image.

[0193] Subsequently, the computing device (100) can determine the user's intent by inputting the generated text caption and the user query together. At this time, the computing device (100) can analyze the user's intent by distinguishing it into a normal learning intent for learning purposes and an inappropriate intent such as cheating, requesting age-inappropriate content, harassment, or bullying.

[0194] Subsequently, the computing device (100) may select a relevant case from among a plurality of policy violation cases stored in a policy violation case database based on user intent. For example, a policy violation case responding to an attempt at academic misconduct, a case prohibiting the provision of age-inappropriate content, or a policy violation case regarding bullying among students may be selected.

[0195] Subsequently, the computing device (100) can determine whether there is a policy violation by comprehensively considering the user's intent and selected policy violation cases. In this process, the computing device (100) can determine whether the user's request is in accordance with educational purposes or whether it violates educational policies and academic ethics.

[0196] At this time, if the computing device (100) determines that there is no policy violation, it may generate and provide a normal response suitable for educational purposes to the user. For example, it may provide an educational response explaining the importance of experiment safety or an educational artificial intelligence response explaining learning concepts.

[0197] On the other hand, if the computing device (100) determines that there is a policy violation, it may restrict the provision of responses, block requests, or provide warning messages. For example, it may refuse to respond to requests for providing test answers or requests to induce cheating, and may provide guidance messages regarding academic honesty or notifications from counselors.

[0198]

[0199] The method described above may be provided as a computer program stored on a computer-readable recording medium for execution on a computer. The medium may continuously store a computer-executable program, or temporarily store it for execution or download. Additionally, the medium may be various recording or storage means in the form of a single or multiple hardware components, and may not be limited to a medium directly connected to a computer system but may exist distributed over a network. Examples of media may include magnetic media such as hard disks, floppy disks, and magnetic tapes; optical recording media such as CD-ROMs and DVDs; magneto-optical media such as floptical disks; and media configured to store program instructions, including ROM, RAM, and flash memory. Furthermore, other examples of media may include recording or storage media managed by app stores that distribute applications or sites and servers that supply or distribute various other software.

[0200] The methods, operations, or techniques of the present disclosure may be implemented by various means. For example, these techniques may be implemented in hardware, firmware, software, or a combination thereof. Those skilled in the art will understand that the various exemplary logical blocks, modules, circuits, and algorithmic steps described in connection with the disclosure herein may be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate such interchangeability between hardware and software, various exemplary components, blocks, modules, circuits, and steps have been generally described above in terms of their functional aspects. Whether such functions are implemented in hardware or in software depends on the design requirements imposed on the specific application and the overall system. Those skilled in the art may implement the functions described in various ways for each specific application, but such implementations should not be construed as departing from the scope of the present disclosure.

[0201] In a hardware implementation, the processing units used to perform the techniques may be implemented in one or more ASICs, DSPs, digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), processors, controllers, microcontrollers, microprocessors, electronic devices, other electronic units designed to perform the functions described in this disclosure, computers, or a combination thereof.

[0202] Accordingly, the various exemplary logic blocks, modules, and circuits described in connection with the present disclosure may be implemented or performed by any combination of general-purpose processors, DSPs, ASICs, FPGAs or other programmable logic devices, discrete gate or transistor logic, discrete hardware components, or those designed to perform the functions described herein. A general-purpose processor may be a microprocessor, but alternatively, the processor may be any conventional processor, controller, microcontroller, or state machine. The processor may also be implemented as a combination of computing devices, for example, a DSP and a microprocessor, a plurality of microprocessors, one or more microprocessors coupled with a DSP core, or any other combination of configurations.

[0203] In firmware and / or software implementations, techniques may be implemented as instructions stored on a computer-readable medium such as random access memory (RAM), read-only memory (ROM), non-volatile random access memory (NVRAM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable PROM (EEPROM), flash memory, compact disc (CD), magnetic or optical data storage devices, etc. The instructions may be executable by one or more processors, and the processor(s) may be enabled to perform specific aspects of the functions described in this disclosure.

[0204] When implemented in software, the techniques described above may be stored on a computer-readable medium as one or more instructions or code, or transmitted through a computer-readable medium. Computer-readable media include both computer storage media and communication media, including any medium that facilitates the transmission of a computer program from one place to another. Storage media may be any available media accessible by a computer. As a non-limiting example, such computer-readable media may include RAM, ROM, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage or other magnetic storage devices, or any other medium accessible by a computer that can be used to transfer or store desired program code in the form of instructions or data structures. Additionally, any connection is appropriately referred to as a computer-readable medium.

[0205] For example, if software is transmitted from a website, server, or other remote source using coaxial cable, fiber optic cable, twisted pair cable, digital subscriber line (DSL), or wireless technologies such as infrared, radio, and microwave, coaxial cable, fiber optic cable, twisted pair cable, digital subscriber line, or wireless technologies such as infrared, radio, and microwave are included within the definition of a medium. As used herein, disk and disc include CD, laser disc, optical disc, DVD (digital versatile disc), floppy disk, and Blu-ray disc, wherein disks usually play data magnetically, whereas discs play data optically using a laser. The above combinations should also be included within the scope of computer-readable media.

[0206] The software module may reside in RAM memory, flash memory, ROM memory, EPROM memory, EEPROM memory, registers, a hard disk, a removable disk, a CD-ROM, or any other known form of storage medium. An exemplary storage medium may be connected to a processor so that the processor can read information from the storage medium or write information to the storage medium. Alternatively, the storage medium may be integrated into the processor. The processor and the storage medium may exist within an ASIC. The ASIC may exist within a user terminal. Alternatively, the processor and the storage medium may exist as separate components within the user terminal.

[0207] Although the embodiments described above have been described as utilizing aspects of the subject matter disclosed herein in one or more standalone computer systems, the present disclosure is not limited thereto and may be implemented in conjunction with any computing environment, such as a network or a distributed computing environment. Furthermore, aspects of the subject matter in the present disclosure may be implemented in a plurality of processing chips or devices, and storage may be similarly affected across a plurality of devices. Such devices may include PCs, network servers, and portable devices.

[0208] Although the present disclosure has been described in relation to some embodiments, various modifications and changes may be made without departing from the scope of the present disclosure as understood by a person skilled in the art to which the invention of the present disclosure pertains. Furthermore, such modifications and changes should be considered to fall within the scope of the claims appended to this specification.

Claims

1. In a method performed by a computing device, A step of obtaining user input from a user that includes one or more queries; A step of determining the user's intention regarding one or more queries based on the user input obtained above; A step of determining whether there is a policy violation based on the above-determined intent; and Based on whether the above-determined policy has been violated, the method includes the step of providing the user with a response corresponding to the acquired user input. Method for providing an AI response service through user intent-based policy violation judgment.

2. In Paragraph 1, The user input obtained above is, Includes content related to one or more of the above queries, The step of determining the user's intent above is, A step of converting the above content into text; and A method comprising the step of determining the user's intent based on the converted text and one or more queries. Method for providing an AI response service through user intent-based policy violation judgment.

3. In Paragraph 2, The step of converting the above content into text is, If the above content is an image, the method includes the step of converting the image into text by performing image captioning on the image through a vision natural language processing-based image captioning model. Method for providing an AI response service through user intent-based policy violation judgment.

4. In Paragraph 2, The step of converting the above content into text is, If the above-mentioned acquired user input includes two or more different contents, the method comprises the step of generating two or more texts corresponding to each of the two or more contents by individually converting each of the two or more contents. Method for providing an AI response service through user intent-based policy violation judgment.

5. In Paragraph 2, The step of determining the user's intent based on the converted text and one or more queries is: A step of generating a first prompt containing the converted text and a command indicating one or more queries and intent determinations; and A method comprising the step of deriving a result of determining the user's intent regarding one or more queries as result data by inputting the first prompt generated above into a large-scale language model. Method for providing an AI response service through user intent-based policy violation judgment.

6. In Paragraph 2, The step of determining whether the above policy has been violated is, A step of generating a second prompt containing the converted text, the one or more queries, and a command instructing the determination of the determined user's intent and whether there is a policy violation; and A step comprising the step of deriving a result determining whether there is a policy violation regarding the determined user's intent as result data by inputting the generated second prompt into a large-scale language model. Method for providing an AI response service through user intent-based policy violation judgment.

7. In Paragraph 6, The step of generating the second prompt above is, A step of selecting a policy violation case corresponding to the user's intent determined above from among a plurality of policy violation cases already stored in a database; and A step comprising adding the above-mentioned selected policy violation case to the above-mentioned generated second prompt, Method for providing an AI response service through user intent-based policy violation judgment.

8. In Paragraph 7, The step of selecting the above policy violation cases is, A step of generating a first vector by vectorizing the user's intent determined above; A step of generating a plurality of second vectors by vectorizing each of the plurality of policy violation cases already stored in the above database; A step of calculating the similarity between the first vector generated above and the plurality of second vectors generated above; and A step comprising selecting a policy violation case corresponding to a second vector whose calculated similarity is greater than or equal to a threshold among a plurality of policy violation cases stored in the database, Method for providing an AI response service through user intent-based policy violation judgment.

9. In Paragraph 1, The step of providing the above response is, If it is determined that there is no policy violation based on the user's intent determined above, a step of generating an answer to one or more of the above queries through a large-scale language model; and A step comprising providing the generated answer to the user as a response corresponding to the user input obtained above, Method for providing an AI response service through user intent-based policy violation judgment.

10. In Paragraph 9, The step of providing the above-mentioned generated answer is, A method comprising the step of evaluating the validity of the generated answer based on at least one of grounds, tone, profanity, and copyright, and providing the generated answer to the user only when the generated answer is determined to be valid. Method for providing an AI response service through user intent-based policy violation judgment.

11. In Paragraph 1, The step of providing the above response is, If it is determined that there is a policy violation based on the user's intent determined above, the method includes the step of blocking the generation of answers to one or more of the above queries through a large-scale language model and providing a message to the user notifying them of the policy violation. Method for providing an AI response service through user intent-based policy violation judgment.

12. In Paragraph 1, The above method is, A step of identifying sensitive information from at least one of the acquired user input and the response generated in response to the acquired user input; and It further includes a step of preprocessing the identified sensitive information, The above preprocessing is, Removing the identified sensitive information or masking the identified sensitive information Method for providing an AI response service through user intent-based policy violation judgment.

13. In Paragraph 12, The step of identifying the above sensitive information is, A step of dividing at least one of the acquired user input and the response generated in response to the acquired user input into a plurality of text chunks in the form of sentences or contexts; A step of calculating the similarity between text included in a sensitive document database that is pre-built and includes multiple sensitive documents and the divided multiple text chunks; and A step comprising identifying text chunks among the above-described multiple text chunks, wherein the calculated similarity is greater than or equal to a threshold value, as sensitive information. Method for providing an AI response service through user intent-based policy violation judgment.

14. In Paragraph 12, The step of identifying the above sensitive information is, A method comprising the step of identifying as sensitive information a string represented by a predefined regular expression pattern among the text included in at least one of the acquired user input and the response generated in response to the acquired user input. Method for providing an AI response service through user intent-based policy violation judgment.

15. In Paragraph 12, The step of identifying the above sensitive information is, A method comprising the step of extracting an object corresponding to sensitive information from at least one of the acquired user input and the response generated in response to the acquired user input using a neural network-based object recognition model. Method for providing an AI response service through user intent-based policy violation judgment.

16. In Paragraph 12, The step of identifying the above sensitive information is, A step of deriving a plurality of sensitive information extraction results by extracting sensitive information from at least one of the acquired user input and the response generated in response to the acquired user input, based on a plurality of different sensitive information identification methods; and A method comprising the step of identifying sensitive information included in at least one of the acquired user input and the response generated in correspondence with the acquired user input by inputting the derived plurality of sensitive information extraction results into a language model in a single context. Method for providing an AI response service through user intent-based policy violation judgment.

17. Processor; Network interface; Memory; and It includes a computer program that is loaded into the memory and executed by the processor, The above processor is, A computing device that performs the method of claim 1 by executing one or more instructions included in the above computer program.

18. Combined with a computing device, A computer program stored on a non-transient recording medium readable by a computing device to execute the method of paragraph 1.

Images