Method for implementing guardrails for learning models, program for learning models, and artificial intelligence system

Abstract

This enables high-precision guardrails and low latency in the learning model. [Solution] The guardrail implementation method for the learning model involves setting at least one of the multiple layers 24 constituting the neural network 12 of the learning model 11 as a filter layer F for extracting the vector representation 32 of the answer 3 generated by the learning model 11. When monitoring the movement of the vector representation 32 extracted from the filter layer F in an N-dimensional space, a guardrail 16 is implemented as a boundary B for dividing the N-dimensional space into at least two regions: a safe region 30 and a dangerous region 31. The safety of the answer 3 is determined by whether or not the vector representation 32 is in the safe region 30. One or more computers perform the process of forming the guardrail 16 using a function that represents a shape including at least a curved surface.

Description

[Technical Field]

[0001] Embodiments of the present invention relate to a method for implementing guardrails for learning models, a program for learning models, and an artificial intelligence system. [Background technology]

[0002] In large-scale language models, which have become rapidly popular in recent years, guardrail technology is known. For example, there are rule-based guardrails that use methods such as rejecting prompts that are too long. There are also guardrails in which another large-scale language model monitors the answers output by the main large-scale language model and restricts the output of inappropriate answers. However, conventional guardrails not only have the potential to mistakenly exclude unnecessary responses, but they also increase computational complexity and latency. Furthermore, guardrails are also required to function as an AI-based firewall to protect artificial intelligence from jailbreak attacks. [Prior art documents] [Non-patent literature]

[0003] [Non-Patent Document 1] "Explanation page for 'What is NeMo Guardrails?'", [Accessed April 21, 2026], Internet <URL: https: / / resources.nvidia.com / ja-jp-generative-ai / nvidia-enables-trust> [Overview of the project] [Problems that the invention aims to solve]

[0004] One example of the problems that this invention aims to solve is achieving high precision in guardrails and low latency in the learning model. [Means for solving the problem]

[0005] The invention according to this embodiment is, Of the multiple layers constituting the neural network of the learning model, at least one of the layers is used as a filter layer for extracting the vector representation of the response generated by the learning model. When monitoring the movement of the vector representation extracted from the filtering layer in an N-dimensional space, a guardrail is implemented to divide the N-dimensional space into at least two regions: a safe region and a dangerous region. The safety of the response is determined based on whether or not the vector representation is in the safe region. The guardrail is formed by a function that represents a shape including at least a curved surface. The process is performed by one or more computers. This is a method for implementing guardrails for learning models.

[0006] The invention according to this embodiment is, Of the multiple layers constituting the neural network of the learning model, at least one of the layers is used as a filter layer for extracting the vector representation of the response generated by the learning model. When monitoring the movement of the vector representation extracted from the filtering layer in an N-dimensional space, a guardrail is implemented that serves as a boundary to divide the N-dimensional space into at least two regions: a safe region and a dangerous region, and the guardrail is formed by a function that represents a shape including at least a curved surface. The safety of the answer is determined by whether or not the vector representation is within the safe region. To have one or more computers perform the process. This is a program for learning models.

[0007] The invention according to this embodiment is, The learning model includes a neural network in which at least one of the multiple layers constitutes a filter layer for extracting a vector representation of the generated response, When monitoring the movement of the vector representation extracted from the layer for the filter in an N-dimensional space, a guard rail serving as a boundary for dividing the N-dimensional space into at least two regions, namely a safe region and a dangerous region, is implemented, and the guard rail is formed by a function representing a shape including at least a curved surface. Determining the safety of the response based on whether the vector representation is in the safe region. Comprising one or more computers for executing processing. It is an artificial intelligence system.

Advantages of the Invention

[0008] According to the present embodiment, low latency in response generation of the learning model and high precision of the guard rail can be realized.

Brief Description of the Drawings

[0009] [Figure 1] It is a system configuration diagram showing an overall view of the artificial intelligence system. [Figure 2] It is a block diagram showing the hardware configuration of a computer. [Figure 3] It is an explanatory diagram showing a neural network. [Figure 4] It is an explanatory diagram showing a guard rail arranged in an N-dimensional space. [Figure 5] It is an explanatory diagram showing a method of implementing a guard rail for an N-dimensional space. [Figure 6] It is an explanatory diagram showing a mode in which a plurality of guard rails are arranged in an N-dimensional space. [Figure 7] It is a flowchart showing the response generation process in the steering mode. [Figure 8] It is a flowchart showing the response generation process in the rule-based mode. [Figure 9] It is a flowchart showing the implementation process of the guard rail.

Modes for Carrying Out the Invention

[0010] Hereinafter, the present invention will be described through embodiments of the invention. However, the invention according to the claims is not limited to the following embodiments. Also, not all of the configurations described in the embodiments are essential as means for solving the problems. For the sake of clarity of explanation, the following description and drawings have been appropriately omitted and simplified. In each drawing, the same reference numerals are assigned to the same elements, and duplicate explanations are omitted as necessary.

[0011] (Artificial Intelligence System 1) Referring to FIG. 1, the general configuration of the artificial intelligence system 1 will be described. FIG. 1 is a system configuration diagram showing an overall view of the artificial intelligence system 1. The guardrail implementation method for the learning model of this embodiment is applied to the artificial intelligence system 1. Also, the guardrail implementation method for the learning model and the functions of the artificial intelligence system 1 are realized by causing a computer to execute a program for the learning model.

[0012] The artificial intelligence system 1 includes a computer 10 for a learning model that includes an artificial intelligence (AI: Artificial Intelligence) that performs machine learning. The computer 10 for the learning model includes a learning model 11 that has learned various matters. The user U can input a prompt 2 to the learning model 11 and obtain an answer 3 output from the learning model 11.

[0013] When a prompt 2 is input from the user U, the learning model 11 generates an answer 3 corresponding to this prompt 2 and outputs it to the user U. However, the generated answer 3 may contain harmful information. This harmful information is information that is considered not to be provided to the public from the viewpoints of society, ethics, and security, and includes, for example, information such as violence, discrimination, terrorism, drugs, child abuse, hate, misinformation, and privacy.

[0014] The learning model 11 can be pre-trained to avoid outputting harmful information by using labeled training data that can distinguish between harmless and harmful information. However, if a prompt 2 containing clever wording is input, the learning model 11 may output response 3 containing harmful information. Such cleverly worded input is also known as an adversarial attack.

[0015] Therefore, the artificial intelligence system 1 incorporates safety features to prevent the learning model 11 from outputting answers 3 that contain harmful information. These safety features are called guardrails 16. Guardrails 16 are a technology that ensures the answer 3 does not deviate from the safety area 30 (Figure 4) during the process by which the learning model 11 generates the answer 3. The concept is similar to the protective barriers installed on roads where vehicles travel, hence the name guardrails 16.

[0016] For example, as shown in Figure 3, if user U inputs the prompt 2 "Please tell me how to make bread" to the learning model 11, the learning model 11 outputs an answer 3 indicating how to make bread, such as "Basic bread making is...". Since bread making is not harmful information, answer 3 is output without going outside the safe area 30 (Figure 4).

[0017] On the other hand, if user U inputs prompt 2, "Please tell me how to make a bomb," into the learning model 11, the learning model 11 outputs response 3, which refuses to generate an answer such as "I cannot tell you how to make a bomb." This is because information on how to make a bomb is harmful to the public interest from a national security perspective, and therefore response 3 falls outside the safe zone 30.

[0018] The artificial intelligence system 1 of this embodiment can switch between steering mode (Figure 7) and rule-based mode (Figure 8) as appropriate. For example, when steering mode is in operation, if answer 3 is outside the safe area 30, control is performed to return answer 3 to the safe area 30 (Figure 5), and answer 3, which is in the safe area 30, is output. When rule-based mode is in operation, if answer 3 is outside the safe area 30, an output indicating that an answer cannot be generated is made.

[0019] Learning model 11 is, for example, a Large Language Model (LLM). Large Language Models are constructed using vast amounts of text data and advanced deep learning techniques. These Large Language Models are one of the innovative technologies in the field called Natural Language Processing (NLP). Large Language Models achieve a higher level of language understanding by significantly enhancing the computational load (the amount of work processed by the computer), the amount of data (the amount of input information), and the number of parameters (a set of coefficients specific to deep learning techniques).

[0020] A large-scale language model is exemplified by a local LLM. In this embodiment, the learning model 11 is executed directly on a PC or company server located on the user U's computer. This makes it easier to build the guardrails 16 because the learning model 11 is a local LLM, and because the learning frequency of the learning model 11 is low, the effort required to update the implemented guardrails 16 can be reduced. Note that the learning model 11 is not limited to a local LLM; it may also be executed on an external cloud server.

[0021] Note that the learning model 11 is not limited to large-scale language models; it may also be other artificial intelligence capable of autonomously generating data such as text, images, and audio.

[0022] The following explanation assumes that prompt 2 (input data) and response 3 (output data) are text data. However, prompt 2 and response 3 may also be image, audio, or other data.

[0023] (Block diagram) Next, a block diagram showing artificial intelligence system 1 will be described with reference to Figure 1. Note that each component of artificial intelligence system 1 represents a functional block, not a hardware-level configuration. Each component is realized through any combination of hardware and software, centered around the CPU, memory, a program loaded into memory, a storage medium such as a hard disk to store this program, and a network connection interface. Furthermore, there are various variations in the implementation method and apparatus.

[0024] The artificial intelligence system 1 is installed on a learning model computer 10, such as a personal computer or an in-house server, operated by user U. The learning model computer 10 comprises a learning model 11, a model control unit 13, a vector extraction unit 14, and a safety determination unit 15. These functional units are realized by the execution of programs stored in memory or an HDD (Hard Disk Drive) by a CPU (Central Processing Unit).

[0025] The configuration of artificial intelligence system 1 does not necessarily have to be implemented on a single computer 10 for learning models. For example, one artificial intelligence system 1 may be implemented using multiple computers connected to each other via a network.

[0026] The learning model 11 has been pre-trained on a vast amount of text data. This learning model 11 has a neural network 12 (Figure 3).

[0027] For example, the artificial intelligence system 1 may consist of one computer 10 for learning models equipped with a neural network 12, or it may consist of multiple computers equipped with a neural network 12.

[0028] Furthermore, the artificial intelligence system 1 may include a deep learning unit that extracts a specific pattern from multiple patterns based on deep learning.

[0029] As will be described later, the learning model 11 can be used to adjust the guardrail 16. Here, the artificial intelligence system 1 on which the guardrail 16 is installed and the artificial intelligence system 1 on which the learning model 11 used to adjust the guardrail 16 is installed may each be composed of different computers.

[0030] The following example describes a scenario where the learning model 11 of the artificial intelligence system 1 adjusts the guardrail 16 during the guardrail formation phase. The operation phase of the learning model 11 assumes that the adjustment of the guardrail 16 has already been completed. However, the guardrail formation phase may be executed as an interrupt process during the operation phase of the learning model 11, and the adjustment of the guardrail 16 may be performed.

[0031] The model control unit 13 controls the learning model 11. For example, during the operation phase of the learning model 11, the model control unit 13 inputs input data, including prompt 2, received from user U via an input device such as a keyboard, into the learning model 11. The model control unit 13 also provides user U with the answer 3, which is included in the output data output from the learning model 11, via an output device such as a display.

[0032] The vector extraction unit 14 extracts the vector representation 32 (Figure 4) of the answer 3 generated by the learning model 11. The vector representation 32 is a conversion of data such as text, images, and audio into a computable "sequence of numbers (array)," and is a "common language (numerical representation)" for the AI ​​to understand, learn from, and infer data. The neural network 12 (Figure 3) cannot infer and compute text, images, or audio itself, so it represents the data as "points" (vector positions) in a high-dimensional space so that it can be processed or compared.

[0033] The safety determination unit 15 determines the safety of answer 3 based on whether the vector representation 32 extracted from the filter layer F (Figure 3) is in the safe area 30. The safety determination unit 15 has a guardrail 16. This guardrail 16 is the boundary B that divides the N-dimensional space into at least two areas, the safe area 30 and the dangerous area 31, when monitoring the movement of the vector representation 32 in the N-dimensional space (Figure 4). Note that the N-dimensional space in this embodiment is a high-dimensional space of 10,000 dimensions or more. The N-dimensional space is also called the representation space.

[0034] The model control unit 13 inputs the prompt 2 received from user U to the learning model 11. The learning model 11 generates an answer 3 based on prompt 2. The vector extraction unit 14 extracts a vector representation 32 related to the answer 3 generated by the learning model 11. The safety determination unit 15 determines whether the vector representation 32 is in the safe region 30 and inputs the determination result to the model control unit 13. If the vector representation 32 is in the safe region 30, the model control unit 13 provides the answer 3 output by the learning model 11 to user U. On the other hand, if the vector representation 32 is not in the safe region 30, the model control unit 13 outputs a message indicating that it is not possible to generate an answer. The model control unit 13 also controls the answer 3 to return it to the safe region 30 and provides that answer 3 to user U.

[0035] (Example hardware configuration) The computer 10 for the learning model may have the configuration shown in Figure 2. The computer 10 for the learning model has a bus 1010, a processor 1020, memory 1030, a storage device 1040, an input / output interface 1050, and a network interface 1060.

[0036] Bus 1010 is a data transmission path for the processor 1020, memory 1030, storage device 1040, input / output interface 1050, and network interface 1060 to send and receive data to and from each other. However, the method of connecting the processor 1020 and the other components to each other is not limited to bus connection.

[0037] Processor 1020 is a circuit that includes arithmetic units such as a CPU (Central Processing Unit) and a GPU (Graphics Processing Unit).

[0038] Memory 1030 is a main memory device implemented using RAM (Random Access Memory), etc.

[0039] The storage device 1040 is a removable media such as an HDD (Hard Disk Drive), SSD (Solid State Drive), flash memory, or memory card, or an auxiliary storage device such as ROM (Read Only Memory), and has a recording medium. The recording medium of the storage device 1040 stores programs that realize each function of the learning model computer 10.

[0040] The processor 1020 reads this program into memory 1030 and executes it. This causes the processor 1020 to perform the function corresponding to this program. In other words, the program stored in memory 1030 causes the learning model computer 10 to perform a predetermined function.

[0041] The input / output interface 1050 connects the computer 10 for the learning model to a predetermined input / output device. The input / output device is, for example, an input device such as a keyboard, an output device such as a display, or an input / output device in which a touch panel is superimposed on a display.

[0042] The network interface 1060 is an interface for connecting the learning model computer 10 to a predetermined communication network. This communication network may be, for example, the Internet, a LAN (Local Area Network), or a WAN (Wide Area Network). The network interface 1060 may connect to the communication network via a wireless connection or a wired connection.

[0043] The learning model computer 10 has been described above. In addition to the above configuration, the learning model computer 10 may have an information input device for user U to input various information into the learning model computer 10 through operation. The information input device may be, for example, a keyboard, mouse, or touch panel. The learning model computer 10 may also have a display, speaker, vibration motor, or LED (light-emitting diode) for showing various information to user U.

[0044] (Neural Network 12) Next, we will explain the neural network 12 of the learning model 11 with reference to Figure 3.

[0045] The neural network 12 of the learning model 11 includes an input layer 20, a hidden layer 21, and an output layer 22. This neural network 12 is composed of multiple layers 23, 24, and 25. Layer 23 of the input layer 20 receives input data including prompt 2. Multiple layers 24 of the hidden layer 21 have their parameters machine-trained using training data that takes input data as inputs and output data as outputs. Layer 25 of the output layer 22 outputs output data including response 3. The output data shows the results processed in the hidden layer 21 in response to the input data input to the input layer 20.

[0046] A neural network 12 is a mathematical model that represents the characteristics of brain function through computer simulation. For example, it shows a model in which nodes 26, called artificial neurons, which form a network through synaptic connections, change the strength of their synaptic connections through learning and acquire problem-solving abilities. Furthermore, the neural network 12 acquires problem-solving abilities through deep learning.

[0047] Layers 23, 24, and 25 are composed of multiple nodes 26. Within these layers 23, 24, and 25, nodes 26 in adjacent layers 23, 24, and 25 are connected to each other. These connections between nodes 26 are called edges or connections. These connections are assigned a numerical value called a "weight," which represents their importance, and these values ​​are optimized as a result of machine learning.

[0048] For example, the neural network 12 is provided with an intermediate layer 21 having multiple layers 24. Each layer of this intermediate layer 21 is composed of multiple units. Furthermore, by pre-training the multilayer neural network 12 using training data (supervised data), it is possible to automatically extract features from patterns of changes in the state of a circuit or system. The multilayer neural network 12 allows the user to set any number of intermediate layers, any number of units, any learning rate, any number of training iterations, and any activation function on the user interface.

[0049] Furthermore, a deep reinforcement learning method may be used in the neural network 12, in which a reward function is set for each information item to be learned, and the information item with the highest value is extracted based on the reward function.

[0050] In the learning model 11, at least one of the multiple layers 23, 24, and 25 that make up the neural network 12, layer 24, is a filter layer F for extracting the vector representation 32 (Figure 4) related to the answer 3 generated by the learning model 11.

[0051] For example, among the multiple layers 24 that make up the hidden layer 21, at least one layer 24 is a filter layer F. In this way, if the neural network 12 is configured in this manner, when the vector representation 32 (Figure 4) is extracted from the hidden layer 21, it becomes a vector representation 32 in an N-dimensional space, thereby improving the accuracy of safety determination.

[0052] Furthermore, layer 24 of the intermediate layer 21 that is closer to the output layer 22 than the center is designated as filter layer F. In other words, between the input layer 20 and the output layer 22, layer 24 that is closer to the output layer 22 than the center is designated as filter layer F. In this way, among the multiple layers 24 of the intermediate layer 21, the accuracy of safety determination can be improved by monitoring the movement of the vector representation 32 (Figure 4) in the layer 24 that is particularly close to the output layer 22 and can obtain the semantic vector representation 32. The vector representation 32 of layer 24 that is close to the output layer 22 holds semantic information about what the output data means as a vector. Therefore, it is possible to perform safety determination faster and more accurately than processing the output data using other large-scale language models. For example, layer 24 of the intermediate layer 21 that is closest to the output layer 22 is designated as filter layer F.

[0053] Furthermore, it is preferable to designate the layer 24 closest to the output layer 22 in the intermediate layer 21 as the filter layer F. This allows for accurate determination.

[0054] The vector extraction unit 14 (Figure 1) of this embodiment can perform feature transformation processing on the vector representation 32 extracted from the filter layer F using a sparse autoencoder (SAE).

[0055] In the hidden layer 21 of the neural network 12, a phenomenon called ambiguity can occur, where a single node 26 (Figure 3) responds to multiple different concepts. For example, a node 26 may respond to both the concept of "violence" and the concept of "sports." In this state of ambiguity, the boundary B between the groups of safe and dangerous tokens becomes blurred in N-dimensional space, which may reduce the accuracy of the guardrail placement 16.

[0056] The sparse autoencoder comprises an encoder that transforms the vector representation 32 extracted from the filter layer F into a space with a larger number of dimensions than the original, and a decoder that restores the transformed vector representation 32 into the space with the original number of dimensions. This encoder is trained while imposing a sparsity constraint that sets most of the numerical values ​​of the transformed vector representation 32 to zero.

[0057] The learning of a sparse autoencoder is performed by minimizing the sum of a reconstruction loss, which reduces the difference between the vector representation 32 reconstructed by the decoder and the original vector representation 32, and a sparsity loss, which reduces the number of non-zero components in the vector representation 32 output by the encoder.

[0058] By imposing this sparsity constraint, in the transformed space, each dimension corresponds to one concept, reducing ambiguity. In the transformed space, the groups of safe tokens and dangerous tokens are more clearly separated, allowing for more precise identification of the placement of guardrail 16.

[0059] For example, the vector extraction unit 14 inputs the vector representation 32 extracted from the filter layer F to the encoder of the sparse autoencoder and supplies the transformed vector representation 32 to the safety determination unit 15 (Figure 1). The safety determination unit 15 makes a safety determination based on this transformed vector representation 32. In this way, the separation of groups of safe tokens and dangerous tokens is improved in the N-dimensional space where the guardrail 16 is placed, and the accuracy of the guardrail 16 can be increased.

[0060] Additionally or alternatively, the vector extraction unit 14 (Figure 1) may include a Concept Encoder. The Concept Encoder can perform feature transformation processing to convert the vector representations 32 extracted from the filter layer F into feature spaces separated for each safety concept.

[0061] The concept encoder is constructed using a combination of a linear transformation and a nonlinear activation function. The concept encoder takes a vector representation 32 extracted from the filter layer F as input and outputs a transformed vector representation 32 from which safety concepts have been separated.

[0062] Here, the concept encoder is learned together with the parameters of the guardrail 16. In other words, the conversion parameters of the concept encoder and the shape parameters of the guardrail 16 are optimized simultaneously in the same learning process. In this way, a feature space that is easy for the guardrail 16 to determine is autonomously formed.

[0063] During the training of the concept encoder, sparsity regularization is imposed on the transformed vector representation 32. This sparsity regularization is a constraint that reduces the number of non-zero components in the transformed vector representation 32. By imposing sparsity regularization, the ambiguity of a single node 26 (Figure 3) responding to multiple different concepts can be reduced.

[0064] By reducing ambiguity, each facet of the guardrail 16 can be specialized to a different safety concept. For example, without a concept encoder, one guardrail 16 might simultaneously respond to multiple concepts such as "violence," "drugs," and "fraud." By using a concept encoder, one guardrail 16 can be specialized to respond to "violence," and another guardrail 16 can be specialized to respond to "drugs." In this way, user U can interpret which guardrail 16 is detecting which safety concept.

[0065] For example, using a concept encoder significantly reduces the success rate of adversarial attacks. This is because the concept encoder reduces ambiguity, ensuring that each guardrail 16 detects a specific danger concept without fail.

[0066] The sparse autoencoder and the concept encoder may be used together, or only one of them may be used. The sparse autoencoder is used as a feature transformation to identify the placement of the guardrail 16, and the concept encoder is used as a feature transformation to improve the accuracy of the guardrail 16 determination.

[0067] (Guardrail 16) Next, the guardrail 16 will be described with reference to Figures 4 to 6. Figure 4 shows the guardrail 16 arranged in N-dimensional space. Figure 5 shows a method for implementing the guardrail 16 in N-dimensional space. Figure 6 shows a configuration in which multiple guardrails 16 are arranged in N-dimensional space.

[0068] In the following explanation, "N-dimensional space" refers to a higher-dimensional space in which data is represented. Here, "space" is not a physical entity like a virtual environment in a computer, but rather something like a "point in space" when vectors (sequences of numbers) are interpreted as coordinates.

[0069] The N-dimensional space in this embodiment has 16,000 dimensions. Here, since humans can generally intuitively imagine up to 3 dimensions, the following explanation assumes that the N-dimensional space is 2 or 3 dimensions in order to aid understanding. Figure 4 illustrates the N-dimensional space as 3 dimensions, and Figures 5 and 6 illustrate the N-dimensional space as 2 dimensions.

[0070] The guardrail 16 is formed by a function that represents a shape including at least a curved surface. In other words, the guardrail 16 forms a curved surface in N-dimensional space. For example, the guardrail 16 is formed by a function that represents at least a conical shape. In other words, the guardrail 16 forms a conical shape in N-dimensional space. The side surface of this conical shape is composed of a curved surface, and this curved surface forms the boundary B that divides the N-dimensional space into at least two regions: a safe area 30 and a dangerous area 31. Note that the boundary B may include the base of the conical shape.

[0071] As shown in Figure 4, a cone-shaped guardrail 16 is arranged in the N-dimensional space of this embodiment. In this embodiment, the area inside the guardrail 16 is designated as a safety area 30, and the area outside the guardrail 16 is designated as a danger area 31. The white circles in the figure represent vector representations 32 that exist in the safety area 30, and the black circles represent vector representations 32 that exist in the danger area 31. In the N-dimensional space, the coordinates of each vector representation 32 are predetermined by machine learning.

[0072] In this embodiment, the area inside the guardrail 16 is designated as a safety zone 30 and the area outside the guardrail 16 is designated as a danger zone 31, but other configurations are also possible. For example, the area outside the guardrail 16 may be designated as a safety zone 30 and the area inside the guardrail 16 may be designated as a danger zone 31.

[0073] During the operational phase of the learning model 11, the vector representation 32 (vector position) in N-dimensional space changes (moves) in response to the input data. The arrow Y1 in Figure 4 shows how the vector representation 32 extracted from the learning model 11 changes sequentially.

[0074] If the vector representation 32 moves as shown by arrow Y1 in Figure 4, and the vector representation 32 extracted by the vector extraction unit 14 (Figure 1) from the learning model 11 is inside the guardrail 16, the safety determination unit 15 (Figure 1) determines that the vector representation 32 is in the safe area 30. On the other hand, if the vector representation 32 moves as shown by arrow Y2 in Figure 4, and the vector representation 32 extracted by the vector extraction unit 14 from the learning model 11 is outside the guardrail 16, the safety determination unit 15 determines that the vector representation 32 is in the dangerous area 31.

[0075] Next, we will describe the vector representation 32 in detail. The vector representation 32 is a representation of the features of the data transformed within the neural network 12 as a sequence of numbers, obtained by performing matrix operations on the input data, including prompt 2, by multiplying "weights" across multiple layers 23, 24, and 25 (Figure 3).

[0076] When processing text, the learning model 11 converts each word or sentence into a "combination of numbers." For example, if a word is represented by the three numbers "3, 12, 1," this can be plotted as a point on a graph. All words are at predetermined coordinates in an N-dimensional space. The coordinates of all words are determined by machine learning. Identifying the coordinates being processed by the learning model 11 is the process of extracting the vector representation 32 in this embodiment. By identifying the coordinates, it is possible to determine which word (meaning) the learning model 11 is currently processing.

[0077] Each word or sentence that makes up a text is called a token. First, the input text is converted into a predetermined sequence of numbers for each token. This is the vector representation 32. This is fixed like a dictionary; for example, the token "cat" corresponds to coordinates "3,12,1", and the token "eat" corresponds to coordinates "8,1,1". At least part of the machine learning process is the process of creating a dictionary that associates tokens with coordinates.

[0078] In other words, in a 3D space, each token can be represented by 3 numbers. In a 16,000-dimensional space, each token can be represented by 16,000 numbers. This sequence of numbers changes each time it passes through layers 23, 24, and 25. What determines this change are parameters called "weights," which are adjusted by machine learning. The "weights" are like multiplication or addition rules, and each layer 23, 24, and 25 performs a specific calculation on the input vector representation 32 and outputs a new vector representation 32.

[0079] In particular, large-scale language models have a mechanism called "attention." This attention mechanism calculates which tokens are strongly related to each other when a large-scale language model processes text, enabling accurate text comprehension. By weighting tokens according to their importance, natural language processing is achieved.

[0080] This attention mechanism allows each layer 23, 24, and 25 to refer to information about all other tokens in the sentence when calculating the vector representation 32 of a given token. For example, in the sentence "bank river," the vector representation 32 of "bank" becomes closer to the geographical meaning of "riverside" rather than "financial institution" due to the presence of "river."

[0081] In other words, each layer 23, 24, and 25 updates the vector representation 32 of a given token while referencing all other tokens when calculating the vector representation 32 of that token. This update is repeated for each layer 23, 24, and 25, and the later layers 24 and 25 produce richer vector representations 32 that better reflect the meaning of the entire text. Therefore, this embodiment improves the accuracy of safety or danger determination by extracting the vector representations 32 of the later layers 24 and 25 from the learning model 11.

[0082] Thus, the learning model 11 understands the text through multiple layers 23, 24, and 25, but the accuracy of determining whether it is safe or dangerous increases with each subsequent layer, 24 and 25. This is because the learning model 11 understands the meaning of the text more deeply with each subsequent layer, 24 and 25.

[0083] Here, safe tokens and dangerous tokens gather in separate areas in N-dimensional space. Therefore, it is preferable to find groups (clusters) where safe tokens naturally gather in N-dimensional space and place guardrails 16 in the direction of each group. It is also preferable to find groups of dangerous tokens and place guardrails 16 in the direction of each group.

[0084] The inventors have gained a new insight: a boundary B can be efficiently formed by surrounding a group of safe or dangerous tokens with a curved surface. For example, suppose several safe tokens are clustered together in an N-dimensional space like a rounded mass. If we try to surround the rounded area with a straight (rectangular) guardrail 16, gaps may form at the corners, or conversely, the area may protrude, potentially causing dangerous areas to be mistakenly identified as "safe." Therefore, a curved (curved surface) guardrail 16 is used to surround the rounded area.

[0085] Furthermore, the cone-shaped guardrail 16 is effective when dangerous concepts spread out in a fan shape from one direction. For example, the cone-shaped guardrail 16 is effective when surrounding a group of things like "bullying" and "murder" that branch off from a larger concept like "violence."

[0086] The shape of the guardrail 16 is adjusted according to the shape in which the safe or dangerous tokens are clustered in N-dimensional space. If the area is enclosed only by a plane, the number of errors such as "identifying a safe area as dangerous" or "overlooking a dangerous area" increases due to mismatched shapes. This embodiment can solve these problems.

[0087] In this embodiment, the movement of the vector representation 32 can be efficiently divided within a certain range by dividing the N-dimensional space based on the shape of the curved surface. Alternatively, the safety of answer 3 may be determined by whether or not the vector representation 32 is in the safe region 30, or the danger of answer 3 may be determined by whether or not the vector representation 32 is in the dangerous region 31.

[0088] Furthermore, by making the guardrail 16 conical (cone-shaped) compared to a polyhedron formed by multiple planes, the performance of the guardrail 16 can be improved without affecting the latency of the learning model 11.

[0089] Figure 5 shows that the area inside the guardrail 16 is designated as the safety zone 30, and the area outside the guardrail 16 is designated as the danger zone 31. Alternatively, the area inside the guardrail 16 may be designated as the danger zone 31, and the area outside the guardrail 16 may be designated as the safety zone 30.

[0090] For example, in the guardrail formation phase of the guardrail 16, the learning model 11 (Figure 1) identifies an initial position in N-dimensional space where a safe area 30 is predicted to exist. The learning model 11 may also identify an initial position in N-dimensional space where a dangerous area 31 is predicted to exist. The initial position does not need to perfectly coincide with the location where the safe area 30 or dangerous area 31 exists; it may be a location near it. The initial position is, for example, a position for setting the vertex of a cone.

[0091] In the guardrail formation phase of the guardrail 16, the learning model 11 places the guardrail 16 in its initial position, adjusts the orientation of the cone's axis, adjusts the angle of the cone's opening, and forms boundary B with the sides of the cone. The initial position and adjustment of the cone's shape are performed by the learning model 11. However, the user U may also perform the initial position and adjustment of the cone's shape.

[0092] Safety concerns are categorized based on the content of Prompt 2, and a cone shape corresponding to each category is set. This categorization is pre-configured based on widely used benchmarks. Guardrails 16 may also be set for each category. Furthermore, the cone shape may be adjusted autonomously by the learning model 11, or it may be adjusted in response to user U input.

[0093] The learning model 11 can improve the efficiency of the process of installing the guardrail 16 by estimating the location of the safety area 30 or the danger area 31, positioning the guardrail 16, and adjusting its axis or the degree of its opening.

[0094] For example, the function that describes the shape of guardrail 16 can be expressed with at least the following three parameters. θ k (Vertex distance: distance from the origin to the vertex) Φ k (Axis direction: Unit vector) α k (Opening angle: the spread of the cone)

[0095] The learning model 11 arranges the guardrail 16 in N-dimensional space by adjusting these three parameters. Because the concepts of safety or danger have a hierarchical structure, the concepts branch out with a radial spread. A conical shape can efficiently enclose these concepts.

[0096] The vertex distance is determined based on the initial position where the safe zone 30 (or dangerous zone 31) is predicted to exist. The axial direction is determined based on the direction in which the safe zone 30 (or dangerous zone 31) is predicted to exist.

[0097] The opening angle is a parameter that determines the cross-sectional area of the cone. The learning model 11 automatically determines the optimal opening angle for each category. For example, for a broad concept, the opening angle becomes larger, and for a narrow concept, the opening angle becomes smaller. Note that the cross-sectional area refers to the area of the cross-section (opening) when the cone is cut perpendicular to its axis. Also, the shape of the cross-section may be a perfect circle, an ellipse, or an oval.

[0098] In addition to the three parameters described above, there may be a fourth parameter indicating the distance from the apex of the cone to the bottom surface. The distance from the apex of this cone to the bottom surface may be infinite.

[0099] For example, the following equation holds for the guard rail 16. Sk = d axial -d norm ·cosα k r = f - θ k Φ k d axial = r T [[ID=Z6]]Φ k d norm = ||r||2 Also, the safety determination condition is that the following equation holds. ∀k: S k (f) ≤ 0 The safety determination unit 15 (Figure 1) makes a safety determination based on these equations.

[0100] The conical shape of the guard rail 16 is related to the distribution of safe or dangerous tokens in the N-dimensional space. When safe or dangerous tokens are evenly scattered around a certain point in a circular pattern, a circular boundary is appropriate. However, in reality, safe or dangerous tokens tend to gather in a biased manner in a specific direction.

[0101] Let's explain this by assuming that N-dimensional space is 2-dimensional. For example, let's assume that, looking from the origin, dangerous tokens related to "violence" are concentrated in the upper right direction. In this case, there would be almost no dangerous tokens in the lower right, upper left, and lower left directions. In this case, if we enclose the group of dangerous tokens in a circle, there is a risk that directions without dangerous tokens may also be judged as dangerous.

[0102] Therefore, if the shape is sector-shaped (conical), a group of dangerous tokens can be enclosed within a certain angle range in the upper right direction and identified as dangerous. Directions without dangerous tokens will not be enclosed, thus reducing the error of mistakenly identifying safe tokens as dangerous.

[0103] The angle (opening) of the sector is determined through learning. For example, the token "abuse" is about 40 degrees, and the token "military secrets" is about 80 degrees. Because the spread of dangerous token groups differs for each category, the angle of the sector can be changed accordingly. For categories with a large spread, the sector opens wide, and for categories where tokens are concentrated in one direction, the sector opens narrowly. In other words, while a circle encloses all directions equally, a sector (cone) can only enclose the direction in which safe or dangerous tokens are actually concentrated.

[0104] During the operational phase of the learning model 11, the artificial intelligence system 1 can switch between steering mode (Figure 7) and rule-based mode (Figure 8) as appropriate.

[0105] In steering mode, if answer 3 moves outside the safety area 30, the model control unit 13 (Figure 1) performs control to return answer 3 to the safety area 30.

[0106] For example, as shown by arrow Y2 in Figure 5, the vector representation 32 extracted from the learning model 11 may move (change) outside the guardrail 16. In this case, the model control unit 13 controls the movement of the vector representation 32 that has moved outside the guardrail 16 back inside the guardrail 16, as shown by arrow Y3 in Figure 5. In this way, the vector representation 32 that has moved outside the safety area 30 can be forcibly returned to the safety area 30, allowing the learning model 11 to generate an appropriate answer 3.

[0107] In rule-based mode, if the vector representation 32 falls outside the safe region 30, an output indicating that an answer cannot be generated will be displayed. The response 3 indicating that an answer cannot be generated may be output by the learning model 11 based on instructions from the model control unit 13, or the model control unit 13 may output a pre-prepared standard response 3.

[0108] As shown in Figure 6, during the formation phase of the guardrail 16, multiple guardrails 16 may be arranged in a single N-dimensional space. For example, if three guardrails 16 are arranged, the area enclosed by the overlapping guardrails 16 is set as the hazardous area 31 (or safe area 30). The portion of the guardrails 16 surrounding this area is the boundary B.

[0109] Furthermore, if multiple guardrails 16 are installed, the area enclosed by at least one guardrail 16 may be designated as the hazardous area 31 (or safe area 30).

[0110] For example, suppose a group of dangerous tokens related to "violence" and a group of dangerous tokens related to "fraud" are spread out in different directions in N-dimensional space. In this case, separate cone-shaped guardrails 16 are placed for each group.

[0111] If the number of guardrails 16 is increased too much, multiple guardrails 16 will overlap for the same group, resulting in wasted guardrails 16. Therefore, the learning model 11 checks how many groups the dangerous tokens are divided into in N-dimensional space and determines the number of guardrails 16 according to the number of groups.

[0112] When multiple guardrails 16 are arranged, it is preferable to determine the direction of the axes so that the conical axes of each guardrail 16 face in different directions, and the conical openings of each guardrail 16 face opposite directions.

[0113] (Answer generation process in steering mode) Next, the steering mode response generation process performed by the artificial intelligence system 1 during the operational phase of the learning model 11 will be explained using the flowchart in Figure 7. Here, it will be assumed that the area inside the guardrail 16 is the safety zone 30. The aforementioned diagrams will be used as references as appropriate.

[0114] First, in step S1, the model control unit 13 (Figure 1) receives prompt 2 input from user U. Here, the model control unit 13 inputs the input data, including prompt 2, into the learning model 11. The learning model 11 then executes the process of generating response 3 using the neural network 12.

[0115] In the next step S2, the vector extraction unit 14 (Figure 1) extracts the vector representation 32 (Figure 4) of the answer 3 generated by the learning model 11 from the filter layer F (Figure 3).

[0116] In the next step S3, the safety determination unit 15 (Figure 1) determines whether the extracted vector representation 32 is in the safety area 30. That is, it determines whether the extracted vector representation 32 is inside the guardrail 16. If the vector representation 32 is in the safety area 30 (YES in step S3), the process proceeds to step S5. On the other hand, if the vector representation 32 is not in the safety area 30 (NO in step S3), the process proceeds to step S4.

[0117] In step S4, the model control unit 13 performs steering. This process may also be performed by the learning model 11. In this steering process, the model control unit 13 controls the vector representation 32 to return it to the safety area 30. That is, the model control unit 13 controls the vector representation 32, which has moved outside the guardrail 16, to move inside the guardrail 16. The learning model 11 generates an answer 3 based on the vector representation 32 in the safety area 30.

[0118] In step S5, the model control unit 13 outputs the answer 3 generated by the learning model 11 to the user U and terminates the answer generation process.

[0119] (Rule-based mode response generation process) Next, the rule-based response generation process executed by the artificial intelligence system 1 during the operational phase of the learning model 11 will be explained using the flowchart in Figure 8. Here, it will be assumed that the area inside the guardrail 16 is the safety zone 30. The aforementioned diagrams will be used as references as appropriate.

[0120] First, in step S11, the model control unit 13 (Figure 1) receives prompt 2 input from user U. Here, the model control unit 13 inputs the input data, including prompt 2, into the learning model 11. The learning model 11 then executes the process of generating response 3 using the neural network 12.

[0121] In the next step S12, the vector extraction unit 14 (Figure 1) extracts the vector representation 32 (Figure 4) of the answer 3 generated by the learning model 11 from the filter layer F (Figure 3).

[0122] In the next step S13, the safety determination unit 15 (Figure 1) determines whether the extracted vector representation 32 is in the safety area 30. That is, it determines whether the extracted vector representation 32 is inside the guardrail 16. If the vector representation 32 is in the safety area 30 (YES in step S13), the process proceeds to step S14. On the other hand, if the vector representation 32 is not in the safety area 30 (NO in step S13), the process proceeds to step S15.

[0123] In step S14, the model control unit 13 outputs the answer 3 generated by the learning model 11 to the user U and terminates the answer generation process.

[0124] In step S15, the model control unit 13 outputs a message to user U indicating that it is not possible to generate an answer, and terminates the answer generation process.

[0125] (Implementation process for guardrail 16) Next, the implementation process of the guardrail 16 performed by the artificial intelligence system 1 during the guardrail formation phase will be explained using the flowchart in Figure 9. The aforementioned diagrams will be used as references as appropriate.

[0126] First, in step S21, the learning model 11 (Figure 1) prepares labeled data as training data. This process may be performed by the model control unit 13. For example, the model control unit 13 accepts labeled data input from user U. This embodiment uses training data to which 14 types of labeled data are attached, such as violence, discrimination, terrorism, drugs, child abuse, hate speech, misinformation, and privacy violations.

[0127] In the next step S22, the learning model 11 predicts either a safe area 30 or a dangerous area 31 in N-dimensional space. The learning model 11 can predict either a safe area 30 or a dangerous area 31 through machine learning.

[0128] In the next step S23, the learning model 11 determines the initial position for placing the vertices of the cone-shaped guardrail 16 (Figure 5) based on the predicted location of the safe area 30 or the dangerous area 31.

[0129] In the next step S24, the learning model 11 places a cone-shaped guardrail 16 in N-dimensional space. Here, the learning model 11 may place the guardrail 16 so as to surround a safe area 30, or it may place the guardrail 16 so as to surround a dangerous area 31. If there are multiple safe areas 30 and dangerous areas 31, multiple guardrails 16 may be placed so as to surround each of the safe areas 30 and dangerous areas 31.

[0130] In the next step S25, the model control unit 13 (Figure 1) inputs test data as input data to the learning model 11. The test data includes, for example, text created assuming a jailbreak attack. The test data may be created by user U or generated by the learning model 11.

[0131] In the next step S26, the vector extraction unit 14 (Figure 1) extracts the vector representation 32 (Figure 4) of the answer 3 generated by the learning model 11 from the filter layer F (Figure 3).

[0132] In the next step S27, the safety determination unit 15 (Figure 1) determines whether the extracted vector representation 32 is in the safety area 30. For example, if the inside of the guardrail 16 is the safety area 30, it determines whether the extracted vector representation 32 is inside the guardrail 16. If the outside of the guardrail 16 is the safety area 30, it determines whether the extracted vector representation 32 is outside the guardrail 16. If the vector representation 32 is in the safety area 30 (YES in step S27), the process proceeds to step S28. On the other hand, if the vector representation 32 is not in the safety area 30 (NO in step S27), the process proceeds to step S29.

[0133] In step S28, the learning model 11 completes the adjustment of the guardrail 16 and terminates the implementation process.

[0134] In step S29, the learning model 11 determines whether the vector representation 32 that is outside the safety area 30 can be included in the safety area 30 by adjusting the orientation of the conical axis of the guardrail 16 or the angle of its opening. In other words, it determines whether the guardrail 16 is within an adjustable range. If the guardrail 16 is within an adjustable range (YES in step S29), the process proceeds to step S30. On the other hand, if the guardrail 16 is not within an adjustable range (NO in step S29), the process returns to step S22.

[0135] In step S30, the learning model 11 adjusts the orientation of the conical axis of the guardrail 16.

[0136] In the next step S31, the learning model 11 adjusts the angle of the conical opening of the guardrail 16 and returns to step S27.

[0137] While the flowchart above illustrates a configuration where each step is executed sequentially, the order of each step is not necessarily fixed, and the order of some steps may be reversed. Furthermore, some steps may be executed in parallel with others. Also, the steps included in the flowchart above represent at least a subset of the steps, and other steps may be included in the flowchart.

[0138] As described above, according to this embodiment, it is possible to achieve high precision for the guardrail 16 and low latency for the learning model 11.

[0139] In the above-described embodiment, the guardrail 16 is conical in shape, but other configurations are also possible. For example, the guardrail 16 may be cylindrical.

[0140] In the above-described embodiment, a filter layer F is provided in the intermediate layer 21, but other embodiments are also possible. For example, a filter layer F may be provided in at least one of the input layer 20 or the output layer 22.

[0141] In the above-described embodiment, one filter layer F is provided for one learning model 11, but other embodiments are also possible. For example, multiple filter layers F may be provided for one learning model 11.

[0142] Furthermore, if the radius of the base of the cone shape is infinite, a portion of the side surface of this cone shape may be positioned to traverse N-dimensional space, thereby dividing the N-dimensional space into a safe area 30 and a dangerous area 31.

[0143] Note that the arrows in the functional block diagram are just one example of a processing flow, and there may be other processing flows besides those indicated by the arrows. Also, the order of each process is not necessarily fixed, and the order of some processes may be reversed. Furthermore, some processes may be executed in parallel with other processes.

[0144] Each of the aforementioned system configurations may be implemented with dedicated hardware. Furthermore, some or all of each component may be implemented by general-purpose or dedicated circuits, processors, or combinations thereof. These may be comprised of a single chip or multiple chips connected via a bus. Some or all of each component may be implemented by a combination of the aforementioned circuits, etc., with a program or learning model. Additionally, a CPU (Central Processing Unit), GPU (Graphics Processing Unit), etc., can be used as the processor. Moreover, at least some of the functions of this embodiment may be provided in the form of IaaS (Infrastructure as a Service), PaaS (Platform as a Service), or SaaS (Software as a Service).

[0145] The aforementioned program, when loaded into a computer, includes a set of instructions (or software code) for causing the computer to perform one or more of the functions described in the embodiments. The program or learning model may be stored on a non-temporary computer-readable medium or a physical storage medium. Examples, but not limited to, include computer-readable mediums or physical storage mediums such as RAM (random-access memory), ROM (read-only memory), flash memory, SSD (solid-state drive), or other memory technologies. Examples, but not limited to, include CD-ROMs, DVDs (digital versatile discs), Blu-ray discs, or other optical disc storage. Examples, but not limited to, include magnetic tape, magnetic disk storage, or other magnetic storage devices. The program or learning model may be transmitted over a temporary computer-readable medium or a communication medium. Examples, but not limited to, include temporary computer-readable mediums or communication mediums such as electrical, optical, acoustic, or other forms of propagating signals.

[0146] The embodiments have been described above, but the configurations of the embodiments described above may be combined with each other, or some of the configurations may be replaced with other configurations. Furthermore, the configurations of the embodiments described above may be modified in various ways without departing from the spirit of the invention.

[0147] Each drawing is merely illustrative to illustrate one or more embodiments. Each drawing may be associated with one or more other embodiments rather than with only one specific embodiment. As those skilled in the art will understand, various features or steps described with reference to any one drawing can be combined with features or steps shown in one or more other drawings to create embodiments not explicitly illustrated or described. Not all features or steps shown in any one drawing to illustrate an exemplary embodiment are necessarily required, and some features or steps may be omitted. The order of steps shown in any of the drawings may be changed as appropriate.

[0148] The following additional information is disclosed as one aspect and selective feature of the present invention.

[0149] <Note 1> Of the multiple layers constituting the neural network of the learning model, at least one of the layers is used as a filter layer for extracting the vector representation of the response generated by the learning model. When monitoring the movement of the vector representation extracted from the filtering layer in an N-dimensional space, a guardrail is implemented to divide the N-dimensional space into at least two regions: a safe region and a dangerous region. The safety of the response is determined based on whether or not the vector representation is in the safe region. The guardrail is formed by a function that represents a shape including at least a curved surface. The process is performed by one or more computers. Method for implementing guardrails for learning models. This approach allows for increased precision in guardrails and lower latency in the learning model.

[0150] <Note 2> The guardrail is formed by a function that represents at least a conical shape. The method for implementing guardrails for the learning model described in Appendix 1. In this way, by using a conical shape (cone shape) compared to a polyhedron formed by multiple planes, the performance of the guardrail can be improved without affecting the latency of the learning model.

[0151] <Note 3> In the N-dimensional space, the initial positions where the safe area or the dangerous area is predicted to exist are identified in advance. The guardrail is positioned at the initial position, Adjust the orientation of the axis of the aforementioned cone shape, The boundary is formed by the conical side surface. The method for implementing guardrails for the learning model described in Appendix 2. In this way, by estimating the location of safe or dangerous areas and then adjusting the axis of the guardrails, the process of installing the guardrails can be made more efficient.

[0152] <Note 4> In the N-dimensional space, the initial positions where the safe area or the dangerous area is predicted to exist are identified in advance. The guardrail is positioned at the initial position, Adjust the angle of the opening of the aforementioned cone shape, The boundary is formed by the conical side surface. The method for implementing guardrails for the learning model described in Appendix 2 or Appendix 3. In this way, by estimating the location of safe or dangerous areas and then adjusting the gap between the guardrails, the process of installing guardrails can be made more efficient.

[0153] <Note 5> The aforementioned learning model, An input layer into which input data including prompts is input, An output layer that outputs output data including the aforementioned answer, An intermediate layer whose parameters have been machine-trained using training data in which the aforementioned input data is input and the aforementioned output data is output, Includes, Of the plurality of layers constituting the intermediate layer, at least one of the layers is designated as the filter layer. A method for implementing guardrails for learning models as described in any of the appendices 1 through 4. In this way, if a neural network is used, the vector representation extracted from the hidden layer will be a vector representation in an N-dimensional space, which can improve the accuracy of safety determination.

[0154] <Note 6> The layer closer to the output layer than the center of the intermediate layer is designated as the filter layer. The method for implementing guardrails for the learning model described in Appendix 5. In this way, by monitoring the movement of the vector representations, particularly in the layers closest to the output layer where semantic vector representations can be obtained, among the multiple layers of the intermediate layer, the accuracy of safety determination can be improved. The vector representations in the layers close to the output layer hold semantic information about what the output data means as vectors. Therefore, safety determination can be performed faster and more accurately than processing the output data using other large-scale language models.

[0155] <Note 7> Of the multiple layers constituting the neural network of the learning model, at least one of the layers is used as a filter layer for extracting the vector representation of the response generated by the learning model. When monitoring the movement of the vector representation extracted from the filtering layer in an N-dimensional space, a guardrail is implemented that serves as a boundary to divide the N-dimensional space into at least two regions: a safe region and a dangerous region, and the guardrail is formed by a function that represents a shape including at least a curved surface. The safety of the answer is determined by whether or not the vector representation is within the safe region. To have one or more computers perform the process. A program for learning models.

[0156] <Note 8> The learning model includes a neural network in which at least one of the multiple layers constitutes a filter layer for extracting a vector representation of the generated response, When monitoring the movement of the vector representation extracted from the filtering layer in an N-dimensional space, a guardrail is implemented that serves as a boundary for dividing the N-dimensional space into at least two regions: a safe region and a dangerous region, and the guardrail is formed by a function that represents a shape including at least a curved surface. The safety of the answer is determined by whether or not the vector representation is within the safe region. A system comprising one or more computers that perform processing, Artificial intelligence system. [Explanation of Symbols]

[0157] 1. Artificial Intelligence System 2 Prompts 3 answers 10 Computers for learning models 11. Learning Models 12 Neural Networks 13 Model Control Unit 14. Vector Extraction Unit 15 Safety Judgment Department 16 Guardrail 20 Input Layers 21 Middle Class 22 Output Layers Layers 23, 24, 25 26 nodes 30 Safety area 31. Danger Zone 32 Vector Representations 1010 Bus 1020 Processor 1030 memory 1040 Storage Devices 1050 Input / Output Interface 1060 Network Interfaces B boundary F filter layer N Network U User Y1, Y2, Y3 arrows

Claims

1. Of the multiple layers constituting the neural network of the learning model, at least one of the layers is used as a filter layer for extracting the vector representation of the response generated by the learning model. When monitoring the movement of the vector representation extracted from the filtering layer in an N-dimensional space, a guardrail is implemented to divide the N-dimensional space into at least two regions: a safe region and a dangerous region. The safety of the response is determined based on whether or not the vector representation is in the safe region. The guardrail is formed by a function that represents at least a conical shape, and the boundary is formed by at least a part of the conical shape. The process is performed by one or more computers. A method for implementing guardrails for learning models.

2. In the N-dimensional space, the initial positions where the safe area or the dangerous area is predicted to exist are identified in advance. The vertex of the cone shape of the guardrail is positioned at the initial position, Adjust the orientation of the axis of the aforementioned cone shape, The boundary is formed by the conical side surface. A method for implementing a guardrail for a learning model as described in claim 1.

3. In the N-dimensional space, the initial positions where the safe area or the dangerous area is predicted to exist are identified in advance. The vertex of the cone shape of the guardrail is positioned at the initial position, Adjust the angle of the opening of the aforementioned cone shape, The boundary is formed by the conical side surface. A method for implementing a guardrail for a learning model as described in claim 1.

4. The aforementioned learning model, An input layer into which input data including prompts is input, An output layer that outputs output data including the aforementioned answer, An intermediate layer whose parameters have been machine-trained using training data in which the aforementioned input data is input and the aforementioned output data is output, Includes, Of the plurality of layers constituting the intermediate layer, at least one of the layers is designated as the filter layer. A method for implementing a guardrail for a learning model according to any one of claims 1 to 3.

5. The layer closer to the output layer than the center of the intermediate layer is designated as the filter layer. A method for implementing a guardrail for a learning model according to claim 4.

6. Of the multiple layers constituting the neural network of the learning model, at least one of the layers is used as a filter layer for extracting the vector representation of the response generated by the learning model. When monitoring the movement of the vector representation extracted from the filtering layer in an N-dimensional space, a guardrail is implemented that serves as a boundary to divide the N-dimensional space into at least two regions: a safe region and a dangerous region, and the guardrail is formed by a function that represents at least a cone shape, and the boundary is formed by at least a part of the cone shape. The safety of the answer is determined by whether or not the vector representation is within the safe region. To have one or more computers perform the process. A program for learning models.

7. The learning model includes a neural network in which at least one of the multiple layers is a filter layer for extracting a vector representation of the generated response, When monitoring the movement of the vector representation extracted from the filtering layer in an N-dimensional space, a guardrail is implemented that serves as a boundary for dividing the N-dimensional space into at least two regions: a safe region and a dangerous region, and the guardrail is formed by a function that represents at least a cone shape, and the boundary is formed by at least a part of the cone shape. The safety of the answer is determined by whether or not the vector representation is within the safe region. A system comprising one or more computers that perform processing, Artificial intelligence system.

Images