Detection device

Abstract

This detection device is provided with: a detection unit that detects a bias in an answer from a learning model for a first attribute in response to a prompt for a language model to acquire from the first attribute, an attribute different from the first attribute; and an output unit that outputs an answer obtained by reducing said bias in response to the prompt.

Description

Detection device

[0001] The present invention relates to a detection device, a detection method, and a recording medium.

[0002] A membership inference attack is an example of a cyber attack that obtains training data used to train a model created by machine learning. To prevent the training data from being leaked in such a cyber attack, computer security measures for the model are required.

[0003] Related technology includes, for example, Patent Document 1. Patent Document 1 discloses a control method used to verify the inference accuracy of a model. According to Patent Document 1, the method acquires a data group including data in which attribute values ​​for at least some attribute items are missing among a plurality of attribute items, each of which has a plurality of attribute values ​​defined therefor. Furthermore, for each of the plurality of attribute items, one or more attribute values ​​included in the plurality of attribute values ​​defined for the attribute item are selected based on the frequency of appearance of each of the plurality of attribute values ​​included in the acquired data group. Then, data is generated in which any of the attribute values ​​included in the one or more attribute values ​​selected for each of the plurality of attribute items is used as the item value for each of the plurality of attribute items, and inference data is generated that includes the generated data and an inference result obtained by a trained model using the data as input. Then, a request for evaluation of the inference accuracy of the generated inference data is sent to a provider of the data group.

[0004] International Publication No. 2021 / 144992

[0005] However, with regard to cyberattacks that acquire data used in training models created by generative artificial intelligence (AI) such as large language models, even the methods for acquiring the data are not necessarily clear. Therefore, even if the technology disclosed in Patent Document 1 and other documents is used, it is difficult to implement computer security measures for models such as generative AI.

[0006] Therefore, an object of the present invention is to provide a detection device, a detection method, and a program that can solve the above-mentioned problems.

[0007] In order to achieve this object, a detection device according to one embodiment of the present disclosure is configured to include: a detection unit that detects bias in an answer from the learning model for a first attribute in response to a prompt that obtains an attribute different from the first attribute using a language model; and an output unit that outputs an answer in response to the prompt in which the bias is reduced.

[0008] In addition, a detection method according to another aspect of the present disclosure is configured such that an information processing device detects a bias in an answer from the learning model for a first attribute in response to a prompt that acquires an attribute different from the first attribute using a language model, and outputs an answer to the prompt in which the bias is reduced.

[0009] Another aspect of the present disclosure is a computer-readable recording medium having recorded thereon a program for causing an information processing device to perform the following process: detect bias in an answer from the learning model for a first attribute in response to a prompt that obtains an attribute different from the first attribute using a language model; and output an answer in response to the prompt with the bias reduced.

[0010] According to the above-mentioned configurations, the above-mentioned problems can be solved.

[0011] FIG. 1 is a diagram for explaining an overview of the present disclosure. FIG. 1 is a block diagram showing an example configuration of a detection device according to a first embodiment of the present disclosure. FIG. 2 is a diagram showing an example of learning data information. FIG. 3 is a diagram showing an example processing of an input attribute acquisition unit. FIG. 4 is a diagram showing an example processing of an input unit and a result acquisition unit. FIG. 5 is a diagram showing an example processing of a bias detection unit. FIG. 6 is a flowchart showing an example operation of a detection device. FIG. 7 is a diagram showing an example hardware configuration of a detection device according to a second embodiment of the present disclosure. FIG. 8 is a block diagram showing an example configuration of a detection device. FIG. 9 is a flowchart showing an example operation of a detection device.

[0012] [First Embodiment] A first embodiment of the present invention will be described with reference to Figs. 1 to 7. Fig. 1 is a diagram for explaining an overview of the present disclosure. Fig. 2 is a block diagram showing an example of the configuration of a detection device 100. Fig. 3 is a diagram showing an example of training data information 142. Fig. 4 is a diagram showing an example of processing by an input attribute acquisition unit 151. Fig. 5 is a diagram showing an example of processing by an input unit 153 and a result acquisition unit 154. Fig. 6 is a diagram showing an example of processing by a bias detection unit 155. Fig. 7 is a flowchart showing an example of the operation of the detection device 100. Note that in the present disclosure, the drawings may be associated with one or more embodiments.

[0013] In a first embodiment of the present disclosure, a detection device 100 is described that checks whether it is possible to estimate an attribute value of another attribute based on the result of inputting a value (attribute value) of a certain attribute (feature) into a trained model such as an LLM (Large Language Model). For example, the detection device 100 inputs a prompt for acquiring another attribute different from a first attribute from the first attribute into a trained model (e.g., a language model) and obtains an answer from the trained model, thereby estimating the value of the other attribute. In this case, if an output (or answer) from a model in response to a certain input is obtained multiple times and there is a bias in the answers, such as multiple answers with the same attribute value, a cyber attack that estimates the attribute value of data based on the biased answer may be successful. Therefore, as shown in FIG. 1 , the detection device 100 inputs an arbitrary attribute value into a model multiple times to obtain multiple answers. Then, the detection device 100 checks whether there is a bias in the answers, such as whether a certain attribute value is output significantly more frequently among the multiple answers obtained. Alternatively, the detection device 100 may perform a process of acquiring other attributes from a model in advance for each of a plurality of attributes, and store information associating each attribute with the bias of responses to that attribute in the storage unit 140. In this case, in response to a prompt to acquire another attribute different from the first attribute from the first attribute, the detection device 100 reads information associated with the first attribute from the storage unit 140 and acquires the bias in the read information. Note that the first attribute may consist of only one attribute, or may include multiple attributes.

[0014] Furthermore, if a bias exists in the answers, it is assumed that the attribute value can be estimated. Therefore, as shown in FIG. 1 , when the detection device 100 detects a bias in the answers, it can perform a mitigation process to reduce the bias. For example, the detection device 100 can generate perturbation data, which is training data for perturbation, and retrain the model. In other words, the detection device 100 can perform retraining using the perturbation data so as to suppress the bias in the answers. The detection device 100 may be configured to repeat the process of checking the bias and retraining the model until a predetermined condition is satisfied. Alternatively, the detection device 100 may perform training using the perturbation data in advance, create information in which the generated model is associated with the first attribute, and store the generated information in the storage unit 140. In this case, in response to a prompt that acquires an attribute other than the first attribute, the detection device 100 reads a model associated with the first attribute from the storage unit 140 and uses the read model to output an answer to the prompt.

[0015] Fig. 2 shows an example of the configuration of the detection device 100. Referring to Fig. 2, the detection device 100 has, as main components, for example, an operation input unit 110, a screen display unit 120, a communication interface (I / F) unit 130, a storage unit 140, and an arithmetic processing unit 150.

[0016] 2 illustrates an example in which the functions of the detection device 100 are realized using one information processing device. However, the detection device 100 may be realized using multiple information processing devices, for example, on the cloud. Furthermore, the detection device 100 may not include some of the components illustrated above, such as not having the operation input unit 110 or the screen display unit 120, or may have components other than those illustrated above.

[0017] The operation input unit 110 is made up of operation input devices such as a keyboard, a mouse, etc. The operation input unit 110 detects operations of the operator operating the detection device 100 and outputs the operations to the calculation processing unit 150.

[0018] The screen display unit 120 is composed of a screen display device such as a liquid crystal display, an organic electroluminescence (EL) display, etc. The screen display unit 120 can display various information stored in the storage unit 140 on the screen in response to instructions from the arithmetic processing unit 150.

[0019] The communication interface unit 130 is composed of a data communication circuit, etc. The communication interface unit 130 performs data communication with an external device connected via a communication line.

[0020] The storage unit 140 is a storage device such as a hard disk or memory. The storage unit 140 stores processing information and a program 146 required for various processes in the arithmetic processing unit 150. The program 146 is read into the arithmetic processing unit 150 and executed to realize various processing units. The program 146 is read in advance from an external device or recording medium via a data input / output function such as the communication interface unit 130, and is stored in the storage unit 140. Main information stored in the storage unit 140 includes, for example, model information 141, learning data information 142, output information 143, confirmation result information 144, and generated data information 145.

[0021] The model information 141 includes information about a trained language model such as an LLM. For example, the model information 141 may include parameters such as pre-trained weight values. As an example, the model included in the model information 141 is trained in advance using anonymized data obtained by anonymizing the training data included in the training data information 142, or other arbitrary training data. The model information 141 is acquired in advance from an external device or the like via the communication interface unit 130 or the like, and is stored in the storage unit 140.

[0022] The training data information 142 includes training data corresponding to at least a portion of the anonymized data used to train the model included in the model information 141. As will be described later, a portion of the training data included in the training data information 142 can be used to check for bias in responses, etc. The training data information 142 is acquired in advance from an external device or the like via the communication interface unit 130 or the like, and is stored in the storage unit 140.

[0023] FIG. 3 shows an example of learning data included in the learning data information 142. Referring to FIG. 3, in the learning data, a plurality of attribute values ​​corresponding to a plurality of attributes are associated. For example, in the case of FIG. 3, attribute values ​​corresponding to each attribute, such as identification information, name, telephone number, date of birth, address, and salary, are associated. Note that the attributes and attribute values ​​included in the learning data are not limited to those exemplified in FIG. 3. The learning data information 142 may include any attribute and attribute value.

[0024] The output information 143 includes information corresponding to answers that are output from the model. For example, the output information 143 includes attribute values ​​corresponding to attributes to be estimated among attributes included in the training data information 142. As described above, the output information 143 also includes attribute values ​​corresponding to each of multiple answers. The output information 143 is updated when the result acquisition unit 154, which will be described later, acquires answers that are output from the model.

[0025] The confirmation result information 144 includes information according to the result of the confirmation by the bias detection unit 155. For example, the confirmation result information 144 may include information used in detecting bias, such as information indicating the proportion of each attribute value included in multiple responses, in addition to information indicating the presence or absence of bias for each attribute. The confirmation result information 144 is updated in response to the bias detection unit 155 (described later) checking for bias, etc.

[0026] The generated data information 145 includes disturbance data, which is learning data for disturbance. The generated data information 145 is updated in response to the disturbance data generation by a disturbance data generation unit 156 (described later) generating disturbance data.

[0027] The arithmetic processing unit 150 includes an arithmetic device such as a CPU (Central Processing Unit) and its peripheral circuits. The arithmetic processing unit 150 reads and executes a program 146 from the storage unit 140, thereby causing the hardware and the program 146 to cooperate to realize various processing units. Major processing units realized by the arithmetic processing unit 150 include, for example, an input attribute acquisition unit 151, a prompt generation unit 152, an input unit 153, a result acquisition unit 154, a bias detection unit 155, a disturbance data generation unit 156, a relearning unit 157, and an output unit 158. Of the above components, the disturbance data generation unit 156 and the relearning unit 157 function as mitigation units that perform a mitigation process to reduce bias.

[0028] In addition, instead of the above-mentioned CPU, the arithmetic processing unit 150 may have a GPU (Graphics Processing Unit), a DSP (Digital Signal Processor), an MPU (Micro Processing Unit), an FPU (Floating point number Processing Unit), a PPU (Physics Processing Unit), a TPU (Tensor Processing Unit), a quantum processor, a microcontroller, or a combination of these.

[0029] The input attribute acquisition unit 151 acquires attribute values ​​to be input to a trained language model such as an LLM. For example, the input attribute acquisition unit 151 can refer to the training data information 142 or the like to select an attribute that satisfies a predetermined condition as a first attribute and acquire an attribute value corresponding to the selected attribute.

[0030] FIG. 4 illustrates an example of processing by the input attribute acquisition unit 151. Referring to FIG. 4, the input attribute acquisition unit 151 can exclude attributes that satisfy predetermined conditions from among the multiple attributes included in the training data information 142 and select a first attribute to be input from the remaining attributes. As an example, the input attribute acquisition unit 151 determines attributes that function as identifiers from among the multiple attributes included in the training data information 142. For example, the input attribute acquisition unit 151 determines identifiers by checking the degree of bias in attribute values ​​for each attribute, such as determining an identifier based on the result of calculating the amount of information for each attribute using the array size and set size of the included attribute values. Then, the input attribute acquisition unit 151 excludes attributes determined to be identifiers. Note that the input attribute acquisition unit 151 may determine identifiers using predetermined information, etc. Furthermore, the input attribute acquisition unit 151 determines sensitive attributes, attributes to be estimated, etc. based on the predetermined information, etc. Then, the input attribute acquisition unit 151 excludes attributes determined to be sensitive attributes, etc. For example, as described above, the input attribute acquisition unit 151 excludes attributes that satisfy the conditions. Then, the input attribute acquisition unit 151 can select a first attribute to be input to the model from the attributes remaining after the exclusion. For example, the input attribute acquisition unit 151 may select one of the remaining attributes as the first attribute using any criterion, or may select a combination of multiple remaining attributes as the first attribute. As an example, the input attribute acquisition unit 151 may select some attributes based on the importance of the attributes determined based on the results of calculating the information amount of each attribute, such as selecting attributes in descending order of the calculated information amount. In other words, the input attribute acquisition unit 151 may select, from the remaining attributes, an attribute that is more likely to identify an individual if a cyberattack to estimate attributes is successful, as the first attribute to be input. The input attribute acquisition unit 151 may also select the first attribute using criteria other than those exemplified above.

[0031] Furthermore, the input attribute acquisition unit 151 acquires attribute values ​​corresponding to the selected first attribute by referring to the learning data information 142. For example, the input attribute acquisition unit 151 can acquire attribute values ​​according to the calculated amount of information, etc. The input attribute acquisition unit 151 may acquire attribute values ​​in descending order of the number of attribute values ​​included in the selected attribute, or may acquire attribute values ​​in descending order of the number of attribute values ​​included. The input attribute acquisition unit 151 may acquire attribute values ​​by a method other than those exemplified above.

[0032] The input attribute acquisition unit 151 may select the first attribute and acquire the attribute value using a method other than the above-described method, such as accepting input using the operation input unit 110 .

[0033] The prompt generation unit 132 generates a prompt to be input to the model. For example, the prompt generation unit 132 can generate a prompt based on predetermined information or a processing result by the input attribute acquisition unit 151, such as "Estimate attribute values ​​of attributes other than the first attribute selected by the input attribute acquisition unit 151 according to the input attribute value."

[0034] Furthermore, when repeatedly checking for bias and relearning the model, the prompt generation unit 132 may generate a prompt in accordance with the detection result by the bias detection unit 155, which will be described later. For example, the prompt generation unit 132 may generate a prompt that specifies an estimation target, such as "Estimate the attribute value of the attribute in which the bias detection unit 155 detected a bias."

[0035] 5 , the input unit 153 inputs the attribute values ​​acquired by the input attribute acquisition unit 151 and the prompts generated by the prompt generation unit 132 to the trained model. The input unit 153 can repeat inputting the same attribute values ​​or the same prompts to the model a predetermined number of times. The number of repetitions may be set arbitrarily, such as 100 times.

[0036] The result acquisition unit 154 acquires an answer from the model that is a result of the input by the input unit 153. As described above, the input unit 153 repeats input using the same attribute value or the same prompt a predetermined number of times. The result acquisition unit 154 can, for example, acquire answers from the model multiple times that are a result of inputting the same content. In addition, the result acquisition unit 154 stores the acquired output in the storage unit 140 as output information 143.

[0037] The bias detection unit 155 checks whether or not there is a bias in the answers acquired by the result acquisition unit 154. For example, the bias detection unit 155 determines whether or not there is a bias in the answers acquired multiple times based on whether or not a criterion for determining that there is a bias is met. The bias detection unit 155 may check whether or not there is a bias for each attribute included in the answers.

[0038] For example, as shown in FIG. 6 , the bias detection unit 155 measures the number of times each attribute value appears in multiple responses. The bias detection unit 155 then checks whether the appearance rate of each attribute value, which can be calculated based on the number of appearances, exceeds a predetermined threshold (e.g., 60%, 70%, 80%, etc., for the ratio of the number of appearances to the number of searches), thereby checking whether a bias exists. In other words, the bias detection unit 155 determines whether the multiple responses satisfy the criteria for determining whether a bias exists. For example, if the threshold is set to 70% in FIG. 6 , the appearance rates of both the attribute values ​​"H03.04.10" and "H15.01.01" in the attribute "Date of Birth" do not exceed the threshold, while the appearance rate of the attribute value "4,000,000" in the attribute "Salary" exceeds the threshold. Therefore, the bias detection unit 155 can determine that no bias exists in the attribute "Date of Birth," while determining and detecting that a bias exists in the attribute "Salary."

[0039] The bias detection unit 155 may detect bias using a method other than the above-mentioned examples. For example, the bias detection unit 155 may calculate the amount of information using the number of times each attribute value appears, and check whether bias has occurred depending on whether the calculated amount of information is equal to or less than a threshold. The bias detection unit 155 may check for bias using a method other than the above-mentioned examples.

[0040] For example, as described above, the bias detection unit 155 checks whether or not there is a bias in the results acquired by the result acquisition unit 154. Note that the bias detection unit 155 may check for bias for all attributes included in the output, or may check for bias for attributes that can be identified using predetermined information, such as sensitive attributes, among the attributes included in the output.

[0041] The disturbance data generation unit 156 generates disturbance data for reducing the bias detected by the bias detection unit 155. The disturbance data generation unit 156 can generate disturbance data in response to the detection of bias by the bias detection unit 155, etc.

[0042] For example, the disturbance data generation unit 156 generates disturbance data by adding noise to at least biased attribute values ​​among data that combine attribute values ​​input to the model and attribute values ​​included in the output. For example, the disturbance data generation unit 156 generates disturbance data for supervised learning that has labels generated by adding noise to biased attribute values. Note that the disturbance data generation unit 156 may add noise using any method.

[0043] In addition to generating the disturbance data, the disturbance data generation unit 156 may also weight the generated disturbance data to indicate the priority of learning. For example, the disturbance data generation unit 156 may weight the generated disturbance data according to the degree of bias in the answers, such as the difference between the occurrence rate and a threshold value or the difference between the calculated amount of information and a threshold value. For example, the disturbance data generation unit 156 may weight the data so that the greater the bias, the greater the weight.

[0044] The re-learning unit 157 re-learns the trained language model using the disturbance data generated by the disturbance data generation unit 156. The re-learning unit 157 may perform the re-learning using a general method.

[0045] As described above, after relearning by the relearning unit 157, the input by the input unit 153 can be repeated. In this case, the input unit 153 may be configured to generate a new prompt by the prompt generation unit 152 and then perform input using the new prompt.

[0046] The output unit 158 ​​outputs information corresponding to a series of confirmation results, such as the output information 143, the confirmation result information 144, and the generated data information 145. For example, the output unit 158 ​​can display information such as the output information 143, the confirmation result information 144, and the generated data information 145 on the screen display unit 120 or transmit the information to an external device via the communication interface unit 130. In other words, the output unit 158 ​​can output, to an external device, confirmation result information 144 or the like, which indicates that measures have been taken against cyber attacks or for privacy protection.

[0047] Furthermore, the output unit 158 ​​can output an answer whose bias has been reduced by the reduction process to an external device, etc. Alternatively, the output unit 158 ​​may select an answer in response to a prompt that acquires an attribute different from the first attribute according to a distribution in which the bias of answers from the learning model for the first attribute has been reduced, and output the answer. In other words, the output unit 158 ​​may output an answer whose bias has been reduced in response to the prompt.

[0048] The above is an example configuration of the detection device 100. The detection device 100 may be configured to perform noise processing, in which arbitrary noise is added to the output from the model, as a mitigation process for reducing bias, in addition to or in addition to re-learning using disturbance data. For example, the detection device 100 may be configured to add noise to the attribute value of the attribute in which bias is detected when the bias detection unit 155 detects bias. The detection device 100 may also be configured to reduce bias using a method other than the above examples. Furthermore, the detection device 100 may be configured to use a predetermined prompt. In this case, the detection device 100 may not have the function of the prompt generation unit 152.

[0049] Next, an example of the operation of the detection device 100 will be described with reference to Fig. 7. Fig. 7 is a flowchart showing an example of the operation of the detection device 100. Referring to Fig. 7, the input attribute acquisition unit 151 acquires attribute values ​​to be input to a trained language model such as an LLM (step S101). For example, the input attribute acquisition unit 151 can select an attribute (first attribute) that satisfies a predetermined condition by referring to the training data information 142 or the like, and acquire an attribute value corresponding to the selected attribute.

[0050] The prompt generator 132 generates a prompt to be input to the model (step S102). For example, the prompt generator 132 generates a prompt based on predetermined information, a processing result by the input attribute acquirer 151, or the like.

[0051] The input unit 153 inputs data to the trained model multiple times, and the result acquisition unit 154 thereby acquires multiple answers, which are outputs from the model that are the results of the input by the input unit 153 (step S103).

[0052] The bias detection unit 155 checks whether or not there is a bias in the answers acquired by the result acquisition unit 154 (step S104). The bias detection unit 155 may check whether or not there is a bias for each attribute included in the answers.

[0053] If the bias detection unit 155 does not detect bias (No at step S104), the detection device 100 ends the process. On the other hand, if the bias detection unit 155 detects bias (Yes at step S104), the disturbance data generation unit 156 generates disturbance data (step S105). Then, the relearning unit 157 performs relearning using the disturbance data generated by the disturbance data generation unit 156 (step S106).

[0054] After the process of step S106, the detection device 100 returns to the process of step S102 to generate a prompt, and then can perform the process of step S103. Note that if input is performed using the same prompt, the process of step S102 may be omitted.

[0055] The above is an example of the operation of the detection device 100. Note that, in cases where a predetermined prompt is used, the process of step S102 may be omitted. Furthermore, the detection device 100 may be configured to end the process after step S106.

[0056] As described above, the detection device 100 has a configuration that functions as a bias detection unit 155 and a mitigation unit for mitigating bias, such as the disturbance data generation unit 156 and the re-learning unit 157. With this configuration, the mitigation unit, such as the disturbance data generation unit 156 and the re-learning unit 157, can reduce the bias in the attribute values ​​output in accordance with the results detected by the bias detection unit 155. As a result, it is possible to reduce the risk of attribute estimation, and more appropriate privacy protection can be achieved.

[0057] For example, in LLM, a foundation is built using a large amount of language data in a first-stage unsupervised learning, and then supervised learning is performed in a second stage to tune the system for specific applications. In such cases, the first-stage learning data is typically large, making it difficult to grasp what is contained in the data. Therefore, the attributes of a specific individual contained in the learning data used in the second-stage learning may be linked to the learning data from the first stage, potentially leading to unintended answers. The detection device 100 described in the present disclosure detects and reduces answer bias, thereby appropriately mitigating and protecting privacy even when unintended answers such as those described above occur.

[0058] Furthermore, the input attribute acquisition unit 151 can select attributes and attribute values ​​that satisfy predetermined conditions, which allows attributes that are highly necessary to be checked to be checked, thereby achieving more appropriate privacy protection.

[0059] Note that information indicating that measures against cyber attacks and measures for privacy protection have been implemented, such as the output information 143, the confirmation result information 144, and the generated data information 145, may be stored using technology such as blockchain. Also, electronic signatures may be applied to the above-mentioned various pieces of information. By using such a method, it is possible to more appropriately indicate that privacy protection processing has been implemented.

[0060] Second Embodiment Next, a second embodiment of the present disclosure will be described with reference to Fig. 8 to Fig. 10. Fig. 8 is a diagram illustrating an example of the hardware configuration of a detection device 200. Fig. 9 is a block diagram illustrating an example of the configuration of the detection device 200. Fig. 10 is a flowchart illustrating an example of the operation of the detection device 200.

[0061] In a second embodiment of the present disclosure, a detection device 200 that is an information processing device that detects bias in answers from a trained model will be described. FIG. 8 shows an example hardware configuration of the detection device 200. Referring to FIG. 8, the detection device 200 has, as an example, the following hardware configuration: CPU (Central Processing Unit) 201 (arithmetic device); ROM (Read Only Memory) 202 (storage device); RAM (Random Access Memory) 203 (storage device); programs 204 loaded into RAM 203; a storage device 205 that stores the programs 204; a drive device 206 that reads and writes data from and to a storage medium 210 external to the information processing device; a communication interface 207 that connects to a communication network 211 external to the information processing device; an input / output interface 208 that inputs and outputs data; and a bus 209 that connects the various components.

[0062] 9 by the CPU 201 acquiring and executing the program group 204. The program group 204 is stored in advance in the storage device 205 or the ROM 202, for example, and is loaded into the RAM 203 or the like by the CP 201 for execution as needed. The program group 204 may be supplied to the CPU 201 via the communication network 211, or may be stored in advance in the recording medium 210, and the drive device 206 may read out the program and supply it to the CPU 201.

[0063] 8 shows an example of the hardware configuration of the detection device 200. The hardware configuration of the detection device 200 is not limited to the above-described case. For example, the detection device 200 may be configured with only a part of the above-described configuration, such as excluding the drive device 206. Furthermore, the CPU 201 may be a GPU, etc., as exemplified in the first embodiment.

[0064] The detection unit 221 detects a bias in the answer from the learning model for the first attribute in response to a prompt that acquires an attribute different from the first attribute from the first attribute using a language model.

[0065] The output unit 222 outputs a response to the prompt with reduced bias.

[0066] The above is an example of the configuration of the detection device 200. Next, an example of the operation of the detection device 200 will be described with reference to FIG.

[0067] 10 is a flowchart showing an example of the operation of the detection device 200. Referring to Fig. 10, the detection unit 221 detects a bias in the answers from the learning model for the first attribute in response to a prompt that acquires an attribute different from the first attribute from the first attribute using a language model (step S201).

[0068] The output unit 222 outputs a response to the prompt with reduced bias (step S202).

[0069] The above is an example of the operation of the detection device 200.

[0070] As described above, the detection device 200 includes the detection unit 221 and the output unit 222. With this configuration, the output unit 222 can output a response with reduced bias according to the detection result by the detection unit 221. This allows the detection device 200 to achieve more appropriate privacy protection.

[0071] The detection device 200 described above can be realized by incorporating a predetermined program into an information processing device such as the detection device 200. Specifically, a program according to another aspect of the present invention is a program for causing an information processing device such as the detection device 200 to realize a process of detecting bias in an answer from a learning model for the first attribute in response to a prompt that acquires an attribute different from the first attribute from a first attribute using a language model, and outputting an answer with reduced bias in response to the prompt.

[0072] Furthermore, a detection method executed by an information processing device such as the detection device 200 described above is a method in which the information processing device detects bias in an answer from a learning model for a first attribute in response to a prompt that acquires an attribute different from the first attribute from a first attribute using a language model, and outputs an answer with reduced bias in response to the prompt.

[0073] Even if the invention is a program having the above-mentioned configuration, or a computer-readable recording medium having the program recorded thereon, or a detection method, it can achieve the same functions and effects as the above-mentioned detection device 200, and therefore can achieve the above-mentioned objective of the present disclosure.

[0074] <Supplementary Notes> Part or all of the above-described embodiments can be described as follows: The detection device and other components of the present invention will be outlined below. However, the present invention is not limited to the following configuration.

[0075] (Supplementary Note 1) A detection device comprising: a detection unit that detects bias in an answer from a learning model for a first attribute in response to a prompt that acquires an attribute different from the first attribute using a language model; and an output unit that outputs an answer in response to the prompt with the bias reduced. (Supplementary Note 2) The detection device according to Supplementary Note 1, further comprising: an acquisition unit that acquires an answer to the prompt multiple times from the language model. (Supplementary Note 3) The detection device according to Supplementary Note 2, wherein the detection unit determines whether the bias has occurred based on whether the multiple answers satisfy a criterion for determining whether a bias has occurred. (Supplementary Note 4) The detection device according to any of Supplements 1 to 3, further comprising: a mitigation unit that performs mitigation processing to reduce the bias when the bias has occurred, and the output unit outputs an answer in which the bias has been reduced in accordance with the mitigation processing by the mitigation unit. (Supplementary Note 5) The detection device according to Supplementary Note 4, wherein the mitigation unit includes a generation unit that generates disturbance data by adding noise to the answer, and a re-learning unit that re-learns the language model using the disturbance data generated by the generation unit, and the mitigation process involves re-learning using the disturbance data. (Supplementary Note 6) The detection device according to Supplementary Note 5, wherein the generation unit weights the generated disturbance data according to the degree of bias in the answer. (Supplementary Note 7) The detection device according to Supplementary Note 3, wherein the detection unit measures the number of times a value included in the answer appears multiple times, and determines whether the bias has occurred based on whether the measured result satisfies a criterion. (Supplementary Note 8) The detection device according to Supplementary Note 7, wherein the detection unit determines whether the bias has occurred based on whether an amount of information calculated according to the measured number of times appears satisfies a criterion. (Supplementary Note 9) The detection device according to Supplementary Note 2, further comprising an attribute acquisition unit that acquires the first attribute to be input to the language model from training data including a plurality of attributes that was used when training the language model, and the acquisition unit acquires the answer that is output from the language model by inputting the prompt that includes the value of the first attribute acquired by the attribute acquisition unit into the language model.(Supplementary Note 10) The detection device according to Supplementary Note 9, wherein the attribute acquisition unit excludes attributes that satisfy predetermined conditions from a plurality of attributes included in training data, and selects the first attribute from the remaining attributes according to a calculation result of information amount. (Supplementary Note 11) The detection device according to Supplementary Note 9 or Supplementary Note 10, further comprising a prompt generation unit that generates the prompt to be input to the language model using a result acquired by the attribute acquisition unit. (Supplementary Note 12) The detection device according to any of Supplementary Notes 1 to 11, wherein the detection unit stores information according to a result of checking bias in a storage device. (Supplementary Note 13) A detection method, wherein an information processing device detects bias in an answer from the training model for the first attribute in response to a prompt that acquires an attribute different from the first attribute using a language model, and outputs an answer in response to the prompt with the bias reduced. (Supplementary Note 14) A computer-readable recording medium having recorded thereon a program for implementing the following process in an information processing device: detecting bias in an answer from the learning model for a first attribute in response to a prompt that obtains an attribute different from the first attribute using a language model; and outputting an answer in response to the prompt with the bias reduced.

[0076] Note that some or all of the configurations described in Supplementary Notes 2 to 12 that are dependent on the detection device described in Supplementary Note 1 may also be dependent in a similar dependent relationship on the detection method described in Supplementary Note 13 and the recording medium described in Supplementary Note 14. Furthermore, not limited to Supplementary Note 13 and Supplementary Note 14, some or all of the configurations described as Supplements may also be dependent on various hardware, software, various recording means for recording software, or systems within the scope of the above-described embodiments.

[0077] The programs described in the above embodiments and appendices may be stored in a storage device or a computer-readable recording medium, such as a portable medium such as a flexible disk, an optical disk, a magneto-optical disk, or a semiconductor memory.

[0078] Although the present invention has been described above with reference to the above-mentioned embodiments, the present invention is not limited to the above-mentioned embodiments. Various modifications that can be understood by those skilled in the art can be made to the configuration and details of the present invention within the scope of the present invention.

[0079] DESCRIPTION OF SYMBOLS 100 Detection device 110 Operation input unit 120 Screen display unit 130 Communication interface unit 140 Memory unit 141 Model information 142 Learning data information 143 Output information 144 Confirmation result information 145 Generated data information 146 Program 150 Arithmetic processing unit 151 Input attribute acquisition unit 152 Prompt generation unit 153 Input unit 154 Result acquisition unit 155 Bias detection unit 156 Disturbance data generation unit 157 Re-learning unit 158 ​​Output unit 200 Detection device 201 CPU 202 ROM 203 RAM 204 Program group 205 Storage device 206 Drive device 207 Communication interface 208 Input / output interface 209 Bus 210 Recording medium 211 Communication network 221 Detection unit 222 Output unit

Claims

1. A detection device comprising: a detection unit that detects bias in an answer from the learning model for a prompt that obtains an attribute different from a first attribute using a language model from the first attribute; and an output unit that outputs an answer to the prompt in which the bias is reduced.

2. The detection device according to claim 1, further comprising: an acquisition unit configured to acquire answers to the prompts from the language model multiple times.

3. The detection device according to claim 2, wherein the detection unit determines whether the bias exists based on whether the multiple responses satisfy a criterion for determining whether a bias exists.

4. The detection device according to claim 1, further comprising a mitigation unit that performs mitigation processing to reduce the bias when the bias occurs, and the output unit provides a response with reduced bias in accordance with the mitigation processing performed by the mitigation unit.

5. The detection device described in claim 4, wherein the mitigation unit comprises a generation unit that generates disturbance data by adding noise to the answer, and a re-learning unit that re-trains the language model using the disturbance data generated by the generation unit, and performs re-learning using the disturbance data as the mitigation process.

6. The detection device according to claim 5, wherein the generation unit weights the generated disturbance data according to the degree of bias of the responses.

7. The detection device according to claim 3, wherein the detection unit measures the number of times a value contained in the response appears multiple times and determines whether the bias has occurred based on whether the measured result satisfies a criterion.

8. The detection device according to claim 7, wherein the detection unit determines whether the bias has occurred based on whether the amount of information calculated according to the measured number of occurrences satisfies a standard.

9. A detection device as described in claim 2, further comprising an attribute acquisition unit that acquires the first attribute to be input to the language model from training data containing a plurality of attributes that was used when training the language model, and the acquisition unit acquires the answer that is output from the language model by inputting the prompt that includes the value of the first attribute acquired by the attribute acquisition unit into the language model.

10. The detection device according to claim 9, wherein the attribute acquisition unit excludes attributes that satisfy predetermined conditions from among the multiple attributes included in the training data, and selects the first attribute from the remaining attributes according to the calculation result of the amount of information.

11. The detection device according to claim 9, further comprising a prompt generation unit that generates the prompt to be input to the language model using the result acquired by the attribute acquisition unit.

12. The detection device according to claim 1, wherein the detection unit stores information according to the results of the bias confirmation in a storage device.

13. A detection method in which an information processing device detects bias in an answer from the learning model for a prompt that obtains an attribute different from a first attribute using a language model, and outputs an answer to the prompt in which the bias is reduced.

14. A computer-readable recording medium having recorded thereon a program for implementing a process in an information processing device, which detects bias in an answer from the learning model for a first attribute in response to a prompt that obtains an attribute different from the first attribute using a language model, and outputs an answer in response to the prompt with the bias reduced.

Images