System, method executed by system, and program

Abstract

According to the present invention, a training data set is edited to improve the quality (accuracy) of a model when the format of the training data is an original data format (and not a text data format). A text data conversion unit 102 converts original data 132 in an original data format into text data 134 corresponding to the original data. A policy determination unit 105 generates a policy 162 on the basis of a policy determination model 161 and a text data set 133 corresponding to the original data. A policy execution unit 106 generates a text data set 141 for generation by editing the text data set 133 corresponding to the original data on the basis of the policy 162. An original data conversion unit 108 converts text data 142 for generation into generated data 144 in the original data format. The generated data 144 is training data for training a discrimination model 165 for discriminating a class 191 of data 148 in the original data format by machine learning.

Description

System, method executed by the system, program

[0001] The present disclosure relates to techniques for compiling training datasets used to train models through machine learning (e.g., techniques for data augmentation and data generation).

[0002] The quality (accuracy) of a model trained by machine learning may depend on the quality of the training dataset used for training. It is considered useful to form a high-quality training dataset if there is a certain amount of training data having each of the various properties that the training data may have. However, due to reasons such as the cost of collecting the training data, it may not be possible to collect a sufficient amount of training data to form a high-quality training dataset. For example, in a collected training dataset, there may be more than enough training data having one property, an appropriate amount of training data having another property, and an insufficient amount of training data having yet another property.

[0003] For these reasons, techniques for editing training datasets have been known. Examples of techniques for editing training datasets include data augmentation and data generation. Data augmentation is a technique for creating or adding new training data by changing (editing) certain training data included in a training dataset. Data generation includes not only a technique for generating training data through data augmentation, but also a technique for generating new training data not included in the training dataset. Examples of techniques for editing training datasets (e.g., data augmentation and data generation) include those disclosed in Non-Patent Document 1 and Non-Patent Document 2. Non-Patent Document 1 discloses a technique for automatically augmenting training data (AutoAugmentation) for determining the class of image data (classifying image data (image classification)). Non-Patent Document 2 discloses a technique for automatically augmenting text data (Text AutoAugmentation) for determining the class of text data (classifying text data (text classification)).

[0004] Vanchinbal Chinbat, Seung-Hwan Bae, "GA3N : Generative adversarial AutoAugment network", Pattern Recognition, Elsevier Ltd., 2022, no.127, 11pagesShuhuai Ren, Jinchao Zhang, Lei Li, Xu Sun, Jie Zhou, "Text AutoAugment : Learning Composition Augmentation Policy for Text Classification", [online], 2021, arXiv, arxiv.org, [searched on November 20, 2020], Internet<URL: https: / / arxiv.org / pdf / 2109.00523.pdf>

[0005] A model trained by machine learning may handle data having a combination of multiple attribute values ​​(including class values). For example, in a discriminant model (classification model, identification model) for discriminating between classes indicating the types of objects contained in image data (e.g., classes indicating cars, people, animals (e.g., dogs, cats, horses), and non-living things (e.g., balls)), each piece of image data handled by the discriminant model may have a combination of multiple attribute values ​​(including class values). Specifically, the image data handled by the discriminant model may have class values ​​indicating the types of objects to be classified (e.g., cars, people, animals (e.g., dogs, cats, horses), and non-living things (e.g., balls)), as well as attribute values ​​that are not class values. Here, the attribute values ​​that are not class values ​​may be, for example, attribute values ​​that indicate the detailed type, shape, size, or color (more detailed than a class) of the object to be classified, or attribute values ​​that indicate the type (e.g., road, garden, sky), shape, size, or color of an object (e.g., background) that is not to be classified. For example, image data representing a "black car driving on a white road" has, as its attribute values, a combination of the class value "car" and attribute values ​​other than the class value, "black as the car's color," "road as the background," and "white as the road's color." Furthermore, for example, image data representing a "black car driving on a gray road" has, as its attribute values, a combination of the class value "car" and attribute values ​​other than the class value, "black as the car's color," "road as the background," and "gray as the road's color." While the above is an example of a discriminant model that handles image data, the same can also be said for a discriminant model that handles audio data. For example, in audio data handled by a discriminant model, the content of a person's voice may be treated as a class value, and other attributes (e.g., the volume of the voice, the wavelength of the voice, the volume of sounds other than audio (e.g., music or noise), and the wavelength of sounds other than audio) may be treated as attribute values ​​other than the class value. Hereinafter, data formats other than text data formats, such as image data and audio data, may be referred to as "original data formats."

[0006] To improve the quality (accuracy) of a model that handles data in an original data format having a combination of multiple attribute values ​​(including class values), it is useful to ensure that a training dataset for the model contains a certain number of training data sets for each combination of multiple attribute values ​​(including class values) and to balance the number of training data sets among the combinations. Alternatively, even if the number of training data sets among combinations of multiple attribute values ​​(including class values) is imbalanced in the training dataset, it is useful to balance the quality (accuracy) provided by the model (trained with the training data) among the combinations to some extent. In other words, it is useful to ensure that the quality (accuracy) of the model is not unacceptably low for data having a specific combination of multiple attribute values ​​(including class values). For example, in a training dataset for training a discriminant model that discriminates the class of image data, simply having enough training data sets for each class value (e.g., a value indicating one of cars, people, animals (e.g., dogs, cats, horses), and non-living things (e.g., balls)) may not result in sufficient class discrimination accuracy for the trained discriminant model. For example, suppose a discriminant model is trained so that it can identify the class "car" when image data containing cars is input. In this scenario, it is useful to have a wide variety of training data, which is image data containing cars, with various attribute values ​​(not class values), such as the color of the car, background objects (e.g., roads, gardens, sky), and background color. If the number of training data is insufficient, the accuracy of class identification by the discriminant model may be low for combinations of attribute values ​​(including class values) for which there is an insufficient amount of training data. For example, when image data showing "a white car driving on a black road" is input to the discriminant model, it may be able to correctly identify the class "car," whereas when image data showing "a black car driving on a white road" is input to the discriminant model, it may not be able to correctly identify the class "car."

[0007] However, when the data handled by a model or the training data used to train the model through machine learning is data in an original data format (e.g., image data or audio data) having a combination of multiple attribute values ​​(including class values), conventionally known techniques for editing training datasets are not suitable. For example, the technique disclosed in Non-Patent Document 1 generates new training data by editing (data augmenting) the image data that is the training data. The technique disclosed in Non-Patent Document 1 attempts to improve the accuracy of determining the class of image data (classifying the image data) using automatic data augmentation (AutoAugmentation). However, the technique disclosed in Non-Patent Document 1 gives scant consideration to attribute values ​​other than the class values ​​possessed by the image data. In other words, the technique disclosed in Non-Patent Document 1 provides a certain amount of training data for each combination of multiple attribute values ​​(including class values), and does not give sufficient consideration to balancing the number of training data between the combinations or balancing the quality (accuracy) provided by the model (trained with the training data) between the combinations. Furthermore, the technology disclosed in Non-Patent Document 2 generates new text data as new training data by editing (data augmenting) text data as training data. In the technology disclosed in Non-Patent Document 2, the data handled by the model and the training data for training the model by machine learning are always text data. In other words, the technology disclosed in Non-Patent Document 2 does not handle data in its original format (a format other than text data), such as image data or audio data. Furthermore, the technology disclosed in Non-Patent Document 2 uses multiple attribute values ​​(including class values) when performing automatic data augmentation (Text AutoAugmentation) of text data.) combinations, there is a certain amount of training data, and there is little consideration given to balancing the amount of training data between the combinations, or balancing the quality (accuracy) provided by the models (trained with the training data) between the combinations.

[0008] In light of the above, one of the objectives of the present disclosure may be to edit the training dataset to improve the quality (accuracy) of the model when the format of the data handled by the model or the format of the training data for training the model by machine learning is the original data format (a format other than text data format).

[0009] In order to achieve at least one of the above objects, the present disclosure may include, for example, the following features. One aspect of the present disclosure is a system. The system includes a text data conversion unit, a policy determination unit, a policy execution unit, and an original data conversion unit. The text data conversion unit converts each piece of original data in an original data format included in an original data set into each piece of original data-corresponding text data in a text data format. The policy determination unit generates a policy to be applied to the original data-corresponding text data set based on an original data-corresponding text data set including each piece of original data-corresponding text data and a policy determination model. The policy execution unit edits the original data-corresponding text data set based on the policy to generate a generation text data set that is a set of generation text data. The original data conversion unit converts each piece of generation text data included in the generation text data set into each piece of generation data in the original data format. The generation data is learning data for training, by machine learning, a discriminant model that discriminates classes of data in the original data format.

[0010] The present disclosure provides a method for generating generated data (set) in an original data format, which serves as training data for training a model through machine learning. The method converts the original data in the original data format into text data (set) corresponding to the original data in text data format, and then performs text data editing on the text data (set) corresponding to the original data. The present disclosure then converts the generated text data (set) in text data format into generated data (set) in the original data format. In other words, the present disclosure performs a substantial editing process on the training data after converting it to text data format, rather than performing the editing process on the data in its original data format. Here, when editing information on combinations of attribute values ​​(including class values) contained in the data, editing the data in text data format makes it easier to achieve intended, flexible, or accurate editing, rather than performing the editing process on the data in its original data format. In other words, the present disclosure provides an ability to perform intended, flexible, or accurate editing on information on combinations of attribute values ​​(including class values) contained in the training data, even if the training data is for a model that handles data in the original data format.

[0011] The present disclosure also generates a strategy for a text dataset corresponding to the original data obtained by converting the original dataset into a text data format. Then, the present disclosure performs text data editing on the text dataset corresponding to the original data based on the strategy. Therefore, the present disclosure generates an appropriate strategy depending on the status of the original dataset (text dataset corresponding to the original data) (e.g., the absolute number of original data (text data corresponding to the original data) for each combination of attribute values ​​(including class values), the balance of the number of original data (text data corresponding to the original data) between the combinations, or the balance of model quality (accuracy) between the combinations), and can generate a generating text dataset (generation dataset) that improves the quality (accuracy) of the model based on the strategy.

[0012] As a result, the present disclosure makes it possible to edit a training dataset to improve the quality (accuracy) of a model when the format of the data handled by the model or the format of the training data for training the model through machine learning is the original data format (a format other than text data format).

[0013] Methods and programs that achieve the same processing as that achieved by the above system can also achieve the same effects as the above system. In the form of a program, costs can be reduced in many cases. Programs also make it easier to make design changes to the processing. Features that the present disclosure may have other than those described above, and the effects corresponding to those features, are disclosed in this specification, claims, or drawings.

[0014] 1 shows a basic functional configuration of an embodiment of the present disclosure. 2 shows data augmentation and data generation taking into account combinations of multiple attribute values. 3 shows the computer architecture of a system. 4 shows a detailed functional configuration of an embodiment of the present disclosure. 5 shows a flowchart for training a policy decision model. 6 shows a continuation of the flowchart for training a policy decision model. 7 shows the input and output of the policy decision model. 8 shows the content of editing operations included in a policy. 9 shows text data loss, original data loss, and policy loss. 10 shows a flowchart for operating a policy decision model. 11 shows a functional configuration related to data count display output control. 12 shows a flowchart for data count display output control. 13 shows a functional configuration related to discrimination accuracy display output control. 14 shows a functional configuration related to discrimination accuracy display output control. 15 shows a functional configuration related to discrimination accuracy display output control. 16 shows a flowchart for discrimination accuracy display output control. 17 shows a functional configuration related to discrimination accuracy display output control. 18 shows a flowchart for discrimination accuracy display output control. 19 shows a functional configuration related to discrimination accuracy display output control. 20 shows a flowchart for discrimination accuracy display output control. 21 shows a flowchart for discrimination accuracy display output control. 22 shows a flowchart for discrimination accuracy display output control. 23 shows a functional configuration related to individual information display output control. 24 shows a flowchart for individual information display output control. 24 shows individual information display output. 25 shows a flowchart for discrimination accuracy display output control. 26 shows a flowchart for discrimination accuracy display output control. 27 shows a flowchart for individual information display output control. 28 shows a flowchart for individual information display output control. 29 shows a flowchart for individual information display output control. 29 shows a flowchart for individual information display output control. 29 shows individual information display output. 20 shows a flowchart for individual information display output control. 21 shows a flowchart for individual information display output control. 29 shows individual information display output. 20 shows a flowchart for individual information display output control. 21 shows an individual information display output control. 22 shows an individual information display output control. 23 shows an

[0015] Embodiments of the present disclosure will be described in detail below with reference to the drawings. Note that the embodiments described below do not limit the disclosure according to the claims, and not all of the elements and combinations thereof described in the embodiments are necessarily essential to the solutions of the present disclosure. The following description and drawings are examples for explaining the present disclosure, and appropriate omissions and simplifications have been made for clarity. The present disclosure can be implemented in various other forms. Unless otherwise specified, each component may be singular or plural. The position, size, shape, range, etc. of each component shown in the drawings may not represent the actual position, size, shape, range, etc., in order to facilitate understanding of the invention. Therefore, the present disclosure is not necessarily limited to the position, size, shape, range, etc., disclosed in the drawings. Each system, device, or functional unit of the present disclosure may be integrated into a single piece of hardware, or may be divided into multiple parts that function in cooperation with each other. Several systems, devices, or functional units may be integrated into hardware. Each system, device, or functional unit may be implemented by executing software (programs) on a computer (as shown in FIG. 3). Some of the functions of a system, device, or functional unit may be implemented in hardware (e.g., hardwired logic or a field programmable gate array (FPGA)), with the remaining functions being implemented by executing software (a program). All of the functions of each system, device, or functional unit may be implemented in hardware. Some or all of the steps shown in the flowcharts and the like described in this disclosure may be implemented in hardware. One or more systems, devices, or functional units of the present disclosure may be implemented using one or more hardware resources. For this purpose, each of the systems, devices, or functional units of the present disclosure may be implemented virtually. For example, a virtual computer or container technique may be used.The program of the present disclosure may be included in a concept that generally encompasses software that constructs a specific information processing system (system) or its operating method according to the intended use through the cooperation of software and hardware resources. In other words, the program of the present disclosure is not limited to a specific type or form of program. Furthermore, the program may be initially recorded in a compressed format. Items using the same reference numerals in multiple drawings are similar to each other. In drawings showing flowcharts, rectangular boxes indicate processing steps, and hexagonal boxes indicate conditional branching steps. In drawings showing flowcharts, "step" is abbreviated as "S." Furthermore, the display or output aspects shown in the drawings are merely examples. Within the spirit of the present disclosure, the display or output aspects may differ from those shown in the drawings.

[0016] 1. Data Generation Taking into Account Combinations of Basic Functional Configuration and Attribute Values ​​(FIG. 1) FIG. 1 illustrates a basic functional configuration 100 (and the information handled) of a system according to an embodiment of the present disclosure. Note that not all functional configurations illustrated in FIG. 1 are required. Furthermore, functional configurations other than those illustrated in FIG. 1 may also exist. The system 101 generates a generated data set 143, which is a set of generated data 144. The generated data 144 is learning data for training a discriminant model 165 through machine learning. The discriminant model 165 discriminates the class of data 148 in the original data format and outputs a class 191 (class value) of the discrimination result. In other words, the discriminant model 165 handles a single-class (single-label) discrimination problem (classification problem) or a multi-class (multi-label) discrimination problem (classification problem). The data 148 in the original data format may be data other than text data, such as image data or audio data. The data 148 in the original data format may include not only one or more class values ​​to be discriminated, but also one or more attribute values ​​other than the class values. For example, image data showing "a black car driving on a white road" may have, as attribute values, a combination of the class value "car" and attribute values ​​other than the class value, such as "black as the color of the car," "road as the background," and "white as the color of the road." The generated data 144 is training data in the original data format. The system 101 uses the original dataset 131, which is a set of original data 132 in the original data format (with training information (label information indicating the correct class)), to obtain the generated dataset 143, which is a set of generated data 144 in the original data format (with training information (label information indicating the correct class)). In other words, the system 101 edits the original dataset 131 (e.g., performs data augmentation or data generation processes) to generate the generated dataset 143.

[0017] The system 101 includes functional units, such as a text data conversion unit 102, a strategy determination unit 105, a strategy execution unit 106, and an original data conversion unit 108. Each of these functional units may be realized by executing a program or may be implemented in hardware. The text data conversion unit 102 converts each piece of original data 132 (data other than text data, such as image data or audio data) in an original data format included in an original data set 131 into each piece of original data-corresponding text data 134 in a text data format. For example, if the original data 132 is image data showing "a black car driving on a white road," the text data conversion unit 102 may convert the text data format "a black car driving on a white road" into the original data-corresponding text data 134. The text data conversion unit 102 may use, for example, a text data conversion model trained by machine learning to convert data in the original data format into data in the text data format. This text data conversion model may be capable of associating latent variables contained in data in the original data format with text data describing the data as a result of machine learning training. The policy determination unit 105 generates a policy 162 to be applied to the original data-corresponding text dataset 133 based on an original data-corresponding text dataset 133 consisting of each piece of original data-corresponding text data 134 and a policy determination model 161. Here, the policy 162 refers to a policy for editing the original data-corresponding text dataset 133 in the text data format. The policy 162 may include one or more editing operations. Each editing operation may have the content shown in FIG. 8 (described later). As shown in FIG. 7 (described later), the policy determination unit 105 may input a vector dataset 139 derived from some or all of the original data-corresponding text data 134 included in the original data-corresponding text dataset 133 to the policy determination model 161. As described above, the policy decision unit 105 generates the policy 162. The policy 162 is generated based on the situation (for example, attribute values ​​(including class values)) of the original data set 131 (text data set 133 corresponding to the original data).The strategy execution unit 106 edits the original data-corresponding text dataset 133 based on the strategy 162 to generate the generating text dataset 141. For example, if the strategy 162 consists of one or more editing operations, the strategy execution unit 106 applies each of the editing operations included in the strategy 162 to the original data-corresponding text dataset 133, thereby performing editing in the text data format (text data editing). In this way, to obtain the generating dataset 143 (training dataset) from the original dataset 131, editing is not performed in the original data format, but the strategy execution unit 106 performs editing in the text data format (text data editing). Therefore, when editing combinations of attribute values ​​(including class values) possessed by the training data (e.g., changing class values ​​or attribute values), it is easy to achieve intended editing, such as correcting imbalances, flexible editing, or accurate editing. The original data conversion unit 108 converts each piece of generation text data 142 included in the generation text dataset 141 into each piece of generation data 144 in the original data format. For example, if the generation text data 142 is data in the text data format of "A white dog running on a black road," the original data conversion unit 108 may generate image data representing "A white dog running on a black road" and use the image data as the generated data 144. Note that the original data conversion unit 108 may use, for example, an original data conversion model trained by machine learning to generate data in the original data format based on data in the text data format. Furthermore, as will be described later, if the content of the editing operation included in the strategy 162 instructs the generation of multiple pieces of generated data 144 for a certain piece of generated text data 142, the original data conversion unit 108 may generate multiple pieces of image data that are different from each other but have the content indicated by the certain piece of generated text data 142.The set of generated data 144 generated in the above manner becomes the generated data set 143. Compared to the situation regarding the original data set 131, the situation regarding the generated data set 143 (for example, the absolute number of generated data 144 in the original data format for each combination of attribute values ​​(including class values), the balance of the number of generated data 144 between the combinations, or the balance of the quality (accuracy) of the discriminant model 165 between the combinations) is improved.

[0018] FIG. 2 illustrates a process 200 in which an original data set 131 is edited (e.g., data augmented or generated) to generate a generated data set 143. The editing takes into account the combinations of multiple attribute values ​​(including class values) possessed by the data. The upper left corner of FIG. 2 illustrates a distribution 201 of the number of original data items contained in the original data set 131. Here, each piece of original data 132 has a first attribute value (which may be a class value) of any one of A, B, C, or D, and a second attribute value (which may be a class value) of any one of a, b, or c. (Note that each piece of original data 132 may also have a third attribute value, a fourth attribute value, and so on.) In the example of the distribution 201 of the number of original data items shown in the upper left corner of FIG. 2, a cylindrical graph shows the number of original data items having each combination of a first attribute value and a second attribute value. (Hereinafter, a set of data for each combination of attribute values ​​(including class values) may be referred to as a "subset.") In the example of the distribution 201 of the number of original data pieces shown in the upper left of Figure 2, there is no original data 132 in which the first attribute value is D and the second attribute value is a, and there is no original data 132 in which the first attribute value is D and the second attribute value is c. On the other hand, there is a relatively large number of original data 132 in which the first attribute value is A and the second attribute value is a. As such, in the original dataset 131, there may be an imbalance in the number of original data 132 for each subset. The upper right of Figure 2 shows the discrimination accuracy 202 of the discriminant model 165 (or 166 (see Figure 4)) after being trained by machine learning using the original dataset 131 as learning data. The discrimination accuracy of the discriminant model 165 (or 166) here may indicate the degree of agreement between the class 191 of the data 148 in the original data format discriminated by the discriminant model 165 (or 166) and the correct class (e.g., the class indicated by the teacher information (label)) of the data 148 in the original data format. In the upper right part of Figure 2, a cylindrical graph shows the discrimination accuracy when the discriminant model 165 (or 166) discriminates the class of the verification data 146 included in each subset of the verification data 146 in the original data format included in the verification dataset 145 described below.As shown in the upper left of FIG. 2 , if there is no original data 132 in which the first attribute value is D and the second attribute value is a, and no original data 132 in which the first attribute value is D and the second attribute value is c, the discrimination accuracy in the subset in which the first attribute value is D and the second attribute value is a, and the discrimination accuracy in the subset in which the first attribute value is D and the second attribute value is c, tend to be low, as shown in the upper right of FIG. 2 . The lower left of FIG. 2 shows a distribution 211 of the number of generated data items included in the generated dataset 143. Like the distribution 201 of the number of original data items, the distribution 211 of the number of generated data items is also represented by a cylindrical graph for each subset. It is shown that editing (e.g., data augmentation / data generation) has eliminated the imbalance in the number of generated data items between the subsets in the generated dataset 143. The lower right of FIG. 2 shows the discrimination accuracy 212 of the discriminant model 165 (or 166) after being trained by machine learning using the generated dataset 143 as training data. Similar to the discrimination accuracy 202 in the upper right portion of Fig. 2 , the discrimination accuracy 212 in the lower right portion of Fig. 2 is also represented by a cylindrical graph for each subset. In the example of Fig. 2 , the imbalance between subsets in the discrimination accuracy 212 is corrected in response to the correction of the imbalance between subsets in the distribution 211 of the number of generated data items. It can be said that the policy 162 is generated so as to achieve such a correction of the imbalance. Note that Fig. 2 illustrates a state in which both the correction of the imbalance between subsets in the distribution 211 of the number of generated data items and the correction of the imbalance between subsets in the discrimination accuracy 212 are achieved. Depending on the data format of the original data handled by the discriminant model 165, it may be possible that the correction of the imbalance between subsets in the distribution 211 of the number of generated data items is not sufficiently achieved, but the correction of the imbalance between subsets in the discrimination accuracy 212 is achieved.

[0019] The system 101 according to the embodiment of the present disclosure has the above-described functional configuration, and therefore can provide the effects described in the above-described "Effects of the Invention." Furthermore, the system 101 can widen the scope of automation of processes leading up to the generation of the generated data 144.

[0020] 2. Computer Architecture for Realizing an Embodiment of the Present Disclosure (FIG. 3) FIG. 3 shows a computer architecture 300 for realizing the system 101 according to an embodiment of the present disclosure. To realize the system 101, some or all of the following may be interconnected via an interconnection unit 311: an information processing device 301, a storage device 302, a non-volatile storage medium (storage device) 303, an external storage medium drive 304, an input device 306, a display or output device 307, a communication device 308, an external input / output port 309, and a reading device 310. (Note that some or all of the interconnection unit 311 may be a network. In this case, the system 101 is realized by multiple devices connected via the network.) The information processing device 301 may be, for example, a processor. Examples of such a processor include a CPU, an MPU, or a GPU. Alternatively, the processor referred to here may be another semiconductor device that executes a predetermined process. The information processing device 301 may also be one or more (micro)processors. The storage device 302 may be, for example, a memory. The non-volatile recording medium (recording device) 303 may be, for example, a non-volatile memory (e.g., flash memory) or a non-volatile disk device. The external recording medium drive 304 may be, for example, a disk drive. The input device 306 may be, for example, a mouse, keyboard, imaging device, sensor, touch panel, or pointing device. The display or output device 307 may be, for example, a display, printer, or speaker. The communication device 308 may be, for example, a communication device for wired communication or a communication device for wireless communication. The communication device 308 may be a network interface device (NIC) that controls communication with other systems, devices, terminals, or servers according to a predetermined protocol. The interconnection unit 311 may be, for example, a bus or a crossbar switch. (As described above, part or all of the interconnection unit 311 may be a network.)

[0021] The non-volatile recording medium (recording device) 303 may record various programs included in the program group 331 (for example, programs for realizing the functional configuration related to the present disclosure; for example, various programs for implementing each of the functional units realized in the system 101), various data groups included in the learning-related data group 332, various data groups included in the internal handling data group 333, or various information 334. The program group 331 includes a text data conversion program 102P, a word / part-of-speech extraction program 103, a vector conversion unit 104, a policy decision unit 105, a policy execution unit 106, a text data loss calculation unit 107, an original data conversion unit 108, a discriminant model training unit 109, an original data loss calculation unit 110, a policy decision model training unit 111, a discriminant model operation unit 112, a data number display output control unit 113, a discrimination accuracy display output control unit 114, an original data display output control unit 115, a text data display output control unit 116, and an attribute value display output control unit 117, which are functional units. The program may include a program 103P, a vector conversion program 104P, a policy decision program 105P, a policy execution program 106P, a text data loss calculation program 107P, an original data conversion program 108P, a discriminant model training program 109P, an original data loss calculation program 110P, a policy decision model training program 111P, a discriminant model operation program 112P, a data number display output control program 113P, a discrimination accuracy display output program 114P, an original data display output control program 115P, a text data display output control program 116P, or an attribute display output control program 117P. Note that some of the above programs may be integrated into a single program. Also, any of the above programs may be divided into multiple programs. The learning-related data group 332 may include the original dataset 131, the original data corresponding text dataset 133, the word / part-of-speech data group 135, the strategy determination text dataset 137, the vector dataset 139, the generation text dataset 141, the generation dataset 143, or the verification dataset 145.The internally handled data group 333 may include policy decision model information 161i which is information (which may include model parameters) expressing the policy decision model 161, the policy 162, text data evaluation model information 163i which is information (which may include model parameters) expressing the text data evaluation model 163, text data loss 164, discriminant model information 165i which is information (which may include model parameters) expressing the discriminant model 165, original data loss 167, or policy loss 168. Alternatively, some or all of the various programs included in the program group 331, the various data groups included in the learning-related data group 332, the various data groups included in the internally handled data group 333, or the information of the various information 334 may be acquired from outside the configuration shown in FIG.

[0022] The external recording medium drive 304 can be connected to an external recording medium 305. The external recording medium 305 may be, for example, a portable recording disk (such as a DVD), an IC card, an SD card, a nonvolatile memory (such as a flash memory), or a portable hard disk. Note that various programs included in the program group 331, various data included in the learning-related data group 332, various data included in the internally handled data group 333, or information similar to the information in the various information 334 may be transferred and stored from the external recording medium 305 to the nonvolatile recording medium (recording device) 303 or the storage device 302. The external recording medium 305 may be used to record programs and data handled in the system 101. The external recording medium drive 304 and the external recording medium 305 may be connected to the system 101 shown in FIG. 3 via a network. The various programs included in the program group 331, the various data included in the learning-related data group 332, the various data included in the internal handling data group 333, or the various information 334 may be brought via the communication device 308, the external input / output port 309, the input device 306, or the reading device 310, and recorded or stored in the non-volatile recording medium (recording device) 303 or the memory device 302.

[0023] In order for the architecture of FIG. 3 to function as the system 101, each functional unit within the system 101, or a portion of each functional unit (to execute one or a series of processes (steps)), various programs included in the program group 331 may be loaded into the storage device 302 (e.g., from the non-volatile recording medium (storage device) 303). The loaded program is indicated by 321 in FIG. 3. The information processing device 301 may then execute the program 321 (using, as necessary, various data included in the learning-related data group 332 stored in the non-volatile recording medium (storage device) 303, various data included in the internally handled data group 333, or information in the various information 334). Execution of the program 321 realizes the function of the system 101, each functional unit within the system 101, or a portion of each functional unit (to execute one or a series of processes (steps)). At this time, various buffers 323 temporarily formed in the storage device 302 may also be used as appropriate.

[0024] 3. Processing Performed by an Embodiment of the Present Disclosure Processing performed by the system 101, which is an embodiment of the present disclosure, will now be described. Note that it is not necessary to realize all of the functional configurations described below and perform all of the processing. Furthermore, it is not prohibited to realize functional configurations and perform processing other than the functional configurations and processing described below. Below, processing performed by the system 101 when training the policy decision model 161 by machine learning will be described, mainly with reference to FIGS. 4 to 9 . Furthermore, processing performed by the system 101 when operating the trained policy decision model 161 will be described, mainly with reference to FIG. 10 . In addition, with reference to FIGS. 11 to 20 , display output control processing that the system 101 can perform will be described in association with the generation of the generated dataset 143 based on the original dataset 131.

[0025] 3-1. Processing During Policy Decision Model Training (FIGS. 4-9) Figure 4 shows a functional configuration (detailed functional configuration) 400 (and information handled) related to the processing executed by the system 101 when training the policy decision model 161 through machine learning. Compared to Figure 1, which shows an overview of the functional configuration when operating the trained policy decision model 161, Figure 4 additionally shows functional configurations, etc. used to train the policy decision model 161 (e.g., text data loss calculation unit 107, text data evaluation model 163, text data loss 164, validation dataset 145 (validation data 146), original data loss calculation unit 110, original data loss 167, policy decision model training unit 111, policy loss 168), and also shows functional configurations, etc. that were omitted from Figure 1 (e.g., word / part-of-speech extraction unit 103, vector conversion unit 104, word / part-of-speech data group 135, vector dataset 139, discriminant model training unit 109). In Fig. 4, numbers surrounded by circles are connected to each other. Figs. 5 and 6 show flowcharts of the process executed by the system 101 when training the policy decision model 161 by machine learning. In Figs. 5 and 6, letters (E or F) surrounded by circles are connected to each other. Below, each of the process steps included in the flowcharts of Figs. 5 and 6 will be described with appropriate reference to Figs. 4, 7, 8, and 9.

[0026] In step 501 (text data conversion step) of FIG. 5 , the text data conversion unit 102 converts each of the original data 132 in the original data format contained in the original data set 131 into each of the original data-corresponding text data 134 in the text data format. Details of the text data conversion unit 102 have already been described with reference to FIG. 1 . A collection of the original data-corresponding text data 134 forms the original data-corresponding text data set 133. Note that the upper part of FIG. 7 shows how each of the original data-corresponding text data 134 (original data-corresponding text data (1), original data-corresponding text data (2), etc.) contained in the original data-corresponding text data set 133 is generated corresponding to each of the original data 132 (original data (1), original data (2), etc.) contained in the original data set 131. In step 502 of FIG. 5 , the word / part-of-speech extraction unit 103 analyzes each of the original data-corresponding text data 134 in the text data format into parts of speech, for example, by morphological analysis. The word and part-of-speech extraction unit 103 saves each of the parts of speech (for example, nouns, verbs, adjectives, particles, etc.) after the decomposition as a word and part-of-speech data group 135. Note that words and parts of speech obtained by decomposing text data other than the original data corresponding text data 134 into parts of speech may be added to the word and part-of-speech data group 135. The word and part-of-speech data group 135 is formed from each piece of the original data corresponding text data 134, and by utilizing the word and part-of-speech data group 135 in the text data editing executed by the strategy execution unit 106, it is possible to increase the likelihood that the content of the generation text data 142 obtained by the text data editing will be appropriate.

[0027] In step 503 of Fig. 5 , the vector conversion unit 104 sets all or a randomly sampled portion of the original data corresponding text data 134 included in the original data corresponding text data set 133 as each piece of text data for policy decision. A collection of the text data for policy decision forms a text data set 137 for policy decision. Note that the upper and middle parts of Fig. 7 show how all or a randomly sampled portion of the original data corresponding text data (original data corresponding text data (1), original data corresponding text data (2), ...) are set as each piece of text data for policy decision (text data for policy decision (i), text data for policy decision (ii), ...). The text data set 137 for policy decision is used to form a vector data set 139 to be input to the policy decision model 161. When the policy decision model 161 determines the text data editing policy 162 to be applied to the original data corresponding text dataset 133, the policy decision model 161 does not necessarily need to know all the information about the original data corresponding text dataset 133. In some cases, the policy decision model 161 may only need to know a certain amount of information about the original data corresponding text dataset 133 (for example, the imbalance between subsets (formed by combinations of attribute values ​​(including class values)) in the set of original data corresponding text data 134 included in the original data corresponding text dataset 133). Therefore, the policy decision text dataset 137 may be formed by random sampling as described above. This can reduce the amount of information that needs to be handled when implementing the policy decision model 161. In step 504 of FIG. 5 , the vector conversion unit 104 converts each piece of policy decision text data included in the policy decision text dataset 137 into vector data that can be handled by the policy decision model 161. The vector dataset 139 is formed from the set of vector data.The middle to bottom of Figure 7 shows how vector data (vector data (i), vector data (ii), etc.) is generated corresponding to each of the text data for policy decision-making (text data (i), text data (ii), etc.).

[0028] In step 505 (policy determination step) of FIG. 5 , the policy determination unit 105 inputs each of the vector data included in the vector dataset 139 to the policy determination model 161. The policy determination unit 105 obtains, as an output of the policy determination model 161, a text data editing policy 162 to be applied to the original data-corresponding text dataset 133. As shown in the lower part of FIG. 7 , the policy determination model 161 can receive multiple vector data included in the vector dataset 139 as input. Because the policy determination model 161 is able to receive such input, the policy determination model 161 can know the status of the original data-corresponding text dataset 133, which is the basis of the vector dataset 139 (e.g., the imbalance status between subsets (formed by combinations of attribute values ​​(including class values)) in the set of original data-corresponding text data 134 included in the original data-corresponding text dataset 133). In other words, the policy determination model 161 (after sufficient training) can generate a policy 162 appropriate for the status of the original data-corresponding text dataset 133.

[0029] The strategy 162 output by the strategy decision model 161 may include one or more text data editing operations (editing operations) to be applied to the original data-corresponding text dataset 133. When the strategy 162 includes multiple editing operations, each of the editing operations may be applied to the same original data-corresponding text dataset 133. FIG. 8 shows an example of the content of the editing operations included in the strategy 162. Each editing operation may have one or more of, for example, a "type of text data editing," "designation of original data-corresponding text data to be edited," "number of generation text data to be generated from one original data-corresponding text data," and "number of generation text data to be generated from one generation text data." The format of the information output from the strategy decision model 161 (the format of the strategy 162) may be predetermined so that the strategy 162 including the editing operations having the content shown in FIG. 8 can be generated.

[0030] The information on the "type of text data editing" may indicate one or more of "delete," "replace," "insert," and "replace." "Delete" may mean deleting Na arbitrary words (e.g., nouns, adjectives, particles, etc.) from the original data-related text data 134 to be edited. For example, if Na=2 and the original data-related text data 134 to be edited is "a big white cat in a green garden," the text data (generation text data 142) after text data editing may be "a cat in a green garden." In other words, the two words "white" and "big" have been deleted. Note that words indicating classes (class values) included in the original data-related text data 134 to be edited may be specified not to be subject to "delete." For example, if the original data-related text data 134 to be edited is "a big white cat in a green garden," even if "delete" is specified as the editing operation, the class value "cat" may not be deleted. To achieve this, as will be described later (e.g., in FIG. 9 ), the calculation of the policy loss 168 may include a prohibition policy penalty 941 for imposing a penalty on a policy 162 that includes a content that deletes a word that indicates a class. Alternatively, when the policy 162 output by the policy decision model 161 includes a content that deletes a word that indicates a class, a policy correction unit may be provided that corrects the policy 162 so that it does not include a content that deletes a word that indicates a class. Alternatively, when the policy execution unit 106 generates the generation text data 142, the generation text data 142 from which a word that indicates a class has been deleted (e.g., as a result of "random deletion" that randomly deletes words from the source data-corresponding text data 134) may be deleted from the generation text dataset 141. Alternatively, when the policy execution unit 106 generates the generating text data 142, if the generating text data 142 has words indicating classes deleted (for example, as a result of "random deletion"), the generating text data 142 may be restored to the state before the deletion and used as the generating text data 142 again.As described above, by ensuring that words indicating classes (class values) are not deleted from the text data (generation text data 142) after text data editing, it is possible to prevent the generation of inappropriate learning data (learning data that does not correspond to any class) for training the discriminant model 165 for class discrimination through machine learning. "Swapping" may involve swapping any word (e.g., noun, adjective, particle, etc.) in the original data-corresponding text data 134 to be edited Nb times. For example, if Nb=1 and the original data-corresponding text data 134 to be edited is "A white dog and a brown cat in a green garden," the text data after text data editing (generation text data 142) may be "A brown dog and a white cat in a green garden." In other words, here, the two words "brown" and "white" are swapped with each other once. Note that, to ensure the validity of the edited text data, words of the same part of speech may be swapped. "Swapping" may allow the swapping of particles in Japanese text data, and may allow the swapping of prepositions in English text data. "Insertion" may refer to the insertion of Nc words (e.g., nouns, adjectives, particles, etc.) into the original data-corresponding text data 134 to be edited. For example, if Nc=4 and the original data-corresponding text data 134 to be edited is "a white dog in a green garden," the text data (generation text data 142) after text data editing may be "a white dog, a brown cat, and a running person in a green garden." In other words, four words, "brown," "cat," "run," and "person," are inserted. Note that the number of inserted words Nc in "insertion" does not need to include the number of particles. In the above example, the particle "and" is not included in Nc. "Replacement" may refer to the replacement of Nd words (e.g., nouns, adjectives, particles, etc.) included in the original data-corresponding text data 134 to be edited with other words. For example, if Nd=2 and the original data corresponding text data 134 to be edited is "a black car running on a white road," the text data after editing (generated text data 142) may be "a black horse running on a blue road."That is, in this example, the word "white" is replaced with a word of the same part of speech, "blue," and the word "car" is replaced with a word of the same part of speech, "horse." In this example, the word indicating the class value is changed from "car" to "horse." The words used for word replacement may be obtained from the word and part-of-speech data group 135. Furthermore, the replacement words may be selected from the word and part-of-speech data group 135 so that the parts of speech before and after the replacement are the same. By using the word and part-of-speech data group 135 (derived from the original data corresponding text dataset 133) as the information source for words for "replacement," it is expected that the content of the text data (generation text data 142) after text data editing will be appropriate.

[0031] The information "designation of original data-corresponding text data to be edited" may specify which of the original data-corresponding text data 134 included in the original data-corresponding text dataset 133 is to be the target of text data editing. When all of the original data-corresponding text data 134 included in the original data-corresponding text dataset 133 have been selected as text data for policy decision making, the "designation of original data-corresponding text data to be edited" may include a specifier Ne. The specifier Ne may directly indicate the index of the original data-corresponding text data 134 to be edited. For example, when Ne = 2, the original data-corresponding text data 134 with an index of 2 may be the target of text data editing. Alternatively, Ne may be a numerical value that indirectly designates the original data-corresponding text data 134 to be edited. For example, when the original data-corresponding text dataset 133 includes 100 pieces of original data-corresponding text data 134 and Ne = 0.2, the original data-corresponding text data 134 with an index of 20, which is the product of the multiplication result of 100 and 0.2, may be the target of text data editing. (Note that if the above product is not an integer, decimal points may be rounded down, rounded up, rounded off, or otherwise processed as appropriate.) In this way, when all of the original data-corresponding text data 134 included in the original data-corresponding text dataset 133 are selected as text data for strategy determination, the editing operation included in the strategy 162 can appropriately specify the specific original data-corresponding text data 134 to be edited. Note that instead of the above-mentioned technique of "specifying the original data-corresponding text data to be edited," the strategy 162 may include one editing operation for each of the original data-corresponding text data 134 included in the original data-corresponding text dataset 133. The editing operation may then include flag information indicating whether the original data-corresponding text data 134 associated with the editing operation is to be edited.When the text data for determining a measure is extracted by random sampling from the original data-corresponding text data 134 included in the original data-corresponding text dataset 133, the "designation of the original data-corresponding text data to be edited" may include specifiers K, k, and Nf. The combination of K and k specifies a combination of attribute values ​​(including class values). (If the number of attribute values ​​included in the combination is three or more, the "designation of the original data-corresponding text data to be edited" may further include a specifier for specifying an attribute value.) Nf specifies the number of original data-corresponding text data 134 having the specified combination of attribute values ​​(including class values). For example, when K=B, k=c, and Nf=2 are specified, two original data-corresponding text data 134 having the attribute value combination of B and c are targeted for text data editing. In this way, even when the measure 162 is determined using random sampling, the original data-corresponding text data 134 to be targeted for text data editing can be appropriately selected from the original data-corresponding text dataset 133.

[0032] The information on "the number of pieces of text data for generation generated from one piece of text data corresponding to original data" specifies the number Ng of pieces of text data (text data for generation 142) after text data editing that is generated by text data editing based on one piece of text data corresponding to original data 134 that is the target of text data editing. For example, if Ng=2 (Nd=1) and the original data-related text data 134 to be edited is "a black car driving on a white road," the text data after text data editing (text data for generation 142) may be two pieces of text data: "a green car driving on a white road" and "a blue car driving on a white road." (In this example, the text data "black car driving on a white road," which was the text data before the text data editing, does not need to be included in the generating text data set 141.) Also, for example, if Ng=0 and the original data-corresponding text data 134 to be edited is "black car driving on a white road," the original data-corresponding text data 134 "black car driving on a white road" that exists in the original data-corresponding text data set 133 may be made not to exist in the generating text data set 141, and alternative generating text data 142 may also not exist. Because the editing operation can specify the "number of generating text data to be generated from one piece of original data-corresponding text data," in the text data (generating text data 142) after text data editing, it is possible to adjust the absolute number of generating text data 142 (generating data 144) for each subset (based on a combination of attribute values ​​(including class values)) and correct imbalances in the number of generating text data 142 (generating data 144) between subsets. Furthermore, since the above-mentioned absolute number can be adjusted to be reduced, it is possible to reduce the number of pieces of training data (generated data 144) for training the discriminant model 165 by machine learning within an acceptable range for the accuracy of the discriminant model 165. When the number of pieces of training data is reduced, the resources (storage area, computing resources, and computing time) required for training the model can also be reduced.When the "number of generation text data generated from one piece of text data corresponding to original data" or the "number of generation data generated from one piece of generation text data" described below is specified by an editing operation, the absolute number of generation text data 142 (generation data 144) for each subset (based on a combination of attribute values ​​(including class values)) is adjusted efficiently, and imbalances in the number of generation text data 142 (generation data 144) between subsets are corrected efficiently, so that training by machine learning of the discriminant model 165 using the generation data 144 as learning data can be performed efficiently. As training of the discriminant model 165 becomes more efficient, training of the policy decision model 161 also becomes more efficient.

[0033] The information on the "number of generation data sets generated from one piece of text data for generation" specifies the number Nh of generation data sets generated by the original data conversion unit 108 from one piece of text data for generation 142 generated by editing the text data indicated by the editing operation. For example, if Nh = 3 and the text data (text data for generation 142) after text data editing is "A white dog running on a black road," the original data conversion unit 108 may generate three pieces of image data (generated data 144) representing "A white dog running on a black road." Also, for example, if Nh = 0 and the text data (text data for generation 142) after text data editing is "A white dog running on a black road," the original data conversion unit 108 does not need to generate image data (generated data 144) representing "A white dog running on a black road." The effect of being able to specify the "number of text data sets for generation generated from one piece of text data corresponding to original data" by an editing operation is similar to the effect of being able to specify the "number of text data sets for generation generated from one piece of text data corresponding to original data," as already described. In one editing operation, either "the number of text data for generation generated from one text data corresponding to original data" or "the number of generation data generated from one text data for generation" may be specified, or both may be specified. Furthermore, if neither is specified, it may be assumed that Ng=1 and Nh=1 are implicitly specified.

[0034] 5 , the vector dataset 139 derived from the original data corresponding text dataset 133 is input to the policy decision model 161, and the word and part-of-speech data group 135 (or a vector data group obtained by vector-converting each of the word and part-of-speech data included in the word and part-of-speech data group 135) may also be input to the policy decision model 161. By inputting information from the word and part-of-speech data group 135 to the policy decision model 161, it is expected that the validity of the content of the policy 162 will improve.

[0035] In step 506 (strategy execution step) of Fig. 5, the strategy execution unit 106 generates a generation text dataset 141 by editing the original data corresponding text dataset 133 based on (one or more editing operations included in) the strategy 162. The strategy execution unit 106 has already been described using Fig. 1. Additionally, the strategy execution unit 106 may also use the word and part-of-speech data group 135. The word and part-of-speech data group 135 can be a source of words after replacement when "replacement" is performed as text data editing, for example.

[0036] Steps 501 to 506 in FIG. 5 (and step 509 in FIG. 5 and step 601 in FIG. 6) described above are executed in the same way when training the policy decision model 161 by machine learning and when operating the trained policy decision model 161. (A flowchart of the processing when operating the trained policy decision model 161 is shown in FIG. 10. Steps 1001 to 1006 in FIG. 10 have the same content as steps 501 to 506 in FIG. 5.) In contrast, steps 507 and 508 in FIG. 5 and steps 602 to 607 in FIG. 6, which will be described below, are executed to train the policy decision model 161 by machine learning.

[0037] In step 507 of FIG. 5 , the text data loss calculation unit 107 inputs each piece of generation text data 142 included in the generation text dataset 141 (after vector conversion into vector data, if necessary) to the text data evaluation model 163. The text data evaluation model 163 may output an evaluation value indicating the validity of the content of the input text data (e.g., the appropriateness of the text data, the naturalness of the text data). For example, a high evaluation value may be assigned to text data titled "people walking," and a low evaluation value may be assigned to text data titled "walking car." Note that the text data evaluation model 163 may be a model trained by machine learning using a group of human-created text data as training data. In step 508 of FIG. 5 , the text data loss calculation unit 107 calculates the text data loss 164 based on the evaluation value output by the text data evaluation model 163 for each piece of generation text data 142. In general, the higher the evaluation value (the higher the validity as text data), the lower the text data loss 164. By calculating the text data loss 164 and reflecting the text data loss 164 in the policy loss 168 described below, the perspective of the validity of the content of the generation text data 142 based on the policy 162 can be reflected in the training of the policy decision model 161. If the content of the generation text data 142 is valid, the generation data 144 is likely to be valid as well. An example of the text data loss 164 calculated by the text data loss calculation unit 107 is shown in the upper part of FIG. 9. The text data loss calculation unit 107 may calculate an overall text data loss 911, a subset text data loss 912 for each subset (based on a combination of attribute values ​​(including class values)), and a maximum value 913 and a variance value 914 for the set of subset text data losses 912. The overall text data loss 911 may reflect the evaluation values ​​of all the generation text data 142 included in the generation text dataset 141. If the evaluation value is high (the validity as text data is high), it is reasonable that the text data loss will be low.Therefore, the total text data loss 911 (J_text_all) may be calculated, for example, by the following linear combination formula for the evaluation value (S_text(i)) of each piece of generation text data 142 (where i is the index): J_text_all := Σ(-S_text(i)) (Σ is the sum for all i) Note that the formula for calculating the total text data loss 911 is not limited to the linear combination formula above, and may be any formula as long as it reflects the evaluation values ​​of all generation text data 142 included in the generation text dataset 141 and is set so that the higher the evaluation value (the higher the validity as text data), the lower the text data loss. By calculating the total text data loss 911 (J_text_all) and reflecting this total text data loss 911 (J_text_all) in the policy loss 168 (J_GT) described below, the perspective of the validity of the content of the entire set of generation text data 142 based on the policy 162 can be reflected in the training of the policy decision model 161. In other words, the policy decision model 161 can be trained so that the content of the generation text data 142 included in the generation text dataset 141 is valid overall. The subset text data loss 912 is calculated for each subset based on a combination of attribute values ​​(including class values). The subset text data loss 912 for a certain subset may reflect the evaluation value of the generation text data 142 included in the certain subset. Here, if the evaluation value is high (the validity as text data is high), it is appropriate that the text data loss be low. Therefore, the subset text data loss 912 (J_text_part(j)) of a certain subset (index j) may be calculated, for example, using the following linear combination formula for the evaluation value (S_text(i)) of each piece of generation text data 142 (index i) included in the certain subset (j).J_text_part(j) := Σ(-S_text(i)) (Σ is the sum for i included in subset j) Note that the formula for calculating the subset text data loss 912 is not limited to the above linear combination formula, and any formula may be used as long as it reflects the evaluation value of the generation text data 142 included in the subset (j) of the generation text data set 141, and the higher the evaluation value (the higher the validity as text data), the lower the text data loss. The maximum value 913 (max_J_text_part) of the set of subset text data losses 912 (J_text_part(j)) may indicate the maximum value of the subset text data losses 912 (J_text_part(j)) for each subset (j). The maximum value 913 (max_J_text_part) is calculated, and this maximum value 913 (max_J_text_part) is reflected in the policy loss 168 (J_GT) described below, thereby training the policy decision model 161 so as to prevent generation text data 142 having a specific combination of attribute values ​​(including class values) (included in a specific subset) from specifically becoming text data with low content validity. The variance value 914 (var_J_text_part) of the set of subset text data losses 912 (J_text_part(j)) may indicate the variance value of the set of subset text data losses 912 (J_text_part(j)) for each subset(j). The variance value 914 (var_J_text_part) is calculated, and this variance value 914 (var_J_text_part) is reflected in the policy loss 168 (J_GT) described below, so that the policy decision model 161 can be trained to prevent excessive variation in the degree of validity of the content of the generation text data 142 for each combination (subset) of attribute values ​​(including class values).

[0038] In step 509 (original data conversion step) of FIG. 5 , the original data conversion unit 108 converts each piece of generation text data 142 included in the generation text data set 141 into each piece of generation data 144 in the original data format. Details of the original data conversion unit 108 have already been described using FIG. 1 . Additionally, if the editing operation included in the strategy 162 includes information on the "number of generation data sets generated from one generation text data set," the information on the "number of generation data sets generated from one generation text data set" is also input to the original data conversion unit 108. For the generation text data 142 to which the input information on the "number of generation data sets generated from one generation text data set" applies, the original data conversion unit 108 may generate the number of generation data sets 144 in the original data format indicated by the information on the "number of generation data sets generated from one generation text data set." The set of generation data sets 144 forms the generation data set 143. After step 509, control transitions to step 601 of FIG. 6 .

[0039] In step 601 of FIG. 6 , the discriminant model training unit 109 trains the discriminant model 165 by machine learning using each piece of generated data 144 included in the generated data set 143 as learning data. For example, when supervised learning is implemented, the discriminant model training unit 109 inputs the generated data 144 to the discriminant model 165 and outputs the generated data 144 as a class 191 of the discrimination result (which may be information indicating the probability of each class, in some cases). The discriminant model training unit 109 compares the information on the class 191 of the discrimination result, which is the output of the discriminant model 165, with supervised information about the generated data 144 (label information indicating the correct class of the generated data 144), to obtain a loss related to discrimination (for one piece of generated data 144). The discriminant model training unit 109 trains the discriminant model 165 (for example, updates the model parameters of the discriminant model 165) to eliminate the loss (groups) related to discrimination in one or more pieces of generated data 144. Here, the training results of the discriminant model 165 (for example, the contents of the update of the model parameters of the discriminant model 165) may be recorded in a non-volatile recording medium (recording device) 303 (or the storage device 302) as discriminant model information 165i shown in Fig. 3. Once the discriminant model 165 has been trained by machine learning using each of the generation data 144 included in the generation dataset 143 and the discriminant model 165 has become trained (shown as a trained discriminant model 166 in Fig. 4), control transitions to step 602 to check the discrimination accuracy of the trained discriminant model 166.

[0040] In step 602 of FIG. 6 , the original data loss calculation unit 110 inputs each piece of verification data 146 in the original data format included in the verification dataset 145 to a trained discriminant model 166, and outputs the trained discriminant model 166 as a class 191 of the discrimination result for each piece of verification data 146 (which may be information indicating the probability for each class, in some cases). In step 603 of FIG. 6 , the original data loss calculation unit 110 calculates the original data loss 167 based on the information on the class 191 of the discrimination result for each piece of verification data 146, which is the output of the trained discriminant model 166. For example, for each piece of verification data 146, the original data loss calculation unit 110 compares the information on the class 191 of the discrimination result with the teacher information for the verification data 146 (label information indicating the correct class of the verification data 146), obtains a loss related to discrimination (for one piece of verification data 146), and calculates the original data loss 167 based on the loss related to discrimination. By calculating the original data loss 167 and reflecting the original data loss 167 in the policy loss 168 described below, the training of the policy decision model 161 can reflect the perspective of the validity of the discrimination accuracy of the discriminant model 165 trained using the generated data 144 based on the policy 162. An example of the original data loss 167 calculated by the original data loss calculation unit 110 is shown in the middle part of Figure 9. The original data loss calculation unit 110 may calculate an overall original data loss 921, a subset original data loss 922 for each subset (based on a combination of attribute values ​​(including class values)), and a maximum value 923 and a variance value 924 in the set of subset original data losses 922. The overall original data loss 921 may reflect the loss related to discrimination in all validation data 146 included in the validation dataset 145. The total original data loss 921 (J_ori_all) may be calculated, for example, by the following linear combination formula for the loss (L_ori(m)) related to discrimination of each piece of verification data 146 (where index is m):J_ori_all := Σ(L_ori(m)) (Σ is the sum over all m) The formula for calculating the overall original data loss 921 is not limited to the above linear combination formula, and any formula may be used as long as it reflects the loss related to discrimination of all of the validation data 146 included in the validation dataset 145. By calculating the overall original data loss 921 (J_ori_all) and reflecting this overall original data loss 921 (J_ori_all) in the policy loss 168 (J_GT) described below, the policy decision model 161 can be trained so that the loss related to discrimination of the entire set of validation data 146 included in the validation dataset 145 is generally lowered. The subset original data loss 922 is calculated for each subset based on a combination of attribute values ​​(including class values). The subset original data loss 922 for a certain subset may reflect the loss related to discrimination of the validation data 146 included in the certain subset. The subset original data loss 922 (J_ori_part(n)) of a certain subset (index is n) may be calculated, for example, by the following linear combination formula of the loss (L_ori(m)) related to discrimination of each piece of verification data 146 (index is m) included in the certain subset (n): J_ori_part(n) := Σ(L_ori(m)) (Σ is the sum for m included in subset n) Note that the formula for calculating the subset original data loss 922 is not limited to the above linear combination formula, and any formula may be used as long as it reflects the loss related to discrimination of the verification data 146 included in the subset (n) of the verification dataset 145. The maximum value 923 (max_J_ori_part) of the set of subset original data losses 922 (J_ori_part(n)) may be the maximum value of the subset original data losses 922 (J_ori_part(n)) for each subset (n). The maximum value 923 (max_J_ori_part) is calculated, and the maximum value 923 (max_J_ori_part) is reflected in the policy loss 168 (J_GT) described later, thereby reducing the attribute value (including the class value).The policy decision model 161 can be trained to prevent a decrease in class discrimination accuracy specifically for validation data 146 having a specific combination of attribute values ​​(including class values) (included in a specific subset). The variance value 924 (var_J_ori_part) of the set of subset original data losses 922 (J_ori_part(n)) may be the variance value of the set of subset original data losses 922 (J_ori_part(n)) for each subset (n). By calculating the variance value 924 (var_J_ori_part) and reflecting this variance value 924 (var_J_ori_part) in the policy loss 168 (J_GT) described below, the policy decision model 161 can be trained to prevent excessive variation in the degree of loss related to discrimination of validation data 146 for each combination (subset) of attribute values ​​(including class values).

[0041] In step 604 of Fig. 6, the policy decision model training unit 111 calculates the policy loss 168. The policy loss 168 is used to train the policy decision model 161. Fig. 9 shows an aspect 900 of an information source and a calculation method for calculating the policy loss 168. As shown in Fig. 9, the policy loss 168 (J_GT) is calculated by subtracting the total text data loss 911 (J_text_all), the subset text data loss 912 for each subset (J_text_part(j)), the maximum value 913 (max_J_text_part) of the set of subset text data losses 912, the variance value 914 (var_J_text_part) of the set of subset text data losses 912, the total original data loss 921 (J_ori_all), the subset original data loss for each subset (J_ori_all), the maximum value 915 (max_J_text_part) of the set of subset text data losses 912, the variance value 916 (var_J_text_part) of the set of subset text data losses 912, the total original data loss 921 (J_ori_all), the subset original data loss for each subset (J_ori_all), the maximum value 916 (max_J_text_part) of the set of subset text data losses 912, the variance value 917 (var_J_text_part) of the set of subset text data losses 912, the total original data loss 922 (J_ori_all), the subset original data loss for each subset (J_ori_all), the maximum value 918 (max_J_text_part) of the set of subset text data losses 912, the variance value 919 (var_J_text_part) of the set of subset text data losses 912, the total original data loss 923 (J_ori_all), the variance value 924 (var_J_text_part) The policy loss 168 (J_GT) may be calculated by a function 951 that takes as arguments some or all of the following: the data loss 922 (J_ori_part(n)), the maximum value 923 (max_J_ori_part) of the set of subset original data losses 922, the variance value 924 (var_J_ori_part) of the set of subset original data losses 922, the number of generated data 931 (Num_all) which is the number of generated data 144 included in the generated data set 143, and the prohibited policy penalty 941 (P_abnormal). The policy decision model training unit 111 may calculate the policy loss 168 (J_GT), for example, by the following linear combination formula. In the following formula, t_1, t_2, t_3, t_4, t_5, t_6, t_7, and t_8 are coefficients (parameters) that can be adjusted arbitrarily.J_GT := t_1×J_text_all + t_2×max_J_text_part + t_3×var_J_text_part + t_4×J_ori_all + t_5×max_J_ori_part + t_6×var_J_ori_part + t_7×Num_all + t_8×P_abnormal Among the terms included in the above linear combination formula, the effect of reflecting the terms J_text_all, max_J_text_part, var_J_text_part, J_ori_all, max_J_ori_part, and var_J_ori_part in the policy loss 168 (J_GT) has already been explained. By reflecting the term of the number of generated data 931 (Num_all) in the policy loss 168 (J_GT), the policy decision model 161 can be trained to reduce the number of generated data 931 (if the adverse effect on other terms is minor). Reducing the number of generated data 931 can reduce resources (storage space, computational resources, and computation time) required to train the discriminant model 165 by machine learning. The prohibited policy penalty 941 (P_abnormal) indicates a penalty when the policy 162 includes undesirable content. For example, if one of the editing operations included in the policy 162 generated by the policy determination unit 105 indicates "deletion" as the "type of text data editing" and includes content indicating the deletion of a word indicating a class from the original data-corresponding text data 134, the prohibited policy penalty 941 (P_abnormal) may be set to a high value. By reflecting the prohibited policy penalty 941 (P_abnormal) term in the policy loss 168 (J_GT), training can be performed so that the policy determination model 161 is less likely to output a policy 162 including undesirable content.(Note that, for example, when adopting an aspect in which the format itself of the policy 162 output from the policy decision model 161 can be prevented from including undesirable content, when adopting an aspect in which the already-described policy correction unit is provided, or when the generation text data 142 reflecting undesirable content of the policy 162 is deleted or corrected, the term for the prohibited policy penalty 941 (P_abnormal) may not be included.) In step 605 of Fig. 6 , the policy decision model training unit 111 trains the policy decision model 161 by machine learning based on the policy loss 168 (J_GT). The policy decision model training unit 111 trains the policy decision model 161 (for example, updates the model parameters of the policy decision model 161) to eliminate the policy loss 168 (J_GT). Here, the training results of the policy decision model 161 (for example, the contents of the update of the model parameters of the policy decision model 161) may be recorded in the non-volatile recording medium (recording device) 303 (or the storage device 302) as policy decision model information 161i shown in Fig. 3. Note that the series of processes from step 501 in Fig. 5 to step 604 in Fig. 6 may be executed multiple times (for example, a predetermined number of times) (using different original datasets 131), and then step 605 in Fig. 6 may be executed.

[0042] In step 606 of Fig. 6 , the policy decision model training unit 111 determines whether to further train the policy decision model 161. For example, if there is no longer a set of learning data in the original data format that can serve as the original dataset 131 and that has not yet been used to train the policy decision model 161, the determination result in step 606 may be negative. Alternatively, if the absolute value of the policy loss 168 becomes sufficiently small and it can be said that the policy decision model 161 has been sufficiently trained, the determination result in step 606 may be negative. If the determination result in step 606 is positive, control transitions to step 607. If the determination result in step 606 is negative, a series of processes for training the policy decision model 161 is completed. In step 607 of Fig. 6 , the policy decision model training unit 111 determines a new original dataset 131 to be used for training the policy decision model 161. The policy decision model training unit 111 may, for example, determine a set of learning data in the original data format that can be the original data set 131 and that has not yet been used to train the policy decision model 161 as the new original data set 131. After step 607, control is returned to step 501 of FIG.

[0043] As described above, the policy decision model 161 is trained using the degree of validity of the content of the text data for generation 142 generated based on the policy 162 output by the policy decision model 161 and the loss related to discrimination in the discriminant model 165 trained by the generation data 144 generated based on the policy 162 output by the policy decision model 161. In this way, even if the loss of the policy decision model 161 itself is difficult to imagine (excluding the number of pieces of generation data 931 (Num_all) and the prohibited policy penalty 941 (P_abnormal)), it is possible to train the policy decision model 161.

[0044] 3-2. Processing during operation of policy decision model (FIG. 10) FIG. 10 shows a flowchart of processing executed by the system 101 when operating the trained policy decision model 161. As already explained, among steps 1001 to 1009 included in the flowchart of FIG. 10, steps 1001 to 1008 have the same content as any of the steps in the flowcharts of FIGS. 5 and 6. Specifically, steps 1001, 1002, 1003, 1004, 1005, 1006, 1007, and 1008 have the same content as steps 501, 502, 503, 504, 505, 506, 509, and 601, respectively. By executing steps 1001 to 1008, the discriminant model 165 is trained. In step 1009 of FIG. 10 , the discriminant model operation unit 112 inputs data 148 in the original data format (also referred to as production data. Unlike data in other original data formats, production data usually does not include teacher information (label information indicating the correct class)) to the trained discriminant model 165 (or 166). The discriminant model operation unit 112 obtains a class 191 of the discrimination result for the production data as the output of the discriminant model 165 (or 166). Among the processing steps included in the flowchart of FIG. 10 , steps 1001 to 1007 are a series of processes for generating the generated dataset 143 based on the original dataset 131. When operating the trained policy decision model 161, if the system 101 performs the processes up to generating the generated dataset 143, the system 101 does not need to execute steps 1008 and 1009.

[0045] 3-3. Display Output Control Processing As the generated data set 143 is generated based on the original data set 131, the system 101 may perform several types of display output control. Below, processing related to display output control of the number of data items included in the original data set 131 and the generated data set 143 will be described, mainly with reference to FIGS. 11 to 13 . Furthermore, processing related to display output control of the discrimination accuracy of a discriminant model trained using the original data set 131 and the discrimination accuracy of a discriminant model trained using the generated data set 143 will be described, mainly with reference to FIGS. 14 to 17 . Additionally, processing related to display output control of individual information (e.g., text data) corresponding to the original data included in the original data set 131 and the generated data included in the generated data set 143 will be described, mainly with reference to FIGS. 18 to 20 . Note that while the number of data items, discrimination accuracy, and display output control of individual information (e.g., text data) will be described separately below, these display output controls may be performed substantially simultaneously. Furthermore, the number of data, the discrimination accuracy, and individual information (text data, etc.) may be displayed or output integrally. Some or all of the display outputs shown in Fig. 13, Fig. 17, and Fig. 20, which will be described later, may be simultaneously displayed or output by the display or output device 307. These displays or outputs improve the explainability of the content of the generation process of the generated dataset 143 based on the original dataset 131. Furthermore, these displays or outputs improve the interpretability of the behavior of the discriminant model 165 and the policy decision model 161.

[0046] 3-3-1. Data Count Display Output Control Processing (FIGS. 11-13) FIG. 11 shows a functional configuration 1100 (and handled information) related to display output control of the number of data items included in the original dataset 131 and the generated dataset 143. FIG. 12 shows a flowchart 1200 of processing related to display output control of the number of data items. FIG. 13 shows display or output 1300 of the number of data items. The following description will be given in the order of the processing steps in the flowchart of FIG. 12, with appropriate reference to FIGS. 11 and 13. In step 1201 of FIG. 12, the data number display output control unit 113 implemented (realized) in the system 101 obtains information on the number (n) of original data items for each subset (based on a combination of attribute values ​​(including class values)) included in the original dataset 131 (before the measure is applied). The upper part of FIG. 11 shows the number (n) 1112 of original data for each subset 1111 when each piece of original data 132 has a combination of a first attribute value (which may be any of A, B, C, or D, which may also be a class value) and a second attribute value (which may also be any of a, b, or c, which may also be a class value). As shown in the upper part of FIG. 11 , for example, the number of original data included in a subset of original data 132 having a first attribute value of B and a second attribute value of c is expressed as n[B, c]. The data number display output control unit 113 may acquire information on the number (n) 1112 of original data for each subset (e.g., n[A, a], n[B, a]...n[D, c] shown in the upper part of FIG. 11 ) by, for example, aggregating the original data-corresponding text dataset 133 obtained from the original dataset 131. Alternatively, when the text data conversion unit 102 generates the original data-corresponding text data set 133 from the original data set 131, it may perform an aggregation process on the original data-corresponding text data set 133 and record information on the number (n) 1112 of original data for each subset in the non-volatile recording medium (storage device) 303. Then, the data number display output control unit 113 may acquire information on the number (n) 1112 of original data for each subset. In step 1202 of FIG. 12 , the data number display output control unit 113 acquires information on the number (N) of generated data 144 for each subset included in the generated data set 143 (after the measure has been applied).The central part of FIG. 11 shows the number (N) 1122 of original data for each subset 1121 when each piece of generated data 144 has a combination of a first attribute value (any of A, B, C, or D (which may also be a class value)) and a second attribute value (any of a, b, or c (which may also be a class value)). As shown in the central part of FIG. 11 , for example, the number of original data included in a subset of generated data 144 having a first attribute value of B and a second attribute value of c is expressed as N[B,c]. The data number display output control unit 113 may, for example, acquire information on the number (N) 1122 of generated data for each subset (e.g., N[A,a], N[B,a]...N[D,c] shown in the central part of FIG. 11 ) by aggregating the generation text dataset 141 used to generate the generated dataset 143. (Note that if the strategy 162 specifies the number of pieces of generated data 144 to be generated based on one piece of text data for generation 142, information on that number may also be used in the counting process.) Alternatively, when the strategy execution unit 106 generates the text data set for generation 141, it may perform counting process on the text data set for generation 141 and record information on the number (N) 1122 of generated data for each subset in the non-volatile recording medium (storage device) 303. Then, the data number display output control unit 113 may acquire information on the number (N) 1122 of generated data for each subset.

[0047] In step 1203 of FIG. 12 , the data number display / output control unit 113 controls the display or output device 307 to display or output the number of data items included in the original data set 131 and the generated data set 143. As shown at the bottom of FIG. 11 , the display or output device 307 displays or outputs (X) 1132 the number of data items (original data, generated data items) for each subset (based on a combination of attribute values ​​(including class values)). Here, the display or output (X) 1132 regarding the number of data items may include information regarding an increase or decrease in the number of generated data items relative to the number of original data items. FIG. 13 shows an aspect 1300 of the display or output (X) regarding the number of data items (original data, generated data items) for each subset. In FIG. 13 , the number of original data items (n) 1112 for each subset is represented by a solid cylindrical graph. FIG. 13 also uses a dotted cylindrical graph to show the increase or decrease in the number of generated data (N) 1122 from the number of original data (n) 1112 for each subset. For example, FIG. 13 shows that for a combination of a first attribute value of C and a second attribute value of c, the number of generated data (N[C,c]) was greater than the number of original data (n[C,c]). For example, FIG. 13 shows that for a combination of a first attribute value of A and a second attribute value of a, the number of generated data (N[A,a]) was less than the number of original data (n[A,a]). For example, FIG. 13 shows that for a combination of a first attribute value of D and a second attribute value of c, the number of original data (n[D,c]) was zero, but the number of generated data (n[D,c]) was no longer zero. FIG. 13 shows that, for example, for a combination of a first attribute value A and a second attribute value c, the number of original data (n[A, c]) and the number of generated data (N[A, c]) are the same. As shown in FIG. 13 , if the display or output includes information regarding an increase or decrease in the number of generated data relative to the number of original data, a viewer of the display or output can understand the manner of editing (e.g., data expansion or data generation) performed by the system 101 on the original data set 131. Furthermore, a viewer of the display or output can understand the degree of improvement in the absolute number of data for each subset and the degree to which imbalances in the number of data between subsets have been corrected.The information on the number of data items for each subset can be obtained by a text-based aggregation process for the original data text dataset 133 and the generation text dataset 141. Therefore, the aggregation process can be easily implemented.

[0048] 3-3-2. Processing for Controlling Display Output of Discrimination Accuracy (FIGS. 14-17) FIGS. 14 and 15 show functional configurations 1400 and 1500 (and the information handled) related to the display output control of the discrimination accuracy of the discriminant model 165. FIG. 16 shows a flowchart 1600 of processing related to the display output control of the discrimination accuracy. FIG. 17 shows display or output 1700 of the discrimination accuracy. The following description will be given in the order of the processing steps of the flowchart in FIG. 16, with appropriate reference to FIGS. 14, 15, and 17. Note that the discrimination accuracy of the discriminant model 165 (or 166) handled in FIGS. 14-17 is the discrimination accuracy of the discriminant model 165 (or 166) trained using the original data 132 included in the original dataset 131 (before the policy is applied) as learning data, and the discrimination accuracy of the discriminant model 165 (or 166) trained using the generated data 144 included in the generated dataset 143 (after the policy is applied) as learning data. In the flowchart of Figure 16, steps 1601 to 1604 train a discriminant model 165 using original data 132 included in an original data set 131 (before the policy is applied) as learning data, and obtain information on the discrimination accuracy of the trained discriminant model 165 (or 166), whereas steps 1605 to 1608 train a discriminant model 165 using generated data 144 included in a generated data set 143 (after the policy is applied) as learning data, and obtain information on the discrimination accuracy of the trained discriminant model 165 (or 166). In other words, steps 1601 to 1604 and steps 1605 to 1608 perform similar processing except for the difference in the learning data for the discriminant model 165. The processing steps of the flowchart of Figure 16 will be described in order below.

[0049] In step 1601 of FIG. 16 , the discriminant model training unit 109 trains the discriminant model 165 by machine learning using each piece of original data 132 (with teacher information (label information indicating the correct class)) included in the original dataset 131 (before the policy is applied) as training data. Details of the training of the discriminant model 165 performed by the discriminant model training unit 109 have already been described. Note that in step 1601, the original data 132, rather than the generated data 144, is used as training data. In step 1602 of FIG. 16 , the original data loss calculation unit 110 inputs each piece of validation data 146 included in the validation dataset 145 to the discriminant model 166 trained in step 1601. The original data loss calculation unit 110 obtains information on the class 191 of the discrimination result for each piece of validation data 146 as output from the trained discriminant model 166. In step 1603 of FIG. 16 , the original data loss calculation unit 110 calculates the subset original data loss 922 (denoted as 922o in FIG. 14 for distinction) for each subset (based on a combination of attribute values ​​(including class values)) of the verification data 146 based on information on the class 191 of the discrimination result obtained in step 1602 for each piece of verification data 146 and teacher information for the verification data 146 (label information indicating the correct class). Details of the process by which the original data loss calculation unit 110 calculates the subset original data loss 922 have already been described. Here, the larger the subset original data loss 922o for a given subset, the lower the discrimination accuracy 1410 of the discriminant model 165 for that subset. Based on this relationship, either the original data loss calculation unit 110 or the discrimination accuracy display output control unit 114 may calculate the discrimination accuracy 1410 of the discriminant model 165 for each subset based on the subset original data loss 922o for each subset. In step 1604 of Fig. 16, the discrimination accuracy display output control unit 114 acquires information on the discrimination accuracy 1410 of the discrimination model 165 (or 166) that was trained using the original data 132 included in the original data set 131 (before the policy was applied) as learning data. This information on the discrimination accuracy 1410 is information for each subset. The form of the information on the discrimination accuracy 1410 is shown in the upper part of Fig. 15.15 shows the accuracy (p) 1512 of the discriminant model for each subset 1511 when each piece of validation data 146 has a combination of a first attribute value (which is any one of A, B, C, or D (which can also be a class value)) and a second attribute value (which is any one of a, b, or c (which can also be a class value)). As shown in the upper part of Fig. 15, for example, the accuracy of the discriminant model for a subset of validation data 146 having a first attribute value of B and a second attribute value of c is denoted as p[B,c].

[0050] Steps 1605 to 1608 in Figure 16 are the same as steps 1601 to 1604, except that the learning data for training the discriminant model 165 is generated data 144 instead of the original data 132. Therefore, a description of steps 1605 to 1608 will be largely omitted. However, the subset original data loss generated in step 1607 is denoted as "subset original data loss 922n" in Figure 14. Furthermore, the discrimination accuracy of the discriminant model for each subset handled in step 1608 is denoted as "discrimination accuracy 1420 by the discriminant model after training using the generated dataset" in Figure 14. Furthermore, the form of information on the discrimination accuracy 1420 is shown in the center of Figure 15. 15 shows the accuracy (P) 1522 of the discriminant model for each subset 1521 when each piece of validation data 146 has a combination of a first attribute value (any of A, B, C, or D (which may also be a class value)) and a second attribute value (any of a, b, or c (which may also be a class value)). As shown in the center of Fig. 15, for example, the accuracy of the discriminant model for a subset of validation data 146 having a first attribute value of B and a second attribute value of c is represented as P[B,c].

[0051] In step 1609 of FIG. 16 , the discrimination accuracy display output control unit 114 controls the display or output device 307 to display or output the discrimination accuracy of the discriminant model 165 (or 166) trained using the original dataset 131 and the discrimination accuracy of the discriminant model 165 (or 166) trained using the generating dataset 143. As shown at the bottom of FIG. 15 , the display or output device 307 displays or outputs (Y) 1532 regarding the discrimination accuracy of the discriminant model for each subset (based on a combination of attribute values ​​(including class values)). Here, the discrimination accuracy display or output (Y) 1532 may include content regarding an increase or decrease in the discrimination accuracy of the discriminant model 165 (or 166) trained using the generating dataset 143 in terms of the discrimination accuracy of the discriminant model 165 (or 166) trained using the original dataset 131. FIG. 17 shows an aspect 1700 of display or output (Y) regarding the discrimination accuracy of the discriminant model 165 for each subset. In FIG. 17 , for each subset, the discrimination accuracy (p) 1512 of the discriminant model trained using the original dataset 131 is represented by a solid-line cylindrical graph. Also in FIG. 17 , for each subset, the increase or decrease in the discrimination accuracy (P) 1522 of the discriminant model trained using the generation dataset 143 relative to the discrimination accuracy (p) 1512 of the discriminant model trained using the original dataset 131 is represented by a dotted-line cylindrical graph. For example, for a subset represented by a combination of a first attribute value of C and a second attribute value of c, the discrimination accuracy (P[C,c]) after training using the generation data was higher than the discrimination accuracy (p[C,c]) after training using the original data. For example, for a subset represented by a combination of a first attribute value of A and a second attribute value of a, the discrimination accuracy (P[A,a]) after training using the generation data was lower than the discrimination accuracy (p[A,a]) after training using the original data. FIG. 17 shows that, for example, for a subset represented by a combination of a first attribute value of D and a second attribute value of c, the discrimination accuracy (p[D,c]) after training using the original data was extremely low, but the discrimination accuracy (P[D,c]) after training using the generated data was significantly improved.FIG. 17 shows that, for example, for a subset represented by a combination of a first attribute value A and a second attribute value c, the discrimination accuracy after training using the original data (p[A,c]) is the same as the discrimination accuracy after training using the generated data (P[A,c]). As shown in FIG. 17, if the display or output includes content related to the increase or decrease in the discrimination accuracy of the discriminant model 165 (or 166) trained using the generated data set 143 compared to the discrimination accuracy of the discriminant model 165 (or 166) trained using the original data set 131, a viewer of the display or output can understand the effect of improving the discrimination accuracy of the discriminant model due to editing (e.g., data augmentation or data generation) performed by the system 101 on the original data set 131. Furthermore, a viewer of the display or output can understand the degree of improvement in the discrimination accuracy of the discriminant model for each subset.

[0052] 3.3.3. Display Output Control Processing of Individual Information (Text Data, etc.) (FIGS. 18-20) FIG. 18 shows a functional configuration 1800 (and the information handled) related to display output control of individual information (e.g., data in original data format, data in text data format, attribute values ​​(including class values)) related to the original data 132 included in the original data set 131 and the generated data 144 included in the generated data set 143. (In FIG. 18, alphabets (A and B) enclosed in parentheses indicate that the same items are connected.) FIG. 19 shows a flowchart 1900 of processing related to display output control of the individual information. FIG. 20 shows display or output 2000 of the individual information. The following description will be given in the order of the processing steps of the flowchart in FIG. 19, with appropriate reference to FIGS. 18 and 20. Note that in FIG. 18, the original data display output control unit 115, the text data display output control unit 116, and the attribute value display output control unit 117 are individually indicated. Hereinafter, these display output control units may be collectively referred to as the "individual information display output control unit." Note that, in terms of implementation in the system 101, the original data display output control unit 115, the text data display output control unit 116, and the attribute value display output control unit 117 may be separate functional units or may be a substantially integrated functional unit. In step 1901 of FIG. 19 , the individual information display output control unit acquires information indicating changes, additions, and deletions between the original dataset 131 (before the measure is applied) and the generated dataset 143 (after the measure is applied). In the example of FIG. 18 , when the generated dataset 143 is generated from the original dataset 131, the original data (1) is changed to the generated data (α), the generated data (β) is added using information from the original data (3), and the original data (4) is deleted. In order to obtain information indicating such changes, additions, and deletions, for example, the individual information display output control unit may obtain information managing the data, etc. contained in the original data set 131 or the original data corresponding text data set 133, and also obtain information managing the data, etc. contained in the generated data set 143 or the generated text data set 141, and then compare the information from both.Alternatively, when generating the generation text dataset 141 or the generation data dataset 143, one or both of the policy execution unit 106 and the original data conversion unit 108 may create information indicating the changes, additions, and deletions and record it on the non-volatile recording medium (storage device) 303. The recorded information may then be acquired by the individual information display output control unit. In step 1902 of FIG. 19 , the individual information display output control unit acquires data in the original data format, data in the text data format, and attribute values ​​(including class values) for the changes, additions, and deletions. Here, the data in the original data format is acquired from the original dataset 131 or the generation dataset 143. The data in the text data format is acquired from the original data-corresponding text dataset 133 or the generation text dataset 141. The attribute values ​​(including class values) may be acquired from the original data-corresponding text dataset 133 or the generation text dataset 141. Alternatively, when the text data conversion unit 102 generates the original data-corresponding text data set 133 or when the policy execution unit 106 generates the generation text data set 141, attribute value (including class value) information may be extracted from the generated text data and recorded on the non-volatile recording medium (storage device) 303. The recorded information may then be acquired by the individual information display output control unit. In the example of FIG. 18 , individual information (data in original data format, data in text data format, attribute values ​​(including class values)) for each of the original data (1), original data (4), generated data (α), and generated data (β) is acquired by the individual information display output control unit.

[0053] In step 1903 of FIG. 19 , the individual information display output control unit controls the display or output device 307 to display or output individual information (data in the original data format, data in the text data format, and attribute values ​​(including class values)) about the changes, additions, and deletions. FIG. 20 shows an example 2000 of displaying or outputting individual information (data in the original data format, data in the text data format, and attribute values ​​(including class values)) about the changes, additions, and deletions when a generated data set 143 is generated from the original data set 131. In FIG. 20 , if one piece of original data 132 is changed to generate one piece of generated data 144, the original data 132 may be represented as "deleted data," and the generated data 144 may be represented as "added data." (Therefore, the example in FIG. 20 does not include the notation "changes.") As shown in FIG. 20 , each record (each row shown in FIG. 20 ) has columns for "edit type," "original data format," "text data format," and "attribute values, etc." (Note that some of these columns may not be displayed or output.) The "Edit Type" column indicates whether the record is a deletion or an addition. In a deletion record, the "Original Data Format" column indicates the deleted original data 132. On the other hand, in an addition record, the "Original Data Format" column indicates the added generated data 144. Note that if the original data format is an image format, the image data itself may be displayed or output. If the original data format is an audio format, a hyperlink may be displayed, and when the hyperlink is clicked, audio data may be played from a speaker. The "Text Data Format" column in a deletion record indicates the original data corresponding text data 134 corresponding to the deleted original data 132. The "Text Data Format" column in an addition record indicates the generation text data 142 corresponding to the added generated data 144. The "Attribute values, etc." column indicates the attribute values ​​(including class values) of the deleted original data 132 (text data 134 corresponding to the original data) or the attribute values ​​(including class values) of the added generated data 144 (text data for generation 142).Some means may be used to make it easier to distinguish between deleted records and added records. In the example of Figure 20, deleted records are marked with diagonal lines. Other means are also possible, such as using different background colors or different fonts for deleted and added records. Furthermore, tables showing deleted records and tables showing added records may be displayed or output separately.

[0054] As described above, when the generated dataset 143 is generated from the original dataset 131, in addition to or instead of displaying or outputting data in the original data format (e.g., image data or audio data), for the changes, additions, and deletions, data in text data format or information on attribute values ​​(including class values) is displayed or output, making it easier for a person viewing the display or output to understand the situation regarding attribute values ​​(including class values), etc. before and after editing (data expansion / data generation).

[0055] Although the above shows the display or output of changes, additions, and deletions, the above-mentioned "Original Data Format," "Text Data Format," and "Attribute Values, etc." columns may be displayed or output for the original data 132 included in the original dataset 131 and the generated data 144 included in the generated dataset 143, not limited to changes, additions, and deletions. In this case, it becomes easier to understand the status of attribute values ​​(including class values) in the original dataset 131 and the generated dataset 143. For example, it becomes easier for a viewer of the display or output to understand that data related to a specific attribute value, etc. is lacking.

[0056] 4. Others (Modifications) The present disclosure is not limited to the above-described embodiments and includes various modifications. Part of the configurations and processes of the embodiments may be replaced with the configurations and processes of other conceivable embodiments. The configurations and processes of other conceivable embodiments may be added to the configurations and processes of the embodiments. For example, the present disclosure may include the following modifications of the embodiments.

[0057] (A) Policy Determination Without Using a Policy Determination Model In the above embodiment, the policy 162 used to generate the generation text dataset 141 by editing the original data-corresponding text data 134 is generated by the policy determination model 161 handled by the policy determination unit 105. The policy determination model 161 is trained by machine learning. In a modified example, the policy determination unit 105 may generate the policy 162 without using the policy determination model 161. For example, the policy determination unit 105 may perform statistical processing on a set of original data-corresponding text data 134 included in the original data-corresponding text dataset 133 to grasp the absolute number of original data-corresponding text data 134 for each subset based on a combination of attribute values ​​(including class values) and the imbalance in the number of original data-corresponding text data 134 between the subsets. The policy determination unit 105 may then determine the policy 162 based on the grasped situation. The above-described modified example eliminates the need to construct the policy decision model 161 or train the policy decision model 161, thereby saving resources (human resources, financial resources, and time resources) for developing the system 101 and resources (storage space, computing resources, and computing time) for operating the system 101.

[0058] (B) Discriminant Model Trained by Unsupervised Learning In the above embodiment, the discriminant model 165 is trained by supervised learning. In a modified example, the discriminant model 165 may be trained by unsupervised learning. For example, the discriminant model 165 may be trained by a clustering method using a set of generated data 144 in the original data format as input. The original data loss 167 in this case may be determined by a loss function suitable for the clustering method. The modified example described above enables a wide range of types of discriminant models 165 to be used in the present disclosure. Furthermore, because it is unsupervised learning, it is not necessary to include supervised information (label information indicating the correct class) in the original data 132 or the generated data 144.

[0059] (C) Variations of Subsets In the above embodiment, for example, if the data (original data 132, generated data 144, and verification data 146) has a first attribute value (which may be a class value) that is one of A, B, C, and D, and a second attribute value (which may be a class value) that is one of a, b, and c, the subset formed by the data is defined by the combination of the first attribute value and the second attribute value. In other words, the subset formed by the data (original data 132, generated data 144, and verification data 146) does not necessarily have to be the finest granularity. For example, if the data (original data 132, generated data 144, and verification data 146) has a first attribute value that is one of A, B, C, and D, and a second attribute value that is one of a, b, and c, the subset may be defined based only on the first attribute value. Alternatively, a subset may be newly defined by combining some of the finest subsets that can be formed by combining attribute values ​​(including class values) possessed by the data. The above-described modified example allows for flexible definition of subsets, thereby enabling processing suited to the discrimination problem (classification problem) handled by the discriminant model 165. Furthermore, if the number of attribute values ​​(including class values) possessed by the data (the original data 132, the generated data 144, and the verification data 146) is large, the number of finest subsets that can be formed by combining attribute values ​​(including class values) possessed by the data will also be large. However, the modified example allows for the number of subsets to be adjusted to an appropriate number.

[0060] (D) Variations in Attribute Values ​​In the above description of the embodiment, the attribute values ​​of the data (original data 132, generated data 144, and verification data 146) were described as being discrete values. In a modified example, continuous attribute values ​​may exist. For example, when color information is used as an attribute value, the attribute value may be interpreted as a continuous value that can be expressed by the wavelength of light. In this case, a range of attribute values ​​may be specified, and the above-described subset may be formed. Such modified examples may enable the present disclosure to handle a wide variety of attribute values.

[0061] (E) Calculating Policy Loss Without Using Text Data Loss In the above embodiment, the text data loss 164 is used in calculating the policy loss 168 for training the policy decision model 161. In a modified example, the text data loss 164 does not need to be used in calculating the policy loss 168. Although the modified example described above makes it difficult to utilize the degree of validity of the content of the generation text data 142 for training the policy decision model 161, it is possible to save resources (storage area, computing resources, and computing time) required for training the policy decision model 161.

[0062] (F) Non-use of Word / Part-of-Speech Extraction Unit (Utilization of Word / Part-of-Speech Database) In the above embodiment, the word / part-of-speech data group 135 is generated based on the original data-corresponding text data 134. In a modified example, the system 101 may not generate the word / part-of-speech data group 135 based on the original data-corresponding text data 134, but may instead use a word / part-of-speech database. The word / part-of-speech database plays the role of the word / part-of-speech data group 135. Such a modified example can save resources (computational resources and computational time) related to extracting words and parts of speech.

[0063] (G) Variations in Display or Output of Number of Data Points and Discrimination Accuracy In the above-described embodiment, as shown in Figures 13 and 17, the increase or decrease in the number of data points and discrimination accuracy before and after the application of the policy for each subset based on a combination of attribute values ​​(including class values) was displayed or output as a graph (e.g., a cylindrical graph). In a modified example, either a graph of the number of data points and discrimination accuracy for each subset before the application of the policy (original dataset 131) or a graph of the number of data points and discrimination accuracy for each subset after the application of the policy (generated dataset 143) may be displayed or output. Even with such a modified example, it is possible for a viewer of the display or output to easily grasp the number of training data points and the discrimination accuracy of the discriminant model 165 for each subset based on a combination of attribute values ​​(including class values).

[0064] (H) Generation of Verification Data In the above embodiment, the generated data 144 generated by the system 101 was learning data for training the discriminant model 165 by machine learning. In a modified example, part of the generated data 144 generated by the system 101 may be used as supplementary data for the verification data 146. In this modified example, even if the verification data set 145 initially has an imbalance in the number of pieces of verification data for each subset based on the combination of attribute values ​​(including class values) or an imbalance in the number of pieces of verification data between subsets, by using part of the generated data 144 as supplementary data for the verification data 146, it is possible to form a verification data set 145 in which the imbalance is eliminated.

[0065] (I) Utilization of a Policy Determination Model for Generating Learning Data for Training a Text Data Discrimination Model In the above embodiment, the discrimination model 165 was used to discriminate the class of data 148 in the original data format (e.g., image data or audio data). In a modified example, instead of the discrimination model 165 for the original data format, a text data discrimination model that discriminates the class of data in text data format may be used. Furthermore, the policy determination model 161 may be used to generate a text data editing policy when generating training text data in text data format for training the text data discrimination model by machine learning. In this modified example, the original data corresponding text data 134 is replaced with training text data before text data editing, the generation text data 142 is replaced with training text data after text data editing, and the training text data after text data editing is used as training data for training the text data discrimination model. In this modified example, some or all of the training text data (vector-converted) included in the training text dataset before text data editing is input to the policy determination model 161. Therefore, the strategy decision model 161 can output a strategy for editing text data for a training text dataset before text data editing, taking into account the status of the training text dataset before text data editing (status such as the absolute number of training text data before text data editing for each subset based on a combination of attribute values ​​(including class values), and imbalances in numbers between subsets).

[0066] The technical matters shown in the above-described embodiments of the present disclosure and the modified examples of the embodiments can be combined as appropriate as long as no technical contradiction occurs.

Claims

1. A system comprising: a text data conversion unit that converts each piece of original data in an original data set format into each piece of text data corresponding to the original data in a text data format; an original data-corresponding text data set consisting of each piece of the original data-corresponding text data, and a policy decision unit that generates a policy to be applied to the original data-corresponding text data set based on a policy decision model; a policy execution unit that edits the original data-corresponding text data set based on the policy to generate a generating text data set, which is a set of generating text data; and an original data conversion unit that converts each piece of generating text data included in the generating text data set into each piece of generated data in the original data format, wherein the generated data is learning data for training a discriminant model by machine learning to discriminate classes of data in the original data format.

2. The system according to claim 1, comprising: a text data loss calculation unit that calculates text data loss based on the generation text dataset and a text data evaluation model; a discriminant model training unit that trains the discriminant model by machine learning using each of the generation data sets as learning data; a raw data loss calculation unit that calculates raw data loss based on loss related to discrimination of each class of verification data in the raw data format included in a verification dataset in the discriminant model trained by the discriminant model training unit; and a policy decision model training unit that determines a policy loss, which is the loss of the policy decision model, based on the text data loss and the raw data loss, and trains the policy decision model by machine learning based on the policy loss.

3. A system as described in claim 2, wherein the policy loss is based on any of an overall text data loss based on all of the generation text data included in the generation text dataset, a maximum value of subset text data losses based on each subset of the generation text data included in the generation text dataset, a variance value of the subset text data losses, an overall original data loss based on all of the validation data included in the validation dataset, a maximum value of subset original data losses based on each subset of the validation data included in the validation dataset, or a variance value of the subset original data losses.

4. A system as described in claim 3, wherein the subset of the generating text data or the subset of the verification data is formed based on class values, attribute values, or combinations of class values ​​or attribute values ​​for one or more of the classes associated with the generating text data or the verification data, respectively, or one or more attributes associated with the generating text data or the verification data, respectively.

5. A system as described in claim 3, wherein the strategy loss is based on a sum of the total text data loss, the maximum value of the subset text data loss, the variance value of the subset text data loss, the total original data loss, the maximum value of the subset original data loss, and the variance value of the subset original data loss multiplied by a coefficient.

6. A system according to claim 1, wherein the strategy determination unit generates the strategy that does not include content for deleting words indicating the class that are included in the text data corresponding to the original data.

7. A system as described in claim 1, wherein the strategy determination unit generates a strategy to be applied to the original data-corresponding text dataset based on some or all of the original data-corresponding text data contained in the original data-corresponding text dataset and the strategy determination model.

8. The system of claim 2, wherein the policy loss is based on the number of generated data generated based on the policy.

9. A system as described in claim 1, wherein the policy decision unit generates the policy including information specifying which of the original data corresponding text data included in the original data corresponding text dataset is to be subject to text data editing performed by the policy execution unit.

10. A system as described in claim 1, wherein the policy decision unit generates the policy including information specifying the number of generation data that the original data conversion unit will generate based on the generation text data generated by the policy execution unit by editing text data.

11. A system as described in claim 1, comprising a data number display output control unit that controls to display or output the number of original data or generated data for each subset of the original data or generated data formed based on class values, attribute values, or combinations of class values ​​or attribute values ​​for one or more of the classes associated with the original data or the generated data, respectively, or one or more attributes associated with the original data or the generated data, respectively, and the data number display output control unit controls to display or output, for each subset, content indicating an increase or decrease in the number of data in the number of generated data included in the subset after the measure is applied, compared to the number of original data included in the subset before the measure is applied.

12. A system as described in claim 2, comprising a discrimination accuracy display output control unit that controls to display or output information indicating the discrimination accuracy of the discriminant model for each subset of the validation data formed based on class values, attribute values, or combinations of class values ​​or attribute values ​​for one or more of the classes associated with each of the validation data, or one or more attributes associated with each of the validation data, wherein the discrimination accuracy display output control unit controls to display or output, for each subset, content indicating an increase or decrease in discrimination accuracy of the discriminant model trained by machine learning using each of the generation data included in the generation dataset after the measure is applied as learning data, as viewed from the discrimination accuracy of the discriminant model trained by machine learning using each of the original data included in the original dataset before the measure is applied as learning data.

13. A system as described in claim 1, comprising a text data display output control unit that controls to display or output a part or all of the generation text data or the text data corresponding to the original data, which corresponds to a change, addition or deletion of data in the original data format in the set of generated data contained in the generated data set after the measure is applied, as viewed from the set of original data contained in the original data set before the measure is applied.

14. A method executed by a system, comprising: a text data conversion step of converting each piece of original data in an original data format included in an original dataset into each piece of text data corresponding to the original data in a text data format; a policy determination step of generating a policy to be applied to the original data corresponding text dataset based on an original data corresponding text dataset consisting of each piece of the original data corresponding text data and a policy decision model; a policy execution step of editing the original data corresponding text dataset based on the policy to generate a generation text dataset, which is a set of generation text data; and an original data conversion step of converting each piece of generation text data included in the generation text dataset into each piece of generation data in the original data format, wherein the generation data is learning data for training a discriminant model by machine learning to discriminate classes of data in the original data format.

15. A program for causing a system to execute an original data conversion step of converting each piece of original data in an original data format included in an original data set into each piece of text data corresponding to the original data in a text data format; a policy determination step of generating a policy to be applied to the original data corresponding text data set based on an original data corresponding text data set consisting of each piece of the original data corresponding text data and a policy determination model; a policy execution step of editing the original data corresponding text data set based on the policy to generate a generation text data set which is a set of generation text data; and an original data conversion step of converting each piece of generation text data included in the generation text data set into each piece of generation data in the original data format, wherein the generation data is learning data for training a discriminant model by machine learning that discriminates classes of data in the original data format.

Images