Information processing method and information processing system

Abstract

In an information processing method executed by a computer, first data is input to a recognizer to obtain first recognition result data, second data is input to obtain second recognition result data, a first difference between reference data in recognition processing of the first data by the recognizer and the first recognition result data is calculated, first variance data and a first weight as a weight of the first variance data are calculated based on the first difference, a second difference between reference data in recognition processing of the second data by the recognizer and the second recognition result data is calculated, second variance data and a second weight as a weight of the second variance data are calculated based on the second difference, and the recognizer is trained based on the first variance data, the second variance data, the first weight, and the second weight. The first recognition result data and the second recognition result data are data of tensors with an order of 1 or more.

Description

Technical Field

[0001] The present disclosure relates to an information processing method executed by a computer and an information processing system executing the information processing method. Background Art

[0002] In order to stabilize the training of a discriminator (hereinafter also referred to as a recognizer) in a Generative Adversarial Network (GAN), a method using normalization of weights has been proposed (see Patent Document 1).

[0003] Prior Art Literature

[0004] Patent Literature

[0005] Patent Document 1: International Publication No. 2019 / 004350

[0006] Non-patent literature

[0007] Non-patent literature 1: Jun-Yan Zhu, Taesung Park, Phillip Isola, and Alexei A. Efros. "Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks", in IEEE International Conference on Computer Vision (ICCV), 2017 Summary of the invention

[0008] Problems to be solved by the invention

[0009] In the above-mentioned method, it may be difficult to stabilize the training (learning) of the classifier depending on the data used for training.

[0010] The present disclosure provides an information processing method and the like that can enable the training of the identifier to converge more stably in such a case.

[0011] Means for solving problems

[0012] An information processing method according to one embodiment of the present disclosure is an information processing method executed by a computer, wherein first data and second data as simulated data based on the first data are obtained, the first data are input to a recognizer and first recognition result data are obtained, a first difference between reference data in a recognition process of the first data by the recognizer and the first recognition result data is calculated, first variance data and a first weight as a weight of the first variance data are calculated based on the first difference, the second data are input to the recognizer and second recognition result data are obtained, a second difference between reference data in a recognition process of the second data by the recognizer and the second recognition result data is calculated, second variance data and a second weight as a weight of the second variance data are calculated based on the second difference, and the recognizer is trained based on the first variance data and the second variance data, and the first weight and the second weight, wherein the first recognition result data and the second recognition result data are data of tensors having an order of 1 or more.

[0013] In addition, an information processing system involved in one embodiment of the present disclosure comprises: an acquisition unit, which acquires first data and second data as simulated data based on the first data; a weight calculation unit, which calculates a first difference between first recognition result data obtained by inputting the first data into a recognizer and reference data in a recognition process for the first data, calculates a second difference between second recognition result data obtained by inputting the second data into the recognizer and reference data in a recognition process for the second data, calculates first variance data and a first weight as a weight of the first variance data based on the first difference, and calculates second variance data and a second weight as a weight of the second variance data based on the second difference; an error calculation unit, which calculates error data used for training the recognizer based on the first variance data and the second variance data, and the first weight and the second weight; and a training unit, which trains the recognizer using the error data, the first recognition result data and the second recognition result data being data of tensors with an order of 1 or more.

[0014] In addition, these general or specific aspects may be implemented by devices, integrated circuits, or computer-readable recording media such as CD-ROMs, in addition to the above-mentioned methods and systems, or by any combination of devices, systems, integrated circuits, methods, computer programs, and recording media.

[0015] Effects of the Invention

[0016] According to the information processing method and the like according to the present disclosure, even when using data for which it has been difficult to make training converge, it is possible to make the training of the classifier converge more stably. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 This is a functional block diagram of the information processing system according to the embodiment in the learning phase.

[0018] Figure 2 This is a functional block diagram of the information processing system according to the embodiment in the inference phase.

[0019] Figure 3 The following is a graph schematically showing a function representing the weights set in the above-mentioned learning phase.

[0020] Figure 4 This is a flowchart showing an example of the sequence of operations of the information processing method executed in the above-mentioned information processing system for training a recognizer. DETAILED DESCRIPTION

[0021] (Knowledge that is the basis of this disclosure)

[0022] The present inventors have found that the following problems occur with respect to the conventional method proposed above.

[0023] In the previous method, in order to prevent the weight of the recognizer from becoming too large, the weight is normalized so that the output value of the recognizer does not become a deviation value. More specifically, the singular value of the weight matrix with the weights of each layer of the recognizer as elements is calculated, and the weight matrix is ​​normalized using the norm of the singular value. Then, the normalized weight matrix is ​​updated based on the error of the output of the recognizer.

[0024] However, in this conventional method, there are cases where it is difficult for the identification device to converge in learning.

[0025] In GAN, for example, there is a so-called CycleGAN that is used for the purpose of conversion between sets of images of different types. CycleGAN is a GAN of the PatchGAN method and the LSGAN (Least Squares GAN) method. The output of the recognizer of the PatchGAN method is not a scalar value that takes a value of 0 or 1, but a matrix, which has as elements the values ​​of the judgment results of which image is a simulated image output by the generator and which image is not output from the generator for each of the multiple small areas (patches) divided into the entire input image. In addition, the recognizer of the LSGAN method is trained based on the variance between the matrix output by the recognizer and the matrix representing the correct answer in the recognition (a matrix in which 0 or 1 is arranged). In such a CycleGAN, even a generator trained using a data set of images that are not aligned shows a suitable conversion result (see non-patent document 1). Therefore, high practicality is expected in applications where, for example, it is difficult to obtain a fully aligned dataset of the amount required for training in reality.

[0026] However, in the data of the image that can be obtained as the unaligned data set, there is a high possibility that it contains greater noise than the aligned data set. The so-called noise here is caused by, for example, the image quality, the degree of focus, or the changes in the color tone. In addition, in the case where the data set is a person image, the range of the body shown in the image that changes according to the posture, occlusion or composition of the person shown, or the changes in objects other than the person such as the person's belongings or background can also be cited as examples of the cause of the noise. Such noise affects the training of the recognizers of the above-mentioned PatchGAN method and LSGAN method. Specifically, it makes the training of the recognizer unstable, which makes it difficult to obtain a generator that generates images of the desired quality. In addition, in the above-mentioned conventional methods, there is a problem that cannot be solved in the stabilization of the training of the recognizer caused by such noise.

[0027] An information processing method according to one embodiment of the present disclosure proposed in view of such a problem is an information processing method executed by a computer, wherein first data and second data as simulated data based on the first data are obtained, the first data are input into a recognizer and first recognition result data are obtained, a first difference between reference data in a recognition process of the first data by the recognizer and the first recognition result data is calculated, first variance data and a first weight as a weight of the first variance data are calculated based on the first difference, the second data are input into the recognizer and second recognition result data is obtained, a second difference between reference data in a recognition process of the second data by the recognizer and the second recognition result data is calculated, second variance data and a second weight as a weight of the second variance data are calculated based on the second difference, and the recognizer is trained based on the first variance data and the second variance data, and the first weight and the second weight, wherein the first recognition result data and the second recognition result data are data of tensors having an order of 1 or more.

[0028] Thus, when the noise included in the training data of the classifier is greater than a certain level, the influence of the noise on the training result can be suppressed, and the training can be converged more stably.

[0029] Alternatively, the larger the absolute value of the first difference is, the smaller the first weight is, so as to reduce the influence of the first variance data on the training of the identifier; and the larger the absolute value of the second difference is, the smaller the second weight is, so as to reduce the influence of the second variance data on the training of the identifier.

[0030] Furthermore, when the absolute value of the first difference exceeds a threshold value, the first weight may be set to zero, and when the absolute value of the second difference exceeds a threshold value, the second weight may be set to zero.

[0031] Alternatively, the second data may be generated and output by a generator based on the first data, and the generator may be trained based on the first variance data and the second variance data, and the first weight and the second weight.

[0032] As a result, the training of the generator that competes with the above-mentioned recognizer in GAN converges more stably.

[0033] Furthermore, the first data may be image data.

[0034] In addition, an information processing system involved in one embodiment of the present disclosure comprises: an acquisition unit, which acquires first data and second data as simulated data based on the first data; a weight calculation unit, which calculates a first difference between first recognition result data obtained by inputting the first data into a recognizer and reference data in a recognition process for the first data, calculates a second difference between second recognition result data obtained by inputting the second data into the recognizer and reference data in a recognition process for the second data, calculates first variance data and a first weight as a weight of the first variance data based on the first difference, and calculates second variance data and a second weight as a weight of the second variance data based on the second difference; an error calculation unit, which calculates error data used for training the recognizer based on the first variance data and the second variance data, and the first weight and the second weight; and a training unit, which trains the recognizer using the error data, the first recognition result data and the second recognition result data being data of tensors with an order of 1 or more.

[0035] Thus, when the noise included in the training data of the classifier is greater than a certain level, the influence of the noise on the training result can be suppressed, and the training can be converged more stably.

[0036] In addition, these general or specific aspects may be implemented by devices, integrated circuits, or computer-readable recording media such as CD-ROMs, in addition to the above-mentioned methods and systems, or by any combination of devices, systems, integrated circuits, methods, computer programs, and recording media.

[0037] Hereinafter, an embodiment of an information processing method and an information processing system involved in one embodiment of the present disclosure will be described with reference to the accompanying drawings. The embodiment shown here represents a specific example of the present disclosure. Therefore, the numerical values, shapes, constituent elements, configurations and connection methods of constituent elements, and steps (processes) and the order of steps shown in the following embodiments are examples and do not limit the present disclosure. In addition, the constituent elements in the following embodiments that are not recorded in the independent claims are constituent elements that can be added arbitrarily. In addition, each figure is a schematic diagram and is not necessarily a strict illustration.

[0038] (Implementation Method)

[0039] [1. Composition]

[0040] Figure 1 and Figure 2This is a functional block diagram showing an example of the functional configuration of an information processing system involved in the implementation mode. These information processing systems are configured by using one or more information processing devices (computers) each having a processor and a memory and executing a program, and CycleGAN is implemented. The functional configuration of the information processing system for the learning phase is Figure 1 As shown in the figure, the functions used in the inference phase are composed of Figure 2 The components of the functional configurations shown in the blocks are implemented, for example, by a part or all of the processors executing one or more programs stored in a part or all of the memory.

[0041] [1-1. Structure used in the training phase]

[0042] like Figure 1 As shown, the information processing system 10A according to the embodiment includes a first conversion unit 11A, a determination unit 12 , a weight calculation unit 13 , a first error calculation unit 14 , a training unit 15 , a second conversion unit 16 , and a second error calculation unit 17 .

[0043] The first conversion unit 11A performs a prescribed conversion on the real image obtained by the information processing system 10A to generate a fake image and output it. The prescribed conversion is, for example, a change in the image quality or style of the image. The change in the style of the image is, for example, to make the input real-scene image look like a painting by a prescribed painter or style, or conversely to make the input oil painting image or the image based on CG (Computer Graphics) look like a real-scene image. In addition, as another example of a prescribed conversion, the color contained in the image is changed in a prescribed direction, for example, to make the input image of a natural landscape look like an image taken with the same composition in different seasons. As another similar example, a specific subject contained in the input image can be cited, by mainly changing the color or pattern to make it look like another subject. More specifically, it is a conversion that makes a chestnut horse reflected in the image look like a zebra, or makes an apple look like an orange. Regarding these conversions performed by the first conversion unit 11A, if another expression is used, it can also be said that the basic structure of the input image (real image) is retained, and an image (pseudo image) is generated by simulating different styles, landscapes of different seasons, or the appearance of different subjects, and output. Such a first conversion unit 11A is one of the two generators (Generator) included in CycleGAN installed by the information processing system 10A, and is a generation model of a neural network for the purpose of the above-mentioned conversion. In addition, the data of the real image is an example of the first data in this embodiment, and the data of the pseudo image is an example of the second data in this embodiment.

[0044] The determination unit 12 performs a recognition process for determining whether the image received as input is a true image or a false image generated by the first conversion unit 11A, and outputs the result. The recognition process is performed in the manner of the above-mentioned PatchGAN, and the result of the recognition process is output in the form of a matrix, the matrix having values ​​indicating the likelihood of each small area being a true image or a false image as elements. For example, the matrix is ​​as follows: the element corresponding to the small area determined as a true image takes 1, the element corresponding to the small area determined as a false image takes 0, and the element corresponding to the other small areas takes a value greater than 0 and less than 1 corresponding to the determination result of whether each small area is a false image or a true image. Such a determination unit 12 is a recognizer in CycleGAN installed by the information processing system 10A, and is a recognition model of a neural network for the purpose of recognition as described above. Hereinafter, among the data (hereinafter also referred to as recognition result data) indicating the result of the recognition process output by the determination unit 12, the recognition result data outputted after receiving the input of a true image is also referred to as the first recognition result data, and the recognition result data outputted after receiving the input of a false image is also referred to as the second recognition result data.

[0045] The weight calculation unit 13 calculates the difference between the recognition result data output by the determination unit 12 after performing the recognition process and the data representing the correct answer in the recognition process (hereinafter also referred to as reference data). In addition, the weight calculation unit 13 calculates the weight and variance of each element of the matrix of the recognition result data output by the determination unit 12 based on the difference. The reference data has the same size as the recognition result data, and is a matrix in which all elements are 1 or all elements are 0. Following the example used in the above description of the determination unit 12, a matrix in which all elements are 1 represents the correct answer of the matrix output by the determination unit 12 when a true image is input. In addition, a matrix in which all elements are 0 represents the correct answer of the matrix output by the determination unit 12 when a false image is input. In addition, the weight and variance calculated by the weight calculation unit 13 for the recognition result output by the determination unit 12 to which the true image is input are also referred to as the first weight and the first variance, respectively, below. The weight and variance calculated by the weight calculation unit 13 for the recognition result output from the determination unit 12 to which the fake image is input are hereinafter referred to as a second weight and a second variance, respectively.

[0046] The first error calculation unit 14 calculates the error of the determination unit 12 based on the first weight, the first variance, the second weight, and the second variance.

[0047] The calculation of the above-mentioned weights by the weight calculation unit 13 and the calculation of the error of the determination unit 12 by the first error calculation unit 14 using the weights will be described later using an example.

[0048] The training unit 15 performs training of the determination unit 12 using the error calculated by the first error calculation unit 14 .

[0049] The second conversion unit 16 is a generator of the other side of the generation model of the neural network in CycleGAN implemented by the information processing system 10A. The second conversion unit 16 receives the input of the pseudo image generated and output by the first conversion unit 11A. Then, based on the pseudo image, the second conversion unit 16 performs conversion to restore the true image before conversion to the pseudo image, and outputs the image generated by the conversion.

[0050] The second error calculation unit 17 calculates an error based on the difference between the image output by the second conversion unit 16 and the image corresponding to the correct answer of the image, that is, the real image before being converted into the pseudo image output by the first conversion unit 11A. The error is input to the training unit 15 and used for the training of the first conversion unit 11A.

[0051] In the information processing system 10A, each of these components is implemented by one or more information processing devices constituting the information processing system 10A.

[0052] In addition, the true image set is a set of images that have not been subjected to any conversion processing for simulation as described above by the first conversion unit 11A, and is composed of a plurality of still images or a moving image including a plurality of frames. The information processing system 10A can obtain the data of the true image set by using a reading device to read data recorded in a non-volatile recording medium such as a DVD (Digital Versatile Disc) or a semiconductor memory, or can obtain the data of the true image set by receiving an input of an image signal from a camera. Alternatively, the information processing system 10A can also be further provided with a communication device, and obtain the data of the true image set via a signal received by the communication device.

[0053] [1-2. Structure for the reasoning stage]

[0054] like Figure 2 As shown, the information processing system 10B involved in the embodiment includes a conversion unit 11B which is a generation model obtained by machine learning. Specifically, the conversion unit 11B is the first conversion unit 11A that has been repeatedly trained for the above-mentioned predetermined conversion in the information processing system 10A, and can be treated as the conversion unit 11B if, for example, the evaluation related to such conversion performance reaches the expected benchmark. The conversion unit 11B performs a predetermined conversion on the unconverted image and outputs a conversion-completed image. For example, if the conversion unit 11B receives an input of a real-scene image as an unconverted image, it converts the real-scene image to generate a conversion-completed image that looks like a painting in a predetermined style and outputs it.

[0055] In the information processing system 10B, the conversion unit 11B is implemented by one or more information processing devices constituting the information processing system 10B. The information processing device constituting the information processing system 10B may be common to the information processing device constituting the information processing system 10A, and the conversion unit 11B may be the first conversion unit 11A itself trained to converge to a certain degree or more. In addition, the information processing device constituting the information processing system 10B may be different from the information processing device constituting the information processing system 10A. For example, the first conversion unit 11A may be located on a plurality of desktop computers constituting the information processing system 10A, and the conversion unit 11B may be located on a microcontroller possessed by a mobile body such as a car, a portable information terminal, or a household electrical appliance. The conversion unit 11B in this case may also be a configuration obtained by making the first conversion unit 11A lightweight (quantized).

[0056] [2. Suppression of the influence of deviation values]

[0057] In the training stage of the previous GAN, the variance calculated based on the difference between the output of the recognizer and the reference data representing the correct answer is used for the training of the recognizer. In contrast, in the training stage of the information processing system 10A involved in the present embodiment, a process for suppressing the influence of the deviation value contained in the data of the recognition object in the training of the determination unit 12 of the recognizer is further implemented. A specific example of this process is described below. In this example, Tukey's biweight estimation method, which is one of the robustness estimation methods, is used in the setting of the weights.

[0058] In the information processing system 10A, as described above, the weight calculation unit 13 calculates the difference between the recognition result data output by the determination unit 12 after performing the recognition process and the reference data in the recognition process. Furthermore, the weight calculation unit 13 calculates the weight (first weight, second weight) and variance (first variance, second variance) of each element of the matrix output by the determination unit 12 based on the difference as follows.

[0059] When the true image x is input to the determination unit 12 as the recognition target, the elements of the first recognition result data output by the determination unit 12 are represented by D 1 (x), and let each element of the reference data be R 1 , the weight calculation unit 13 can obtain the first difference d as the difference between the first recognition result data and the reference data through the calculation represented by the following formula: 1 .

[0060] d 1 =D 1 (x)-R 1

[0061] In this case, the reference data is a matrix in which all elements have a value of 1, so R 1 =1.

[0062] Here, if the weight calculation unit 13 represents the first difference d 1 The size of the absolute value, that is, the threshold of the boundary of whether the absolute value is acceptable (hereinafter referred to as the error tolerance value) is set to T, then the calculation represents the difference d 1 The corresponding first weight function is as follows t(d 1 ).

[0063] d 1 <-T: t(d 1 )=0

[0064] [Number 1]

[0065] -T≤d 1 ≤T: t(d 1 )={1-(d 1 / T) 2} 2

[0066] T<d 1 The situation: t(d 1 )=0

[0067] Figure 3 The function t(d 1 ). Figure 3 It can be seen that the first weight is set as follows: 1 The absolute value of increases from 0 and decreases from 1 and approaches 0. 1 If the absolute value exceeds the allowable error value T, it is zero.

[0068] Similarly, when the following pseudo image G(z) is input to the determination unit 12 as a recognition target, the elements of the second recognition result data output by the determination unit 12 are set to D 2 (G(z)), and let each element of the reference data be R 2 , the weight calculation unit 13 can obtain the second difference d as the difference between the second recognition result data and the reference data through the calculation represented by the following formula: 2 , where the pseudo image G(z) is an image output by the first conversion unit 11A as a generator that receives the input of the true image z.

[0069] d 2 =D 2 (G(z))-R 2

[0070] In this case, the reference data is a matrix in which all elements have a value of 0, so R 2 =0.

[0071] In addition, if the second difference d 2 The absolute value of the error tolerance is set to T, which means that the second difference d 2 The corresponding second weight function t(d 2 ) is also related to the function t(d 1 ) is also calculated as follows.

[0072] d 2 <-T: t(d 2 )=0

[0073] [Number 2]

[0074] -T≤d 2 ≤T: t(d 2 )={1-(d 2 / T) 2} 2

[0075] T<d 2 The situation: t(d 2 )=0

[0076] The function representing the second weight t(d 2 ) is also schematically shown in the graph Figure 3 That is, the second weight is set as follows: 2 The absolute value of increases from 0 and decreases from 1 and approaches 0. 2 If the absolute value exceeds the allowable error value T, it is zero.

[0077] The weight calculation unit 13 further calculates the variance of the recognition result of the determination unit 12. Specifically, for each element of the matrix as the recognition result data, based on the first difference D obtained above, 1 (x)-1Calculate the first variance (D 1 (x)-1) 2 , based on the second difference D 2 (G(z))-0Calculate the second variance (D 2 (G(z))-0) 2 .

[0078] Next, the first error calculation unit 14 calculates the error of the determination unit 12 based on the first weight, the first variance, the second weight, and the second variance calculated as described above. Specifically, each element of the first variance is multiplied by d 1 The value of the first weight (t(d 1)). The result is also referred to as the true image error. In addition, each element of the second variance is multiplied by the value of d 2 The value of the corresponding second weight (t(d 2 )). Hereinafter, the result is also referred to as a pseudo image error. Then, the result obtained by adding the true image error and the pseudo image error is obtained as the error of the determination unit 12. The error of the determination unit 12 is used by the training unit 15 for training the determination unit 12.

[0079] The significance of applying the weights set as described above to the variance is as follows. The size (absolute value) of the first difference or the second difference indicates the size of the deviation of the judgment of each part of the data of the recognition object (a small area in the above-mentioned image example) from the correct answer. In addition, the greater the deviation from the correct answer, the smaller the weights set as described above. If such weights are applied to the variance used for the training of the recognizer, the greater the deviation from the correct answer of each part of the data input to the determination unit 12 for training, the smaller the error added to the training. In other words, the greater the deviation from the correct answer, the more the influence on the training is suppressed. Here, the part of the data input to the determination unit 12 for training containing the deviation value can be judged as a large deviation from the correct answer. Therefore, the influence of the deviation value contained in the data of the recognition object on the training of the recognizer can be suppressed. In addition, the greater the degree of deviation from the deviation value, the stronger the suppression effect. In the above example, the weight is set to zero for the part of the judgment that the deviation from the correct answer exceeds the threshold, so the influence on the training of the recognizer becomes zero.

[0080] In the above example, Tukey's biweight estimation method is used to suppress the influence of outliers, but the method of suppressing the influence of outliers is not limited to this. Another M estimation method, which is a robust estimation method capable of setting the variance as described above, may also be used.

[0081] [3. Operation of the information processing system]

[0082] The operation of the information processing method for training the determination unit 12 as a classifier executed in the information processing system 10A will be described using a sequence example. Figure 4 This is a flowchart showing an example of the sequence of operations of the information processing system 10A that executes the information processing method.

[0083] (Step S10) The determination unit 12 as a classifier receives input of an image acquired by the information processing system 10A from a set of true images.

[0084] (Step S11) The determination unit 12 performs recognition processing for each small area of ​​the image received as input to determine whether it is a real image or a fake image, calculates a matrix based on the result and outputs it. Here, it is assumed that the image to be recognized is a real image, and for convenience, the calculated matrix is ​​called the first output matrix.

[0085] (Step S12) The weight calculation unit 13 calculates the first weight as the weight of each element using the first output matrix calculated in step S11. For the calculation method of the first weight, refer to the example given in the above-mentioned "2. Suppression of the influence of outliers".

[0086] (Step S13) The weight calculation unit 13 calculates a first variance, which is a variance of each element of the first output matrix calculated in step S11. For the calculation method of the first variance, refer to the above-mentioned "2. Suppression of the influence of outliers".

[0087] (Step S20 ) The determination unit 12 receives input of a pseudo image generated by the first conversion unit 11A as a generator by converting a real image.

[0088] (Step S21) The determination unit 12 performs recognition processing for each small area of ​​the image received as input to determine whether it is a real image or a fake image, calculates a matrix based on the result and outputs it. Here, it is assumed that the image to be recognized is a fake image, and for convenience, the calculated matrix is ​​called the second output matrix.

[0089] (Step S22) The weight calculation unit 13 calculates the second weight as the weight of each element using the second output matrix calculated in step S21. For the calculation method of the second weight, refer to the example given in the above-mentioned "2. Suppression of the influence of outliers".

[0090] (Step S23) The weight calculation unit 13 calculates the second variance, which is the variance of each element of the second output matrix calculated in step S21. For the calculation method of the second variance, refer to the above-mentioned "2. Suppression of the influence of outliers".

[0091] (Step S30 ) The first error calculation unit 14 multiplies the first weight calculated in step S12 by the first variance calculated in step S13 to calculate a true image error.

[0092] (Step S31 ) The first error calculation unit 14 multiplies the second weight calculated in step S22 by the second variance calculated in step S23 to calculate a pseudo image error.

[0093] (Step S32 ) The first error calculation unit 14 adds the true image error calculated in step S30 and the false image error calculated in step S31 to calculate the error of the determination unit 12 .

[0094] (Step S33) The training unit 15 performs training of the determination unit 12 as a classifier using the error calculated in step S32.

[0095] In addition, the content of the information processing method performed by the information processing system 10A is not limited to the above. For example, the training of the first conversion unit 11A as a generator is also performed by the training unit 15. This training is performed, for example, by using the above-mentioned error calculated by the second error calculation unit 17. In addition, in the training of the first conversion unit 11A, the first weight, the first variance data, the second weight, and the second variance data may also be used.

[0096] [4. Supplementary matters]

[0097] The information processing method and information processing system involved in one or more embodiments of the present disclosure are not limited to the description of the above-mentioned embodiments. As long as it does not deviate from the purpose of the present disclosure, the various modifications thought of by those skilled in the art to the above-mentioned embodiments may also be included in the embodiments of the present disclosure. Such modified examples and other supplementary matters for the description of the embodiments are given below.

[0098] (1) The information processing system involved in the above-mentioned embodiment is described by taking a system in which CycleGAN is installed as an example, but is not limited to this. The information processing system involved in one embodiment of the present disclosure can also be applied to other types of GANs such as PatchGAN and LSGAN, such as ComboGAN.

[0099] (2) The information processing system involved in the above-mentioned embodiment is described by taking a system for converting and recognizing images as an example, but the processing object of the information processing system is not limited to image data. Other examples of the processing object include sensor data such as voice, distance point group, pressure, temperature, humidity, smell, and language data.

[0100] (3) The information processing system according to the above embodiment is described by taking the example of the recognition result in matrix form, but the present invention is not limited thereto. The information processing system of the present disclosure can be applied to information processing that processes recognition result data that is tensor data of order 1 or higher.

[0101] (4) Some or all of the functional components of each of the above-mentioned information processing systems may also be constituted by a system LSI (Large Scale Integration). System LSI is a super-multifunctional LSI manufactured by integrating multiple components on a single chip. Specifically, it is a computer system composed of a microprocessor, ROM (Read-Only Memory), RAM (Random Access Memory), etc. Computer programs are stored in ROM. The microprocessor operates according to the computer program, and the system LSI achieves the functions of each component.

[0102] In addition, although it is set as system LSI here, it is sometimes called IC, LSI, super LSI, and special LSI depending on the difference in integration. In addition, the method of forming an integrated circuit is not limited to LSI, and it can also be realized by a dedicated circuit or a general-purpose processor. It is also possible to use an FPGA (Field Programmable Gate Array) that can be programmed after manufacturing the LSI, or a reconfigurable processor that can reconfigure the connection and setting of the circuit unit inside the LSI.

[0103] Furthermore, if integrated circuit technology that replaces LSI emerges due to advances in semiconductor technology or other derived technologies, the technology can also be used to integrate functional modules. Application of biotechnology is possible.

[0104] (5) One aspect of the present disclosure is not limited to the above-mentioned information processing systems, but may also be an information processing method having as steps processing performed by characteristic components included in the information processing system. The information processing method may be, for example, using Figure 4 An information processing method described in a flowchart of the present invention. In addition, one embodiment of the present invention may be a computer program that causes a computer to execute each characteristic step included in the information processing method. In addition, one embodiment of the present invention may be a computer-readable non-volatile recording medium that records such a computer program.

[0105] Industrial Applicability

[0106] The present disclosure can be used for training a recognizer in a GAN.

[0107] Description of reference numerals:

[0108] 10A, 10B Information Processing System

[0109] 11A 1st conversion section

[0110] 11B Conversion Unit

[0111] 12. Judgment Department

[0112] 13 Weight calculation unit

[0113] 14.1st Error Calculation Unit

[0114] 15 Training Department

[0115] 16. Second Conversion Section

[0116] 17 The second error calculation unit.

Claims

1. An information processing method executed by a computer, obtaining first data and second data which is simulated data based on the first data, inputting the first data into the recognizer and obtaining first recognition result data, calculating a first difference between reference data in a recognition process of the first data by the recognizer and the first recognition result data, calculating first variance data and a first weight as a weight of the first variance data based on the first difference, inputting the second data into the recognizer and obtaining second recognition result data, calculating a second difference between reference data in a recognition process of the second data by the recognizer and the second recognition result data, calculating second variance data and a second weight serving as a weight of the second variance data based on the second difference, The identifier is trained based on the first variance data and the second variance data, and the first weight and the second weight, The first recognition result data and the second recognition result data are tensor data with an order greater than or equal to 1, The larger the absolute value of the first difference is, the smaller the first weight is made to reduce the influence of the first variance data on the training of the identifier. The larger the absolute value of the second difference is, the smaller the second weight is made to reduce the influence of the second variance data on the training of the identifier. When the absolute value of the first difference exceeds a threshold value, the first weight is set to zero, When the absolute value of the second difference exceeds a threshold value, the second weight is set to zero.

2. The information processing method according to claim 1, The second data is generated and output by a generator based on the first data, The generator is trained based on the first variance data and the second variance data, and the first weight and the second weight.

3. The information processing method according to claim 1 or 2, The first data is image data.

4. An information processing system comprising: an acquisition unit that acquires first data and second data that is simulated data based on the first data; a weight calculation unit that calculates a first difference between first recognition result data obtained by inputting the first data into the recognizer and reference data in a recognition process for the first data, calculates a second difference between second recognition result data obtained by inputting the second data into the recognizer and reference data in a recognition process for the second data, calculates first variance data and a first weight as a weight of the first variance data based on the first difference, and calculates second variance data and a second weight as a weight of the second variance data based on the second difference; an error calculation unit that calculates error data used for training the identifier based on the first variance data and the second variance data, and the first weight and the second weight; as well as A training unit uses the error data to train the identifier. The first recognition result data and the second recognition result data are tensor data with an order greater than or equal to 1, The weight calculation unit performs: The larger the absolute value of the first difference is, the smaller the first weight is made to reduce the influence of the first variance data on the training of the identifier. The larger the absolute value of the second difference is, the smaller the second weight is made to reduce the influence of the second variance data on the training of the identifier. When the absolute value of the first difference exceeds a threshold value, the first weight is set to zero, When the absolute value of the second difference exceeds a threshold value, the second weight is set to zero.

Images