Information processing device, information processing method, and program

Abstract

In order to prevent, in image recognition, an image of an unregistered object from being falsely recognized as an image of a registered object, an information processing device (1) comprises: an acquisition means (11) that acquires an original image which belongs to one of a plurality of classes; a determination means (12) that determines a parameter which specifies an image generation method; an image generation means (13) that uses the parameter which has been determined by the determination means to generate a new image from the original image; and a data generation means (14) that generates data which includes the new image and a label given to the new image, said label differing from that of the class to which the original image belongs.

Description

[Technical Field]

[0001] The present invention relates to an information processing device, an information processing method, allusion to law and programs. [Background technology]

[0002] Techniques for applying image recognition processing to target images are known. For example, Patent Document 1 discloses a training data generation device that can automatically generate training data for performing machine learning to evaluate images in which a repair process has been performed on a missing area. Furthermore, Patent Document 2 discloses an information processing device that suppresses the generation of redundant training data when generating new training data using existing training data. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Publication No. 2017-058930 [Patent Document 2] Japanese Patent Publication No. 2020-091737 Summary of the Invention [Problem to be solved by the invention]

[0004] As disclosed in Patent Documents 1 and 2, there is a need to improve the classification accuracy of image classification devices. However, even if learning is performed using available learning data, there is a problem in that the classification accuracy does not improve as much as expected.

[0005] One aspect of the present invention has been made in consideration of the above-mentioned problems, and one example of its purpose is to provide a technology that prevents an image of an unregistered object from being mistakenly recognized as an image of a registered object during image recognition. [Means for solving the problem]

[0006] An information processing device according to one aspect of the present invention comprises an acquisition means for acquiring an original image belonging to one of a plurality of classes, a determination means for determining parameters that define an image generation method, an image generation means for generating a new image from the original image using the parameters determined by the determination means, and a data generation means for generating data including the new image and a label assigned to the new image, the label being of a class different from the class to which the original image belongs.

[0007] (delete)

[0008] (delete)

[0009] An information processing method according to one aspect of the present invention includes obtaining an original image belonging to one of a plurality of classes by at least one processor, determining parameters that define an image generation method, generating a new image from the original image using the determined parameters, and generating data including the new image and a label assigned to the new image, the label being of a class different from the class to which the original image belongs.

[0010] (delete)

[0011] A program according to one aspect of the present invention is a program for causing a computer to function as an information processing device, causing the computer to function as an acquisition means for acquiring an original image belonging to one of a plurality of classes, a determination means for determining parameters that define an image generation method, an image generation means for generating a new image from the original image using the parameters determined by the determination means, and a data generation means for generating data including the new image and a label assigned to the new image, the label being of a class different from the class to which the original image belongs.

[0012] (delete) [Effects of the Invention]

[0013] According to one aspect of the present invention, it is possible to provide a technology that prevents an image of an unregistered object from being mistakenly recognized as an image of a registered object in image recognition. [Brief explanation of the drawings]

[0014] [Figure 1] 1 is a block diagram showing a configuration of an information processing device according to a first exemplary embodiment of the present invention. [Figure 2] 1 is a flowchart showing the flow of an information processing method according to the first exemplary embodiment. [Figure 3] 1 is a block diagram showing a configuration of an information processing system according to a first exemplary embodiment. [Figure 4] FIG. 10 is a block diagram showing the configuration of an information processing device according to a second exemplary embodiment of the present invention. [Figure 5] 10 is a diagram illustrating a method for generating a new image executed by an information processing device according to an exemplary embodiment 2. FIG. [Figure 6] 10 is a flowchart showing the flow of an information processing method according to the second exemplary embodiment. [Figure 7] FIG. 10 is a block diagram showing the configuration of an information processing device according to a third exemplary embodiment of the present invention. [Figure 8] 11 is a flowchart showing the flow of an information processing method S3 according to the third exemplary embodiment. [Figure 9] 11 is a flowchart showing the flow of an information processing method S4 according to the third exemplary embodiment. [Figure 10] 11 is a flowchart showing the flow of an information processing method S5 according to the third exemplary embodiment. [Figure 11] FIG. 10 is a block diagram showing the configuration of an information processing device according to a fourth exemplary embodiment of the present invention. [Figure 12] FIG. 1 is a schematic diagram of a configuration of a target model to be trained. [Figure 13] FIG. 10 is a block diagram showing the configuration of an information processing device according to a fifth exemplary embodiment of the present invention. [Figure 14] FIG. 10 is a block diagram showing the configuration of an information processing device according to a sixth exemplary embodiment of the present invention. [Figure 15] FIG. 1 is a schematic diagram of an object model configuration having two processing layers. [Figure 16] FIG. 10 is a block diagram showing the configuration of an information processing device according to a seventh exemplary embodiment of the present invention. [Figure 17] 13 is a flowchart showing the flow of an information processing method S6 according to the seventh exemplary embodiment. [Figure 18] 10 is a graph showing the accuracy rate of classifiers trained using different training data. [Figure 19] FIG. 1 is a configuration diagram for realizing an information processing device or the like by software. DETAILED DESCRIPTION OF THE INVENTION

[0015] Exemplary Embodiment 1 A first exemplary embodiment of the present invention will be described in detail with reference to the drawings. This exemplary embodiment is a basic form of the exemplary embodiments described below.

[0016] (Configuration of information processing device 1) The configuration of an information processing device 1 according to this exemplary embodiment will be described with reference to Fig. 1. Fig. 1 is a block diagram showing the configuration of the information processing device 1. As shown in Fig. 1, the information processing device 1 includes an acquisition unit 11, a determination unit 12, an image generation unit 13, and a data generation unit 14.

[0017] The acquisition unit 11 is one form of the "acquisition means" set forth in the claims, the determination unit 12 is one form of the "determination means" set forth in the claims, the image generation unit 13 is one form of the "image generation means" set forth in the claims, and the data generation unit 14 is one form of the "data generation means" set forth in the claims.

[0018] The acquisition unit 11 acquires an original image belonging to one of a plurality of classes. The source from which the acquisition unit 11 acquires the original image is not limited. For example, the acquisition unit 11 may acquire an image recorded in an external database, or an image recorded in a memory (not shown) of the information processing device 1. The original image is assigned a label corresponding to the class to which the original image belongs. The image acquired by the acquisition unit 11 is called an original image. The acquisition unit 11 transmits the acquired original image to the determination unit 12.

[0019] When the determination unit 12 receives an original image from the acquisition unit 11, it determines parameters that define an image generation method. The image generation method is a method performed by the image generation unit 13 to generate a new image from the original image. The parameters include, for example, a parameter that defines a method of changing the image and a parameter that defines the degree of change to be made to the original image in the image change method. The determination unit 12 determines one or more parameters for one original image. The determination unit 12 transmits the original image and the determined parameters to the image generation unit 13.

[0020] The image generation unit 13 generates a new image from the original image using the parameters determined by the determination unit 12. Specifically, when the image generation unit 13 receives the original image and parameters from the determination unit 12, it generates a new image by making predetermined changes to the original image based on the parameters. The predetermined changes include, for example, changes in hue, text, and style. The new image generated by the image generation unit 13 is an image that is similar to the original image but belongs to a different class. The class to which the new image belongs is different from the class to which the original image belongs, but the content of the image is similar to that of the original image, so it is also called a pseudo-class. In other words, the image generation unit 13 generates a new image of a pseudo-class similar to the original image. The image generation unit 13 transmits the label of the original image and the generated new image to the data generation unit 14. The image generation unit 13 may also transmit the parameters used to generate the new image to the data generation unit 14.

[0021] When the data generation unit 14 receives a new image from the image generation unit 13, it determines a label to be assigned to the new image, which label belongs to a class different from the class to which the original image belongs. The data generation unit 14 generates data including the new image and a label assigned to the new image, which label belongs to a class different from the class to which the original image belongs. In other words, the set of the new image and the label assigned thereto is referred to as data. The data generation unit 14 may generate data including parameters.

[0022] 1, the acquisition unit 11, the determination unit 12, the image generation unit 13, and the data generation unit 14 are illustrated as being arranged together as a single information processing device 1, but this is not necessarily the case. That is, at least some of these may be arranged separately and connected to each other via wire or wirelessly so that information can be communicated. Furthermore, at least some of these may be arranged on the cloud.

[0023] The information processing device 1 may also include at least one processor, which may be configured to read stored programs and function as an acquisition unit 11, a determination unit 12, an image generation unit 13, and a data generation unit 14. Such a configuration will be described later.

[0024] As described above, the information processing device 1 according to this exemplary embodiment is configured to include the acquisition unit 11, the determination unit 12, the image generation unit 13, and the data generation unit 14. Therefore, the information processing device 1 according to this exemplary embodiment can generate new images belonging to a pseudo-class. Then, the generated new images can be used to train a classifier that identifies images of articles. Therefore, an effect can be obtained in which, in image recognition, an image of an unregistered object can be prevented from being mistakenly recognized as an image of a registered object.

[0025] (Information processing method S1) Next, an information processing method S1 executed by the information processing device 1 according to this exemplary embodiment will be described with reference to Fig. 2. Fig. 2 is a flow diagram showing the flow of the information processing method S1. As shown in Fig. 2, the information processing method S1 includes the following steps:

[0026] (Step S11) In step S11, at least one processor (acquisition unit 11) acquires an original image belonging to one of a plurality of classes.

[0027] (Step S12) In step S12, at least one processor (determining unit 12) determines parameters that define an image generation method.

[0028] (Step S13) In step S13, at least one processor (image generating unit 13) determines from the original image Department 12 generates a new image using the determined parameters.

[0029] (Step S14) In step S14, at least one processor (data generator 14) generates data including a new image and a label assigned to the new image, the label being of a class different from the class to which the original image belongs. The generated data is recorded in a predetermined database.

[0030] Furthermore, the data manufacturing method executed by the information processing device 1 includes the following steps, similar to the information processing method S1: the data manufacturing method includes the steps of: acquiring an original image belonging to one of a plurality of classes, determining parameters that define an image generation method, generating a new image from the original image using the determined parameters, and generating data that includes the new image and a label assigned to the new image, the label being of a class different from the class to which the original image belongs, by at least one processor.

[0031] As described above, the information processing method S1 and data production method according to this exemplary embodiment employ a configuration in which at least one processor acquires an original image belonging to one of multiple classes, determines parameters defining an image generation method, generates a new image from the original image using the determined parameters, and generates data including the new image and a label assigned to the new image, the label being of a class different from the class to which the original image belongs. In other words, the information processing method S1 according to this exemplary embodiment can generate training data that can train a classifier to identify images of objects. Therefore, the effect of image recognition is achieved, which is that it is possible to prevent images of unregistered objects from being mistaken for images of registered objects.

[0032] (Configuration of Information Processing System 2) Next, the information processing system 2 according to this exemplary embodiment will be described with reference to the drawings. Fig. 3 is a block diagram showing the configuration of the information processing system 2 according to this exemplary embodiment.

[0033] As shown in FIG. 3, the information processing system 2 includes an acquisition unit 11, a determination unit 12, an image generation unit 13, a data generation unit 14, and a database 25. The acquisition unit 11, the determination unit 12, the image generation unit 13, and the data generation unit 14 are as described above in the information processing device 1. The acquisition unit 11, the determination unit 12, the image generation unit 13, the data generation unit 14, and the database 25 are connected to each other via a network N including the Internet so that they can communicate information with each other. Note that it is not necessary for all of these units to be connected via the network N, and some may be directly connected wirelessly or via a wire. Furthermore, at least some of these units may be located on the cloud.

[0034] The acquisition unit 11 acquires an original image from a database 25. A plurality of images classified into a plurality of classes are recorded in the database 25. For example, in the example shown in FIG. 3, images classified into different classes from class A to class Z are recorded. Class A stores a plurality of images A1, A2, ... Am that belong to the same class A, and each image is assigned a label with an item name such as A. Class Z stores a plurality of images Z1, Z2, ... Zn that belong to the same class Z, and each image is assigned a label with an item name such as Z. In other words, classes are classified by labeling each item to be identified by the classifier.

[0035] The data generated by the data generation unit 14 is recorded in the database 25. Alternatively, the data generated by the data generation unit 14 may be recorded in a database different from the database 25. As an example, the data generated by the data generation unit 14 is data in which a label A' is assigned to an image A1' generated by the image generation unit 13 from an original image A1. FIG. 3 illustrates a state in which the image A1' assigned the label A' is recorded in the database 25 as a class A'.

[0036] In the information processing system 2 having the above configuration, the same effects as those obtained by the information processing device 1 described above can be obtained.

[0037] Exemplary Embodiment 2 A second exemplary embodiment of the present invention will be described in detail with reference to the drawings. Components having the same functions as those described in the first exemplary embodiment are given the same reference numerals, and their description will be omitted as appropriate. In this exemplary embodiment, an information processing device 3 that identifies product classes will be described as an example.

[0038] For example, in retail stores and the like, identification devices are being introduced for use in inventory management, price management, and the like. These identification devices identify products using images of product packaging. Retail stores need to handle a large number of new types of products or products with new packaging. New types of products or products with new packaging (collectively referred to as "new products") can be identified by registering the product type and an image of the package as a new class in the identification device.

[0039] However, it is difficult to register images of all new products that arrive daily into a classification device. Therefore, it is desirable to train the classification device so that it can identify new, unregistered products as new products that do not belong to any existing registered class. However, it is not easy to collect training data to train the classification device so that it can identify products that are similar in appearance to registered products as being different from the registered products.

[0040] The information processing device 1 according to this exemplary embodiment is a device that generates data for training a product class identification device (classifier). This identification device is, for example, a device that identifies whether an image belongs to one of the already registered product classes or whether it is an image that does not belong to any of the registered product classes. A class refers to a group to which images of substantially the same product belong, and each class is assigned a different label. Classes are set for each specific type of product, and each class is assigned a label, for example, the product name. However, products with the same product name but updated packaging are treated as products of a different class, and the class is assigned a different label.

[0041] Generally, product packages are composed of designs that do not have specific features such as cats or cars, but combine indefinite shapes, patterns, text, colors, etc. Furthermore, many product packages have designs in which only a portion of the design has been changed. Therefore, in order to train an image classifier that classifies whether a product package is the same as or different from a product in an already registered class, it is preferable to train the image classifier using images of product packages that are similar to the images of product packages in the registered class but belong to a different class. The information processing device 1 is a device that generates images for such training. Images of product packages are also referred to as product images.

[0042] (Configuration of information processing device 3) 4 is a block diagram showing the configuration of an information processing device 3 according to exemplary embodiment 2. The information processing device 3 includes an acquisition unit 11, a determination unit 12, an image generation unit 13, a data generation unit 14, and a dissimilarity determination unit 35.

[0043] As an example, the acquisition unit 11 acquires an original product image (hereinafter also simply referred to as an "original image") belonging to one of a plurality of registered product classes (hereinafter also simply referred to as a "class") from a database of product images. The database stores a plurality of product images classified into one of a plurality of classes. The acquisition unit 11 transmits the acquired original image to the determination unit 12.

[0044] When the determination unit 12 receives the original image from the acquisition unit 11, it determines parameters that define a method for generating a new product image (hereinafter also simply referred to as a "new image"). Alternatively, when the determination unit 12 receives the original image and parameters from the dissimilarity determination unit 35, it changes the parameters. After determining or changing the parameters, the determination unit 12 transmits the original image and the parameters to the image generation unit 13.

[0045] Upon receiving the original image and the parameters from the determination unit 12, the image generation unit 13 generates a new image from the original image using the parameters. After generating the new image, the image generation unit 13 transmits the original image and the new image to the dissimilarity determination unit 35.

[0046] The dissimilarity determination unit 35 derives the dissimilarity between the original image and the new image generated from the original image and compares it with a first threshold. The dissimilarity determination unit 35 is one form of the "dissimilarity determination means" described in the claims. If the dissimilarity between the original image and the new image is smaller than the first threshold, the dissimilarity determination unit 35 transmits the original image and parameters to the determination unit. If the dissimilarity between the original image and the new image is equal to or greater than the first threshold, the dissimilarity determination unit 35 transmits the label of the original image and the new image to the data generation unit 14.

[0047] When the data generation unit 14 receives the label of the original image and a new image from the dissimilarity determination unit 35, it generates data including the new image and a label assigned to the new image that is of a class different from the class to which the original product image belongs.

[0048] (Image generation method) Next, a method by which the image generation unit 13 generates a new image from an original image will be described with reference to the drawings. FIG. 5 is a diagram showing an example of a method for generating a new image executed by the image generation unit 13 of the information processing device 3. The image generation unit 13 generates a new image using at least one of converting at least a portion of colors, replacing at least a portion of characters, style conversion, interpolation using an image generation model, and replacing or superimposing a portion of an image. Specifically, the image generation unit 13 generates a new image using parameters determined by the determination unit 12. The parameters include a method parameter M that specifies a method for generating a new image, and a transformation parameter T that specifies a transformation value, degree of transformation, or transformation range of the image transformation by the method M when the new image is generated using the method M.

[0049] For example, the method parameter M may be a color conversion method M1 that converts colors, a character replacement method M2 that replaces characters with other characters, a style conversion method M3 that converts color combinations while preserving the overall shape and lines, an image interpolation method M4 that uses an image generation model, an image replacement method M5 that replaces a part of an image or superimposes another image or pattern on a part of an image, etc. The determination unit 12 first determines the method parameter M, and then determines a conversion parameter T that specifically specifies the conversion value, or the degree or range of conversion, for each of these methods.

[0050] Color conversion method M1 is a method of expressing the colors of an original image in HSV format and changing the hue, saturation, value, contrast, etc. (not shown). For example, the hues of the original image are arranged in a circular pattern in HSV format, and then converted into colors rotated by a predetermined angle, thereby generating a new image with a different hue. When hue is used in color conversion method M1, conversion parameter T1 is the angle by which the circularly arranged hues are rotated. The hues are arranged clockwise in the order of red → green → blue, and the colors of the original image are converted according to the angle of rotation.

[0051] Character replacement method M2 is a method for generating a new image 2012 by replacing character (string) portions in an original image 2011 with other characters (strings), as shown in 201 in Fig. 5. When character replacement method M2 is used, conversion parameters T2 include the proportion of characters (strings) to be replaced, the type of characters (strings) after replacement, and the font.

[0052] As shown in 202 of FIG. 5, style conversion method M3 is a method of combining an original image 2021 with another image 2022 to generate a new image 2023. As style conversion method M3, for example, AdaIN (Adaptive Instance Normalization) can be used to generate a new image. When using style conversion method M3, conversion parameters T3 include the type of the other image, the type of style, and color space values.

[0053] Image interpolation method M4 is a method for generating an intermediate image by combining two images with varying amounts of features. In the example shown in 203 of FIG. 5, the upper part 2031 of the figure shows an image combining the features of the handwritten digits 3 and 2, with the feature of 3 increasing toward the left and the feature of 2 increasing toward the right. The lower part 2032 of the figure similarly shows an image combining the features of the handwritten digits 5 and 6, with the feature of 5 increasing toward the left and the feature of 6 increasing toward the right. When using image interpolation method M4, the conversion parameter T4 is the ratio of the feature amounts of the two images. The ratio of the feature amounts of the two images may be determined based on the discrimination ability of a trained classifier. Alternatively, the ratio of the feature amounts of the two images may be determined based on the pattern of package changes.

[0054] Image replacement method M5 is a method of replacing a partial area of ​​an image with a different image or pattern, or a method of superimposing a different image or pattern on a partial area of ​​an image. In the example shown in 204 of FIG. 5, a star mark 2043 is superimposed on an original image 2041 to generate a new image 2042. When a different image or pattern is superimposed, an alpha blending method or the like can be used. When image replacement method M5 is used, conversion parameters T5 include the proportion of the partial area, the designation of the different image or pattern, an alpha value, etc.

[0055] The image generation unit 13 may generate multiple new images from one original image. For example, the image generation unit 13 may generate multiple new images from one original image using multiple image generation methods, or may generate multiple new images using the same image generation method by changing the transformation parameter T.

[0056] As described above, the image generation unit 13 can generate a new image from an original image using various methods, and the methods are not limited to those described above. Note that the image generation unit 13 may use a trained model using, for example, a neural network. In particular, when the image generation unit 13 employs, for example, the style conversion method M3 or the image interpolation method M4, it is preferable that the image generation unit 13 use a trained model using a neural network.

[0057] (Dissimilarity derivation method) Next, a method for deriving the dissimilarity and the first threshold will be described. The dissimilarity is derived as a numerical value, and the numerical value is compared with a preset first threshold. The method for deriving the dissimilarity is not limited, but the following method can be used, for example.

[0058] The dissimilarity between the original image and the new image can be derived using, for example, a neural network. For example, the dissimilarity determination unit 35 may input the two images, the original image and the new image, into a trained image recognition neural network such as VGG16, derive the average or total value of the differences in the outputs of multiple layers, and use this as the dissimilarity. Alternatively, the dissimilarity determination unit 35 may perform character recognition using a neural network, derive the degree of mismatch between characters in the images, and use this as the dissimilarity.

[0059] As a method that does not use a neural network, the dissimilarity determination unit 35 may derive the average or total value of the differences in pixel values ​​between two images and use this as the dissimilarity. Alternatively, the dissimilarity may be determined by having an evaluator (user) determine the dissimilarity. For example, the dissimilarity determination unit 35 displays the two images on a display, prompts the evaluator to input the dissimilarity between the two images within a preset range of values, and determines the value input by the user as the dissimilarity. The range of dissimilarity to be input may be a normalized range of values, for example, where 0 is set when the user determines that the two images are the same product packaging, and 1 is set when the user determines that the two images are clearly different product packaging.

[0060] When the user determines that the degree of difference is small enough that the new image is determined to be almost identical to the original image, it is preferable to have the image generation unit 13 generate a new image with a greater degree of difference. Therefore, when the degree of difference is smaller than a first threshold, the determination unit 12 changes the parameters so that the degree of difference becomes greater. To change the parameters to increase the degree of difference, for example, in the case of color conversion method M1, the rotation angle can be increased. In addition, in the case of character replacement method M2, the number of characters to be converted can be increased (increased), or the character type, such as hiragana, katakana, or kanji, can be changed. In addition, in the case of image replacement method M5, the area of ​​the replacement region can be increased.

[0061] Note that there are cases where it is unclear whether changing the parameters will increase the degree of dissimilarity. Therefore, the determination unit 12 may randomly change the parameters when the degree of dissimilarity is smaller than the first threshold. The determination unit 12 determines the degree of dissimilarity of a new image generated using the randomly changed parameters, and if the degree of dissimilarity increases, the parameters can be used continuously or further increased.

[0062] Furthermore, if the degree of dissimilarity between an original image and a new image generated from the original image using a certain parameter is smaller than a first threshold, the determination unit 12 may determine not to use that parameter. For example, if color conversion method M1 is used as method parameter M and its conversion parameter T is a hue rotation amount of "90 degrees," and the dissimilarity is smaller than the first threshold, the determination unit 12 may determine not to use that conversion parameter T. In this case, the determination unit 12 may use a hue rotation amount of "180 degrees" as the conversion parameter T. By making such a determination, it is possible to reduce the possibility of generating a new image with a small degree of dissimilarity.

[0063] The first threshold is set in advance based on a method for deriving the dissimilarity. For example, the first threshold may be set after accumulating data on the degree to which a new image generated from an original image using certain parameters differs from the original image. Alternatively, the first threshold may be set to the dissimilarity when a user compares the original image with the new image and determines that the images are different. Furthermore, the first threshold may be changed based on the results of training an image classifier.

[0064] (Effects of information processing device 3) As described above, the information processing device 3 according to this exemplary embodiment has a configuration in which it further includes a dissimilarity determination means that derives a dissimilarity between an original image and a new image and compares it with a first threshold value, in addition to the configuration according to the above-described information processing device 1 or 2. Therefore, the information processing device 3 according to this exemplary embodiment has the effect of reducing the possibility of generating a new image that is almost identical to the original image, in addition to the effect achieved by the information processing device 1 according to exemplary embodiment 1.

[0065] (Information processing method S2) Next, an information processing method S2 executed by the information processing device 3 according to this exemplary embodiment will be described with reference to Fig. 6. Fig. 6 is a flow diagram showing the flow of the information processing method S2. As shown in Fig. 6, the information processing method S2 includes the following steps:

[0066] (Step S21) In step S21, the acquisition unit 11 acquires an original image that belongs to one of a plurality of registered classes.

[0067] (Step S22) In step S22, the determination unit 12 determines (or changes) parameters that define the image generation method.

[0068] (Step S23) In step S23, the image generating unit 13 extracts the determined Department12 generates a new image using the determined (or changed) parameters.

[0069] (Step S24) In step S24, the dissimilarity determination unit 35 determines whether the dissimilarity between the original image and the new image is smaller than a first threshold. If it is determined in step S24 that the dissimilarity is smaller than the first threshold (step S24: Y), the process returns to step S22, and the determination unit 12 changes the parameters. On the other hand, if it is determined in step S24 that the dissimilarity is not smaller than the first threshold (step S24: N), the process proceeds to step S25.

[0070] (Step S25) In step S25, the data generator 14 generates data including a new image and a label assigned to the new image, the label being of a class different from the class to which the original image belongs. The generated data is recorded in a predetermined database.

[0071] As described above, if it is determined in step S24 that the dissimilarity is smaller than the first threshold value (step S24: Y), the determination unit 12 may decide not to use the parameter without returning to step S22.

[0072] (Effect of information processing method S2) As described above, the information processing method S2 according to this exemplary embodiment has the same configuration as the information processing method S1 according to exemplary embodiment 1, but further includes step S24 in which the dissimilarity determination unit 35 determines whether the dissimilarity between the original image and the new image is smaller than the first threshold value. Therefore, the information processing method S2 according to this exemplary embodiment has the same effect as the information processing method S1 according to exemplary embodiment 1, and also has the effect of reducing the possibility of generating a new image that is almost identical to the original image.

[0073] Exemplary Embodiment 3 A third exemplary embodiment of the present invention will be described in detail with reference to the drawings. Note that components having the same functions as those described in the first and second exemplary embodiments are denoted by the same reference numerals, and their description will not be repeated.

[0074] (Configuration of information processing device 4) 7 is a block diagram showing the configuration of an information processing device 4 according to exemplary embodiment 3. The information processing device 4 includes an acquisition unit 11, a determination unit 12, an image generation unit 13, a data generation unit 14, a dissimilarity determination unit 35, a classification unit 45, and an output unit 46. The acquisition unit 11, the determination unit 12, the image generation unit 13, and the data generation unit 14 are the same as the respective units described in embodiment 2, and therefore description thereof will be omitted.

[0075] The dissimilarity determination unit 35 has the same functions as the dissimilarity determination unit 35 of the information processing device 3 described above, but differs in that it derives the dissimilarity between the original image and the new image, and if the dissimilarity is equal to or greater than a first threshold, transmits the original image and the new image together with the parameters to the classification unit 45. Note that if the dissimilarity is smaller than the first threshold, the dissimilarity determination unit 35 transmits the original image and the parameters to the determination unit. This is similar to the processing performed by the dissimilarity determination unit 35 of the information processing device 3 described above.

[0076] The classifying unit 45 includes a model 451 for classifying an image. The classifying unit 45 is one form of the "first classifying means" described in the claims.

[0077] The output unit 46, for example, outputs the identification result derived by the identification unit 45 to the outside. The output unit 46 is a wired or wireless output interface. Specifically, it is an output terminal or the like for wired use, or a communication transmitter conforming to the Bluetooth (registered trademark) standard or the Wi-Fi (registered trademark) standard for wireless use. The identification result output from the output unit 46 is displayed on a display, for example.

[0078] The classification unit 45 will be described in detail below. The classification unit 45 derives a classification result by inputting a new image into a model 451 that classifies images. As an example, the model 451 that classifies images derives a similarity indicating the degree of similarity between the input new image and the original image. In this case, the classification result is the similarity between the input new image and the original image. The model 451 is an image classification model to be learned. In particular, the model 451 is preferably an image classification model to be learned using images generated by the information processing device 1, 3, or 4.

[0079] Furthermore, the classification unit 45 compares the derived similarity with a second threshold. If the classification result of the classification unit 45 indicates that the similarity between the new image and the original image is smaller than the second threshold, the determination unit 12 changes the parameters so that the similarity between the new image and the original image increases. Specifically, if the result indicates that the similarity is smaller than the second threshold, the classification unit 45 transmits the original image and the parameters to the determination unit 12. When the determination unit 12 receives the original image and the parameters from the classification unit 45, it changes the parameters so that the similarity between the new image and the original image increases. As parameter changes to increase the similarity, for example, in the case of color conversion method M1, the rotation angle can be reduced. Furthermore, in the case of character replacement method M2, the number of characters to be converted can be reduced. Furthermore, in the case of image replacement method M5, the replacement area can be reduced.

[0080] The method of deriving the similarity can be the same as the dissimilarity deriving method executed by the dissimilarity determining unit 35 described in the second exemplary embodiment. However, unlike the dissimilarity, the similarity becomes smaller as the degree of dissimilarity between the two increases. The second threshold is set in advance based on the method of deriving the similarity.

[0081] When the result shows that the similarity is smaller than the second threshold, the reason why the determination unit 12 changes the parameters so that the similarity between the new image and the original image is larger is to make the image generation unit 13 generate images suitable for training the image classifier. The reason is that even if the image classifier is trained using images with small similarity (large dissimilarity), the image classifier will not acquire the ability to distinguish images with large similarity, and in order to train the image classifier to acquire the ability to distinguish images with large similarity, it is necessary to train it using images with large similarity.

[0082] When an image classifier that classifies classes of items (products, etc.) is used as model 451, the classification result by the classification unit 45 includes the class into which the new image is classified. If the classification result indicates that the new image has been classified into a class different from the class to which the original image belongs, it is preferable that the determination unit 12 change the parameters so that the similarity between the new image and the original image increases. Specifically, if the classification result by the classification unit 45 indicates that the new image has been classified into a class different from the class to which the original image belongs, the classification unit 45 transmits the original image and the parameters to the determination unit 12. When the determination unit 12 receives the original image and the parameters from the classification unit 45, it changes the parameters so that the similarity between the new image and the original image increases. This configuration allows the image generation unit 13 to generate images suitable for training the image classifier.

[0083] Alternatively, it is preferable that the classification result by the classification unit 45 includes the class into which the new image has been classified and the reliability of the classification into that class. If the classification result indicates that the new image has been classified into a class different from the class to which the original image belongs and the reliability of the classification into the different class is greater than a third threshold, it is preferable that the determination unit 12 change the parameters so that the similarity between the new image and the original image is increased.

[0084] Specifically, if the classification result indicates that the new image is classified into a class different from the class to which the original image belongs and the reliability of the classification into the different class is greater than a third threshold, the classification unit 45 transmits the original image and parameters to the determination unit 12. When the determination unit 12 receives the original image and parameters from the classification unit 45, it changes the parameters so that the similarity between the new image and the original image increases. With this configuration, the parameters can be changed only when the reliability is greater than the third threshold, allowing the image generation unit 13 to efficiently generate a suitable image. The reliability of the classified class is, for example, the probability that the new image will be classified into a certain class. The third threshold is set in advance based on a reliability derivation method.

[0085] Depending on the classification result of the classification unit 45, the determination unit 12 may change the parameters so as to increase the similarity between the new image and the original image. This contradicts the aforementioned change in the parameters by the determination unit 12 so as to decrease the dissimilarity between the new image and the original image, depending on the classification result of the dissimilarity determination unit 35. In this exemplary embodiment, the reason for providing the functions of the dissimilarity determination unit 35 and the classification unit 45, which have contradictory roles, is as follows. That is, if only the dissimilarity determination unit 35 is provided, only images with large dissimilarity from the original image will be generated, which may make it impossible to learn to distinguish images with small dissimilarity. Conversely, if only the classification unit 45 is provided, only images with large similarity to the original image will be generated, which may make it impossible to learn to distinguish images with small similarity. By providing both the dissimilarity determination unit 35 and the classification unit 45, it is possible to generate various types of training images necessary for appropriate training.

[0086] The similarity may be derived using a neural network as described above, through image analysis, or through user judgment. However, if the classifier to be trained is to identify artificial differences in objects such as product packaging, it is preferable for the user to determine the level of discrimination they wish to achieve. Therefore, it is preferable for the first, second, and third thresholds to be set by the user according to the level of discrimination they wish to achieve.

[0087] (Effects of information processing device 4) As described above, the information processing device 4 according to the present exemplary embodiment 3 has a configuration in which, in addition to the configuration of the above-described information processing devices 1 to 3, it further includes a classification unit 45 that derives a classification result by inputting a new image into the image classification model 451. Therefore, according to the information processing device 4 according to the present exemplary embodiment 3, in addition to the effects achieved by the information processing devices 1 to 3 according to the exemplary embodiment 1, it is possible to obtain an effect of being able to generate various types of images necessary for appropriate learning.

[0088] (Information processing method S3) Next, an information processing method S3 executed by the information processing device 4 will be described with reference to the drawings. Fig. 8 is a flowchart showing the flow of the information processing method S3 according to this exemplary embodiment. As shown in Fig. 8, steps S31, S32, S33, and S34 of the information processing method S3 are the same as steps S21, S22, S23, and S24 of the information processing method S2 described above.

[0089] If it is determined in step S34 that the dissimilarity is smaller than the first threshold (step S34: Y), the process returns to step S32, and the determination unit 12 changes the parameter. On the other hand, if it is determined in step S34 that the dissimilarity is equal to or greater than the first threshold (step S34: N), the process proceeds to step S35.

[0090] (Step S35) In step S35, the classification unit 45 determines whether the classification result by the model 451 indicates that the similarity between the new image and the original image is smaller than a second threshold value.

[0091] In step S35, if it is determined that the similarity between the new image and the original image is smaller than the second threshold value (step S35: Y), the process returns to step S32 and the determination is made. Department 12 changes the parameters so that the similarity between the new image and the original image increases. If it is determined in step S35 that the similarity between the new image and the original image is equal to or greater than the second threshold (step S35: N), the process proceeds to step S36.

[0092] (Step S36) In step S36, the data generating unit 14 generates data in which the new image is labeled with a label of a class different from the class to which the original image belongs, and the generated data is recorded in a predetermined database.

[0093] Next, an information processing method S4 executed by the information processing device 4 will be described with reference to the drawings. Fig. 9 is a flowchart showing the flow of the information processing method S4 according to this exemplary embodiment. As shown in Fig. 9, steps S41, S42, S43, and S44 of the information processing method S4 are the same as steps S31, S32, S33, and S34 of the information processing method S3 described above.

[0094] (Step S45) In step S44, if it is determined that the degree of difference is equal to or greater than the first threshold value (step S 44 :N), the process proceeds to step S45. In step S45, the classification unit 45 determines whether or not the classification result by the model 451 indicates that the new image has been classified into a class different from the class to which the original image belongs.

[0095] In step S45, if it is determined that the new image has been classified into a class different from the class to which the original image belongs (step S45: Y), the process returns to step S42 and the decision is made. Department 12 changes the parameters so that the similarity between the new image and the original image increases. If it is determined in step S45 that the new image is not classified into a class different from the class to which the original image belongs (step S45: N), the process proceeds to step S46.

[0096] (Step S46) In step S46, the data generating unit 14 generates data in which the new image is labeled with a label of a class different from the class to which the original image belongs, and the generated data is recorded in a predetermined database.

[0097] Next, an information processing method S5 executed by the information processing device 4 will be described with reference to the drawings. Fig. 10 is a flowchart showing the flow of the information processing method S5 according to this exemplary embodiment. As shown in Fig. 10, steps S51, S52, S53, and S54 of the information processing method S5 are the same as steps S41, S42, S43, and S44 of the information processing method S4 described above.

[0098] (Step S55) If it is determined in step S54 that the dissimilarity is equal to or greater than the first threshold (step S54: N), the process proceeds to step S55. In step S55, the classification unit 45 determines whether the classification result by the model 451 indicates that the new image is classified into a class different from the class to which the original image belongs, and whether the reliability of the classification into the different class is greater than a third threshold.

[0099] In step S55, if it is determined that the new image is classified into a class different from the class to which the original image belongs, and the reliability of the classification into the different class is greater than the third threshold (step S55: Y), the process returns to step S52 and a decision is made. Department12 changes the parameters so that the similarity between the new image and the original image increases.

[0100] In step S55, if it is determined that the new image is not classified into a class different from the class to which the original image belongs, or that the new image is classified into a class different from the class to which the original image belongs but the reliability of the classification into the different class is not greater than the third threshold (step S55: N), proceed to step S56.

[0101] (Step S56) In step S56, the data generating unit 14 generates data in which the new image is labeled with a label of a class different from the class to which the original image belongs, and the generated data is recorded in a predetermined database.

[0102] (Effects of information processing methods S3, S4, and S5) As described above, in the information processing methods S3 to S5 according to this exemplary embodiment, in addition to the configuration of the information processing method S1 according to exemplary embodiment 1, the classification unit 45 is configured to include steps S35, S45, and S55 for determining whether the similarity between a new image and the original image is smaller than a second threshold value as a classification result by the model 451. Therefore, according to the information processing methods S3 to S5 according to this exemplary embodiment, in addition to the effect achieved by the information processing method S1 according to exemplary embodiment 1, the effect of being able to generate various types of images necessary for appropriate learning can be obtained.

[0103] Exemplary Embodiment 4 A fourth exemplary embodiment of the present invention will be described in detail with reference to the drawings. Note that components having the same functions as those described in the first to third exemplary embodiments are denoted by the same reference numerals, and their description will not be repeated.

[0104] (Configuration of information processing device 5) 11 is a block diagram showing the configuration of an information processing device 5 according to exemplary embodiment 4. The information processing device 5 includes an acquisition unit 11, a determination unit 12, an image generation unit 13, a data generation unit 14, a learning unit 55, and a database 56. The acquisition unit 11, the determination unit 12, the image generation unit 13, and the data generation unit 14 are the same as the respective units described in embodiments 2 and 3.

[0105] As shown in Fig. 11, the learning unit 55 includes a target model 551 to be trained. The target model 551 is a classifier that identifies a product class from an image of the product. The database 56 contains a data generator. Department A plurality of new images generated from the original image by the image processing unit 14 are recorded together with the original image. The learning unit 55 is one form of the "learning means" described in the claims.

[0106] The learning unit 55 generates data Department Specifically, the learning unit 55 learns the target model 551 by referring to the data generated by the data generation unit 14. Department 14, acquires a new image generated by the image recognition unit 14, and inputs it to the target model 551. Then, the target model 551 is trained so that the classification result output by the target model 551 is correct. A correct classification result is a result in which, when a new image is input, the image does not belong to any of the classes to which the original image registered in the database 56 belongs. Note that the learning unit 55 may acquire an original image from the database 56, input it to the target model 551, and train it to output a correct class. Furthermore, the target model 551 may be the same classification model as the model 451 of the classification unit 45 described in the information processing device 4.

[0107] FIG. 12 is a schematic diagram of the configuration of a target model 551 to be trained. As shown in FIG. 12, the target model 551 is a convolutional neural network consisting of multiple layers. When a new image belonging to class A' is input to the target model 551, the target model 551 outputs the class to which this image is thought to belong and its reliability. Note that class A of the output result indicates the class to which the original image of the new image belonging to class A' belongs. Class A'' indicates the class to which another new image generated from the same original image belongs. Class K indicates a class of the original image that is different from class A.

[0108] The learning unit 55 calculates a loss value of the output of the target model 551, and trains the target model 551 so as to reduce the loss value. The loss value is, for example, the total reliability of classes other than the correct answer. For example, if the reliability of class A is 0.10, the reliability of class A' (correct answer) is 0.80, the reliability of class A'' is 0.05, and the reliability of class K is 0.05, the loss value is 0.2. Training the target model 551 by the learning unit 55 means updating the weights of the function formulas in each layer of the convolutional neural network so as to reduce the loss value.

[0109] (Effects of information processing device 5) As described above, the information processing device 5 according to the fourth exemplary embodiment has the configuration of the above-described information processing devices 1 to 4, and further has a data generation function. Department The information processing device 5 according to the fourth exemplary embodiment further includes a learning unit 55 that learns an object model 551 by referring to the data generated by the information processing device 14. Therefore, in addition to the effects of the information processing devices 1 to 4 according to the first to third exemplary embodiments, the information processing device 5 according to the fourth exemplary embodiment can achieve the effect of learning an object model using the newly generated image.

[0110] Exemplary Embodiment 5 A fifth exemplary embodiment of the present invention will be described in detail with reference to the drawings. Note that components having the same functions as those described in the first to fourth exemplary embodiments are denoted by the same reference numerals, and their description will not be repeated.

[0111] (Configuration of information processing device 6) 13 is a block diagram showing the configuration of an information processing device 6 according to exemplary embodiment 5. The information processing device 6 includes a classification target image acquisition unit 61, a determination unit 12, an image generation unit 13, a data generation unit 14, a learning unit 55, a second classification unit 66, a database 67, and an input / output unit 68. The determination unit 12, the image generation unit 13, the data generation unit 14, the learning unit 55, and the database 67 are the same as the respective units described in embodiment 4.

[0112] The classification target image acquisition unit 61 acquires a classification target image. The classification target image may be an image recorded in the database 67, or may be an image stored outside the information processing device 6. The classification target image acquisition unit 61 acquires an image stored outside the information processing device 6 via the input / output unit 68. The second classification unit 66 is provided with a trained model 661, which is the target model 551 trained by the learning unit 55. The second classification unit 66 performs a classification process on the classification target image by inputting the classification target image acquired by the classification target image acquisition unit 61 into the trained model 661 trained by the learning unit 55.

[0113] For example, when an image is input, the trained model 661 outputs a class to which the image may belong along with a confidence level. The second identification unit 66 may output whether the image corresponds to one of the registered classes or does not correspond to any of the registered classes along with the confidence level. The input / output unit 68 is an interface for acquiring images from the outside or outputting the identification results to the outside.

[0114] (Effects of information processing device 6) As described above, the information processing device 6 according to the present exemplary embodiment 5 has the same configuration as the information processing devices 1 to 5 described above, but further includes a second classification unit 66 that performs classification processing on the classification target image by inputting the classification target image acquired by the classification target image acquisition unit 61 into the target model 551 (trained model 661) trained by the learning unit 55. Therefore, the information processing device 6 according to the present exemplary embodiment 5 has the same effect as the information processing devices 1 to 5 according to the exemplary embodiments 1 to 4, and also has the effect of being able to classify images using the trained target model.

[0115] Exemplary Embodiment 6 A sixth exemplary embodiment of the present invention will be described in detail with reference to the drawings. Note that components having the same functions as those described in the first to fifth exemplary embodiments are denoted by the same reference numerals, and their description will not be repeated.

[0116] (Configuration of information processing device 7) FIG. 14 is a block diagram showing the configuration of an information processing device 7 according to a sixth exemplary embodiment. The information processing device 7 includes an acquisition unit 71, a learning unit 72, and a database 73. The acquisition unit 71 acquires training data including a plurality of images, class labels assigned to each of the plurality of images, and identification information assigned to at least some of the plurality of images, the identification information being for identifying an image generation process for the at least some of the images. The acquisition unit 71 acquires the training data from the database 73, for example. The learning unit 72 includes a target model 721, which is a model to be trained. The learning unit 72 trains the target model 721 by referring to the training data acquired by the acquisition unit 71. In other words, the learning unit 72 inputs the training data to the target model 721 and trains the target model 721 so as to reduce the loss value of the output classification result. The database 73 stores the training data.

[0117] As an example, the target model 721 may include two layers made up of a convolutional neural network, as shown in FIG. 15. One of the layers is a common layer 7211 that is applied regardless of the identification information, and the other layer is branch layers 7212 and 7213 that are selectively applied depending on the identification information. The branch layer 7212 is trained to have a high ability to distinguish between images of patterns with changed hues. On the other hand, the branch layer 7213 is trained to have a high ability to distinguish between images of patterns with changed characters.

[0118] Identification information assigned to an image is information that indicates the type of image that the image is. As an example, identification information is information that indicates the method by which the image was generated. For example, as shown in FIG. 15, image a'(H) of class A', which was generated by changing the hue of image a of class A, is assigned identification information H. Also, image b'(L) of class B', which was generated by changing the characters from the original image of class B (not shown), is assigned identification information L. Note that image a of class A is not assigned identification information.

[0119] When an image a'(H) is input to the target model 721, image processing is performed using a common layer 7211 and a branching layer 7212, as indicated by the solid line in the figure. The learning unit 72 trains the target model 721 so that the total loss value (Loss1) of the output values ​​output from the branching layer 7212 becomes small.

[0120] On the other hand, when image b'(L) is input to the target model 721, image processing is performed using the common layer 7211 and the branching layer 7213, as indicated by the thick dashed line in the figure. The learning unit 72 trains the target model 721 so as to reduce the total loss value (Loss2) of the output values ​​output from the branching layer 7213.

[0121] When image a is input to the target model 721, image processing may be performed using both the common layer 7211 and the branch layers 7212 and 7213, as shown by the thin dashed line in the figure. The learning unit 72 trains the target model 721 so that the sum (Loss) of the total loss value (Loss1) of the output values ​​output from the branch layer 7212 and the total loss value (Loss2) of the output values ​​output from the branch layer 7213 becomes small.

[0122] In this way, by using an image processing layer that is suitable for an image depending on the identification information such as how the image was generated, the accuracy of image identification can be further improved.

[0123] (Effects of information processing device 7) As described above, the information processing device 7 according to this exemplary embodiment includes an acquisition unit 71 that acquires training data including a plurality of images, class labels assigned to each of the plurality of images, and identification information assigned to at least some of the plurality of images, the identification information being for identifying image generation processing for the at least some of the images, and a learning unit 72 that trains a target model 721 by referring to the training data acquired by the acquisition unit 71. The target model 721 includes a common layer 7211 that is applied regardless of the identification information, and branch layers 7212 and 7213 that are selectively applied depending on the identification information. Therefore, the information processing device 7 according to this exemplary embodiment has the advantage of being able to improve classification accuracy by changing the image processing path depending on the characteristics of the image, in addition to the advantages achieved by the information processing device 1 according to the exemplary embodiment 1.

[0124] Exemplary Embodiment 7 A seventh exemplary embodiment of the present invention will be described in detail with reference to the drawings. Note that components having the same functions as those described in the first to sixth exemplary embodiments are denoted by the same reference numerals, and their description will not be repeated.

[0125] (Configuration of information processing device 8) FIG. 16 is a block diagram showing the configuration of an information processing device 8 according to a seventh exemplary embodiment. The information processing device 8 includes a classification target image acquisition unit 81, a classification unit 82, and an output unit 83. The classification target image acquisition unit 81 acquires a classification target image. The classification target image acquisition unit 81 may acquire the classification target image from a memory (not shown) or an external database (not shown). The classification unit 82 includes a trained model 821, and performs classification processing on the classification target image by inputting the classification target image acquired by the classification target image acquisition unit 81 into the trained model 821, and outputs the results of the classification processing. The output unit 83 is an interface that outputs the results of the classification processing output by the trained model 821 to the outside.

[0126] The trained model 821 is a model trained using training data including an image labeled with a first class and an image generated from the image labeled with the first class and labeled with one or more second classes different from the first class. The first class corresponds to the class of the original image described above. The second class image corresponds to the pseudo class described above.

[0127] That is, the trained model 821 is an image recognition model trained using data generated by the above-described information processing devices (information processing systems) 1 to 4. Alternatively, the trained model 821 is a model equivalent to the target model 551 trained by the training unit 55 of the above-described information processing device 5, the trained model 661 provided in the second classification unit 66 of the information processing device 6, or the target model 721 trained by the training unit 72 of the information processing device 7. With this configuration, the information processing device 8 can classify the acquired classification target image.

[0128] The information output by the classification unit 82 and its output format are arbitrary. As a result of the classification process, the classification unit 82 may output information regarding to which of a first class and one or more second classes the classification target image belongs. For example, the classification unit 82 may output the first class and / or the second class to which the classification target image may belong, and the reliability (probability, etc.) of each class. With this configuration, it is possible to output multiple possible classes and their reliability.

[0129] Alternatively, when the output of the trained model 821 indicates that the classification target image belongs to one or more second classes, the classification unit 82 may output information indicating that the classification target image belongs to the first class as a result of the classification process. For example, when the output of the trained model 821 indicates that the classification target image belongs to one of the second classes (pseudo-classes), the classification unit 82 may output only the identified class. In this case, the output is simple.

[0130] (Flow of image inference method S6) Next, the flow of the information processing method (inference method) S6 for image classes executed by the information processing device 8 will be described with reference to the drawings. Fig. 17 is a flowchart showing the flow of the information processing method S6. As shown in Fig. 17, the information processing method S6 includes the following steps:

[0131] (Step S61) In step S61, the classification target image acquisition unit 81 acquires a classification target image.

[0132] (Step S62) In step S62, the classification unit 82 performs classification processing on the classification target image by inputting the classification target image acquired by the classification target image acquisition unit 81 into the trained model 821. The trained model 821 is as described above.

[0133] (Step S63) In step S63, the classification unit 82 (or the output unit 83) outputs the result of the classification process performed by the classification unit 82. Furthermore, the output unit 83 may output the classification result to the outside.

[0134] (Configuration and effects of information processing device 8 and inference method S6) The information processing device 8 according to the seventh exemplary embodiment includes a classification target image acquisition unit 81 that acquires a classification target image, and a classification unit 82 that performs classification processing on the classification target image by inputting the classification target image acquired by the classification target image acquisition unit 81 into a trained model 821. The inference method S6 includes acquiring the classification target image, inputting the classification target image acquired by the classification target image acquisition unit 81 into the trained model 821, and Part 81 The method includes inputting the classification target image acquired by the image processing unit 100, and performing classification processing on the classification target image.

[0135] Therefore, according to the information processing device 8 and the inference method S6 of this exemplary embodiment 7, it is possible to obtain the effect of being able to classify the image to be classified using the trained model 821 trained using a new image. [Example]

[0136] Next, an example will be described. Fig. 18 is a graph showing the accuracy rate of a classifier trained using only original image data and the accuracy rate of a classifier trained using the original image as well as a new image generated from the original image. In this example, the classifier was made to classify whether an input image belonged to one of 50 classes registered in the classifier, or whether it did not belong to any of the registered classes. The accuracy rate is the proportion of images that were correctly classified as the registered class under conditions in which the classifier parameters were set so that the rate of misclassifying images of a non-registered class as images of a registered class was 5% or less.

[0137] The bar graph on the left side of the graph in Figure 18 shows the accuracy rate of the classifier trained using only the original images of the registered class. The bar graph on the right side shows the accuracy rate of the classifier trained using new images as well. As shown in Figure 18, the accuracy rate of the classifier trained using only the original images of the registered class was 0.5, but the accuracy rate of the classifier trained using new images as well improved to 0.71.

[0138] As described above, it has been found that new images generated using the information processing device according to this exemplary embodiment serve as effective training data for training a classifier.

[0139] [Software implementation example] Some or all of the functions of the information processing devices 1, 3 to 8 and the information processing system 2 (collectively referred to as "information processing device 1, etc.") may be realized by hardware such as an integrated circuit (IC chip), or by software.

[0140] In the latter case, the information processing device 1, etc., is realized by, for example, a computer that executes instructions of a program, which is software that realizes each function. An example of such a computer (hereinafter referred to as computer C) is shown in FIG. 19. The computer C includes at least one processor C1 and at least one memory C2. The memory C2 stores a program P for operating the computer C as the information processing device 1, etc. In the computer C, the processor C1 reads and executes the program P from the memory C2, thereby realizing each function of the information processing device 1, etc.

[0141] The processor C1 may be, for example, a central processing unit (CPU), a graphics processing unit (GPU), a digital signal processor (DSP), a micro processing unit (MPU), a floating point number processing unit (FPU), a physics processing unit (PPU), a microcontroller, or a combination thereof. The memory C2 may be, for example, a flash memory, a hard disk drive (HDD), a solid state drive (SSD), or a combination thereof.

[0142] The computer C may further include a RAM (Random Access Memory) for expanding the program P during execution and for temporarily storing various data. The computer C may also include a communication interface for transmitting and receiving data to and from other devices. The computer C may also include an input / output interface for connecting input / output devices such as a keyboard, mouse, display, and printer. Furthermore, the program P can be recorded on a non-transitory tangible recording medium M that can be read by the computer C. Such a recording medium M can be, for example, a tape, a disk, a card, a semiconductor memory, or a programmable logic circuit. The computer C can acquire the program P via such a recording medium M. The program P can also be transmitted via a transmission medium. Such a transmission medium can be, for example, a communication network or broadcast waves. The computer C can also acquire the program P via such a transmission medium.

[0143] [Appendix 1] The present invention is not limited to the above-described embodiments, and various modifications are possible within the scope of the claims. For example, embodiments obtained by appropriately combining the technical means disclosed in the above-described embodiments are also included in the technical scope of the present invention.

[0144] [Appendix 2] Some or all of the above-described embodiments can also be described as follows: However, the present invention is not limited to the following described aspects.

[0145] (Appendix 1) An information processing device comprising: an acquisition means for acquiring an original image belonging to one of a plurality of classes; a determination means for determining parameters that define an image generation method; an image generation means for generating a new image from the original image using the parameters determined by the determination means; and a data generation means for generating data including the new image and a label assigned to the new image, the label being of a class different from the class to which the original image belongs. According to the above configuration, in image recognition, it is possible to prevent an image of an unregistered object from being mistakenly recognized as an image of a registered object.

[0146] (Appendix 2) 2. The information processing device according to claim 1, further comprising a dissimilarity determining means for deriving a dissimilarity between the original image and the new image and comparing it with a first threshold value. According to the above configuration, it is possible to reduce the possibility that a new image that is almost identical to the original image will be generated.

[0147] (Appendix 3) 3. The information processing device according to claim 2, wherein the determining means changes the parameter when the dissimilarity derived by the dissimilarity determining means is smaller than the first threshold value. According to the above configuration, it is possible to reduce the possibility that a new image with a small degree of difference will be generated.

[0148] (Appendix 4) 4. The information processing device according to claim 3, wherein the determining means changes the parameter so that the degree of difference becomes larger when the degree of difference is smaller than the first threshold value. According to the above configuration, it is possible to reduce the possibility that a new image with a small degree of difference will be generated.

[0149] (Appendix 5) 4. The information processing device according to claim 3, wherein the determining means randomly changes the parameter when the degree of difference is smaller than the first threshold value. According to the above configuration, it is possible to reduce the possibility that a new image with a small degree of difference will be generated.

[0150] (Appendix 6) 6. The information processing device according to any one of claims 1 to 5, further comprising a classification unit that derives a classification result by inputting the new image into a model that classifies images. According to the above configuration, it is possible to generate various types of images necessary for appropriate learning.

[0151] (Appendix 7) The information processing device described in Appendix 6, characterized in that when the classification result indicates that the similarity between the new image and the original image is smaller than a second threshold, the determination means changes the parameters so that the similarity between the new image and the original image is increased. According to the above configuration, it is possible to generate various types of images necessary for appropriate learning.

[0152] (Appendix 8) the classification result includes a class into which the new image has been classified; The information processing device described in Appendix 6, characterized in that when the classification result indicates that the new image has been classified into a class different from the class to which the original image belongs, the determination means changes the parameters so that the similarity between the new image and the original image is increased. According to the above configuration, it is possible to generate various types of images necessary for appropriate learning.

[0153] (Appendix 9) The information processing device described in Appendix 8, characterized in that the classification result includes the class into which the new image has been classified and the confidence level regarding the classification into that class, and when the classification result indicates that the new image has been classified into a class different from the class to which the original image belongs and the confidence level regarding the classification into the different class is greater than a third threshold, the determination means changes the parameters so that the similarity between the new image and the original image is increased. According to the above configuration, it is possible to generate various types of learning images necessary for appropriate learning.

[0154] (Appendix 10) The information processing device described in any one of appendices 1 to 9, characterized in that the image generation means generates the new image using at least one of converting at least a portion of the color, replacing at least a portion of the characters, style conversion, interpolation using an image generation model, and replacing or superimposing a portion of the image. According to the above configuration, a new image can be generated from an original image using various methods.

[0155] (Appendix 11) 11. The information processing device according to any one of appendices 1 to 10, further comprising: a learning means for learning a target model by referring to the data generated by the data generating means. According to the above configuration, the target model can be trained using the generated new image.

[0156] (Appendix 12) 12. The information processing device according to claim 11, further comprising: a classification target image acquisition means for acquiring a classification target image; and a second classification means for performing a classification process on the classification target image by inputting the classification target image acquired by the classification target image acquisition means into the target model learned by the learning means. According to the above configuration, it is possible to classify the classification target image using the trained target model.

[0157] (Appendix 13) 1. An information processing apparatus comprising: an acquisition means for acquiring learning data including a plurality of images, class labels assigned to each of the plurality of images, and identification information assigned to at least some of the plurality of images, the identification information being for identifying an image generation process for the at least some of the images; and a learning means for training a target model by referring to the learning data acquired by the acquisition means, wherein the target model comprises a common layer that is applied regardless of the identification information, and a branch layer that is selectively applied in accordance with the identification information. According to the above configuration, the image processing path is changed in accordance with the characteristics of the image, thereby improving the classification accuracy.

[0158] (Appendix 14) an information processing device comprising: a classification target image acquisition means for acquiring a classification target image; and a classification means for performing a classification process on the classification target image by inputting the classification target image acquired by the classification target image acquisition means into a model trained using learning data including images labeled with a first class and images generated from the images labeled with the first class, the images being labeled with one or more second classes different from the first class. According to the above configuration, the information processing device can classify the image to be classified using a trained model trained using a new image.

[0159] (Appendix 15) The information processing device described in Appendix 14, characterized in that the classification means outputs, as a result of the classification process, information regarding whether the image to be classified belongs to the first class or the one or more second classes. This configuration allows multiple possible classes to be output along with their confidence levels.

[0160] (Appendix 16) The information processing device described in Appendix 14, characterized in that the classification means outputs information indicating that the image to be classified belongs to the first class as a result of the classification process when the output of the model indicates that the image to be classified belongs to any of the one or more second classes. According to the above configuration, a simple output result can be output.

[0161] (Appendix 17) An information processing method including: at least one processor acquiring an original image belonging to one of a plurality of classes; determining parameters that specify an image generation method; generating a new image from the original image using the determined parameters; and generating data including the new image and a label assigned to the new image, the label being of a class different from the class to which the original image belongs. According to the above configuration, it is possible to generate training data that can train a classifier to identify images of articles, and therefore, in image recognition, it is possible to prevent images of unregistered objects from being mistakenly recognized as images of registered objects.

[0162] (Appendix 18) A data production method including at least one processor acquiring an original image belonging to one of a plurality of classes, determining parameters that specify an image generation method, generating a new image from the original image using the determined parameters, and generating data including the new image and a label assigned to the new image, the label being of a class different from the class to which the original image belongs. According to the above configuration, it is possible to produce training data that can train a classifier that can identify images of articles, and therefore, in image recognition, it is possible to prevent images of unregistered objects from being mistakenly recognized as images of registered objects.

[0163] (Appendix 19) A program for causing a computer to operate as the information processing device according to any one of appendices 1 to 16, the program causing the computer to function as each of the means.

[0164] (Appendix 20) A computer-readable non-transitory recording medium having the program described in Appendix 19 recorded thereon.

[0165] (Appendix 21) A model is trained using training data including an image to be classified, the image being assigned a label of a first class, and an image generated from the image to which the label of the first class is assigned, the image being assigned a label of one or more second classes different from the first class, The above-mentioned obtained 1. An information processing method comprising: inputting an image to be classified, and performing a classification process on the image to be classified. According to the above configuration, it is possible to classify the image to be classified using a trained model trained using a new image.

[0166] [Appendix 3] Some or all of the above-described embodiments can also be expressed as follows. An information processing device comprising at least one processor that executes an acquisition process for acquiring an original image belonging to one of a plurality of classes, a determination process for determining parameters that define an image generation method, an image generation process for generating a new image from the original image using the parameters determined by the determination means, and a data generation process for generating data including the new image and a label assigned to the new image, the label being of a class different from the class to which the original image belongs.

[0167] The information processing device may further include a memory that stores a program for causing the processor to execute the acquisition process, the preliminary determination process, the image generation process, and the data generation process. The program may be recorded on a computer-readable, non-transitory, tangible recording medium. [Explanation of symbols]

[0168] 1,3,4,5,6,7,8... Information processing device 2. Information Processing System 11,71...Acquisition part 12. Decision section 13. Image generation unit 14. Data generation section 25,56,67,73···Database 35...Difference determination unit 45,82 Identification unit 46,83... Output section 55,72···Learning Department 61, 81... Image acquisition unit for identification 66 Second Identification Unit 68...Input / output section

Claims

1. an acquisition means for acquiring an original image belonging to one of a plurality of classes; determining means for determining parameters defining an image generation method; an image generating means for generating a new image from the original image using the parameters determined by the determining means; a data generating means for generating data including the new image and a label assigned to the new image, the label being of a class different from the class to which the original image belongs; a difference determination means for deriving a difference between the original image and the new image and comparing the difference with a first threshold value; When the degree of difference is smaller than the first threshold, the determining means changes the parameter so that the degree of difference becomes larger. Information processing device.

2. An acquisition means for acquiring an original image belonging to any one of a plurality of classes; determining means for determining parameters defining an image generation method; an image generating means for generating a new image from the original image using the parameters determined by the determining means; a data generating means for generating data including the new image and a label assigned to the new image, the label being of a class different from the class to which the original image belongs; a difference determination means for deriving a difference between the original image and the new image and comparing the difference with a first threshold value; The determining means randomly changes the parameter when the degree of difference is smaller than the first threshold. Information processing device.

3. The image recognition system further includes a first recognition unit for deriving a recognition result by inputting the new image into a model for recognizing an image.

3. The information processing device according to claim 1 or 2.

4. When the classification result indicates that the similarity between the new image and the original image is smaller than a second threshold, the determining means changes the parameters so that the similarity between the new image and the original image increases. The information processing device according to claim 3 .

5. the classification result includes a class into which the new image has been classified; When the classification result indicates that the new image has been classified into a class different from the class to which the original image belongs, the determining means changes the parameters so that the similarity between the new image and the original image increases. The information processing device according to claim 3 .

6. At least one processor Obtaining an original image that belongs to one of multiple classes; determining parameters that define an image generation method; generating a new image from the original image using the determined parameters; generating data including the new image and a label assigned to the new image, the label being of a class different from the class to which the original image belongs; deriving a dissimilarity between the original image and the new image and comparing it with a first threshold; and if the derived dissimilarity is smaller than the first threshold value in the comparing, changing the parameter in the determining step so that the dissimilarity is increased. An information processing method including:

7. At least one processor: Obtaining an original image that belongs to one of multiple classes; determining parameters that define an image generation method; generating a new image from the original image using the determined parameters; generating data including the new image and a label assigned to the new image, the label being of a class different from the class to which the original image belongs; deriving a dissimilarity between the original image and the new image and comparing it with a first threshold; and randomly varying the parameter in the determining step if the derived dissimilarity in the comparing step is less than the first threshold. An information processing method including:

8. A program for causing a computer to function as an information processing device, the program comprising: an acquisition means for acquiring an original image belonging to one of a plurality of classes; determining means for determining parameters defining an image generation method; an image generating means for generating a new image from the original image using the parameters determined by the determining means; a data generating means for generating data including the new image and a label assigned to the new image, the label being of a class different from the class to which the original image belongs; and a difference determination unit that derives a difference between the original image and the new image and compares the difference with a first threshold value; When the dissimilarity calculated by the dissimilarity determining means is smaller than the first threshold, the determining means changes the parameter so that the dissimilarity becomes larger. program.

9. A program for causing a computer to function as an information processing device, the program comprising: an acquisition means for acquiring an original image belonging to one of a plurality of classes; determining means for determining parameters defining an image generation method; an image generating means for generating a new image from the original image using the parameters determined by the determining means; a data generating means for generating data including the new image and a label assigned to the new image, the label being of a class different from the class to which the original image belongs; and a difference determination unit that derives a difference between the original image and the new image and compares the difference with a first threshold value; When the dissimilarity calculated by the dissimilarity determining means is smaller than the first threshold, the determining means randomly changes the parameter. program.

Images