Information processing device, information processing method and program

Abstract

To improve the distinguish precision among categories of an object that becomes likely to be incorrectly distinguished when input data is in a certain condition.SOLUTION: An information processing device for learning a model that distinguishes the category of an object in an image includes: attribute obtaining means for obtaining an attribute from the image; and generating means for generating, when learning the model, a group of a plurality of categories based on the attribute obtained by the attribute obtaining means.SELECTED DRAWING: Figure 6

Description

Technical Field

[0001] The present invention relates to an information processing apparatus, an information processing method, and a program.

Background Art

[0002] In recent years, in the learning of a machine learning model (hereinafter referred to as a category classifier) for identifying categories, a technique has been proposed to improve the identification rate of a subset by dividing categories to be learned in advance into several groups and performing learning. In Non-Patent Document 1, a method is proposed to suppress the imbalance of teacher data by dividing the groups of categories with low appearance frequencies of teacher data for learning a category classifier and groups other than that, and calculating the loss function at the time of learning for each group. Further, in Non-Patent Document 2, a method is proposed to improve the classification accuracy between categories to be misclassified by grouping categories based on the likelihood magnitude of each category identified by the category classifier and performing learning with the grouped loss function (softmax function).

Prior Art Documents

Non-Patent Documents

[0003]

Non-Patent Document 1

Non-Patent Document 2

[0004] However, it is difficult to improve the classification accuracy between categories of objects that are likely to be misclassified under specific conditions such as when the size of the object to be identified is small. For example, there are cases where a small airplane in an image is likely to be misclassified as a bird, or cases where a dog is likely to be misclassified as a cat when the brightness of the image is low. Thus, misclassification between categories may easily occur under specific conditions.

[0005] An object of the present invention is to improve the discrimination accuracy between categories of objects that are likely to be misclassified under specific conditions of input data. [Means for Solving the Problems]

[0006] The present invention is an information processing apparatus for learning a model for identifying the category of an object included in an image, comprising: an attribute acquisition means for acquiring an attribute from the image; and a generation means for generating a group by a plurality of categories when learning the model based on the attribute acquired by the attribute acquisition means.

Advantages of the Invention

[0007] According to the present invention, it is possible to improve the discrimination accuracy between categories in which input data is likely to be mis-identified under specific conditions.

Brief Description of the Drawings

[0008]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Embodiments for Carrying Out the Invention

[0009] Hereinafter, embodiments will be described with reference to the accompanying drawings.

[0010] 〔Embodiment 1〕 In this embodiment, a system for training a machine learning model (hereinafter referred to as a category identifier) for identifying the category of an object in an input image will be described. In this system, using the attribute parameters obtained from the image, a group of categories to be applied when training the category identifier is generated, and the generated group is used to train the category identifier. The input data is not limited to images, and is not particularly limited such as documents and time series data.

[0011] FIG. 1 shows an example of the hardware configuration of the information processing apparatus according to this embodiment. The information processing apparatus 1 includes a CPU 11, a ROM 12, a RAM 13, a secondary storage device 14, an input device 15, and a display device 16. These components are mutually connected via a connection bus 17. The CPU (Central Processing Unit) 11 controls the entire information processing apparatus 1. By executing the control program stored in the ROM 12 and the like, the processing of the flowchart described later is realized. Note that, instead of the CPU or together with the CPU, a GPU (Graphics Processing Unit) may be used.

[0012] The ROM 12 is a non-volatile memory and stores a control program and various parameter data. The RAM 13 is a volatile memory and temporarily stores images, control programs, and their execution results. The secondary storage device 14 is a rewritable secondary storage device such as a hard disk or a flash memory, and stores various data used in the flowcharts described later. For example, it stores input data, a control program, a training dataset, and a processing result. These information is output to the RAM 13 and used by the CPU 11 for executing the control program. The input device 15 is a keyboard, a mouse, a touch panel device, etc., and inputs various user instructions. The display device 16 is a monitor or the like, and displays a processing result, an image, etc.

[0013] In this embodiment, the processes described below will be implemented in software using the CPU 11, but part or all of the processes described below may be implemented in hardware. As the hardware, a dedicated circuit (ASIC), a processor (reconfigurable processor, DSP), or the like can be used. Further, the information processing apparatus 1 has a communication unit for communicating with an external apparatus, and may acquire input data, a control program, a learning data set, etc. from the external apparatus via the communication unit, and may output a processing result or the like to the external apparatus via the communication unit.

[0014] FIG. 2 shows a functional configuration example of the information processing apparatus according to this embodiment. The information processing apparatus 1 includes an acquisition unit 201, an attribute acquisition unit 202, a category identification unit 203, a group generation unit 204, a teacher data set acquisition unit 205, and a learning unit 206. By executing a control program stored in the ROM 12 or the like by the CPU 11, the functions of the respective functional units shown in FIG. 2 are realized.

[0015] The acquisition unit 201 acquires an image from an external apparatus or the secondary storage device 14. The attribute acquisition unit 202 acquires attribute parameters from the image. In this embodiment, the attribute is the size of the region of the object in the image. The attribute parameter is a value of the attribute, and in this embodiment, it is a value of the size of the region of the object. Details of the function of the attribute acquisition unit 202 will be described later with reference to FIG. 3.

[0016] The category identification unit 203 identifies the category of the object in the image. In this embodiment, the categories are assigned serial numbers from the first to the Nth, and the N categories (the first category, the second category, ···, the Nth category) are identified. The category identification unit 203 includes a category identifier for identifying the object in the image into the first to the Nth categories.

[0017] The group generation unit 204 generates information on a group of categories that are likely to be mis-identified under specific conditions by aggregating the pairs of the attribute parameters and the category identification results of each image. Details of the function of the group generation unit 204 will be described later with reference to FIG. 4. The teacher dataset acquisition unit 205 acquires teacher datasets from the first category to the Nth category prepared in advance from an external device or the secondary storage device 14. The learning unit 206 performs learning of the category identifier included in the category identification unit 203 using the teacher dataset as input data. Details of the processing performed by the learning unit 206 will be described later with reference to FIG. 6.

[0018] FIG. 3 shows a functional configuration example of the attribute acquisition unit 202 according to the present embodiment. The attribute acquisition unit 202 includes a detection unit 301 and a size acquisition unit 302.

[0019] The detection unit 301 detects the region of an object in the image with respect to the image acquired by the acquisition unit 201. The region of the object may be a rectangle surrounding the object or a region within a closed curve surrounding the object boundary. As a method for detection, as shown in Non-Patent Document 3, there is a method of using a learning model that has been pre-learned to detect a region of a specific category.

[0020] The size acquisition unit 302 acquires the size from the region of the object detected by the detection unit 301. The size may be the width or height when the region of the object is a rectangle, or the area of the rectangle. When it is a region within a closed curve surrounding the object boundary, it may be the count of pixels within that region. In the above manner, the attribute acquisition unit 202 acquires the size of the region of the object in the image, which is the attribute parameter of the present embodiment.

[0021] FIG. 4 shows a functional configuration example of the group generation unit 204 according to the present embodiment. The group generation unit 204 includes an image dataset acquisition unit 401, a misidentification determination unit 402, and a group determination unit 403.

[0022] The image dataset acquisition unit 401 acquires, from an external device or the secondary storage device 14, an image dataset including images of N categories that can be identified by the category identification unit 203 and information on the categories of objects in the images. This image dataset may be a teacher dataset for training the category identifier of the category identification unit 203 described later, or may be prepared separately therefrom.

[0023] The misidentification determination unit 402 acquires, using the attribute acquisition unit 202, the attribute parameters of each image in the image dataset (in this embodiment, the size of the region of the object in the image). Further, the misidentification determination unit 402 acquires, using the category identification unit 203, the category identification result of the object in each image in the image dataset. Here, the category identification result represents, for example, the likelihoods of the N categories that can be identified by the category identifier. From the category identification result, it is possible to make a determination that the category with the highest likelihood is likely to be the category of the object in the image.

[0024] The misidentification determination unit 402 aggregates the pairs of the attribute parameters and the category identification results of each image in the image dataset to determine the combination of the attribute parameters (object size) and the category that the category identifier is likely to misidentify. The misidentification determination unit 402 divides the object size into a predetermined range (for example, large size, medium size, small size), and for the images of the first category for each size interval, when the category identifier misidentifies to a category other than the first category, it counts the misidentified category. Then, for each misidentified category, it calculates the probability (misidentification rate) of misidentifying to that category. Similarly, it repeats from the second category to the images of the Nth category, counts the misidentified categories for each size interval, and calculates the misidentification rate.

[0025] The group determination unit 403 determines the category group according to the misidentification rate calculated by the misidentification discrimination unit 402. FIG. 5(a) is an example of a graph showing the misidentification rate for each misidentified category with respect to the image of the first category in the small size interval. In the example of FIG. 5(a), the misidentification rate 502 of the L-th category and the misidentification rate 503 of the M-th category exceed a predetermined misidentification rate threshold 504 (for example, 10%). In this case, the group determination unit 403 adds the L-th and M-th categories to the group for the first category in the attribute of the small size interval. In this way, the group determination unit 403 generates a group including the categories that are likely to be misidentified (here, the L-th and M-th categories) for the first category under specific attribute conditions (here, the small size interval). Note that in the present embodiment, a category with a misidentification rate equal to or higher than the threshold is determined as a category that is likely to be misidentified, but the method for determining a category that is likely to be misidentified is not limited to the method using the misidentification rate as described above.

[0026] Similarly, the group determination unit 403 determines groups for the second category to the N-th category in the small size interval. FIG. 5(b) is an example of a graph showing the misidentification rate for each misidentified category with respect to the image of the second category in the small size interval. In the example of FIG. 5(b), there is no category whose misidentification rate exceeds a predetermined misidentification rate threshold 506 (for example, 10%). In this case, the group determination unit 403 does not generate a group for the second category in the small size interval.

[0027] Similarly, the group determination unit 403 generates groups of categories that are likely to be misidentified for the first category to the N-th category in the medium size interval and the large size interval. The information on the category groups generated by the group generation unit 204 is held in the secondary storage device 14 or the like.

[0028] FIG. 6 is a flowchart showing the learning process of the category identifier according to the present embodiment. The learning unit 206 learns the category identifier using a teacher data set of the first category to the Nth category prepared in advance and information on the category groups generated by the group generation unit 204. The information on the category groups generated by the group generation unit 204 is, for example, data in which a category and a category that is likely to be misidentified under a specific attribute condition (for example, an object is small in size) are set. The information on the category groups generated by the group generation unit 204 is obtained by aggregating the set of the attribute parameters of the images of each category in the image data set and the category identification result. In the following description, the notation of the steps (steps) is omitted by attaching S at the head of each step (step).

[0029] In S501, the learning unit 206 acquires a teacher data set for learning the category identifier using the teacher data set acquisition unit 205. The teacher data set may be stored in the secondary storage device 14 or the like. The teacher data set acquisition unit 205 may acquire the teacher data set from the secondary storage device 14 or from an external device. In S502, the learning unit 206 acquires the category identifier included in the category identification unit 203. In the present embodiment, the category identifier is a machine learning model for identifying the category of an object in an image. The machine learning model is a multi-layer neural network model. However, it is not limited to the multi-layer neural network model, and known machine learning models such as random forest and AdaBoost may be used. The model parameters of the category identifier are stored in the secondary storage device 14 or the like. In S503, the learning unit 206 acquires a mini-batch from the teacher data set acquired in S501. A mini-batch is an input image to the category identifier composed of one or more images.

[0030] In S504, the learning unit 206 acquires the correct category corresponding to each image in the mini-batch. In S505, the learning unit 206 uses the attribute acquisition unit 202 to acquire the attribute parameters of each image in the mini-batch. In this embodiment, the size of the region of the object in each image is acquired. Note that when the teacher dataset pre-includes information representing the size of the region of the object in the image, that information may be acquired. The attribute acquisition unit 202 functions as an attribute acquisition means.

[0031] In S506, the learning unit 206 performs inference processing on each image in the mini-batch using the category identification unit 203. Specifically, the learning unit 206 calculates the likelihoods (logits) for the first to Nth categories obtained by inputting each image into the category identification model. Thereby, the category identification result of the object in each image is acquired. The category identification result includes the likelihoods (logits) of all categories. The category identification unit 203 functions as a category identification means. In S507, the learning unit 206 first targets one image in the mini-batch of images. The learning unit 206 acquires the information of the category group generated by the group generation unit 204 from the secondary storage device 14 or the like. The learning unit 206 functions as a group acquisition means. Then, when there is a category that is likely to be mis-identified under specific attribute conditions for the correct category of the target image (acquired in S504) and the attribute parameters of the target image (acquired in S505) satisfy the specific attribute conditions, the learning unit 206 generates a group. The learning unit 206 functions as a group generation means. The generated group includes the correct category and one or more categories that are likely to be mis-identified. In this way, a category group is generated for each image in the mini-batch. That is, the learning unit 206 switches the group to be applied when calculating the value of the loss (loss function) in the next step according to the correct category and the attribute parameters of each image in the mini-batch. On the other hand, when the above conditions are not satisfied, the learning unit 206 does not generate a group. For example, when the attribute parameters of the target image (acquired in S505) do not satisfy specific conditions, the learning unit 206 does not generate a group. That is, the learning unit 206 controls whether to generate a group according to the correct category and the attribute parameters.

[0032] In S508, the learning unit 206 calculates the value of the loss (loss function) using the category identification result and category group of each image in the mini-batch. In the present embodiment, for calculating the value of the loss, the group softmax function used in Non-Patent Document 1 or Non-Patent Document 2 is used. When no group is generated, the learning unit 206 takes the sum of the likelihoods of all categories using the normal softmax function, whereas when a group is generated, the learning unit 206 calculates the value of the loss by taking the sum of the likelihoods of the categories in the group using the group softmax function. By applying the group softmax function, learning is performed such that the features between the grouped categories are separated, and misidentification between the grouped categories is more likely to be suppressed.

[0033] Note that, in the same manner as the method of applying the softmax function between the grouped categories, in known loss functions obtained by extending the softmax function such as SphereFace and ArcFace, the sum of the grouped categories may also be used. Further, when calculating the loss, not only the value of the group softmax function but also a value obtained by multiplying the value of the normal softmax function by a certain weight may be added. Thereby, while maintaining the discrimination accuracy between all categories, it becomes possible to suppress misidentification of categories that are likely to occur under specific attribute conditions (for example, when the object is small in size).

[0034] In S509, the learning unit 206 applies the error backpropagation method to the loss calculated in S508 to calculate the gradient, and obtains the update amount of the model parameters of the category discriminator. In S510, the learning unit 206 updates the model parameters of the category discriminator. Specifically, a known learning method for a multi-layer neural network may be applied, and detailed description thereof is omitted. In S511, the learning unit 206 outputs the category discriminator with updated model parameters to the category discrimination unit 203. The learning unit 206 determines the model parameters of the category discriminator by repeatedly executing the processes of S501 to S511 as described above until the value of the loss and the discrimination accuracy converge. Thereafter, the processing of the series of flowcharts is terminated.

[0035] Note that, according to the progress of learning, the group generation unit 204 may re-generate groups using the correct category (obtained in S504), the attribute parameters (obtained in S505), and the inference result (obtained in S506) for each image in the mini-batch. The learning unit 206 may apply new groups starting from the next learning.

[0036] According to the first embodiment as described above, by learning the category identifier using a group of categories that are likely to be mis-identified under specific attribute conditions, it is possible to improve the discrimination accuracy between categories under those conditions.

[0037] [Embodiment 2] In this embodiment, a method for efficiently performing learning by converting an image to correct the attribute parameters and increasing the number of images having specific attribute parameters will be described. Hereinafter, parts common to the first embodiment will be omitted from the description, and the description will focus on the differences from the first embodiment.

[0038] FIG. 7 shows a functional configuration example of the information processing apparatus according to this embodiment. In the functional configuration of FIG. 7, it is different from the functional configuration of FIG. 2 in that it has a correction amount calculation unit 601 and an image conversion unit 602. Further, in this embodiment, the attribute acquisition unit 202 acquires not only the size of the region of the object in the image but also the luminance value, the amount of blur, the amount of shake, etc. of the image as attribute parameters. Details of the functions of the attribute acquisition unit 202 will be described later with reference to FIG. 8.

[0039] The correction amount calculation unit 601 calculates the correction amount of the attribute parameters acquired by the attribute acquisition unit 202. The image conversion unit 602 converts the image so that the attribute parameters are corrected by the correction amount calculated by the correction amount calculation unit 601.

[0040] FIG. 8 shows a functional configuration example of the attribute acquisition unit 202 according to this embodiment. In the functional configuration of FIG. 8, it is different from the functional configuration of FIG. 3 in that it has a luminance acquisition unit 701, a blur amount calculation unit 702, and a shake amount calculation unit 703.

[0041] The luminance acquisition unit 701 acquires the luminance value of the image. For example, it is the luminance value in a general YUV color space. Also, the attribute parameter may be the average of the luminance values in the image, or may be the average of the luminance values within the region of the object detected by the detection unit 301.

[0042] The blur amount calculation unit 702 estimates the blur amount of the object in the image. As a method for estimating the blur amount, for example, there is a method of using a machine learning model (hereinafter referred to as a blur amount estimator) for estimating the blur amount. The blur amount estimator is learned, for example, by preparing teacher data that combines an image blurred by applying a Gaussian blur to an image of an object without blur and the intensity of the Gaussian blur within the region of the object in that image. Also, at the time of inference, the intensity of the Gaussian blur within the region of the object is estimated.

[0043] The shake amount calculation unit 703 estimates the shake amount of the object in the image. As a method for estimating the shake amount, similar to the blur amount calculation unit 702, there is a method of using a machine learning model (hereinafter referred to as a shake amount estimator) for estimating the shake amount. The shake amount estimator is learned, for example, by preparing teacher data that combines an image obtained by applying a motion blur filter to an image of an object without shake and the intensity of that filter. Also, at the time of inference, the intensity of the motion blur filter within the region of the object is estimated.

[0044] In the present embodiment, the group generation unit 204 generates information on a group of categories that are likely to be misidentified under specific attribute conditions by aggregating the pairs of attribute parameters and category identification results of each image. When performing aggregation with a plurality of attribute parameters as in the present embodiment, the specific attribute condition may be not only the condition of one attribute parameter but also a condition combining a plurality of attribute parameters. For example, it may be a condition combining the size of the region of the object (for example, a small size interval) and the luminance value (for example, low luminance).

[0045] FIG. 9 is a flowchart showing the learning process of the category identifier according to the present embodiment. The flowchart of FIG. 9 mainly differs from the flowchart of FIG. 6 in that the processing steps of S801 and S802 are executed between the processing steps of S505 and S506. Hereinafter, the processing steps different from the flowchart of FIG. 6 will be described, and the description of the same processing steps will be omitted.

[0046] In S505, the learning unit 206 uses the attribute acquisition unit 202 to acquire the attribute parameters of each image in the mini-batch acquired in S503. In the case of the present embodiment, not only the size of the region of the object in the image but also the luminance value, the amount of blur, and the amount of shake are acquired respectively. When the teacher data set pre-includes information representing the size of the region of the object in the image and the luminance value, the amount of blur, and the amount of shake, that information may be acquired.

[0047] In S801, the learning unit 206 uses the correction amount calculation unit 601 to calculate the correction amount of the attribute parameters acquired in S505. As a method for the correction amount calculation unit 601 to calculate the correction amount, there is a method using the statistics of the attribute parameters of the teacher data set. For example, the correction amount calculation unit 601 uses the attribute acquisition unit 202 to acquire the attribute parameters for each image in the teacher data set, divides the attribute parameters into a predetermined range, and calculates the appearance frequency for each divided interval.

[0048] Then, the correction amount calculation unit 601 calculates a correction amount such that the attribute parameters of the images in the mini-batch acquired in S503 will be attribute parameters in a range with a low appearance frequency when converted in S802 with a certain probability. For example, when the appearance frequency in the small size interval is low, a correction amount for the size of the region of the object (for example, a resize value such that the object becomes in the small size interval) is calculated so that an image in the small size interval appears with a probability exceeding the appearance frequency. Note that the processing step of calculating the correction amount in S801 may be executed every time an image is acquired in S503, or may be executed with a preset probability.

[0049] For example, if the occurrence frequency of the small size interval is 3% in the original teacher dataset and it is desired to change it to 30%, the correction amount calculation unit 601 calculates the correction amount of the attribute parameter so that the acquired mini-batch of images becomes the small size interval with a probability of 30%. Similarly, for other parameters such as the luminance value, the amount of blur, and the amount of shake, the correction amount of the attribute parameter is calculated so that the range of the attribute parameter with a low occurrence frequency appears with a certain probability. In this way, in the present embodiment, the group generation unit 204 generates a group using the teacher dataset obtained by increasing the ratio of the attribute parameters in the range where the occurrence frequency is low.

[0050] As described above, as a method other than the method using the statistics of the attribute parameters of the teacher dataset, a method of calculating the correction amount of the attribute parameter so that it becomes the attribute parameter of the group of categories that are easily misclassified obtained by the group generation unit 204 is also conceivable. For example, when a group associated with a certain range of luminance values is generated, the correction amount calculation unit 601 calculates the correction amount such that the luminance value of the acquired image becomes the luminance value in the range associated with the group.

[0051] In S802, the learning unit 206 uses the image conversion unit 602 to apply the correction amount of the attribute parameter acquired in S801 to the mini-batch of images acquired in S503 and perform conversion. When the attribute parameter is the size of the object region, the image is resized based on the correction amount. When it is luminance, the luminance value in the YUV color space of the image is converted. When it is the amount of blur and the amount of shake, the correction amount is added to the intensity of the amount of blur and the amount of shake of the image to convert the image.

[0052] Thereafter, the processing steps after S506 are executed. In S507, the learning unit 206 uses the group generation unit 204 to generate a group for each image in the mini-batch based on the corrected value of the attribute parameter. Then, in S508, for a target image whose correct category is the first category, if there is a category that is likely to be misclassified as the first category under specific attribute conditions and the attribute parameter of the target image satisfies the specific attribute conditions, the learning unit 206 generates the group. Also, for a target image whose correct category is the M-th category, if the M-th category is likely to be misclassified as the first category in a small size interval and the size of the object region in the target image is in the small size interval, the learning unit 206 generates a group with the first category and the M-th category. Further, for a target image whose correct category is the L-th category, if the L-th category is likely to be misclassified as the first category at low brightness and the target image is at low brightness, the learning unit 206 generates a group with the first category and the L-th category. Note that not only the condition of one attribute parameter but also the condition combining multiple attribute parameters may be used. Then in S508, the learning unit 206 calculates the value of the loss by applying the group softmax function to each group formed during the mini-batch and taking the sum thereof.

[0053] The learning unit 206 repeatedly executes the processes of S501 to S511 as described above until the value of the loss and the discrimination accuracy converge, thereby determining the model parameters of the category discriminator. Then the processing of the series of flowcharts is terminated.

[0054] Note that the group generation unit 204 may be determined to eliminate the misclassification between categories under specific attribute conditions from the misclassification result calculated using the category discriminator during learning and remove the misclassified category from the group. Then, the learning unit 206 may apply a new group from the next learning.

[0055] According to the second embodiment as described above, by adjusting the appearance frequency of images under the conditions of specific attribute parameters, generating a group of categories that are easily mis-identified, and training a category identifier, the training efficiency can be improved and the discrimination accuracy between categories can be enhanced.

[0056] 〔Embodiment 3〕 In this embodiment, a system will be described that identifies the category of an object in an image, generates a group of categories based on the identified category and the attribute parameters of the image, and applies an analysis task according to the group. The analysis tasks are prepared for each identified category. Specifically, as an analysis task prepared for the category of a person, there is a task of detecting the joints of the person, and as an analysis task prepared for the category of an animal, there is a task of detecting the face of the animal. Hereinafter, the parts common to the first embodiment will be omitted from the description, and the description will focus on the differences from the first embodiment.

[0057] FIG. 10 shows an example of the functional configuration of the information processing apparatus according to this embodiment. As shown in FIG. 10, the information processing apparatus 1 according to this embodiment has an acquisition unit 201, an attribute acquisition unit 202, and a category identification unit 203, which are the same as those in the first embodiment.

[0058] The information processing apparatus 1 according to this embodiment further has an analysis task application unit 905. The analysis task application unit 905 includes an analysis task execution unit 906 for the first category, an analysis task execution unit 907 for the second category, ···, and an analysis task execution unit 908 for the Nth category. Hereinafter, it is assumed that the category identification unit 203 includes a category identifier that has been trained by the method of the first embodiment or the second embodiment, etc. Further, the group acquisition unit 902 acquires information on the group of categories from a secondary storage device 14 or the like. The information on the group of categories is information on a group of categories that are easily mis-identified under specific attribute conditions and is generated using the category identifier included in the category identification unit 203.

[0059] In this embodiment, the acquisition unit 201 acquires an image 901 in which an object of an unknown category is imaged. Next, the attribute acquisition unit 202 acquires the attribute parameters of the image 901. Here, for the sake of easy understanding, the size of the region of the object in the image is estimated and acquired using the detection unit 301 and the size acquisition unit 302.

[0060] The category identification unit 203 identifies the category of the object in the acquired image. The group acquisition unit 902 acquires information on a group including the identified category in a specific region size classification using the size of the acquired object region and the identified category. Here, the region size is divided into a large size and a small size interval.

[0061] Hereinafter, it is assumed that the size of the acquired object region belongs to the small size interval and the identified category is the nth category. At this time, if there is no group including categories other than the nth category for the nth category, the analysis task application unit 905 applies the analysis task execution unit of the nth category and outputs the analysis result 909. On the other hand, when there is a group including the mth category for the nth category in the small size interval of the attribute parameters, the nth category identified by the category identification unit 203 may be misidentified and may actually be the mth category.

[0062] Therefore, the analysis task application unit 905 applies not only the analysis task execution unit of the nth category but also the analysis task execution unit of the mth category. That is, when the attribute condition to which the acquired attribute parameters belong is given and there is a group including the identified category, the analysis task application unit 905 also applies the analysis task execution units of other categories belonging to the group. By applying a plurality of analysis tasks in consideration of the possibility of such misidentification, a desired analysis result 909 may be obtained. Also, the analysis task application unit 905 may operate so as not to apply any analysis task execution unit when the attribute condition to which the acquired attribute parameters belong is given and there is a group including the identified category. This can avoid the execution of an incorrect analysis task.

[0063] According to the third embodiment as described above, even if there is a category that is likely to be misidentified under specific attribute conditions, a more appropriate analysis task can be applied to the input data.

[0064] [Other Embodiments] As described above, the present invention has been described together with embodiments. However, the above embodiments are merely examples of specific embodiments in implementing the present invention, and the technical scope of the present invention should not be construed in a limited manner by these. That is, the present invention can be implemented in various forms without departing from its technical idea or its main features.

[0065] The present invention can also be realized by supplying a program that realizes one or more functions of the above-described embodiments to a system or device via a network or a storage medium, and having one or more processors in the computer of the system or device read and execute the program. It can also be realized by a circuit (for example, ASIC) that realizes one or more functions.

[0066] The disclosure of each of the above embodiments includes the following configurations, methods, and programs. (Configuration 1) An information processing apparatus for learning a model for identifying the category of an object included in an image, an attribute acquisition means for acquiring an attribute from the image, a generation means for generating a group of a plurality of categories when learning the model based on the attribute acquired by the attribute acquisition means, An information processing apparatus characterized by comprising the above. (Configuration 2) an identification means for identifying the category of an object included in the image using the model, a learning means for learning the model based on the identification result by the identification means and the group of categories generated by the generation means, The information processing apparatus according to Configuration 1, further comprising the above. (Configuration 3) It further has group acquisition means for acquiring information on a group of categories that are likely to be mis-identified under predetermined attribute conditions. The generation means generates the group using the information on the group acquired by the group acquisition means, and the information processing apparatus according to Configuration 1 or 2. (Configuration 4) When the attribute acquired by the attribute acquisition means satisfies a predetermined attribute condition, the group generated by the generation means includes the correct category of the object and one or more categories that are likely to be mis-identified with the correct category under the predetermined attribute condition, and the information processing apparatus according to any one of Configurations 1 to 3. (Configuration 5) The group acquisition means acquires the information on the group by aggregating the mis-identification rates of the categories for each attribute for the data set of each category, and the information processing apparatus according to Configuration 3. (Configuration 6) It further has conversion means for converting the image so as to correct the attribute acquired from the image, The identification means identifies the category of the converted object, The generation means generates the group based on the corrected attribute, and the information processing apparatus according to Configuration 2. (Configuration 7) The conversion means calculates the correction amount of the attribute by using the statistics of the attribute for the data set of each category to be learned, and the information processing apparatus according to Configuration 6. (Configuration 8) It further has group acquisition means for acquiring information on a group of categories that are likely to be mis-identified under predetermined attribute conditions. The conversion means calculates the correction amount of the attribute so that the attribute acquired from the image satisfies the predetermined attribute condition, and the information processing apparatus according to Configuration 7. (Configuration 9) The attribute is the size of the region of the object in the image, and the information processing apparatus according to any one of Configurations 1 to 8. (Configuration 10) The information processing apparatus according to any one of Configurations 1 to 9, wherein the attribute is the luminance of the image. (Configuration 11) The information processing apparatus according to any one of Configurations 1 to 10, wherein the attribute is the blur in the image. (Configuration 12) The information processing apparatus according to any one of Configurations 1 to 11, wherein the attribute is the defocus in the image. (Configuration 13) The information processing apparatus according to Configuration 2, wherein the learning means applies a loss function to the category belonging to the group. (Configuration 14) The generation means controls whether to generate the group according to the attribute acquired by the attribute acquisition means. The information processing apparatus according to Configuration 2, wherein the learning means applies a loss function to the category belonging to the group when the group is generated by the generation means, and applies a loss function to all categories when the group is not generated by the generation means. (Configuration 15) An information processing apparatus for applying an analysis task for each category to an input image, Attribute acquisition means for acquiring an attribute from the input image; Identification means for identifying the category of an object included in the input image; Group acquisition means for acquiring information on a group of categories that are likely to be mis-identified under a predetermined attribute condition; Application means for applying an analysis task to the input image based on the attribute acquired by the attribute acquisition means, the identified category, and the information on the group; An information processing apparatus, characterized by comprising the above. (Configuration 16) The application means applies, to the input image, an analysis task for the first category and an analysis task for categories other than the first category belonging to the group when the attribute acquired by the attribute acquisition means satisfies the predetermined attribute condition and the first category, which is the identified category, belongs to the group, in the information processing apparatus according to Configuration 15. (Configuration 17) The application means does not apply an analysis task to the input image when the attribute acquired by the attribute acquisition means satisfies the predetermined attribute condition and the first category, which is the identified category, belongs to the group and there are categories other than the first category belonging to the group, in the information processing apparatus according to Configuration 15. (Method) An information processing method for training a model for identifying categories of objects included in an image, an attribute acquisition step of acquiring an attribute from the image, a generation step of generating a group by a plurality of categories when training the model based on the attribute acquired in the attribute acquisition step, characterized by including. (Program) A program for causing a computer of an information processing apparatus for training a model for identifying categories of objects included in an image to function as an attribute acquisition means for acquiring an attribute from the image, and a generation means for generating a group by a plurality of categories when training the model based on the attribute acquired by the attribute acquisition means. ​

Claims

1. An information processing apparatus for training a model for identifying the category of an object included in an image, comprising: attribute acquisition means for acquiring an attribute from the image; generation means for generating a group by a plurality of categories when training the model based on the attribute acquired by the attribute acquisition means; An information processing apparatus characterized by comprising the above.

2. identification means for identifying the category of an object included in the image using the model; learning means for training the model based on the identification result by the identification means and the group of categories generated by the generation means; The information processing apparatus according to claim 1, further comprising the above.

3. further comprising group acquisition means for acquiring information on a group of categories that are likely to be misidentified under a predetermined attribute condition; The information processing apparatus according to claim 1, wherein the generation means generates the group using the information on the group acquired by the group acquisition means.

4. When the attribute acquired by the attribute acquisition means satisfies a predetermined attribute condition, the group generated by the generation means includes the correct category of the object and one or more categories that are likely to be misidentified with the correct category under the predetermined attribute condition. The information processing apparatus according to claim 1, characterized by this.

5. The information processing apparatus according to claim 3, wherein the group acquisition means acquires the information on the group by tabulating the misidentification rate of each category for each attribute for each category dataset.

6. further comprising conversion means for converting the image so as to correct the attribute acquired from the image; the identification means identifies the category of the converted object; The information processing apparatus according to claim 2, wherein the generation means generates the group based on the corrected attribute.

7. The information processing apparatus according to claim 6, wherein the conversion means calculates the correction amount of the attribute by using the statistics of the attribute for each category dataset to be trained.

8. further comprising group acquisition means for acquiring information on a group of categories that are likely to be misidentified under a predetermined attribute condition; The information processing apparatus according to claim 7, wherein the conversion means calculates the correction amount of the attribute so that the attribute acquired from the image satisfies the predetermined attribute condition.

9. The information processing apparatus according to claim 1, wherein the attribute is the size of the area of an object in the image.

10. The information processing apparatus according to claim 1, wherein the attribute is the luminance of the image.

11. The information processing apparatus according to claim 1, wherein the attribute is the blur in the image.

12. The information processing apparatus according to claim 1, wherein the attribute is the defocus in the image.

13. The information processing apparatus according to claim 2, wherein the learning means applies a loss function to the categories belonging to the group.

14. The generation means controls whether to generate the group according to the attribute acquired by the attribute acquisition means. The information processing apparatus according to claim 2, wherein when the group is generated by the generation means, the learning means applies a loss function to the categories belonging to the group, and when the group is not generated by the generation means, the learning means applies a loss function to all categories.

15. An information processing apparatus for applying an analysis task for each category to an input image, comprising: an attribute acquisition means for acquiring an attribute from the input image; an identification means for identifying the category of an object included in the input image; a group acquisition means for acquiring information on a group of categories that are likely to be mis-identified under a predetermined attribute condition; an application means for applying an analysis task to the input image based on the attribute acquired by the attribute acquisition means, the identified category, and the information on the group; The information processing apparatus characterized by comprising the above.

16. When the attribute acquired by the attribute acquisition means satisfies the predetermined attribute condition and a first category which is the identified category belongs to the group, the application means applies, to the input image, an analysis task for the first category and an analysis task for categories other than the first category belonging to the group. The information processing apparatus according to claim 15.

17. The information processing apparatus according to claim 15, wherein the application means does not apply an analysis task to the input image when the attribute acquired by the attribute acquisition means satisfies the predetermined attribute condition and the first category, which is the identified category, belongs to the group, and a category other than the first category belongs to the group.

18. An information processing method for training a model that identifies categories of objects included in an image, comprising: an attribute acquisition step of acquiring an attribute from the image; a generation step of generating a group of a plurality of categories when training the model based on the attribute acquired in the attribute acquisition step; The information processing method is characterized by including the above.

19. A computer of an information processing apparatus for training a model that identifies categories of objects included in an image, an attribute acquisition means for acquiring an attribute from the image; a generation means for generating a group of a plurality of categories when training the model based on the attribute acquired by the attribute acquisition means; A program for causing the computer to function as such.

Images