Information processing apparatus, information processing method, and program

Abstract

To achieve learning that can improve recognition accuracy.SOLUTION: An information processing apparatus has: training means that trains a learning model corresponding to predetermined recognition processing by using learning data; evaluation means that performs recognition processing using the learning model on data for verification associated with the learning model; and adjustment means that, on the basis of an area falsely recognized from the data for verification in the recognition processing performed by the evaluation means, performs at least one of predetermined adjustment processing on the learning data and adjustment processing of changing settings in learning.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to an information processing technique relating to machine learning. [Background technology]

[0002] There is a technology called machine learning in which computers and other machines learn the contents of data such as images and sounds and perform recognition processing. Here, the issue that recognition processing deals with is called a recognition task, and the learning model, which is a mathematical model for learning and executing a recognition task, is called a recognition model. An example of a recognition task is an object detection task, which detects (recognizes) a specific object (person, animal, vehicle, etc.) from an image. Neural networks are known as a technology for learning and executing recognition tasks.

[0003] Data Augmentation (hereinafter, DA) is known as a type of technique for improving detection accuracy (recognition accuracy) in neural networks. DA is a technique for artificially increasing training image data by making various changes to training images. In other words, DA can increase the apparent variation of training images, thereby improving generalization performance during training. In addition, Patent Document 1 proposes a technique for maintaining detection accuracy even when a target object is partially hidden by synthesizing an image of an obstructing object so that the target object is hidden.

[0004] In addition, there are methods disclosed in Non-Patent Document 1 and Patent Document 2 as methods for suppressing false detection (false recognition) during object detection. Non-Patent Document 1 discloses that a model trained for the purpose of object detection is used to perform a process of detecting a target area from a training image, and a partial image including a falsely detected area is preferentially used as a hard negative sample in the training image. According to the method of Non-Patent Document 1, it is possible to suppress the occurrence of false detection by performing re-training using a training image including the hard negative sample. Patent Document 2 discloses a learning method for object detection that has a main task and a subtask, each of which learns a different detection target. And, in the method of Patent Document 2, a false detection map is created in the main task in the training image, and the false detection map is used to weight the loss of the subtask, thereby making it possible to suppress false detection in the main task. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] Patent Publication No. 2021-33707 [Patent Document 2] JP 2020-91543 A [Non-patent literature]

[0006] [Non-Patent Document 1] Abhinav Shrivastava, Abhinav Gupta, Ross Girshick, “Training Region-Based Object Detectors with Online Hard Example Mining”, in CVPR, 2016 Summary of the Invention [Problem to be solved by the invention]

[0007] However, when DA is applied to training images, an image may be generated in which parts of the original image that should remain unchanged are changed, or an image that does not exist in reality may be generated. For this reason, when training is performed using training images to which DA has been applied, over-training may occur more easily, making optimization difficult and improving generalization performance. In other words, a learning model trained using training images to which DA has been applied may result in a decrease in recognition accuracy in the recognition process.

[0008] Furthermore, in the case of learning using the methods of Non-Patent Document 1 and Patent Document 2, learning may proceed in a way that suppresses features that are effective for correct detection. In this case, the occurrence of false positives (false recognition) is suppressed, but non-detection cases in which areas that should be correctly detected cannot be detected tend to occur. In other words, in this case as well, the recognition accuracy in the recognition process may decrease.

[0009] Therefore, an object of the present invention is to make it possible to realize learning that can improve recognition accuracy. [Means for solving the problem]

[0010] The information processing device of the present invention is characterized by having a learning means that uses learning data to learn a learning model corresponding to a predetermined recognition processing, an evaluation means that performs the recognition processing using the learning model on verification data corresponding to the learning model, and an adjustment means that performs at least one of a predetermined adjustment processing on the learning data and an adjustment processing that changes the settings during the learning based on an area that is misrecognized from the verification data in the recognition processing by the evaluation means. Effect of the Invention

[0011] According to the present invention, it is possible to realize learning that can improve recognition accuracy. [Brief description of the drawings]

[0012] [Figure 1] FIG. 2 is a diagram illustrating an example of a functional configuration of an information processing apparatus according to the first and second embodiments. [Diagram 2] FIG. 4 is a functional block diagram of an adjustment unit. [Diagram 3] 4 is a flowchart of a process executed by the information processing apparatus of the first embodiment. [Figure 4] FIG. 13 is an explanatory diagram of an example of inference based on a recognition task. [Diagram 5] 1A and 1B are explanatory diagrams of a learning image, a teacher frame, a false detection region, and a similar region. [Figure 6] 11A and 11B are diagrams illustrating an example of a predetermined adjustment process for a similar region. [Figure 7] 10 is a flowchart of a process executed by an information processing apparatus according to a second embodiment. [Figure 8] FIG. 13 is a diagram illustrating an example of a functional configuration of an information processing apparatus according to a third embodiment. [Figure 9] 13 is a flowchart of a process executed by an information processing apparatus according to a third embodiment. [Figure 10] FIG. 1 illustrates an example of the configuration of a computer capable of implementing each embodiment. [Figure 11] FIG. 1 is a diagram illustrating an example of the configuration of an information processing device that uses a trained recognition model. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0013] Hereinafter, an embodiment of the present invention will be described with reference to the drawings. The following embodiments do not limit the present invention, and not all of the combinations of features described in the present embodiments are necessarily essential to the solution of the present invention. The configuration of the embodiment may be appropriately modified or changed depending on the specifications of the device to which the present invention is applied and various conditions (conditions of use, environment of use, etc.). In addition, the embodiment may be configured by appropriately combining parts of the embodiments described below. In the following embodiments, duplicated descriptions of the same or similar configurations and processing steps will be omitted.

[0014] In this embodiment, as described above, a problem that recognition processing in machine learning deals with is called a recognition task, and a learning model that is a mathematical model for learning and executing a recognition task is called a recognition model. Recognition tasks include the object detection task for detecting a specific object (person, animal, vehicle, etc.) from an image described above, and a region detection task for performing object detection in pixel units of an image called semantic region division. There are also various recognition tasks such as an object category identification task for identifying a category (human, animal, vehicle, etc.) of an object (subject) in an image, a tracking task for searching and tracking a specific subject, and a scene type recognition task for recognizing a type of scene (city, mountain, coast, etc.). This embodiment can be applied to any of these recognition tasks. In the following embodiment, an object detection task for detecting a whole body of a person will be described as an example of a recognition task. In the following description, an object detection task for a whole body of a person will be referred to as a human whole body task. Of course, this is only one example, and this embodiment can be applied to various object detection tasks such as the upper body of a person, a person's face, an animal, a vehicle, an insect, a tree, a building, and a road.

[0015] In addition, in this embodiment, a neural network is taken as an example of a technology for learning and executing a recognition task. A deep (large number of layers) multi-layer neural network is also called a deep neural network (hereinafter, DNN). In particular, a deep convolutional neural network is called a deep convolutional neural network (hereinafter, DCNN). DCNN is known to have high performance (recognition accuracy, recognition performance).

[0016] For example, in the case of hard-negative learning disclosed in the above-mentioned Non-Patent Document 1 and Patent Document 2, all areas that were erroneously detected (erroneously recognized) during learning are weighted and re-learned. However, if an area that should be correctly detected is included in the erroneously detected area, learning proceeds so as to suppress features that are effective for correct detection. In this case, the occurrence of erroneous detection in the recognition task is suppressed, but the drawback is that correct detection becomes weaker, and undetected cases where areas that should be correctly detected cannot be detected tend to occur, increasing the possibility of a decrease in recognition accuracy (detection accuracy) overall.

[0017] <First embodiment> Therefore, the information processing device 100 of the first embodiment has the functional configuration shown in Fig. 1, and can realize learning that improves recognition accuracy by performing the processing described below. Fig. 1 is a functional block diagram showing the functional configuration of the information processing device 100 of the first embodiment.

[0018] In the information processing device 100 shown in Fig. 1, the training data 101 is a data set having a plurality of training image data for training the recognition model 106 and training data (correct answer data) corresponding to the training images. The training data is given as information on a frame surrounding a detection target. Specifically, the training data is given by the image coordinate values ​​of the center position of the frame and the size (height, width) of the frame on the image. Hereinafter, the frame of the training data is referred to as the training frame.

[0019] The learning unit 102 is a functional unit that uses the training image data and teacher data contained in the training data 101 to learn a recognition model 106 associated with a recognition task. The recognition model 106 is, for example, a DNN, and is a learning model (mathematical model) that outputs a recognition result of a recognition target in response to at least one of an image and a feature amount as input.

[0020] The verification data 103 is a data set having a plurality of verification image data and correct answer data corresponding to the verification images, and is data prepared for calculating the accuracy of the recognition process (recognition accuracy) in the recognition model 106. The verification data 103 has the same configuration as the training data 101 in that it is composed of image data and correct answer data (teacher data). In this embodiment, in order to distinguish the verification data 103 from the training data 101, the teacher data in the verification data 103 is called correct answer data, and the frame of the correct answer data is called a correct answer frame. Since the verification data 103 is data intended to verify the recognition model 106 trained by the training unit 102, image data other than the training image data included in the training data 101 is used for the verification image data. Note that when the training data is insufficient, cross-validation may be used.

[0021] The evaluation unit 104 evaluates the recognition model 106 trained by the learning unit 102, using the verification image data and the correct answer data of the verification data 103. The evaluation unit 104 includes a calculation unit 107, a comparison unit 108, and a result storage unit 109 as components for evaluating the recognition model 106 trained by the learning unit 102.

[0022] The calculation unit 107 is an accuracy acquisition unit that performs inference on a verification image included in the verification data 103 using the recognition model 106 learned by the learning unit 102, and compares the inference result with correct answer data corresponding to the verification image to calculate the recognition accuracy. The calculation unit 107 calculates a likelihood that indicates the certainty of a detection target (a whole human body) during inference, and also determines whether a detection frame detected from the verification image by inference is a correctly recognized detection frame (a correct detection frame) or an erroneously recognized detection frame (a false detection frame). Note that the recognition accuracy is represented by a comparable evaluation measure such as a loss function value for the verification data of a region recognized (detected) by inference, a correct answer rate, a recall rate, or a precision rate. Hereinafter, the recognition accuracy will be described as being represented by a correct answer rate of a detected region.

[0023] The comparison unit 108 compares the recognition accuracy calculated by the calculation unit 107 with a target accuracy preset for the recognition model, and determines whether to perform an update process for improving the recognition accuracy or to end the process based on the comparison result. The comparison result by the comparison unit 108 may be, for example, the difference between the accuracy values ​​of the recognition accuracy and the target accuracy, or the ratio of the accuracy values. In the case of the present embodiment, the comparison result is described as the difference in accuracy values. Then, when it is determined that the update process is to be performed, the comparison unit 108 extracts an image within the false detection frame in which the likelihood representing the likelihood of the detection target object calculated by the calculation unit 107 at the time of inference is higher than a predetermined likelihood threshold value from the verification image as a false detection area image and holds it. The false detection area image held in the result holding unit 109 is then sent to the adjustment unit 105.

[0024] As described above, the evaluation unit 104 performs recognition processing on the verification data 103 using the recognition model 106 to evaluate the accuracy of the recognition model 106, and based on the evaluation results, retains images of false positive areas within the false positive frame where the likelihood representing the probability that the object is the detected object is higher than a likelihood threshold.

[0025] The adjustment unit 105 is a learning adjustment unit for adjusting the learning in the learning unit 102. In the present embodiment, the adjustment unit 105 performs at least one of a predetermined adjustment process for the learning data and an adjustment process for changing the settings during learning, based on the false detection area image acquired by the evaluation unit 104. The predetermined adjustment process for the learning data and the adjustment process for changing the settings during learning will be described in detail later.

[0026] 2(a) is a block diagram showing an example of the functional configuration of the adjustment unit 105 in the first embodiment. The adjustment unit 105 according to the first embodiment includes an extraction unit 201, a data update unit 202, and a parameter update unit 203.

[0027] The extraction unit 201 extracts a region similar to the false positive region image stored in the result storage unit 109 of the evaluation unit 104 from the training image included in the training data 101. That is, the extraction unit 201 performs inference on the training data 101 in order to identify a region in the training image of the training data 101 in which the false positive region image stored in the evaluation unit 104 and the CNN feature amount are similar. Hereinafter, a region extracted from the training image as having similar feature amounts by inference is referred to as a similar region, and the degree of similarity therebetween at the time of inference is referred to as an inferred similarity. The extraction unit 201 compares the inferred similarity with a preset similarity threshold, and determines whether to perform or end adjustment processing depending on the comparison result.

[0028] When a similar region is found within the teacher frame of the learning data, the data update unit 202 performs a process of reducing the feature amount of the similar region. Hereinafter, the process of reducing the feature amount is referred to as a feature amount reduction process. The details of the feature amount reduction process will be described later. When there is a similar region outside the teacher frame of the learning data, the parameter update unit 203 weights the set value (set parameter) of the loss of the similar region based on the inferred similarity. The details of the weighting process for the loss based on the inferred similarity will be described later.

[0029] 3 is a flowchart showing the flow of processing executed by the information processing device 100 according to the first embodiment. In the following explanation of each flowchart, the symbol S represents a processing step (process).

[0030] In S301, the learning unit 102 performs a learning process of the recognition model 106. During the learning process, the learning unit 102 first calculates a loss by comparing the output of the DNN when a learning image is input with the teacher data. This loss is a value indicating an error in the output. Hereinafter, the output of the DNN is referred to as an estimated map. The loss can be calculated using a loss function set in advance. For example, the learning unit 102 calculates the cross-entropy error shown in formula (1) for each pixel as the loss function E in the recognition task, and further calculates the loss by taking the sum of all pixels as shown in formula (2). Note that the loss function is not limited to this, and can be appropriately selected according to the detection target.

[0031] E i =-y i logp i -(1-y i )log(1-p i ) Formula (1)

[0032]

number

[0033] In formula (1), the cross entropy E between the coordinate p of the inference map and the coordinate y of the teacher data is calculated for each pixel i, and formula (2) indicates that the sum of all pixels is taken as the loss. The learning unit 102 updates the weight coefficients of each layer of the DNN based on the loss calculated as described above. For example, the learning unit 102 can update the weight coefficients by using backpropagation (BP) or the like based on the calculated loss. The learning unit 102 repeats these processes to sequentially update the weight coefficients of each layer. This gradually reduces the loss. That is, the estimated map approaches the teacher data. The learning unit 102 can perform the learning process of the recognition model 106 in this manner.

[0034] Next, in S302, the calculation unit 107 of the evaluation unit 104 performs inference and evaluation using the recognition model 106 of the recognition task for the verification image of the verification data 103, and outputs the results. Here, as the inference result, information on a frame representing the position of a detection target object by the recognition task for each verification image of the verification data 103 and a likelihood representing the probability of the detection target object for each frame are obtained. In addition, the calculation unit 107 also determines whether the detection frame detected from the verification image by inference is a correct detection frame or a false detection frame. Note that the calculation unit 107 may output an image in which the detection frame is superimposed on each verification image as the inference result. In addition, the evaluation result is the detection accuracy (recognition accuracy) of the recognition task when the inference result is matched with the correct answer data of the verification data 103. Here, as an example, the detection accuracy of the recognition task, that is, the detection accuracy of the human whole body task, is 70%.

[0035] Next, in S303, the comparison unit 108 of the evaluation unit 104 compares the detection accuracy output by the calculation unit 107 with a target accuracy t that is preset for the recognition task. If the detection accuracy of the recognition task is equal to or lower than the target accuracy (detection accuracy≦t), the processing of the evaluation unit 104 proceeds to S304. On the other hand, if the detection accuracy of the recognition task is higher than the target accuracy, that is, if the detection accuracy of the recognition task has achieved the target accuracy t, the processing of the flowchart in Fig. 3 ends. Here, the target accuracy of the recognition task, that is, the target accuracy t for the detection accuracy of the human whole body task, is assumed to be 80%.

[0036] When the process proceeds to S304, the result holding unit 109 extracts and holds, from the verification image, an image within a false positive frame in which the likelihood representing the likelihood of the detection target object obtained during inference of the recognition task for the verification image in S302 is higher than a predetermined likelihood threshold, as a false positive region image. Note that the result holding unit 109 may hold multiple false positive region images, and the number of images to be held is not limited. In this embodiment, it is assumed that the result holding unit 109 holds one false positive region image.

[0037] FIG. 4(a) is a diagram showing an example of a verification image of verification data in a human whole body task, that is, an evaluation image 400 used when the evaluation unit 104 evaluates the recognition model 106. The correct answer frame of the evaluation image 400 is assumed to be the correct answer frame 401 shown in FIG. 4(b). Also, an image 402 in FIG. 4(c) shows the result of inference of the human whole body task by the calculation unit 107 in S302 for the evaluation image 400, and detection frames 403 and 404 are assumed to be frames detected as a human whole body. In the example of FIG. 4(c), a flag pattern having similar characteristics to the pattern of the clothes worn by a person is erroneously detected in the inference of the human whole body task. In the example of FIG. 4(c), the detection frame 403 detected as a human region has a high degree of agreement with the correct answer frame 401 in FIG. 4(b), so the calculation unit 107 determines that the detection frame 403 is a correct detection frame. On the other hand, in the example of Fig. 4(c), detection frame 404 is detected in an area where no correct answer frame exists, so calculation unit 107 determines that detection frame 404 is an erroneous detection frame. Therefore, result storage unit 109 stores the image within detection frame 404 that has been determined to be an erroneous detection frame as erroneous detection area image 405 as shown in Fig. 4(d).

[0038] Next, in S305, the extraction unit 201 of the adjustment unit 105 performs inference to extract, from the training images of the training data 101, areas (similar areas) whose CNN features are similar to the false positive area images stored in the result storage unit 109.

[0039] FIG. 5(a) is a diagram showing an example of a training image 500 of training data. The training image 500 has a training frame 501 as shown in FIG. 5(b). The result of inferring a region in which the false positive region image stored in the result storage unit 109 and the CNN feature amount are similar for the training image 500 is the inference result 502 in FIG. 5(c). In the inference result 502 in FIG. 5(c), it is assumed that a similar region 503 and a similar region 504 are obtained as regions in which the false positive region image stored in the result storage unit 109 and the CNN feature amount are similar. As a method of determining a similar region, there is a method of inferring a similar region from the closeness of the CNN feature amount when the false positive region image 405 and the training image 500 are passed through the recognition model 106. In this case, for example, the cosine similarity between one-dimensional vectors of the CNN feature becomes the inferred similarity. Here, it is assumed that the inferred similarity of the similar region 503 is, for example, 250, and the inferred similarity of the similar region 504 is, for example, 210. It should be noted that the method of determining the similar region is not limited to this example, and there is also a method of training a CNN detector using the false positive region image 405 as training data and performing detection on the training image 500. In this case, the likelihood representing the likelihood of the detection target object becomes the inferred similarity.

[0040] Next, in S306, the extraction unit 201 compares the inferred similarity calculated in S305 with a preset similarity threshold s, and determines whether or not to perform processing for adjusting the learning of the learning unit 102 by the adjustment unit 105 based on the result of the comparison. That is, if there is at least one similar region whose inferred similarity is equal to or greater than the similarity threshold (inferred similarity≧s) as a result of the comparison, the extraction unit 201 determines to perform processing for adjusting the learning, and proceeds to S307. On the other hand, if there is no similar region whose inferred similarity is equal to or greater than the similarity threshold as a result of the comparison by the extraction unit 201, the processing in the information processing device 100 ends. Here, it is assumed that the similarity threshold s is set to 200. In the case of the above-mentioned example, the inferred similarity of the similar region 503 is 250, and the inferred similarity of the similar region 504 is 210, both of which exceed the similarity threshold s of 200, so the extraction unit 201 proceeds to S307.

[0041] In S307, the extraction unit 201 determines the relationship between the position of the similar region inferred in S305 and the position of the teacher frame of the learning image. Then, the process in the adjustment unit 105 branches depending on the determination result of the relationship between these positions. For example, if the position of the similar region is within the teacher frame of the learning image, the process in the adjustment unit 105 proceeds to S308, whereas if the position of the similar region is outside the teacher frame of the learning image, the process in the adjustment unit 105 proceeds to S309.

[0042] When the process proceeds to S308, the data update unit 202 performs feature reduction processing (processing to reduce features) on the similar regions present within the teacher frame of the learning image, and updates the learning data 101 using the image data after the feature reduction processing. In this embodiment, an example of the feature reduction processing is a process of replacing the similar regions with other image information in order to reduce edge information and shading information present in the image. Examples of the feature reduction processing include, but are not limited to, black painting, which sets all RGB luminance values ​​of the similar regions to 0, white painting, which sets the luminance values ​​to the maximum value (255 for an 8-bit image), and replacement with random luminance values.

[0043] 6(a) is a diagram showing an example of an image 601 in which a black painting process that sets all RGB luminance values ​​to 0 is applied as a feature reduction process to reduce the feature amount of a similar region 503 in a learning image 500. The data update unit 202 adds a teacher frame to the image 601 after the feature reduction process in the same position and size as the learning image 500, and replaces the learning image 500 and the image 601 and the associated teacher data in the learning data 101. By performing such a data update process, it becomes difficult to learn a region similar to an erroneous detection region image as a correct detection, so that erroneous detections that are a problem in practical use can be efficiently suppressed.

[0044] When the process proceeds to S309, the parameter update unit 203 performs an adjustment process of changing the set value during learning based on the inferred similarity acquired in S305 for the loss of the similar region existing outside the teacher frame of the learning image. That is, the parameter update unit 203 performs an adjustment process of changing the set parameter during learning in the learning unit 102 so that the loss of the similar region existing outside the teacher frame of the learning image is weighted based on the inferred similarity. Specifically, for the similar region 504 in FIG. 5(c), which is an example of a similar region outside the teacher frame, the parameter update unit 203 weights the loss by the inferred similarity acquired in S305. Here, the loss in this embodiment can be expressed in the form of Equation (3) in which the cross entropy error E for each pixel is weighted by the weight value w and an arbitrary coefficient α.

[0045]

number

[0046] For this reason, the parameter update unit 203 sets the weight w to 1 (i.e., no change) for pixels other than the similar region, and sets the weight w to the inferred similarity for pixels in the similar region, so that the loss of the similar region becomes large. By performing such a learning setting change process by updating the parameters, it becomes possible to learn to preferentially suppress regions similar to the erroneous detection region image, and it becomes possible to efficiently suppress erroneous detections that are problematic in practical use.

[0047] As described above, the adjustment unit 105 performs at least one of the update process of the learning data and the update process of the settings during learning. After that, the information processing device 100 returns to the process of S301, and the learning unit 102 performs learning again.

[0048] In the information processing device 100, by repeatedly performing the above-mentioned processes of S301 to S309, it is possible to learn the human whole-body task with high recognition accuracy in the DNN that learns the human whole-body task. As a result, it is possible to obtain a correct recognition result even for difficult cases in learning the human whole-body task, such as cases where objects or patterns other than humans are erroneously detected, or cases where objects are not detected due to clothing patterns or postures.

[0049] <Second embodiment> Next, as a second embodiment, an example in which data augmentation (DA) processing is performed on training images of training data will be described. DA processing is a method of artificially increasing the number of learning images by adding blur, shaking, image synthesis, rotation, translation, enlargement / reduction, up / down / left / right inversion, noise addition, edge change, color tone change, color temperature change, brightness change, shading change, etc. to the learning image. In this embodiment, DA processing is exemplified by shading change and rotation of an image area, but is not limited thereto. DA processing is applied to the entire learning image or a part of the image area of ​​the learning image, but in this case, an image may be generated in which a part that should be unchanged in the original image is changed, or the image may be converted into an image that does not exist in reality. When learning is performed using a learning image that has been subjected to such DA processing, overlearning may occur easily, making optimization difficult, which may result in the generalization performance not improving.

[0050] In the second embodiment, an example will be described in which the learning data is updated by performing DA processing on the learning image based on a similar area extracted from the learning data as an area similar to a false positive area image by inference of the verification data similar to the first embodiment, thereby enabling the recognition accuracy to be improved. In the following second embodiment, only functions and processing different from the above-mentioned first embodiment will be described, and illustration and description of functions and processing similar to the first embodiment will be omitted.

[0051] FIG. 2B is a block diagram showing the functional configuration of the adjustment unit 105 in the information processing device 100 of the second embodiment. In the second embodiment, the adjustment unit 105 includes an extraction unit 204 , a DA unit 205 , and a data update unit 206 . The extraction unit 204 is similar to the extraction unit 201 in the first embodiment, and extracts, from the learning data, a region similar to an erroneously detected region image based on inference of the verification data, as a similar region.

[0052] The DA unit 205 performs DA processing on the similar region that the extraction unit 204 extracts from the learning image of the learning data as a region similar to the erroneous detection region image. The data update unit 206 updates the learning data 101 so as to add image data that has been subjected to the DA process by the DA unit 205 .

[0053] 7 is a flowchart showing the flow of processing performed by the information processing device 100 of the second embodiment. Note that the processing from S701 to S706 is similar to the processing from S301 to S306 of the first embodiment, and therefore a description thereof will be omitted.

[0054] In the second embodiment, if the extraction unit 204 compares the inferred similarity with the similarity threshold s in S706 and there is at least one similar region whose inferred similarity is greater than or equal to the similarity threshold (estimated similarity ≧ s), the processing of the adjustment unit 105 proceeds to S707. In S707, the DA unit 205 performs a DA process on the similar region that the extraction unit 204 extracted from the learning image as a region similar to the erroneous detection region image.

[0055] FIG. 6(b) is a diagram showing an example of an image 602 after DA processing is performed on the similar regions 503 and 504 in FIG. 5(c) extracted as regions similar to the false detection region image from the learning image 500 shown in FIG. 5(a). The example in FIG. 6(c) shows an example in which the DA processing involves changing the luminance of the similar region 503 and rotating the similar region 504. That is, the DA processing that changes the luminance of the similar region results in a state in which the shading of the pattern in the similar region 603 in FIG. 6(b) has been changed compared to the similar region 503 in FIG. 5(c). Also, the DA processing that rotates the similar region results in a state in which the pattern in the similar region 604 in FIG. 6(b) has been rotated compared to the similar region 506 in FIG. 5(c).

[0056] Next, in S708, the data updating unit 206 adds a teacher frame to the image 602 that has been subjected to the DA processing in the same position and size as the training image 500, and updates the image 602 so as to add it to the training data 101.

[0057] The information processing device 100 of the second embodiment can improve the recognition accuracy of difficult cases and the like while reducing deterioration in the overall accuracy of the recognition task by repeatedly performing the processes of S701 to S708 described above. That is, by performing the above processes, the information processing device 100 of the second embodiment can learn using learning data with an increased variation in the learning image region to suppress false detection, thereby making it possible to suppress overlearning. Furthermore, according to the second embodiment, it is possible to pinpoint increase the variation of features effective for correct detection, making it possible to improve generalization performance while suppressing deterioration in the overall accuracy.

[0058] <Third embodiment> In the third embodiment, an example is described in which recognition accuracy is improved by replacing some areas between the learning data and a data set prepared in advance as data different from the learning data, based on similar areas to a false positive area image similar to that of the first embodiment.

[0059] Fig. 8 is a block diagram showing an example of the functional configuration of an information processing device 800 according to the third embodiment. Note that learning data 801 to recognition model 806 in Fig. 8 correspond to learning data 101 to recognition model 106 in Fig. 1, and calculation unit 807 to result holding unit 809 correspond to calculation unit 107 to result holding unit 109 in Fig. 1, respectively, and therefore their explanations will be omitted. The information processing device 800 of the third embodiment differs from the information processing device 100 of the first embodiment in that replacement data 810 is added so as to be connected to the adjustment unit 805.

[0060] The replacement data 810 is made up of a plurality of image data sets showing various kinds of objects and creatures different from the learning data 101. For example, various open data may be used as the replacement data 810, but the replacement data 810 is not limited to this. In addition, as described below, a label is added in advance to exclude a portion of the image data of the replacement data 810 that would obviously make the image unnatural if replaced with a partial region of the learning image, so as to exclude the portion from the processing target of this embodiment.

[0061] 2C is a block diagram showing the functional configuration of the adjustment unit 805 in the third embodiment. The adjustment unit 805 in the third embodiment includes an extraction unit 207, an area holding unit 208, a replacement unit 209, and a data update unit 210.

[0062] In order to identify an area similar to the erroneous detection area image, the extraction unit 207 performs inference on the learning data 801 and the replacement data 810. In this embodiment, the degree of similarity inferred by the extraction unit 207 on the learning data 801 and the replacement data 810 is called an inferred similarity. The extraction unit 207 compares the inferred similarity with a preset similarity threshold, and determines whether to continue or end the process based on the comparison result.

[0063] The region holding unit 208 holds images within the similar regions extracted by the extraction unit 207 from the images included in the learning data and the replaced data as regions similar to the erroneously detected region image. The replacement unit 209 performs processing to replace the images held in the area holding unit 208. Details of the image replacement processing will be described later. The data updating unit 210 updates the learning data 101 by adding image data that has been subjected to image replacement processing by the replacement unit 209 .

[0064] 9 is a flowchart showing the flow of processing performed by the information processing device 800 of the third embodiment. Note that the processing of S901 to S904 is similar to the processing of S301 to S304 of the first embodiment, and therefore the description thereof will be omitted. The processing of the information processing device 800 of the third embodiment proceeds to S905 after the comparison unit 808 performs the processing of S904.

[0065] In S905, the extraction unit 207 of the adjustment unit 805 performs inference to extract areas whose CNN features are similar to the false positive area images stored in the result storage unit 109 from the training images of the training data 801 and the images of the replaced data. The results of inference on the learning data 801 are similar to those shown in Fig. 5(a) to Fig. 5(c) of the first embodiment, and therefore will not be described further. Fig. 5(d) is a diagram showing an example in which inference is performed on an image example 505 included in replacement data 810, and a similar region 506 is specified as a region similar to the erroneously detected region image. It should be noted that the inference similarity of the similar region 506 at this time is assumed to be 230.

[0066] Next, in S906, the extraction unit 207 compares the inferred similarity calculated by the inference in S905 with a preset similarity threshold. If there is at least one similar region in the learning data whose inferred similarity is equal to or greater than the similarity threshold (inferred similarity≧s), the extraction unit 207 advances the process to S907. On the other hand, if there is no similar region in the learning data whose inferred similarity is equal to or greater than the similarity threshold, the process of the information processing device 800 ends. If there is a similar region in the learning data whose inferred similarity is equal to or greater than the similarity threshold, in S906, the extraction unit 207 stores images of the similar regions in the learning data and the replaced data whose inferred similarity is equal to or greater than the similarity threshold in the region storage unit 208. Here, for example, it is assumed that the similarity threshold s is 200, the inferred similarity of the similar region 503 is 250, the inferred similarity of the similar region 504 is 210, and the inferred similarity of the similar region 506 is 230. In this case, since the inferred similarity between similar regions 503, 504, and 506 exceeds the similarity threshold s of 200, the process proceeds to S907, and the images of similar regions 503, 504, and 506 are stored in the region storage unit 208.

[0067] Next, in S907, the replacement unit 209 replaces the images of the similar regions held in the region holding unit 208 in S906, and creates new image data.

[0068] FIG. 6(c) is a diagram showing an example of an image 605 in which the similar region 503 and the similar region 504 in FIG. 5(c) and the similar region 506 in FIG. 5(d) are replaced based on the learning image 500 shown in FIG. 5(a). The region 606 in FIG. 6(c) is obtained by replacing the similar region 503 in the image in FIG. 5(c) with the similar region 506 in the image of the replaced data in FIG. 5(d). The region 607 in FIG. 6(c) is obtained by replacing the similar region 504 in the image in FIG. 5(c) with the similar region 503 in FIG. 5(c). Note that the replacement of the similar regions may be random, and for example, when the sizes of the similar regions do not match, they may be replaced after being enlarged, reduced, rotated, or the like. For example, when the aspect ratios do not match, a part of the image may be cropped, or the image may be enlarged or reduced so that the aspect ratios match.

[0069] Next, in S908, the data update unit 210 adds a teacher frame to the image 605 that was replaced in S907 in the same position and size as the learning image 500 (Figure 5 (b)), adds it as learning data, and updates the learning data 801.

[0070] In the description of this embodiment, an example of creating one image has been given, but the present invention is not limited to this. It is also possible to generate multiple images by processing one image by changing the combination of replacing similar regions or using similar regions of other learning data or replacement data. In addition, in the description of this embodiment, an example of replacing similar regions of learning data and replacement data has been given, but the present invention is not limited to this. It is also possible to replace similar regions only in learning data without preparing replacement data.

[0071] As described above, the information processing device 800 of the third embodiment can improve the accuracy of difficult cases while reducing deterioration in the overall accuracy of the recognition process by repeatedly performing the processes of S901 to S908. That is, according to the information processing device 800 of the third embodiment, by replacing the similar areas of the learning data and the replacement data, learning can be performed using learning data with an increased variation of the learning image area to suppress false detection, and overlearning can be suppressed. Furthermore, according to this embodiment, it is possible to pinpoint the variation of features effective for correct detection, and therefore it is possible to improve generalization performance while suppressing deterioration in the overall accuracy.

[0072] FIG. 10 is a diagram showing a basic configuration of a computer capable of realizing the information processing apparatus according to each of the first to third embodiments described above. In Fig. 10, a processor 1001 is, for example, a CPU, which controls the operation of the entire computer and executes the information processing according to each of the above-mentioned embodiments. A memory 1002 is, for example, a RAM, which temporarily stores the information processing program and data according to this embodiment. A computer-readable storage medium 1003 is, for example, a hard disk or a CD-ROM, which stores the program and data for a long period of time. In this embodiment, a program that realizes the function of each unit stored in the storage medium 1003 is read into the memory 1002. Then, the processor 1001 operates according to the program on the memory 1002, thereby realizing the function of each unit in the information processing device according to each of the above-mentioned embodiments.

[0073] 10, an input interface 1004 is an interface for acquiring information from an external device. An output interface 1005 is an interface for outputting information to an external device. A bus 1006 connects the above-mentioned units and enables data exchange.

[0074] FIG. 11 is a diagram showing an example configuration of an information processing device 1100 that performs object detection using at least one of the structure and setting parameters of a recognition model 1103 learned by the information processing device of each of the above-mentioned embodiments, and outputs the detection results. The input unit 1101 is an image acquisition unit that acquires target data for object detection, that is, image data that is target data for human body detection in this embodiment, and sends the acquired image data to the detection unit 1102 . The recognition model 1103 is composed of a DNN, and is a learning model that has already been trained as described in each of the above-mentioned embodiments.

[0075] The detection unit 1102 performs object detection processing on object detection target data input from the input unit 1101 by using at least one of the structure and setting parameters of the recognition model 1103 trained in each of the above-mentioned embodiments. The output unit 1104 outputs the result of object detection for the object detection target data by the detection unit 1102. The output result may be, for example, information on the position and size of the detected frame, but is not limited thereto, and may be appropriately selected according to the intended use of the detection result.

[0076] The present invention can also be realized by a process in which a program for implementing one or more of the functions of the above-described embodiments is supplied to a system or device via a network or a storage medium, and one or more processors in a computer of the system or device read and execute the program. The present invention can also be realized by a circuit (e.g., ASIC) for implementing one or more of the functions. The above-mentioned embodiments are merely examples of the implementation of the present invention, and the technical scope of the present invention should not be interpreted as being limited by these. In other words, the present invention can be implemented in various forms without departing from its technical concept or main features.

[0077] The disclosure of this embodiment includes the following configuration, method, and program. (Configuration 1) A learning means for learning a learning model corresponding to a predetermined recognition process using the learning data; an evaluation means for performing the recognition process using the learning model on verification data corresponding to the learning model; an adjustment means for performing at least one of a predetermined adjustment process for the learning data and an adjustment process for changing settings for the learning process, based on a region that has been erroneously recognized from the verification data in the recognition process by the evaluation means; 13. An information processing device comprising: (Configuration 2) 2. The information processing device according to configuration 1, wherein the evaluation means performs the recognition process using the learning model on an image included in the verification data. (Configuration 3) 3. The information processing device according to configuration 2, wherein the verification data includes an image different from an image included in the training data. (Configuration 4) The evaluation means includes: an accuracy acquisition means for performing the recognition process using the learning model on the verification data and acquiring the recognition accuracy of the learning model; A comparison means for comparing the recognition accuracy of the learning model with a target accuracy; a result storage means for storing an area that is erroneously recognized in the recognition process from the verification data; 4. The information processing device according to any one of configurations 1 to 3, comprising: (Configuration 5) The information processing device described in configuration 4, characterized in that the accuracy acquisition means acquires the recognition accuracy of the learning model based on a result of comparing the result of the recognition processing on the verification data with correct answer data corresponding to the verification data. (Configuration 6) The information processing device according to configuration 4 or 5, characterized in that when the comparison by the comparison means shows that the recognition accuracy of the learning model is equal to or lower than the target accuracy, the result storage means stores an area from the verification data that was misrecognized in the recognition process. (Configuration 7) the accuracy acquisition means acquires a likelihood representing a probability that the verification data is a predetermined recognition target during the recognition process of the verification data; The information processing device according to any one of configurations 4 to 6, characterized in that the result holding means holds, from among the areas of the verification data that have been erroneously recognized in the recognition process, areas whose likelihood is higher than a likelihood threshold value. (Configuration 8) 8. The information processing apparatus according to any one of configurations 1 to 7, wherein the adjustment means has an extraction means for extracting a similar region similar to the erroneously recognized region from the learning data. (Configuration 9) The adjustment means is When the similar region is included in the teacher data of the learning data, a predetermined adjustment process is performed on the learning data; 9. The information processing device according to configuration 8, wherein, when the similar region is not included in teacher data of the learning data, an adjustment process is performed to change settings during the learning. (Configuration 10) The information processing device according to configuration 8, wherein the extraction means determines whether or not to perform the adjustment process by the adjustment means depending on a result of comparing the similarity between the misrecognized area and the similar area with a similarity threshold. (Configuration 11) 11. The information processing apparatus according to configuration 10, wherein the extraction means determines that the adjustment process is to be performed by the adjustment means when there is a similar region in which the similarity is equal to or greater than the similarity threshold value. (Configuration 12) The adjustment means is performing a predetermined adjustment process on the learning data when the similar region in which the similarity is equal to or greater than the similarity threshold is included in teacher data of the learning data; The information processing device according to configuration 11, characterized in that when the similar area in which the similarity is equal to or greater than the similarity threshold is included in the teacher data of the learning data, an adjustment process is performed to change the settings during the learning. (Configuration 13) The adjustment means is As a predetermined adjustment process for the learning data, a process of reducing a feature amount for the similar region of the learning data is performed; 13. The information processing device according to any one of configurations 9 to 12, wherein a process of changing a weighting for a loss during the learning is performed as an adjustment process for changing a setting during the learning. (Configuration 14) The adjustment means is a data updating means for updating the learning data with data obtained by performing an adjustment on the similar region of the learning data to reduce a feature amount; A parameter update means for updating a weighting parameter for the loss during the learning; 14. The information processing device according to claim 13, further comprising: (Configuration 15) The adjustment means is an extraction means for extracting a similar region similar to the erroneously recognized region from the learning data; A DA means for performing data augmentation on the similar region; A data updating means for updating the learning data so as to add data after the data augmentation is performed; 8. The information processing device according to any one of configurations 1 to 7, comprising: (Configuration 16) The adjustment means is an extraction means for extracting a similar region similar to the erroneously recognized region from the learning data and data different from the learning data; an area holding means for holding the similar area; a replacement means for replacing the similar region of the learning data with the similar region held in the region holding means; a data updating means for updating the learning data so as to add data after the replacement of the similar regions; 8. The information processing device according to any one of configurations 1 to 7, comprising: (Configuration 17) image acquisition means for acquiring an image; A detection means for detecting a detection target object from the image by using at least one of a learning model learned by the information processing device according to any one of configurations 1 to 16 and a setting used during the learning; an output means for outputting a detection result by the detection means; 13. An information processing device comprising: (Method 1) A learning process of learning a learning model corresponding to a predetermined recognition process using the learning data; an evaluation step of performing the recognition process using the learning model on verification data corresponding to the learning model; an adjustment step of performing at least one of a predetermined adjustment step for the learning data and an adjustment step for changing settings for the learning based on a region that has been erroneously recognized from the verification data in the recognition process by the evaluation step; 13. An information processing method comprising: (Program 1) A program for causing a computer to function as the information processing device according to any one of configurations 1 to 17. [Explanation of symbols]

[0078] 100: information processing device, 101: learning data, 102: learning unit, 103: verification data, 104: evaluation unit, 105: adjustment unit, 106: recognition model, 107: calculation unit, 108: comparison unit, 109: result storage unit

Claims

1. A learning means for learning a learning model corresponding to a predetermined recognition process using the learning data; an evaluation means for performing the recognition process using the learning model on verification data corresponding to the learning model; an adjustment means for performing at least one of a predetermined adjustment process for the learning data and an adjustment process for changing settings for the learning process, based on a region that has been erroneously recognized from the verification data in the recognition process by the evaluation means; 13. An information processing device comprising:

2. The information processing apparatus according to claim 1 , wherein the evaluation means performs the recognition process using the learning model on an image included in the verification data.

3. The information processing apparatus according to claim 2 , wherein the verification data includes an image different from an image included in the training data.

4. The evaluation means includes: an accuracy acquisition means for performing the recognition process using the learning model on the verification data and acquiring the recognition accuracy of the learning model; A comparison means for comparing the recognition accuracy of the learning model with a target accuracy; a result storage means for storing an area that is erroneously recognized in the recognition process from the verification data; 2. The information processing apparatus according to claim 1, further comprising:

5. The information processing device according to claim 4, characterized in that the accuracy acquisition means acquires the recognition accuracy of the learning model based on a result of comparing the result of the recognition process on the verification data with correct answer data corresponding to the verification data.

6. 5. The information processing apparatus according to claim 4, wherein when the comparison by said comparison means shows that the recognition accuracy of said learning model is equal to or lower than the target accuracy, said result holding means holds an area from said verification data that was misrecognized in said recognition process.

7. the accuracy acquisition means acquires a likelihood representing a probability that the verification data is a predetermined recognition target during the recognition process of the verification data; 5. The information processing apparatus according to claim 4, wherein the result storage means stores, from among the regions erroneously recognized in the recognition process from the verification data, regions in which the likelihood is higher than a likelihood threshold value.

8. 8. The information processing apparatus according to claim 1, wherein the adjustment means comprises an extraction means for extracting a similar region similar to the erroneously recognized region from the learning data.

9. The adjustment means is When the similar region is included in the teacher data of the learning data, a predetermined adjustment process is performed on the learning data; 9. The information processing apparatus according to claim 8, wherein, when the similar region is not included in the teacher data of the learning data, an adjustment process is performed to change a setting for the learning.

10. 9. The information processing device according to claim 8, wherein the extraction means determines whether or not to perform the adjustment process by the adjustment means depending on a result of comparing the similarity between the erroneously recognized area and the similar area with a similarity threshold.

11. 11. The information processing apparatus according to claim 10, wherein the extraction means determines that the adjustment process is to be performed by the adjustment means when there is a similar region in which the similarity is equal to or greater than the similarity threshold value.

12. The adjustment means is performing a predetermined adjustment process on the learning data when the similar region in which the similarity is equal to or greater than the similarity threshold is included in teacher data of the learning data; The information processing apparatus according to claim 11 , characterized in that, when the similar region in which the similarity is equal to or greater than the similarity threshold is included in teacher data of the learning data, an adjustment process for changing settings during the learning is performed.

13. The adjustment means is As a predetermined adjustment process for the learning data, a process of reducing a feature amount for the similar region of the learning data is performed; 13. The information processing apparatus according to claim 12, wherein the adjustment process for changing the settings during the learning comprises a process for changing a weighting for a loss during the learning.

14. The adjustment means is a data updating means for updating the learning data with data obtained by performing an adjustment on the similar region of the learning data to reduce a feature amount; A parameter update means for updating a weighting parameter for the loss during the learning; 14. The information processing apparatus according to claim 13, further comprising:

15. The adjustment means is an extraction means for extracting a similar region similar to the erroneously recognized region from the learning data; A DA means for performing data augmentation on the similar region; A data updating means for updating the learning data so as to add data after the data augmentation is performed; 8. The information processing apparatus according to claim 1, further comprising:

16. The adjustment means is an extraction means for extracting a similar region similar to the erroneously recognized region from the learning data and data different from the learning data; an area holding means for holding the similar area; A replacement means for replacing the similar region of the learning data with the similar region held in the region holding means; a data updating means for updating the learning data so as to add data after the replacement of the similar regions; 8. The information processing apparatus according to claim 1, further comprising:

17. image acquisition means for acquiring an image; a detection means for detecting a detection target object from the image by using at least one of a learning model learned by the information processing device according to claim 1 and a setting used during the learning; an output means for outputting a detection result by the detection means; 13. An information processing device comprising:

18. A learning process of learning a learning model corresponding to a predetermined recognition process using the learning data; an evaluation step of performing the recognition process using the learning model on verification data corresponding to the learning model; an adjustment step of performing at least one of a predetermined adjustment step for the learning data and an adjustment step for changing settings for the learning based on a region that has been erroneously recognized from the verification data in the recognition process by the evaluation step; 13. An information processing method comprising:

19. Computer, A learning means for learning a learning model corresponding to a predetermined recognition process using the learning data; an evaluation means for performing the recognition process using the learning model on verification data corresponding to the learning model; an adjustment means for performing at least one of a predetermined adjustment process for the learning data and an adjustment process for changing settings for the learning process, based on a region that has been erroneously recognized from the verification data in the recognition process by the evaluation means; A program that causes the device to function as an information processing device having the above-mentioned configuration.

Images