Information processor, information processing method and program

Abstract

To make a small scale student model preferentially learn knowledge of a teacher model prioritized from among a plurality of teacher models.SOLUTION: Provided is an information processor having: first extraction means for performing first feature extraction processing on a first image to extract first feature information; second extraction means for performing second feature extraction processing on a second image to extract second feature information; third extraction means for performing third feature extraction processing on the first image or the second image to extract third feature information; setting means for setting priority for the first feature information and the second feature information; comparison means for performing comparison between the first feature information as well as the second feature information and the third feature information on the basis of the first feature information, the second feature information, the third feature information and the priority to output a comparison result; and control means for controlling a parameter related to the third feature extraction processing on the basis of the output of the comparison means.SELECTED DRAWING: Figure 2

Description

[Technical field]

[0001] The present invention relates to an information processing device, an information processing method, and a program. [Background technology]

[0002] In recent years, many technologies have been proposed to extract useful information by performing advanced processing on images of objects captured in captured images. Among these, image recognition technology that uses a multi-layered neural network called a deep net to identify which of a number of pre-registered classes an object in an input image belongs to has been actively researched and developed. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] US Patent Application Publication No. 2021 / 0158156 Summary of the Invention [Problem to be solved by the invention]

[0004] In order to improve the accuracy of classification in classification processing using deep nets, a method called ensemble is being considered, in which multiple trained multi-layered neural networks (also called models) are prepared and their classification results are combined. The ensemble method is a method of obtaining higher accuracy than a single model by preparing multiple models that have been trained independently and using their prediction results or a feature vector that combines the individual feature vectors that they output.

[0005] In addition, when implementing classification processing using deep nets, the amount of processing (calculation amount) is often a problem. In particular, when applying an ensemble method, it is necessary to operate multiple models, and the amount of processing may be several times larger than when using a single model. In order to address the problem of the amount of processing, a technique called distillation is used to train knowledge held by a trained multi-layered neural network into another multi-layered neural network with a different amount of processing. The distillation technique is a technique for training a small-scale model (hereinafter also referred to as a "student model") based on the output data of a trained large-scale model (hereinafter also referred to as a "teacher model"). By using the distillation technique, it is possible to reduce the amount of processing while preventing a deterioration in classification accuracy to some extent.

[0006] Patent Document 1 proposes a technique for distilling an ensemble result of multiple trained models as teacher data to obtain a student model with improved accuracy compared to the ensemble result. In Patent Document 1, distillation is performed on a student model that has a larger number of parameters than the total number of parameters of multiple teacher models.

[0007] Patent Document 1 proposes distillation to a student model with a large number of parameters (large scale deep net), but distillation is generally performed on a student model with a small number of parameters (small scale deep net) to reduce the amount of processing. In such cases, the accuracy achieved by a large scale ensemble may not be achieved by a small scale student model. Patent Document 1 does not take into consideration distillation to such a small scale student model.

[0008] An object of the present invention is to allow a small-scale student model to preferentially learn knowledge from a teacher model that is prioritized among a plurality of teacher models. [Means for solving the problem]

[0009] The information processing device of the present invention is characterized in having a first extraction means for performing a first feature extraction process on a first image to extract first feature information, a second extraction means for performing a second feature extraction process on a second image to extract second feature information, a third extraction means for performing a third feature extraction process on the first image or the second image to extract third feature information, a setting means for setting priorities for the first feature information and the second feature information, a comparison means for comparing the first feature information and the second feature information with the third feature information based on the first feature information, the second feature information, the third feature information, and the priorities, and outputting the comparison result, and a control means for controlling parameters related to the third feature extraction process based on the output of the comparison means. Effect of the Invention

[0010] According to the present invention, it is possible to allow a small-scale student model to preferentially learn knowledge of a teacher model that is prioritized among a plurality of teacher models. [Brief description of the drawings]

[0011] [Figure 1] FIG. 2 is a diagram illustrating an ensemble method in the first embodiment. [Diagram 2] 1 is a diagram illustrating an example of the configuration of an information processing device according to a first embodiment. [Diagram 3] 4 is a diagram for explaining the configuration of a comparison unit and a data flow in the first embodiment. FIG. [Figure 4] 4 is a flowchart showing an example of processing by the information processing device in the first embodiment. [Diagram 5] FIG. 11 is a diagram illustrating an ensemble method in the second embodiment. [Figure 6] FIG. 11 is a diagram illustrating an example of the configuration of an information processing device according to a second embodiment. [Figure 7] 13 is a diagram illustrating the configuration of a comparison unit and a data flow in the second embodiment. FIG. [Figure 8] 11 is a flowchart showing an example of processing by the information processing device in the second embodiment. [Figure 9] FIG. 11 is a diagram illustrating an example of the configuration of an information processing device according to a third embodiment. [Figure 10] FIG. 2 is a diagram illustrating an example of a hardware configuration of an information processing device. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0012] Hereinafter, an embodiment of the present invention will be described with reference to the drawings. This embodiment relates to an image identification technology that identifies which of the pre-registered classes an object present in an input image belongs to. In the following, an example will be described in which the image identification technology of this embodiment is applied to face recognition that identifies which pre-registered person's face the face of a person present in an input image belongs to. In face recognition, for example, a multi-layered neural network called a deep net is used to compare an input face image with face images of multiple people that have been registered in advance, and it is determined who the input face image is. The deep net is also called a deep neural net or deep learning. Note that an example of application to a face recognition system will be described, but the target object in image recognition is not limited to a face, and each embodiment described below can be applied to other target objects.

[0013] <Embodiment 1> In the first embodiment, an example in which the information processing device in the present embodiment is applied to a system that performs face recognition (personal recognition) will be described. In the following, an example in which a priority is used when a neural network that realizes face recognition is trained by distillation will be described.

[0014] First, the ensemble method used in this embodiment will be described. For simplicity of explanation, a case will be described in which two trained models (called a first trained model and a second trained model) trained for face recognition (personal authentication) are used. In the ensemble method used in this embodiment, a final feature vector is obtained by concatenating feature vectors output from each trained model, and face recognition is performed using the final feature vector. For example, when a 512-dimensional feature vector is obtained from each of the two trained models, the final feature vector obtained by concatenating them is a 1024-dimensional vector. This final feature vector is called an ensemble concatenated feature vector.

[0015] With reference to FIG. 1(a) and FIG. 1(b), a case will be described in which such an ensemble method is used to determine whether faces in two images (image A and image B) are the faces of the same person or different people. As shown in FIG. 1(a), the information processing device inputs an image A111 to a first trained model 101 and a second trained model 102, respectively, to extract a first feature vector A112 and a second feature vector A113. The extracted first feature vector A112 and second feature vector A113 are concatenated to obtain a final concatenated feature vector A114. Next, as shown in FIG. 1(b), the information processing device performs a similar process on an image B121, extracts a first feature vector B122 and a second feature vector B123, and concatenates them to obtain a concatenated feature vector B124. Then, the combined feature vector A114 is compared with the combined feature vector B124 to determine whether the face in the image A111 and the face in the image B121 are the face of the same person.

[0016] By applying the ensemble method to multiple feature vectors obtained from multiple models in this way, it is possible to perform face recognition with higher accuracy than with a single model (for example, the first trained model). However, it is necessary to perform feature extraction multiple times (twice in this example), which increases the amount of processing compared to using a single model.

[0017] In this embodiment, by performing distillation learning so as to reproduce the final feature vector obtained by applying the ensemble method, it is possible to obtain a student model that can maintain the accuracy improvement realized by the ensemble method while suppressing an increase in the amount of processing. Furthermore, when the student model learns the knowledge of the teacher model, the student model is made to learn the knowledge of the learned model having the characteristics required for the student model, among the teacher model composed of a plurality of learned models, with priority. Thereby, even in learning using a small-scale student model, it is possible to learn from the teacher model with priority given to the necessary knowledge. For this purpose, a high priority is given to the learned model having the characteristics required by the student model, and the student model is learned based on the priority. Here, the characteristics required for the student model mean characteristics for achieving high accuracy in the use case (scene in which the student model is used) in which the student model is used, and a specific example will be described later.

[0018] The following describes the information processing device in embodiment 1. Fig. 2 is a block diagram showing an example of the configuration of an information processing device 200 in embodiment 1. The information processing device 200 has an image input unit 201, an object detection unit 202, a first feature vector extraction unit 203, a second feature vector extraction unit 204, a third feature vector extraction unit 205, a priority setting unit 206, a comparison unit 207, and an update control unit 208.

[0019] Image data to be learned is input to the image input unit 201. In this embodiment, it is assumed that a reproduced image of a captured recorded image is input to the image input unit 201.

[0020] The object detection unit 202 detects a target object from the image input to the image input unit 201, extracts a region including the target object, and obtains and outputs a detected image of the target object. In this example, the object detection unit 202 detects a face from the input image. The face detection method may be an existing publicly known method. For example, the object detection unit 202 extracts shapes corresponding to components of a face region such as a nose, mouth, and eyes from the input image, and estimates the size of the face from the size of both eyes and the distance between them. Then, the object detection unit 202 may set the region surrounded by a region of the estimated size based on a position corresponding to the center of the nose in the input image as an image of the face region. The image of the face region detected by the object detection unit 202 is normalized to a certain size by a predetermined method and output as a face region image.

[0021] The first feature vector extraction unit 203 performs a first feature extraction process on the detected image of the target object input from the object detection unit 202, and obtains and outputs a first feature vector. The first feature vector extraction unit 203 is an example of a first extraction means, and the first feature vector is an example of first feature information. The first feature vector extraction unit 203 is realized by a neural network in which feature extraction parameters obtained in advance by learning using learning data are set. In other words, the first feature vector extraction unit 203 is a trained model. In this example, the first feature vector extraction unit 203 performs a first feature extraction process on the face region image input from the object detection unit 202, and obtains and outputs a first feature vector.

[0022] The second feature vector extraction unit 204 performs a second feature extraction process on the detected image of the target object input from the object detection unit 202, and obtains and outputs a second feature vector. The second feature vector extraction unit 204 is an example of a second extraction means, and the second feature vector is an example of second feature information. The second feature vector extraction unit 204 is also realized by a neural network in which feature extraction parameters obtained in advance by learning using learning data are set. Therefore, the second feature vector extraction unit 204 is also a trained model. In this example, the second feature vector extraction unit 204 performs a second feature extraction process on the face region image input from the object detection unit 202, and obtains and outputs a second feature vector.

[0023] Both the first feature vector extraction unit 203 and the second feature vector extraction unit 204 are trained models, and in this embodiment, these two trained models constitute a teacher model for distillation learning. The feature extraction parameters of the first feature vector extraction unit 203 and the second feature vector extraction unit 204 are different from each other. This can be realized by using different learning data between the learning data used to train the first feature vector extraction unit 203 and the learning data used to train the second feature vector extraction unit 204. Also, even if the same learning data is used, it can be realized by changing the settings of the hyperparameters during learning.

[0024] In this embodiment, the first feature vector extraction unit 203 is assumed to have been trained using face images with little blur (for example, a face image with a clear face such as an ID photograph). This training image is called the first training data. Therefore, the first feature vector extraction unit 203 can be said to be a model trained to be able to extract feature vectors with high accuracy for a face image with a clear face such as an ID photograph. In this way, the features possessed by the entire training data used to train a certain feature extraction unit are called the attributes of the training data. The attribute of the first training data is that it has little blur.

[0025] Similarly, the second feature vector extraction unit 204 is assumed to have been trained using a facial image with a large amount of blur (a facial image with a large amount of motion). This training image is called the second training data. Therefore, the second feature vector extraction unit 204 can be said to be a model trained to be able to extract a feature vector with high accuracy for a facial image with a large amount of blur, such as an image taken by a surveillance camera. The attribute of the second training data is that it is largely blurred.

[0026] The third feature vector extraction unit 205 is a feature vector extraction unit (so-called student model) of a learning object by distillation, and receives as input the face region image sent from the object detection unit 202. The learning object by the learning method in this embodiment is the feature extraction parameters of this third feature vector extraction unit 205.

[0027] There is no particular restriction on the network structure of the neural network constituting the third feature vector extraction unit 205. However, the size of the third feature vector output by the third feature vector extraction unit 205 must be the same as the concatenated vector of the first feature vector output by the first feature vector extraction unit 203 and the second feature vector output by the second feature vector extraction unit 204. This is because distillation learning is performed so that the third feature vector reproduces the concatenated vector of the first feature vector and the second feature vector. In this embodiment, both the first feature vector and the second feature vector are 512-dimensional, and the third feature vector is 1024-dimensional.

[0028] Furthermore, in this embodiment, the purpose of performing distillation is to obtain a student model that requires less processing than performing feature extraction multiple times (twice in this example). Therefore, in this embodiment, it is necessary to have a network structure in which the processing amount of the third feature vector extraction unit 205 is smaller than the total processing amount of the first feature vector extraction unit 203 and the second feature vector extraction unit 204. The processing amount referred to here can be measured by the number of product-sum operations, the processing time when executed on a certain hardware, the amount of occupied memory, etc.

[0029] The third feature vector extraction unit 205 performs a third feature extraction process on the face region image input from the object detection unit 202, and obtains and outputs a third feature vector. The third feature vector extraction unit 205 is an example of a third extraction means, and the third feature vector is an example of third feature information. The feature extraction parameters of the third feature vector extraction unit 205 are updated from time to time through learning, as described below. Therefore, the feature vector is obtained in this case by a feature vector extraction process using the latest feature extraction parameters at that time.

[0030] The priority setting unit 206 sets priorities for the first feature vector output by the first feature vector extraction unit 203, which is a trained model, and the second feature vector output by the second feature vector extraction unit 204, which is also a trained model. The priorities are set according to a use case in which the third feature vector extraction unit 205, which is a learning target by distillation, is expected to be used.

[0031] In this example, in a use case in which a face image with little blur is assumed to be input to the third feature vector extraction unit 205, the priority setting unit 206 sets a high priority to the first feature vector output by the first feature vector extraction unit 203. For example, a use case in which the third feature vector extraction unit 205 is used is a case in which it is assumed that a person will stop in front of a camera in entrance and exit management using face recognition.

[0032] Conversely, in a use case where it is assumed that a facial image with a large blur is input to the third feature vector extraction unit 205, the priority setting unit 206 sets a high priority to the second feature vector output by the second feature vector extraction unit 204. For example, a use case in which the third feature vector extraction unit 205 is used is a case in which it is desired to search for a specific person using face recognition, particularly using images from a surveillance camera.

[0033] That is, the priority setting unit 206 sets a high priority when the learning image of each teacher model is similar to an image expected to be input in a use case of a student model. In other words, the priority setting unit 206 sets a high priority when the attribute of the learning image and the attribute of the image obtained in the use case are closely related, and sets a low priority when the relationship is low.

[0034] In this embodiment, a use case is assumed in which a face image with little blur is input to the third feature vector extraction unit 205. Therefore, a high priority is set to the first feature vector output from the first feature vector extraction unit 203. The priority is expressed by a relative numerical value, and in this example, a larger numerical value represents a higher priority. As an example, the priority of the first feature vector is set to "2", and the priority of the second feature vector is set to "1".

[0035] The comparison unit 207 receives the first feature vector, the second feature vector, the third feature vector, and the priority, compares the first feature vector and the second feature vector with the third feature vector, and outputs the comparison result. When comparing the feature vectors, the comparison unit 207 performs a weighted comparison based on the priority.

[0036] FIG. 3 is a diagram illustrating the configuration of the comparison unit 207 and the flow of data. A first feature vector 311, a second feature vector 312, a third feature vector 313, and a priority 314 are input to the comparison unit 207. The comparison unit 207 extracts the first 512 dimensions of the input third feature vector 313 (the third feature vector <1> 313-1) and the latter 512 dimensions (the third feature vector <2> 313-2) and

[0037] The distance calculation unit 301 calculates the distance between two input feature vectors. Although there is no particular restriction on the method of distance calculation, it is assumed here that the Euclidean distance (L2 distance) is calculated. The distance calculation unit 301 calculates the distance between the first feature vector 311 and the third feature vector <1> 313-1, and calculates a first Euclidean distance between the second feature vector 312 and the third feature vector <2> 313-2. The two Euclidean distances calculated by the distance calculation unit 301 are input to the weighted distance calculation unit 302. Although two distance calculation units 301 are shown in FIG. 3, it is also possible to calculate two Euclidean distances by performing processing twice using one distance calculation unit 301. Note that as a distance calculation method other than the Euclidean distance, the Manhattan distance (L1 distance), the cosine distance, etc. may be calculated.

[0038] The weighted distance calculation unit 302 weights the two Euclidean distances calculated by the distance calculation unit 301 based on the priority 314, and further calculates the sum. In this example, the priority setting unit 206 sets priorities as shown in the table 315 for the first feature vector and the second feature vector. The weighted distance calculation unit 302 weights the first Euclidean distance calculated using the first feature vector 311 output by the first feature vector extraction unit 203 with a priority 2 according to the set priority. Similarly, the weighted distance calculation unit 302 weights the second Euclidean distance calculated using the second feature vector 312 output by the second feature vector extraction unit 204 with a priority 1. The weighted distance calculation unit 302 further calculates the sum and outputs it as the weighted distance. That is, in this example, the weighted distance calculation unit 302 calculates "2 x first Euclidean distance + 1 x second Euclidean distance" and outputs it as the weighted distance.

[0039] Returning to Fig. 2, the update control unit 208 updates the feature extraction parameters of the third feature vector extraction unit 205 so as to reduce the weighted distance output as the comparison result from the comparison unit 207. A known method may be used for updating the feature extraction parameters. For example, the update control unit 208 may regard the weighted distance as an error, and update the feature extraction parameters of the third feature vector extraction unit 205 using an error backpropagation method so as to reduce the error.

[0040] According to the information processing device 200 configured as shown in Fig. 2, it is possible to perform distillation learning so as to reproduce the final feature vector obtained by applying an ensemble technique to a plurality of trained models. That is, the third feature vector extraction unit 205 can perform distillation learning using a linked feature vector obtained by integrating the feature vectors output from the trained first feature vector extraction unit 203 and second feature vector extraction unit 204 using an ensemble technique as a teacher signal. Furthermore, by calculating the error required for distillation learning as a weighted distance between the linked feature vector and the feature vector calculated by the student model, it is possible to prioritize the feature vector output by the teacher model.

[0041] The weighted distance is calculated using the configuration shown in Fig. 3. In the above example, the first feature vector 311 and the third feature vector <1> The distance (error) between the second feature vector 312 and the third feature vector 313-1 is <2> 313-2 is weighted by twice the distance (error) between the first feature vector and the second feature vector. Learning is performed with the aim of reducing this weighted distance. In order to reduce the weighted distance, it is advantageous to prioritize reducing the heavily weighted distance, so it is expected that learning will proceed so as to prioritize reducing the distance between the first feature vector and the first feature vector, which has a high priority (large weight).

[0042] As a result, the third feature vector extraction unit 205, which is the student model of distillation learning, learns by prioritizing the reproduction of the first 512-dimensional part (the part to be compared with the first feature vector) of the third feature vector it outputs. In other words, learning progresses so that the entire third feature vector reproduces the ensemble concatenated feature vector of the first feature vector and the second feature vector, and at that time, the first half of the third feature vector is given priority over the latter half.

[0043] When the network scale (feature extraction parameter amount) of the third feature vector extraction unit 205, which is the student model, is small, it is conceivable that it would be difficult to distill and learn all of the knowledge possessed by the two teacher models expressed by the ensemble link feature vector. However, even in such a case, according to this embodiment, it is possible to preferentially learn knowledge from the teacher model with high priority (in this example, the first half of the ensemble link feature vector, i.e., the first feature vector).

[0044] In this embodiment, it is expected that the third feature vector extraction unit 205 will be used in a situation where a face image with little blur is input. In order to realize high classification accuracy in actual use, it is desirable to preferentially distill and learn the knowledge of the first vector extraction unit 203 that has been learned about the face image with little blur.

[0045] Next, the operation of the information processing device 200 shown in Fig. 2 will be described with reference to Fig. 4. Fig. 4 is a flowchart showing an example of processing of the information processing device 200 shown in Fig. 2. Fig. 4 shows an example of the operation for one input image. For simplicity of explanation, it is assumed that one input image contains one face image. When there are multiple face images, the processing after face detection can be repeated as many times as the number of faces.

[0046] In step S401, the image input unit 201 inputs a learning image. In step S402, the object detection unit 202 executes a process of detecting a face, which is a target object, for the image (input image) input in step S401, and outputs a face region image including the detected face. Note that if no face is detected in the input image, the subsequent processes are skipped.

[0047] In step S403, the first feature vector extraction unit 203 performs feature vector extraction processing on the face area image obtained in step S402 to obtain a first feature vector. Similarly, the second feature vector extraction unit 204 and the third feature vector extraction unit 205 each perform feature vector extraction processing on the face area image to obtain a second feature vector and a third feature vector.

[0048] In step S404, the comparison unit 207 calculates a weighted distance based on the first feature vector, the second feature vector, and the third feature vector obtained in step S403 and the priorities set in the priority setting unit 206. The procedure for calculating the weighted distance has been described in detail with reference to Fig. 3, and therefore will not be described here.

[0049] In step S405, the update control unit 208 updates the feature extraction parameters of the third feature vector extraction unit 205 based on the weighted distance calculated in step S404. The feature extraction parameters may be updated using, for example, the backpropagation method as described above.

[0050] In step S406, the information processing device 200 uses the updated feature extraction parameters to measure the learning achievement of the third feature vector extraction unit 205 and judges whether the identification accuracy has reached a predetermined accuracy. Here, a verification image set (verification data) is prepared in advance, and the learning achievement is measured by measuring the face recognition accuracy for the verification image set. Therefore, the face recognition accuracy of the verification image set is measured using the third feature vector extraction unit 205, and whether or not the learning is terminated is judged depending on whether or not the accuracy exceeds a predetermined accuracy that has been determined in advance. If it is judged that the face recognition accuracy of the verification image set exceeds the predetermined accuracy (YES in step S406), the learning shown in FIG. 4 is terminated, and if not (NO in step S406), the process proceeds to step S407 to continue the learning. Note that in the flowchart shown in FIG. 4, the process of step S406 is performed every time the feature extraction parameters are updated, but it is common to perform the process every time the feature extraction parameters are updated a predetermined number of times, and this may also be the case in this embodiment.

[0051] In step S407, the information processing device 200 compares the number of parameter updates performed in step S405 with a preset predetermined number threshold. In general, learning is not continued until the condition in step S406 is satisfied, but is realistically terminated under some condition. In step S407, the number of parameter updates is used as a condition for terminating learning. If it is determined that the number of parameter updates exceeds the predetermined number threshold (YES in step S407), the learning shown in FIG. 4 is terminated, and if not (NO in step S407), the process proceeds to step S401 to continue learning. Note that, as a condition for terminating learning, the learning time may be adopted, or the change in the calculated weighted distance may be reduced. In this embodiment, since learning can be prioritized from the knowledge of the teacher model with a high priority, even if learning is terminated midway, it is expected that the knowledge with a high priority is learned by the student model.

[0052] As described above, in this embodiment, priorities are assigned to multiple feature vectors output by multiple trained models, and a student model is trained based on the assigned priorities. This makes it possible to preferentially train the student model with knowledge of a trained model having characteristics required for the student model among a teacher model composed of multiple trained models. This allows the knowledge of the prioritized teacher model to be preferentially trained even if the expressive ability (amount of feature extraction parameters) of the student model is small.

[0053] In the above example, the teacher model is composed of two trained models, but it can be applied to three or more trained models. Also, the example of the magnitude of blur is shown as an attribute of the training image, but the orientation of the face in the image and the presence or absence of an item worn on the face (mask or sunglasses) can also be used as an attribute.

[0054] <Embodiment 2> In the second embodiment, the information processing device in the present embodiment is also applied to a system for performing face recognition (personal recognition). In the first embodiment, there are a plurality of trained teacher models, each having a different feature extraction parameter. The same image is input to the plurality of teacher models. In the second embodiment, an example is described in which there are a plurality of trained teacher models having the same feature extraction parameter. Furthermore, images cut out from the same image with slightly different cutouts are input to each of the plurality of teacher models. In the second embodiment, the teacher models are the same, but the input images are different, so that different feature vectors are output from each teacher model. It is not necessary to have a plurality of exactly the same teacher models, and images cut out in different ways may be input sequentially to a single teacher model to sequentially obtain feature vectors. However, in the following description, for ease of explanation, a case in which there are a plurality of exactly the same teacher models will be described.

[0055] First, the ensemble method used in this embodiment will be described. For the sake of simplicity, a case will be described in which two trained models (called the third trained model and the fourth trained model) trained for face recognition (personal authentication) are used. As described above, the third trained model and the fourth trained model are essentially the same. In the ensemble method performed in this embodiment, a final feature vector is obtained by weighting and averaging the feature vectors output by each trained model, and face recognition is performed using the final feature vector. For example, when a 512-dimensional feature vector is obtained from each of the two trained models, the final feature vector obtained by weighting and averaging them is a 512-dimensional vector. This final feature vector is called the ensemble average feature vector.

[0056] In this embodiment, as an example, face images with different enlargement / reduction ratios are input to a plurality of teacher models. Examples of input images with different enlargement / reduction ratios are shown in FIG. 5(a) and FIG. 5(b). For example, image 501 shown in FIG. 5(a) is an input image to a third trained model, and image 502 shown in FIG. 5(b) is an input image to a fourth trained model. Input image 501 and input image 502 are images cut out from the same face image with different enlargement / reduction ratios. In other words, input image 501 and input image 502 have the same image size, but the size of the face in the image (for example, the distance between the eyes) is different.

[0057] With reference to FIG. 5(c) and FIG. 5(d), a case will be described in which such an ensemble method is used to determine whether faces in two images (images C and D) are the faces of the same person or different people. As shown in FIG. 5(c), the information processing device cuts out a face image such as image 521 from image C and inputs it to a third trained model 511, and similarly cuts out a face image such as image 522 from image C and inputs it to a fourth trained model 512. In this way, the third feature vector C523 and the fourth feature vector C524 are extracted, and the final average feature vector C525 is obtained by weighting and averaging them. Next, as shown in FIG. 5(d), the information processing device performs the same process on image D, extracts a third feature vector D533 and a fourth feature vector D534 from images 531 and 532, and weights and averages them to obtain an average feature vector D535. Then, the average feature vector C525 is compared with the average feature vector D535 to determine whether the face in image C and the face in image D are the face of the same person.

[0058] By applying the ensemble method to multiple feature vectors obtained by inputting such slightly different images into a single model, it is possible to perform face recognition with higher accuracy than when a single image is input into a single model. The reason for this is that the face size in the cut-out image input into the trained model is not necessarily always constant. Although the face is cut out from the input image by a face detector, the face size may vary slightly depending on the state of the face in the image (face direction, face brightness, etc.). Inputting multiple cut-out images into a single model to obtain multiple feature vectors and averaging them leads to the acquisition of feature vectors that are robust to changes in face size, which is expected to result in improved face recognition accuracy.

[0059] Fig. 6 is a block diagram showing an example of the configuration of an information processing device 600 in embodiment 2. In Fig. 6, the same components as those shown in Fig. 2 are denoted by the same reference numerals, and duplicated explanations will be omitted. The information processing device 600 has an image input unit 201, an object detection unit 602, a fourth feature vector extraction unit 603, a fifth feature vector extraction unit 604, a sixth feature vector extraction unit 605, a priority setting unit 606, a comparison unit 607, and an update control unit 208.

[0060] The object detection unit 602, like the object detection unit 202, detects a target object in an input image, extracts an area including the target object, and obtains and outputs a detected image of the target object. However, in this embodiment, the object detection unit 602 generates input images for each feature vector extraction unit by performing normalization to a plurality of object sizes. In this example, the object detection unit 602 performs face detection on the input image data, and generates input images for each feature extraction unit by performing normalization to a plurality of face sizes. That is, the object detection unit 602 creates images with the same size but different face sizes in the images, such as the input image 501 shown in FIG. 5(a) and the input image 502 shown in FIG. 5(b). In this embodiment, the object detection unit 602 creates a face image like the input image 501 and a face image like the input image 502. Then, the object detection unit 602 outputs the input image 501 to a fourth feature vector extraction unit 603 and a sixth feature vector extraction unit 605 , and outputs the input image 502 to a fifth feature vector extraction unit 604 .

[0061] The fourth feature vector extraction unit 603 performs a fourth feature extraction process on the detected image in which the target object input from the object detection unit 602 is a first size, and obtains and outputs a fourth feature vector. The fourth feature vector extraction unit 603 is an example of a first extraction means, and the fourth feature vector is an example of first feature information. The fourth feature vector extraction unit 603 is realized by a neural network in which feature extraction parameters obtained in advance by learning using learning data are set. In other words, the fourth feature vector extraction unit 603 is a trained model. In this example, the fourth feature vector extraction unit 603 performs a fourth feature extraction process on the face region image (face size equivalent to the input image 501) input from the object detection unit 602, and obtains and outputs a fourth feature vector.

[0062] The fifth feature vector extraction unit 604 performs a fifth feature extraction process on the detected image in which the target object input from the object detection unit 602 is of the second size, and obtains and outputs a fifth feature vector. The fifth feature vector extraction unit 604 is an example of a second extraction means, and the fifth feature vector is an example of second feature information. The substance of the fifth feature vector extraction unit 604 is the same as that of the fourth feature vector extraction unit 603. In this example, the fifth feature vector extraction unit 604 performs a fifth feature extraction process on the face region image (face size equivalent to the input image 502) input from the object detection unit 602, and obtains and outputs a fifth feature vector.

[0063] The fourth feature vector extraction unit 603 is a trained model as described above. It is assumed that the training was performed using a training image having a face size like the input image 501. In this case, in the fourth feature vector extraction unit 603, the face size of the training image used during its own training (corresponding to the input image 501) matches the face size of the training image input when used as a teacher model for distillation (corresponding to the input image 501). In contrast, in the fifth feature vector extraction unit 604, the face size of the training image used during its own training (corresponding to the input image 501) does not match the face size of the training image input when used as a teacher model for distillation (corresponding to the input image 502). Therefore, when each model is used alone, it is expected that better face authentication can be performed by using the feature vector output by the fourth feature vector extraction unit 603. However, when the face size input during face authentication is not necessarily constant, something like the fifth feature vector extraction unit 604 may occur.

[0064] The sixth feature vector extraction unit 605 is a feature vector extraction unit (so-called student model) of a learning object by distillation, and receives as input a face region image (face size equivalent to the input image 501) sent from the object detection unit 602. The learning object by the learning method in this embodiment is the feature extraction parameters of this sixth feature vector extraction unit 605.

[0065] There is no particular restriction on the network structure of the neural network constituting the sixth feature vector extraction unit 605. However, the size of the sixth feature vector output by the sixth feature vector extraction unit 605 must be the same as the size of the fourth feature vector and the fifth feature vector output by the fourth feature vector extraction unit 603 and the fifth feature vector extraction unit 604, respectively. This is because distillation learning is performed so that the sixth feature vector reproduces the weighted average of the fourth feature vector and the fifth feature vector. In this embodiment, the fourth feature vector, the fifth feature vector, and the sixth feature vector are all 512-dimensional.

[0066] In this embodiment, the purpose of distillation is to obtain a student model with less processing load than performing feature extraction multiple times (twice in this example). Therefore, it is necessary to make the network structure such that the processing load of the sixth feature vector extraction unit 605 is smaller than the total processing load of the fourth feature vector extraction unit 603 and the fifth feature vector extraction unit 604.

[0067] The sixth feature vector extraction unit 605 performs a sixth feature extraction process on the face region image (face size equivalent to the input image 501) input from the object detection unit 602, and obtains and outputs a sixth feature vector. The sixth feature vector extraction unit 605 is an example of a third extraction means, and the sixth feature vector is an example of third feature information. The feature extraction parameters of the sixth feature vector extraction unit 605 are updated from time to time through learning, as described below. Therefore, in this case, the feature vector is obtained by a feature vector extraction process using the latest feature extraction parameters at that time.

[0068] The priority setting unit 606 sets priorities for the fourth feature vector output by the fourth feature vector extraction unit 603, which is a trained model, and the fifth feature vector output by the fifth feature vector extraction unit 604, which is also a trained model. The priorities are set according to a use case in which the sixth feature vector extraction unit 605, which is a learning target by distillation, is expected to be used.

[0069] For example, if the sixth feature vector extraction unit 605 is expected to be used in an environment where the face condition (direction, brightness, etc.) in the image is expected to be good and it is easy to cut out a face of a certain size, the priority of the fourth feature vector is increased. In other words, if the learning image of the teacher model and the image expected to be input in the use case of the student model are similar, the priority of the feature vector output by the feature vector extraction unit to which an image of a face size similar to that during learning of the teacher model is input is increased. In the example of this embodiment, the priority of the fourth feature vector is increased.

[0070] On the other hand, if the sixth feature vector extraction unit 605 is expected to be used in an environment where the condition of the face in the image is poor and it is difficult to cut out the face to a certain size, the priority of the fourth feature vector is lowered. In other words, if the learning image of the teacher model and the image expected to be input in the use case of the student model are not similar, the priority of the feature vector output by the feature vector extraction unit to which an image of a face size similar to that during learning of the teacher model is input is lowered. In the example of this embodiment, the priority of the fourth feature vector is lowered.

[0071] In this embodiment, the face size in the input image is the attribute of the data. Therefore, when the attribute (face size) of the training image of the teacher model is closely related to the attribute (face size) of the image obtained in the use case, the priority setting unit 606 gives a high priority to the feature vector output by the trained model trained with the image having the attribute (face size).

[0072] In this embodiment, a use case is assumed in which the face size in the image input to the sixth feature vector extraction unit 605 is assumed to be stable. Therefore, a high priority is set to the fourth feature vector output from the fourth feature vector extraction unit 603. The priority is expressed by a relative numerical value, and in this example, a larger numerical value represents a higher priority. As an example, the priority of the fourth feature vector extraction unit 603 is set to "2", and the priority of the fifth feature vector extraction unit 604 is set to "1".

[0073] The comparison unit 607 receives the fourth, fifth, and sixth feature vectors and the priority, compares the fourth and fifth feature vectors with the sixth feature vector, and outputs the comparison result. When comparing feature vectors, the comparison unit 607 performs a weighted comparison based on the priority.

[0074] FIG. 7 is a diagram for explaining the configuration of the comparison unit 607 and the data flow. The comparison unit 607 receives a fourth feature vector 711, a fifth feature vector 712, a sixth feature vector 713, and a priority 714. The weighted average calculation unit 701 calculates a weighted average of the fourth feature vector 711 and the fifth feature vector 712 based on the priority 714. In this example, the priority setting unit 606 sets priorities as shown in a table 715. The weighted average calculation unit 701 weights the fourth feature vector 711 output by the fourth feature vector extraction unit 603 with a priority of 2 according to the set priorities. Similarly, the weighted average calculation unit 701 weights the fifth feature vector 712 output by the fifth feature vector extraction unit 604 with a priority of 1. The weighted average calculation unit 701 further calculates the average (weighted average) of them and outputs it as a weighted average feature vector. That is, in this example, the weighted average calculation unit 701 calculates “(2×fourth feature vector+1×fifth feature vector) / (2+1).” The distance calculation unit 702 calculates the distance between the weighted average feature vector of the two input feature vectors, i.e., the fourth feature vector 711 and the fifth feature vector 712, and the sixth feature vector 713, and outputs the distance.

[0075] According to the information processing device 600 configured as shown in FIG. 6, the error required for distillation learning is calculated as the distance between the weighted average feature vector and the feature vector calculated by the student model, so that the feature vector output by the teacher model can be prioritized. The weighted average feature vector is one in which the characteristics of the feature vector with high priority (highly weighted) are largely reflected. That is, the weighted average feature vector in the above-mentioned example is one in which the characteristics of the fourth feature vector are largely reflected compared to the characteristics of the fifth feature vector. In this embodiment, since it is assumed that the sixth feature vector extraction unit 605 is used in an environment in which it is easy to cut out a face of a roughly constant size, it is preferable to learn a feature vector in which the characteristics of the fourth feature vector are largely reflected.

[0076] Next, the operation of the information processing device 600 shown in Fig. 6 will be described with reference to Fig. 8. Fig. 8 is a flowchart showing an example of processing of the information processing device 600 shown in Fig. 6. Fig. 8 shows an example of the operation for one input image. For simplicity of explanation, it is assumed that one input image contains one face image. When there are multiple face images, the processing after face detection can be repeated as many times as the number of faces.

[0077] In step S801, the image input unit 201 inputs a learning image. In step S802, the object detection unit 602 executes a detection process of a face, which is a target object, on the image input in step S801 (input image), and outputs an image including the detected face. At this time, the object detection unit 602 creates a plurality of images with different face image sizes, and outputs them to the feature vector extraction units 603 to 605. Note that if no face is detected in the input image, the subsequent processes are skipped.

[0078] In step S803, the fourth feature vector extraction unit 603 performs feature vector extraction processing on the image obtained in step S802 to obtain a fourth feature vector. Similarly, the fifth feature vector extraction unit 604 and the sixth feature vector extraction unit 605 also perform feature vector extraction processing on the image obtained in step S802 to obtain a fifth feature vector and a sixth feature vector, respectively.

[0079] In step S804, the comparison unit 607 calculates the distance between the feature vectors based on the fourth, fifth, and sixth feature vectors obtained in step S803 and the priorities set in the priority setting unit 606. The comparison unit 607 first calculates a weighted average feature vector of the fourth and fifth feature vectors. Furthermore, the comparison unit 607 calculates the distance between the feature vectors based on the calculated weighted average feature vector and the sixth feature vector. The procedure for calculating this distance has been described in detail with reference to FIG. 7, and therefore will not be described here.

[0080] In step S805, the update control unit 208 updates the feature extraction parameters of the sixth feature vector extraction unit 605 based on the distance calculated in step S804. The feature extraction parameters may be updated using, for example, the backpropagation method as described above.

[0081] In step S806, the information processing device 600 uses the updated feature extraction parameters to measure the learning achievement of the sixth feature vector extraction unit 605 and judges whether the classification accuracy has reached a predetermined accuracy. The process of step S806 is the same as step S406 in FIG. 4 in the first embodiment, so detailed description will be omitted. If it is judged that the classification accuracy exceeds the predetermined accuracy (YES in step S806), the learning shown in FIG. 8 is terminated, and if not (NO in step S806), the process proceeds to step S807 to continue the learning. Note that, in the flowchart shown in FIG. 8, the process of step S806 is performed every time the feature extraction parameters are updated, but it is common to perform the process every time the feature extraction parameters are updated a predetermined number of times, and this may also be the case in this embodiment.

[0082] In step S807, the information processing device 600 compares the number of parameter updates performed in step S805 with a preset predetermined number threshold. The process of step S807 is similar to step S407 in Fig. 4 in the first embodiment, and therefore detailed description will be omitted. If it is determined that the number of parameter updates exceeds the predetermined number threshold (YES in step S807), the learning shown in Fig. 8 is terminated, and if not (NO in step S807), the process proceeds to step S801 to continue the learning.

[0083] As described above, in this embodiment, priorities are assigned to multiple feature vectors output by multiple trained models, and the student model is trained based on the assigned priorities. This makes it possible to have the student model learn knowledge of feature vectors having characteristics required for the student model with priority. This makes it possible to learn knowledge of prioritized feature vectors with priority even if the expressive ability (amount of feature extraction parameters) of the student model is small.

[0084] In the above example, two types of cut-out images are output from the object detection unit 602, but three or more types are also applicable. Also, although the example of enlargement / reduction is shown as a cut-out method, the position where the cut-out is performed may be shifted.

[0085] In addition, in each of the above-described embodiments, the priority is determined based on the attributes of the learning image of the learned model, but it may be determined based on the evaluation results when the learned model is evaluated. For example, when it is assumed that the student model is used in an environment where images with little blur are obtained, evaluation images with little blur (a group of images that can measure the face recognition accuracy) are prepared, and the priority can be determined based on the evaluation results for the evaluation images. In other words, for the evaluation images, a high priority may be assigned to a learned model that has achieved high accuracy, and a low priority may be assigned to a learned model that has low accuracy.

[0086] <Embodiment 3> Hereinafter, an information processing device that performs image recognition using a feature vector extraction unit obtained by distillation learning as described above will be described. Fig. 9 is a block diagram showing an example of the configuration of an information processing device 900 in embodiment 3. The information processing device 900 has an image input unit 901, an object detection unit 902, a feature vector extraction unit 903, a similarity acquisition unit 904, and a judgment processing unit 905.

[0087] An image to be subjected to classification processing (in this example, an image to be subjected to face recognition processing) is input to the image input unit 901. In this embodiment, it is assumed that a reproduced image of a captured recorded image is input to the image input unit 901.

[0088] The object detection unit 902 detects a target object in the image input from the image input unit 901, extracts an area including the target object from the image, and acquires and outputs a detected image of the target object. In the example of this embodiment, the object detection unit 902 detects a face, which is a target object, in the input image. Any existing publicly known method may be used as the face detection method. The image of the face area detected by the object detection unit 902 is normalized to a fixed size by a predetermined method and output as a face area image.

[0089] The feature vector extraction unit 903 performs feature vector extraction processing on the image input from the object detection unit 902, and obtains and outputs the feature vector of the input image. In the example of this embodiment, the feature vector extraction unit 903 performs feature vector extraction processing on the face region image input from the object detection unit 902, and obtains and outputs the feature vector of the face region image. The feature vector extraction unit 903 is a trained model (student model) obtained by distillation learning using multiple trained models (teacher models), like the third feature vector extraction unit 205 in the first embodiment and the sixth feature vector extraction unit 605 in the second embodiment. In other words, the feature vector extraction unit 903 is a trained model (student model) obtained by distillation learning by weighting multiple trained models (teacher models) according to the priority set for them.

[0090] The similarity acquiring unit 904 acquires the similarity between the two feature vectors based on the feature vectors obtained by the feature vector extracting unit 903. In this embodiment, two face area images to be matched may be input, and the similarity between the two feature vectors obtained from each image by the feature vector extracting unit 903 may be acquired. Alternatively, a registered image and its feature vector may be registered in advance, and a feature vector based on the registered image and one face area image to be matched may be input, and the similarity between the feature vector obtained from the image by the feature vector extracting unit 903 may be acquired.

[0091] The judgment processing unit 905 judges whether the people whose faces appear in the face area images to be compared are the same person or different people based on the similarity of the feature vectors acquired by the similarity acquisition unit 904, and outputs the judgment result.

[0092] <Hardware configuration> 10 is a block diagram showing an example of the hardware configuration of an information processing device in each of the above-mentioned embodiments. The information processing device 1000 has a CPU 1001, a ROM 1002, a RAM 1003, a storage device 1004, an operation unit 1005, a display unit 1006, a communication unit 1007, and a system bus 1008. The CPU 1001, the ROM 1002, the RAM 1003, the storage device 1004, the operation unit 1005, the display unit 1006, and the communication unit 1007 are communicatively connected via the system bus 1008. Note that the information processing device 1000 may further have other configurations.

[0093] A CPU (Central Processing Unit) 1001 controls the entire information processing device 1000 using computer programs and data stored in a ROM 1002 and a RAM 1003, and realizes, for example, each function of the information processing device described above. Note that the information processing device 1000 may have one or more dedicated hardware pieces different from the CPU 1001, and at least a part of the processing by the CPU 1001 may be executed by the dedicated hardware pieces. Examples of the dedicated hardware pieces include an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), a DSP (Digital Signal Processor), and a GPU (Graphics Processing Unit).

[0094] A ROM (Read Only Memory) 1002 stores programs that do not require modification. A RAM (Random Access Memory) 1003 temporarily stores programs and data supplied from a storage device 1004, and data supplied from the outside via a communication unit 1007. The RAM 1003 also functions as a main memory, work area, etc. for the CPU 1001. The storage device 1004 is composed of, for example, a hard disk drive (HDD) or a solid state drive (SSD), and stores various data. The storage device 1004 stores, for example, various data required when the CPU 1001 performs processing related to the programs, and various data obtained by the CPU 1001 performing processing related to the programs.

[0095] The operation unit 1005 is composed of, for example, a keyboard, a mouse, a joystick, a touch panel, etc., and receives operations by a user to input various instructions to the CPU 1001. The display unit 1006 is composed of, for example, a liquid crystal display, an LED, etc., and displays a GUI (Graphical User Interface) for the user to operate the information processing device 1000, processing results by the CPU 1001, etc. Note that the operation unit 1005 and the display unit 1006 may exist as separate devices outside the information processing device 1000. The communication unit 1007 connects the information processing device 1000 to a network and controls communication with other devices, etc.

[0096] <Other embodiments of the present invention> The present invention can also be realized by a process in which a program for implementing one or more of the functions of the above-described embodiment is supplied to a system or device via a network or a storage medium, and one or more processors in a computer of the system or device read and execute the program. The present invention can also be realized by a circuit (e.g., ASIC) for implementing one or more of the functions.

[0097] It should be noted that the above-mentioned embodiments are merely examples of the implementation of the present invention, and the technical scope of the present invention should not be interpreted as being limited by these. In other words, the present invention can be implemented in various forms without departing from its technical concept or main features.

[0098] The disclosure of the present embodiment includes the following configurations, methods, etc. (Configuration 1) a first extraction means for performing a first feature extraction process on a first image to extract first feature information; a second extraction means for performing a second feature extraction process on the second image to extract second feature information; a third extraction means for extracting third feature information by performing a third feature extraction process on the first image or the second image; a setting means for setting priorities for the first feature information and the second feature information; a comparison means for comparing the first feature information and the second feature information with the third feature information based on the first feature information, the second feature information, the third feature information, and the priority, and outputting a comparison result; and a control unit that controls parameters relating to the third feature extraction process based on an output of the comparison unit. (Configuration 2) the first image and the second image are the same image, The information processing device described in configuration 1, characterized in that the comparison means outputs a result of comparing the first feature information with the third feature information, a result of comparing the second feature information with the third feature information, and the comparison result obtained based on the priority. (Configuration 3) The information processing device according to configuration 2, wherein the comparison means weights the distance between the first feature information and the third feature information and the distance between the second feature information and the third feature information based on the priority, and outputs a sum of the weighted distances as the comparison result. (Configuration 4) The first image and the second image are different images, The information processing device according to configuration 1, wherein the comparison means outputs a comparison result obtained by comparing weighted feature information obtained based on the first feature information, the second feature information, and the priority with the third feature information. (Configuration 5) The information processing device according to configuration 4, wherein the comparison means compares feature information obtained from the first feature information and the second feature information, each weighted based on the priority, with the third feature information. (Configuration 6) 6. The information processing device according to any one of configurations 1 to 5, wherein the first extraction means and the second extraction means are trained models. (Configuration 7) 7. The information processing device according to any one of configurations 1 to 6, wherein the setting means sets the priority based on attributes of learning data used in learning of the first extraction means and the second extraction means. (Configuration 8) 7. The information processing device according to any one of configurations 1 to 6, wherein the setting means sets the priority based on an evaluation result when the first extraction means and the second extraction means are evaluated. (Configuration 9) 9. The information processing device according to any one of configurations 1 to 8, wherein the setting means sets the priority based on an attribute of an environment in which the third extraction means is expected to be used. (Configuration 10) A learned extraction means, the learned extraction means being a third extraction means according to any one of configurations 1 to 8; a similarity obtaining means for obtaining a similarity of the feature information extracted by the third extraction means from the two images; and determining means for determining whether or not the objects appearing in the two images are the same object based on the similarity acquired by the similarity acquisition means. (Method 1) An information processing method executed by an information processing device, a first extraction step of performing a first feature extraction process on the first image to extract first feature information; a second extraction step of performing a second feature extraction process on the second image to extract second feature information; a third extraction step of performing a third feature extraction process on the first image or the second image to extract third feature information; a setting step of setting priorities for the first feature information and the second feature information; a comparison step of comparing the first feature information and the second feature information with the third feature information based on the first feature information, the second feature information, the third feature information, and the priority, and outputting a comparison result; and a control step of controlling parameters relating to the third feature extraction process based on an output from the comparison step. (Program 1) The computer of the information processing device a first extraction step of performing a first feature extraction process on the first image to extract first feature information; a second extraction step of performing a second feature extraction process on the second image to extract second feature information; a third extraction step of performing a third feature extraction process on the first image or the second image to extract third feature information; a setting step of setting priorities for the first feature information and the second feature information; a comparison step of comparing the first feature information and the second feature information with the third feature information based on the first feature information, the second feature information, the third feature information, and the priority, and outputting a comparison result; and a control step of controlling parameters related to the third feature extraction process based on an output from the comparison step. [Explanation of symbols]

[0099] 200, 600: Information processing device 201: Image input unit 202, 602: Object detection unit 203-205, 603-605: Feature vector extraction unit 206, 606: Priority setting unit 207, 607: Comparison unit 208: Update control unit

Claims

1. a first extraction means for performing a first feature extraction process on a first image to extract first feature information; a second extraction means for performing a second feature extraction process on the second image to extract second feature information; a third extraction means for extracting third feature information by performing a third feature extraction process on the first image or the second image; a setting means for setting priorities for the first feature information and the second feature information; a comparison means for comparing the first feature information and the second feature information with the third feature information based on the first feature information, the second feature information, the third feature information, and the priority level, and outputting a comparison result; and a control means for controlling a parameter relating to the third feature extraction process based on an output of the comparison means.

2. the first image and the second image are the same image, 2. The information processing device according to claim 1, characterized in that the comparison means outputs a result of comparing the first feature information with the third feature information, a result of comparing the second feature information with the third feature information, and the comparison result obtained based on the priority.

3. 3. The information processing device according to claim 2, wherein the comparison means weights the distance between the first feature information and the third feature information and the distance between the second feature information and the third feature information based on the priority, and outputs a sum of the weighted distances as the comparison result.

4. The first image and the second image are different images, 2. The information processing apparatus according to claim 1, wherein the comparison means outputs a comparison result obtained by comparing weighted feature information obtained based on the first feature information, the second feature information, and the priority with the third feature information.

5. 5. The information processing apparatus according to claim 4, wherein the comparison means compares feature information obtained from the first feature information and the second feature information, each weighted based on the priority, with the third feature information.

6. The information processing apparatus according to claim 1 , wherein the first extraction means and the second extraction means are trained models.

7. 2 . The information processing apparatus according to claim 1 , wherein the setting means sets the priority based on attributes of learning data used in learning by the first extraction means and the second extraction means.

8. 2. The information processing apparatus according to claim 1, wherein the setting means sets the priority based on an evaluation result when the first extraction means and the second extraction means are evaluated.

9. 2. The information processing apparatus according to claim 1, wherein said setting means sets said priority based on an attribute of an environment in which said third extraction means is expected to be used.

10. A learned extraction means, comprising the third extraction means according to claim 1; a similarity obtaining means for obtaining a similarity of the feature information extracted by the third extracting means from the two images; a determining unit that determines whether or not the object appearing in the two images is the same object based on the similarity acquired by the similarity acquiring unit.

11. An information processing method executed by an information processing device, a first extraction step of performing a first feature extraction process on the first image to extract first feature information; a second extraction step of performing a second feature extraction process on the second image to extract second feature information; a third extraction step of extracting third feature information by performing a third feature extraction process on the first image or the second image; a setting step of setting priorities for the first feature information and the second feature information; a comparison step of comparing the first feature information and the second feature information with the third feature information based on the first feature information, the second feature information, the third feature information, and the priority level, and outputting a comparison result; and a control step of controlling parameters relating to the third feature extraction process based on an output from the comparison step.

12. The computer of the information processing device a first extraction step of performing a first feature extraction process on the first image to extract first feature information; a second extraction step of performing a second feature extraction process on the second image to extract second feature information; a third extraction step of performing a third feature extraction process on the first image or the second image to extract third feature information; a setting step of setting priorities for the first feature information and the second feature information; a comparison step of comparing the first feature information and the second feature information with the third feature information based on the first feature information, the second feature information, the third feature information, and the priority level, and outputting a comparison result; and a control step of controlling parameters relating to the third feature extraction process based on an output from the comparison step.

Images