Information processing device, information processing method and program

Abstract

To prevent deterioration of identification performance even if a use environment is changed.SOLUTION: An information processing device according to an embodiment includes one or more input devices, an extraction unit, a specification unit, an environment information acquisition unit, an analysis unit, and a re-registration unit. The one or more input devices acquire input information. The extraction unit extracts a feature amount indicating a feature of a person from the input information by using a person feature model. The specification unit specifies the person by comparing the feature amount with a feature amount indicated by a human dictionary. The environment information acquisition unit acquires environment information from the input information. The analysis unit analyzes a change of the environment information and specification accuracy of the person by the specification unit. The re-registration unit performs control to re-register the person dictionary when environment information is changed or the specification accuracy is less than an accuracy threshold.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] Embodiments of the present invention relate to an information processing apparatus, an information processing method, and a program.

Background Art

[0002] A person identification technique for preventing performance degradation due to changes over time or changes in physical condition has been conventionally known. For example, a method of updating person information when misrecognized by having a user input personal characteristic information when identification fails has been conventionally known.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Non-Patent Documents

[0004]

Non-Patent Document 1

Non-Patent Document 2

Non-Patent Document 3

Summary of the Invention

Problems to be Solved by the Invention

[0005] However, in the conventional technology, it has been difficult to prevent the deterioration of the identification performance when the usage environment changes.

Means for Solving the Problem

[0006] The information processing apparatus according to the embodiment includes one or more input devices, an extraction unit, an identification unit, an environment information acquisition unit, an analysis unit, and a re-registration unit. The one or more input devices acquire input information. The extraction unit extracts a feature amount indicating a feature of a person from the input information using a person feature model. The identification unit identifies the person by comparing the feature amount with the feature amount indicated by a person dictionary. The environment information acquisition unit acquires environment information from the input information. The analysis unit analyzes the change in the environment information and the identification accuracy of the person by the identification unit. The re-registration unit controls re-registration of the person dictionary when there is a change in the environment information or when the identification accuracy is less than an accuracy threshold.

Brief Description of the Drawings

[0007]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7A

Figure 7B

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

Figure 14

Figure 15

Figure 16

Figure 17

Figure 18

Figure 19

[0008] Hereinafter, embodiments of an information processing apparatus, an information processing method, and a program will be described in detail with reference to the accompanying drawings.

[0009] (First Embodiment) In the first embodiment, when registering person-specific information, environmental information such as noise types is also considered to detect changes in the environment and update the person-specific information according to the environment, thereby preventing performance degradation. An information processing apparatus will be described. The information processing apparatus according to the first embodiment may be any device, for example, a personal computer, a smart device (such as a tablet and a smartphone), and a game machine.

[0010] [Example of Functional Configuration] FIG. 1 is a diagram showing an example of the functional configuration of an information processing apparatus 100 according to the first embodiment. The information processing apparatus 100 according to the first embodiment includes an input device 1, a signal acquisition unit 2, an extraction unit 3, a person feature model storage unit 4, a registration unit 5, a person dictionary storage unit 6, a specification unit 7, an environmental information acquisition unit 11, an analysis unit 21, a re-registration unit 31, and an environmental information storage unit 41.

[0011] The input device 1 is a microphone (mic) for acquiring input information, a camera, and the like. The input device 1 is not limited to one, and a plurality of input devices 1 may be provided.

[0012] The signal acquisition unit 2 acquires input information from the input device 1.

[0013] The extraction unit 3 extracts person feature quantities based on the person feature model read from the person feature model storage unit 4 from the acquired input information. For example, when the input device 1 is a microphone and the signal included in the input information is audio, the extraction unit 3 performs feature extraction on the acoustic signals input from the microphone at each time. In this case, the feature quantities to be extracted are MFCC (Mel-Frequency Cepstrum Coefficients), Mel filter bank feature quantities, and the like.

[0014] Furthermore, the extraction unit 3 generates an embedding vector based on the extracted feature quantities by i-vector, d-vector (Non-Patent Document 1), x-vector (Non-Patent Document 2), and their derivative methods.

[0015] Further, when the input device 1 is a camera or the like, the extraction unit 3 may extract the feature amount obtained from the image as a person feature amount.

[0016] The person feature model storage unit 4 stores a person feature model used for extraction of the feature amount. Specifically, the person feature model storage unit 4 stores parameters of a person feature model that extracts features of a person.

[0017] The registration unit 5 stores, for example, the average vector of the person feature amounts of a plurality of frames obtained per unit time in the person dictionary storage unit 6 as a person dictionary indicating the features of a person.

[0018] FIG. 2 is a diagram showing an example of the person dictionary storage unit 6 of the first embodiment. The person dictionary storage unit 6 of the first embodiment includes an id and a person feature amount. The id is identification information (for example, a unique value) for identifying the data of each row. The person dictionary indicates the average vector of the above-described person feature amounts. In the example of FIG. 2, it is an example in which Mr. A is registered at two locations and Mr. B is registered at one location. When Mr. A uses the information processing apparatus 100, for example, one of the two person dictionaries of Mr. A may be selected according to the place where Mr. A uses it. Also, for example, when the person dictionary is selected, Mr. A may be asked to make a sound, and the person dictionary with the best score (the person dictionary with the highest similarity to Mr. A's voice) may be selected.

[0019] Returning to FIG. 1, the specifying unit 7 calculates the similarity between the person dictionary stored in the person dictionary storage unit 6 and the person feature amount extracted from the input signal. As a method for calculating the similarity, methods such as cosine similarity and PLDA (Non-Patent Document 3) are used. When there is a person dictionary whose similarity exceeds a predetermined threshold value, the specifying unit 7 notifies that the person identified by the person dictionary has been specified.

[0020] The environment information acquisition unit 11 acquires environment information indicating the environment of the information processing apparatus 100, and stores the acquired environment information in the environment information storage unit 41. Details of the method for acquiring the environment information will be described later in Modification Example 1 of the first embodiment.

[0021] FIG. 3 is a diagram showing an example of the environmental information storage unit 41 of the first embodiment. The environmental information storage unit 41 of the first embodiment includes an id, noise, and a microphone. The description of the id is the same as that in FIG. 2, so it is omitted. Noise and the microphone are examples of the environmental information of the first embodiment. In the noise, the presence or absence of noise is stored. In the microphone, the type of the microphone (for example, identification information for identifying an input device, etc.) is stored.

[0022] Returning to FIG. 1, the analysis unit 21 includes a determination unit 22 and a monitoring unit 23. The determination unit 22 determines (detects) an environmental change from the environmental information acquired by the environmental information acquisition unit 11. The monitoring unit 23 monitors the accuracy of the specific unit 7 (person specific function).

[0023] Examples of environmental changes include the presence or absence of noise and differences in input devices. As a method for monitoring accuracy, for example, before using the information processing apparatus 100, a value obtained by dividing the number of detections of a user registered in the person dictionary storage unit 6 by the total number of detections including the number of detections of other users misidentified is used. For example, when a user is detected 10 times and another user is detected 1 time by the person dictionary of the user registered in the person dictionary storage unit 6, the accuracy is 10 / 11.

[0024] Also, for example, as a method for monitoring the accuracy of the person specific function by speaker identification, a method of comparing the number of times a person speaks measured by a camera or the like with the number of detections by the person dictionary of the person registered in the person dictionary storage unit 6 may be used.

[0025] Also, for example, as a method for monitoring the accuracy of the person specific function by speaker identification, a method of monitoring that the similarity between the user's voice and the user's person dictionary is decreasing may be used.

[0026] For example, as a method for monitoring the accuracy of the person identification function by speaker identification, a method may be used in which the user is asked to provide feedback on the performance of the person identification function after using the information processing apparatus 100. As a method of providing feedback, for example, a method may be used in which the user evaluates it in five levels using an input device 1 such as a touch panel. Also, for example, as a method of providing feedback, a method of providing a button for notifying that the performance is not good, or a method of recognizing the user's facial expression using an input device 1 such as a camera may be used.

[0027] When the usage environment of the user changes as a result of the analysis by the analysis unit 21, or when the identification accuracy of the user falls below a predetermined threshold, the re-registration unit 31 updates the person dictionary of the user. For example, as a method of updating the person dictionary, a method is used in which the user inputs input information again to the input device 1 and the person dictionary in the person dictionary storage unit 6 is updated.

[0028] When using the person identification function of the information processing apparatus 100 according to the first embodiment, there are two phases: a phase of registering a person dictionary in the person dictionary storage unit 6 and a phase of performing person identification using the registered person dictionary.

[0029] The phase of registering the person dictionary in the person dictionary storage unit 6 is performed by the above-described registration unit 5 before using the person identification function of the information processing apparatus 100.

[0030] The phase of performing person identification using the registered person dictionary will be described with reference to FIG. 4.

[0031] FIG. 4 is a flowchart showing an example of the person identification method according to the first embodiment. First, the information processing apparatus 100 receives an operation input to turn on the power (step S1), and the use of the information processing apparatus 100 is started.

[0032] Next, the specifying unit 7 selects (user selection) a person dictionary to be used for person identification (step S2). For example, the following methods (i) to (iii) and the like are used for user selection.

[0033] (i) A method for receiving, via an input device 1, a selection operation of a registered person dictionary from a user (ii) A method for speculating who is using by narrowing down the user's behavior pattern assuming a certain group of users (iii) A method for speculating using a technique for identifying a person other than the person identification

[0034] In the example of (i), for example, when a microphone is provided as one of the input devices 1, candidates for the person dictionary and their descriptions (for example, descriptions such as the usage environment based on environmental information) are presented on a display device or the like, and the selection of the person dictionary is received by voice input from the user. Also, for example, when a keyboard and a mouse are provided as one of the input devices 1, the selection of the person dictionary is received by operation input from the user.

[0035] Regarding the example of (ii), for example, when the information processing device 100 is a game machine or the like, the specifying unit 7 specifies the user from the login information when logging in to the game machine and selects the person dictionary of the user. Also, for example, when the information processing device 100 is mounted on an automobile or the like, the specifying unit 7 uses the behavior pattern of the user (driver) (for example, on weekdays: driven by Mr. A, on holidays: driven by Mr. B, etc.) as one of the judgments for user selection.

[0036] Regarding the example of (iii), for example, when a camera and a fingerprint authentication sensor are provided as the input device 1, the specifying unit 7 specifies the user from the photographed image and fingerprint information and selects the person dictionary of the user. The example of (iii) is an example of making a user selection using a function different from the person identification function, such as performing image recognition with a camera when the person identification function by the specifying unit 7 is speaker identification using voice.

[0037] Next, the environmental information acquisition unit 11 acquires a part of the environmental information (step S3). Next, the determination unit 22 determines an environmental change score indicating the magnitude of the difference (the magnitude of the environmental change) between the environmental information associated with the person dictionary selected in step S2 (the environmental information at the time of person dictionary registration) and the environmental information acquired in step S3 (step S4). The presence or absence of a difference is determined, for example, as follows: if the same microphone is being used, there is no difference (the environmental change score is less than the threshold), and if the same type of noise is present, there is no difference (the environmental change score is less than the threshold). The details of the environmental change score will be described in Modification Example 1 of the first embodiment.

[0038] When the environmental change score is greater than the threshold (step S4, Yes), the re-registration unit 31 re-registers the person dictionary selected in step S2 (step S5).

[0039] When the environmental change score is less than the threshold (step S4, No), the identification unit 7 identifies (discriminates) the person by comparing the feature amount extracted from the user input with the feature amount indicated by the person dictionary (step S6).

[0040] Also, the environmental information acquisition unit 11 acquires environmental information from the user input at the time of identification in step S6 (step S7).

[0041] Next, the analysis unit 21 (the determination unit 22 and the monitoring unit 23) analyzes the change in the environmental information (the difference between the environmental information at the time of person dictionary registration and the environmental information acquired in step S7) and the person identification accuracy by the identification unit 7 (step S8).

[0042] When the analysis is NG (there is a difference from the environmental information at the time of person dictionary registration or the person identification accuracy is less than the threshold) (step S8, No), the re-registration unit 31 re-registers the person dictionary selected in step S2 (step S9).

[0043] When the analysis is OK (there is no difference from the environmental information at the time of registering the person dictionary, or the identification accuracy of the person is equal to or higher than the threshold value) (step S8, Yes), as long as the information processing apparatus 100 is not turned off (step S10, No), the process returns to step S6. When it is turned off (step S10, Yes), the process ends.

[0044] As described above, in the information processing apparatus 100 of the first embodiment, one or more input devices 1 acquire input information. The extraction unit 3 extracts a feature amount indicating the characteristics of a person from the input information using a person feature model. The identification unit 7 identifies a person by comparing the feature amount with the feature amount indicated by the person dictionary. The environmental information acquisition unit 11 acquires environmental information from the input information. The analysis unit 21 analyzes the change in environmental information and the identification accuracy of the person by the identification unit 7. Then, when there is a change in environmental information or when the identification accuracy is less than the accuracy threshold value, the re-registration unit 31 performs control to re-register the person dictionary.

[0045] According to the first embodiment, it is possible to prevent a decrease in identification performance even when the usage environment changes. For example, even when the feature amount of a person includes noise in the usage environment in addition to the user's voice, or when the feature amount considered other than the user's features due to the frequency characteristics and signal processing of the microphone being used, it is possible to prevent a decrease in identification performance.

[0046] (Modification Example 1 of the First Embodiment) Next, modification example 1 of the first embodiment will be described. In the description of modification example 1, the same description as that of the first embodiment will be omitted, and the parts different from the first embodiment will be described. In modification example 1, a more detailed specific example of the environmental information acquisition unit 11 in FIG. 1 and an example of a calculation method for an environmental change score will be described.

[0047] FIG. 5 is a diagram showing an example of the functional configuration of the environmental information acquisition unit 11 in modification example 1 of the first embodiment. The environmental information acquisition unit 11 receives inputs of a plurality of input information from a plurality of input devices 1 and acquires environmental information from the plurality of input information.

[0048] For example, the plurality of input devices 1 includes a microphone, a GPS receiver that receives GPS (Global Positioning System) information input, a camera that acquires a captured image, and a device that receives operation input from a user (e.g., a keyboard and a mouse), etc.

[0049] The environmental information acquisition unit 11 of Modification 1 includes a microphone information acquisition unit 111, an SNR (Signal Noise Ratio) acquisition unit 112, a noise type acquisition unit 113, a spatial information acquisition unit 114, a captured image acquisition unit 115, a GPS information acquisition unit 116, a total motor number acquisition unit 117, a registered word number acquisition unit 118, an age acquisition unit 119, and a gender acquisition unit 120.

[0050] For example, when the person identification function is a speaker identification technology, a microphone is used as the input device 1 used for speaker identification. The extraction unit 3 extracts a signal (feature quantity) used for person identification from the acoustic signal acquired from the microphone via the signal acquisition unit 2.

[0051] The microphone information acquisition unit 111, the SNR acquisition unit 112, the noise type acquisition unit 113, and the spatial information acquisition unit 114 acquire (estimate) environmental information by a method different from that of the signal acquisition unit 2, and store the acquired environmental information in the environmental information storage unit 41.

[0052] Specifically, the microphone information acquisition unit 111 acquires the frequency characteristics of the microphone from the input information from the microphone. The SNR acquisition unit 112 acquires the SNR from the input information from the microphone. The noise type acquisition unit 113 acquires the type of noise from the input information from the microphone, for example, using a classifier that can classify noise. The spatial information acquisition unit 114 acquires the spatial information of sound (e.g., reverberation and reverberation time, etc.) from the input information from the microphone.

[0053] Also, the captured image acquisition unit 115 acquires a captured image from the input information from the camera. For example, the captured image is a captured image including a person. Also, for example, when the camera is an iris sensor, the captured image includes the iris information of the person.

[0054] The GPS information acquisition unit 116 acquires the position information of the information processing apparatus 100 based on GPS information from the input information from the GPS receiver.

[0055] Also, the total mora number acquisition unit 117 acquires the total mora number from the input information input from the microphone at the time of registering the person dictionary. Similarly, the registered word number acquisition unit 118 acquires the registered word number from the input information input from the microphone at the time of registering the person dictionary.

[0056] Also, the age acquisition unit 119 acquires the age of the user via a keyboard, a mouse, etc. Similarly, the gender acquisition unit 120 acquires the gender of the user via a keyboard, a mouse, etc.

[0057] By acquiring the environmental information as described above, it becomes possible to detect that the environmental information is different at the time of registering the person dictionary and at the time of specifying the person.

[0058] FIG. 6 is a diagram showing an example of environmental information of Modification 1 of the first embodiment. Since the description of the id column is the same as that of FIG. 2, the description is omitted. In the noise type column, for example, the classification result by a classifier that can classify noise is stored. In the microphone column, the device name of the microphone and the frequency characteristics of the microphone are stored. In the SNR column, the average SNR of the noise at the time of registering the person dictionary is stored as the magnitude of the noise. In the camera column, the captured image is stored. Similarly, the above-described other environmental information is also stored in the environmental information storage unit 41 below.

[0059] Next, an example of calculating the environmental change score will be described. For example, after expressing the difference of each environmental information as a scalar value, it is scored by the following formula (1).

[0060] Environmental change score r = c1 * microphone information difference + c2 * spatial information difference + c3 * SNR difference + … + ···(1)

[0061] Here, c1, c2, c3, … are weights, and the method for obtaining the optimal values of c1, c2, c3, … depends on the usage requirements of the information processing apparatus 100 and the like. When the environmental change score is greater than a threshold value (for example, 5 (%) etc.), it is determined that the environment has changed, and re-registration of the person dictionary is performed.

[0062] As an example, a method of representing the difference between each environmental information as a scalar value has been mentioned, but the environmental change score may be defined by a method using a non-linear function such as a neural network. The update method (re-registration method) of the person dictionary can be changed according to the combination of this environmental change score and the result of monitoring the accuracy of person identification.

[0063] Examples of the acquisition method of each environmental information and the calculation method of the difference in FIG. 2 are shown in FIGS. 7A and 7B. FIGS. 7A and 7B are diagrams showing examples of the acquisition method of environmental information and the calculation method of the difference in Modification 1 of the first embodiment. In the examples of FIGS. 7A and 7B, when there are multiple possible calculation methods for the difference, an example 2 of the difference is also described.

[0064] As described above, according to Modification 1, environmental changes can be considered by comprehensively considering more various information.

[0065] Note that the example of the environmental information acquisition unit 11 in FIG. 5 is an example, and environmental information may be acquired by other methods. For example, when the input device 1 is equipped with a sensor for acquiring a person's fingerprint, information indicating the person's fingerprint may be acquired as environmental information.

[0066] (Modification 2 of the First Embodiment) Next, Modification 2 of the first embodiment will be described. In the description of Modification 2, the same descriptions as those in the first embodiment will be omitted, and the parts different from the first embodiment will be described. In Modification 2, a method of inferring a part of the data that could not be acquired when a part of the environmental information of the environmental information acquisition unit 11 in FIG. 1 cannot be acquired will be described.

[0067] FIG. 8 is a diagram showing an example of the functional configuration of the environmental information acquisition unit 11 according to Modification 2 of the first embodiment. The environmental information acquisition unit 11 according to Modification 2 includes an acquisition unit 12 and an estimation unit 13. The acquisition unit 12 acquires environmental information from the input information input from the input device 1.

[0068] The estimation unit 13 estimates environmental information using, for example, environmental information acquired by the acquisition unit 12 so far and stored in the environmental information storage unit 41. As a result, it is possible to more accurately detect environmental changes and update the person dictionary.

[0069] For example, due to some reason (such as when it cannot be used due to a defect), noise and spatial information may not be acquirable by the mounted device (input device 1) of the information processing apparatus 100. In that case, when the estimation unit 13 cannot acquire some data indicating environmental information, the estimation unit 13 estimates the position information indicated by the some data from the position information and the moving speed of the information processing apparatus 100, and estimates the environmental information indicated by the some data that could not be acquired from the estimated position information. Specifically, the estimation unit 13 estimates where it is now (for example, in a car, in a crowded street, at an airport, or in a department store) from the position information and moving speed obtained from the GPS information, etc., and thereby estimates environmental information such as the type of noise and spatial information (for example, reverberation and reverberation time).

[0070] Also, for example, a part of certain continuous time-series data may be missing. When the environmental information is time-series data, the estimation unit 13 estimates the environmental information by interpolating some data from the acquired part of the data indicating the environmental information when some data indicating the environmental information cannot be acquired. For example, when the data from 2 seconds to 4 seconds cannot be obtained out of the data for 10 seconds, the estimation unit 13 interpolates the missing data by approximating the function with a linear function or the like using the data before and after. Although the method of using a linear function is given as an example of the interpolation method, a method of approximating with a non-linear function such as a neural network or linear prediction analysis may also be used.

[0071] For example, as an example of a method for inferring environmental information, a method may be considered in which a plurality of candidates for general-purpose environmental information are prepared in advance, and the inference unit 13 infers the environmental information by selecting one from among the plurality of candidates for environmental information.

[0072] (Second Embodiment) Next, the second embodiment will be described. In the description of the second embodiment, descriptions similar to those of the first embodiment will be omitted, and differences from the first embodiment will be described. In the second embodiment, a plurality of methods for re-registering (updating) the person dictionary are prepared, and control for selecting one or more re-registration methods from among the plurality of re-registration methods will be described.

[0073] [Example of Functional Configuration] FIG. 9 is a diagram showing an example of the functional configuration of the information processing apparatus 100-2 according to the second embodiment. The information processing apparatus 100-2 according to the second embodiment includes an input device 1, a signal acquisition unit 2, an extraction unit 3, a person feature model storage unit 4, a registration unit 5, a person dictionary storage unit 6, a specification unit 7, an environmental information acquisition unit 11, an analysis unit 21, a re-registration unit 31, an input information storage unit 32, a generation unit 33, and an environmental information storage unit 41.

[0074] In the second embodiment, an input information storage unit 32 and a generation unit 33 are further added to the configuration of the information processing apparatus 100 of the first embodiment.

[0075] As methods for re-registering (updating) the person dictionary, for example, there are the following three methods.

[0076] The first method is to have the user re-enter the input information via the input device 1.

[0077] The second is the method by which the re-registration unit 31 requests the generation unit 33 to generate input information. This is a method in which the generation unit 33 superimposes the environmental information of the current usage environment on the input information indicating the user's voice stored in the input information storage unit 32, and re-registers the person dictionary with the superimposed input information. In the case of this method, there is no need for the user's effort, and the re-registration of the person dictionary can be automated. As methods of superimposition, there are methods such as synthesizing noise on the user's voice so as to achieve the current SNR, or reproducing reverberation, etc. with an acoustic simulator.

[0078] For example, when the environmental information includes voice recognition environmental information obtained from the input information input by the microphone, the second method is used. That is, when the environmental change score is greater than the threshold value, the generation unit 33 generates updated data by superimposing data including the voice recognition environmental information on the accumulated input information. Then, the re-registration unit performs control to re-register the person dictionary based on the updated data.

[0079] Specifically, as the voice recognition environmental information, there is, for example, a noise type indicating the type of noise. When the environmental change score is greater than the threshold value, the generation unit 33 generates updated data by superimposing data including noise of the noise type corresponding to the current environment on the accumulated input information.

[0080] Also, for example, when in addition to the noise type, the SNR is also obtained from the input information as the voice recognition environmental information, when the environmental change score is greater than the threshold value, the generation unit 33 generates updated data by superimposing data including noise of the noise type corresponding to the current environment on the accumulated input information at a ratio corresponding to the SNR.

[0081] Also, for example, when the characteristics of the microphone are obtained from the input information as the voice recognition environmental information, when the environmental change score is greater than the threshold value, the generation unit 33 generates updated data by applying a filter that converts the accumulated input information into the microphone characteristics corresponding to the current environment.

[0082] For example, when spatial information of sound is obtained from input information as voice recognition environment information, if the environment change score is greater than a threshold value, the generation unit 33 generates updated data by superimposing data including spatial information corresponding to the current environment on the accumulated input information.

[0083] Note that not limited to voice recognition, even when only a camera image is obtained as environment information in the case of face recognition, the generation unit 33 can generate updated data for improving the performance of face recognition, for example, by clearing the blur of the face part or correcting the lighting conditions.

[0084] The third is a method in which when the re-registration unit 31 determines that improvement of the person feature model for extracting the feature amount of a person is necessary, re-learning is performed by changing the person feature model in the person feature model storage unit 4 or adding learning data used for re-learning the person feature model.

[0085] Next, a method for determining success / failure of registration at the time of re-registration of the person dictionary will be described. There are, for example, the following two methods for determining success / failure of registration at the time of re-registration of the person dictionary.

[0086] (i) The specifying unit 7 specifies a person using each of a plurality of input information and the updated person dictionary, and the monitoring unit 23 monitors a specific accuracy. The re-registration unit 31 determines that the update is successful if the accuracy is equal to or higher than a certain threshold value.

[0087] (ii) The environment information acquisition unit 11 calculates a distribution of feature amounts for each user (see FIG. 10) using, for example, a Gaussian distribution or a von Mises distribution from a plurality of input information. FIG. 10 is a diagram showing an example of an individual distribution model of the second embodiment. The re-registration unit 31 determines that the update of the person dictionary is successful if the feature amount indicated by the updated person dictionary is within the confidence interval of the individual distribution model in FIG. 10.

[0088] With reference to FIG. 11, the functional configuration of the environment information acquisition unit 11 for determining success / failure of re-registration of the person dictionary by the determination method (ii) using a distribution will be described.

[0089] FIG. 11 is a diagram showing an example of the functional configuration of the environmental information acquisition unit 11 of the second embodiment. The environmental information acquisition unit 11 of the second embodiment includes an individual distribution model calculation unit 34 (additional input information generation unit 35, embedded vector generation unit 36, and calculation unit 37).

[0090] The additional input information generation unit 35 increases (inflates) the number of input information by generating additional input information from the input information in the input information storage unit 32 accumulated so far. For example, when the person identification function of the specific unit 7 is a function using a speaker identification technique, the additional input information generation unit 35 generates additional input information by superimposing multiple types of noise on the input information while changing the SNR. Also, for example, the newly generated additional input information may be included by superimposing data including voice recognition environmental information on a plurality of input information used for determination.

[0091] Also, for example, when the speaker identification technique is a method that does not depend on keywords, the additional input information generation unit 35 divides the voice used for generating speaker information into several tens of milliseconds and randomly shuffles the divided voices to generate additional input information. That is, the newly generated additional input information may be included by changing the time series order of the data included in the input information in a plurality of input information used for determination.

[0092] FIG. 12 is a diagram showing an example of the generation of additional input information in the second embodiment. As shown in FIG. 12, the additional input information generation unit 35 generates a plurality of new additional input information by changing the time series order of the data included in the input information.

[0093] Returning to FIG. 11, the embedded vector generation unit 36 extracts a plurality of feature amounts by generating an embedded vector indicating a feature amount from each of the plurality of input information including the additional input information.

[0094] The calculation unit 37 calculates (estimates) a personal distribution model based on a multi-dimensional Gaussian distribution or the like from the extracted plurality of feature amounts, and stores it in the environment information storage unit 41 as one piece of environment information.

[0095] When the personal distribution model in FIG. 10 described above is calculated, hereinafter, the personal distribution model can be used for the processing of the monitoring unit 23 and the re-registration unit 31. Specifically, every time input information is input by the user, it is determined whether the feature amount extracted from the input information is within the confidence interval of the personal distribution model, whereby the success / failure of person identification and the success / failure of re-registration of the person dictionary can be determined.

[0096] FIG. 13 is a flowchart showing Example 1 of the re-registration flow of the person dictionary according to the second embodiment. First, the re-registration unit 31 determines a method for updating the person dictionary (re-registration method) (step S21).

[0097] When the environment change score is greater than the threshold value (that is, the difference is large), the re-registration unit 31 controls the re-registration process of the person dictionary based on the current environment information. Specifically, first, the generation unit 33 superimposes the current environment information on the input information of the user stored in the input information storage unit 32, for example (step S22). Then, the registration unit 5 recalculates the person dictionary in the person dictionary storage unit 6 based on the feature amount extracted by the extraction unit 3 from the input information suitable for the environment (step S25).

[0098] In addition, when the registration information at the time of person registration is insufficient (for example, when the total number of mora or the number of registered words (or registration time) is less than the threshold value), although the performance is good at the time of registration, it often happens that the performance deteriorates at a later date. Therefore, for example, when the registration information is insufficient, the re-registration unit 31 accepts the input of input information from the user via the input device 1, and controls the re-registration process of the person dictionary by supplementing the insufficient registration information (steps S23 and S25).

[0099] For example, when the environmental information includes the total number of moras corresponding to the feature amounts indicated in the person dictionary or the number of registered words corresponding to the feature amounts indicated in the person dictionary, the processes of steps S23 and S25 can be executed. That is, when the re-registration unit 31 re-registers the person dictionary, if the total number of moras is less than the mora number threshold or if the number of registered words is less than the registered word number threshold, control is performed to re-register the person dictionary based on the voice re-input by the microphone.

[0100] In other cases (when there is no environmental change and when there is no lack of registration information), the accuracy of person identification may be improved by updating (re-learning) the person feature model used for extraction.

[0101] For example, in the determination of step S21, as an example of an update method in other cases, an example of determining whether to update the person feature model using the above-described individual distribution model can be considered. In this determination example, for example, when the user is known in advance, the distribution of the user's input information is obtained, and if the feature amount obtained from the current input information is outside the confidence interval of the distribution, the process proceeds to step S24 to update the person feature model for feature extraction (step S24). Then, the registration unit 5 re-calculates the person dictionary in the person dictionary storage unit 6 based on the feature amount extracted using the updated person feature model (step S25).

[0102] Next, the monitoring unit 23 calculates the performance of the person identification function using the re-calculated person dictionary and the input information accumulated in the past. If the accuracy is equal to or higher than the threshold (performance equal to or higher than the standard), it is determined that the re-registration of the person dictionary is successful (step S26, Yes), and the re-registration process ends. On the other hand, if the monitoring unit 23 determines that the specific accuracy is below the threshold, it determines that the re-registration of the person dictionary fails (step S26, No), and the update process returns to step S21 to continue the re-calculation of the person dictionary by other update methods.

[0103] Note that the example of the re-registration flow of the person dictionary in FIG. 13 is merely an example, and another re-registration flow may be used.

[0104] FIG. 14 is a flowchart showing Example 2 of the re-registration flow of the person dictionary according to the second embodiment. For example, as shown in FIG. 14, when it is determined that re-registration is successful (Steps S33, S36, and S39), the person dictionary may be recalculated (updated) in order until re-registration is successful. In the example of FIG. 14, three recalculation patterns are tried in order, and when it is determined that re-registration is successful in any of Steps S33, S36, and S39, the update is completed, and when it is determined that re-registration fails in Step S39, the update fails.

[0105] According to the second embodiment, by simply making a user selection (or user estimation) at the start of use, for example, when identification fails, the person dictionary can be automatically updated according to the usage environment without relying on user input. As a result, each time identification fails, the trouble of the user inputting input information can be saved.

[0106] In addition to the method of overwriting the person dictionary, a method of adding newly generated feature amounts to the past person dictionary can also be considered for updating the person dictionary. Also, methods such as averaging using the feature amounts indicated by the past person dictionary and the newly generated feature amounts, or leaving both the feature amounts indicated by the past person dictionary and the newly generated feature amounts can be considered.

[0107] (Modification Example of the Second Embodiment) Next, a modification example of the second embodiment will be described. In the description of the modification example, descriptions similar to those of the second embodiment will be omitted, and differences from the second embodiment will be described. When the person identification function by the specifying unit 7 of the information processing apparatus 100-2 is used, for example, in a car, noise and the like change depending on whether the car is running, and the environmental information may frequently change greatly. In such a case, re-registration of the person dictionary may occur frequently if the method of the second embodiment is used as it is. In the second embodiment, a countermeasure method for such a case will be described.

[0108] In the modified example, when selecting a person dictionary used for person identification, a plurality of person dictionaries of the number of types of environments that can change frequently (or more than that) are selected. Then, the specifying unit 7 performs person identification using each of the plurality of person dictionaries, and when a person is specified by one or more person dictionaries, it is determined that the person has been specified.

[0109] Alternatively, when the same person is specified by more than half of the person dictionaries, the specifying unit 7 may determine that the person has been specified. Or, instead of performing person identification using a plurality of person dictionaries, environmental information for selecting an appropriate person dictionary is acquired at regular intervals, and at regular intervals, the person dictionary that matches the current environmental information is reselected, and a method of performing person identification may be used.

[0110] (Third Embodiment) Next, the third embodiment will be described. In the description of the third embodiment, the same description as that of the second embodiment will be omitted, and the parts different from the second embodiment will be described. In the third embodiment, the control when a plurality of person dictionaries are stored in the person dictionary storage unit 6 for one user will be described.

[0111] [Example of Functional Configuration] FIG. 15 is a diagram showing an example of the functional configuration of the information processing apparatus 100-3 according to the third embodiment. The information processing apparatus 100-3 according to the third embodiment includes an input device 1, a signal acquisition unit 2, an extraction unit 3, a person feature model storage unit 4, a registration unit 5, a person dictionary storage unit 6, a specifying unit 7, an environmental information acquisition unit 11, an analysis unit 21, a re-registration unit 31, an input information storage unit 32, a generation unit 33, an environmental information storage unit 41, and a selection unit 42.

[0112] In the third embodiment, a selection unit 42 is further added to the configuration of the information processing apparatus 100-2 of the second embodiment.

[0113] When the information processing device 100-3 controlled using the person identification function by the specific unit 7 frequently changes its usage environment, it is conceivable that one user registers a plurality of person dictionaries corresponding to the usage environment in the person dictionary storage unit 6. When the information processing device 100-3 is a portable notebook computer or the like, it is assumed that the usage environment frequently changes.

[0114] When there are a plurality of person dictionaries stored in the person dictionary storage unit 6 for one person, the selection unit 42 selects a person dictionary according to the environment information. Specifically, when selecting a person dictionary during the use of the information processing device 100-3, the selection unit 42 calculates an environment change score using each of the environment information stored at the time of registration of the plurality of person dictionaries and the current environment information. Then, the selection unit 42 can automatically select the person dictionary of the appropriate environment by selecting the person dictionary with the smallest environment change score. For example, the user can save the trouble of selecting a person dictionary by himself / herself by simply inputting the user's identification information such as the user name into the information processing device 100-3.

[0115] (Fourth Embodiment) Next, the fourth embodiment will be described. In the description of the fourth embodiment, the same description as that of the second embodiment will be omitted, and the parts different from the second embodiment will be described. When monitoring the accuracy of person identification, even if the environment information does not change, the accuracy may decrease due to changes over time. In the fourth embodiment, in such a case, a method of automatically updating the feature vector and the person dictionary according to changes over time will be described.

[0116] [Example of Functional Configuration] FIG. 16 is a diagram showing an example of the functional configuration of the information processing device 100-4 according to the fourth embodiment. The information processing device 100-4 according to the fourth embodiment includes an input device 1, a signal acquisition unit 2, an extraction unit 3, a person feature model storage unit 4, a registration unit 5, a person dictionary storage unit 6, a specific unit 7, an environment information acquisition unit 11, an analysis unit 21, a re-registration unit 31, an input information storage unit 32, a generation unit 33, an environment information storage unit 41, and a change-over-time processing unit 51-1.

[0117] In the fourth embodiment, a time-dependent change processing unit 51-1 is further added to the configuration of the information processing apparatus 100-2 of the second embodiment.

[0118] For example, the time-dependent change processing unit 51-1 processes the change over time of the feature amounts indicated by the person dictionary of a person by calculating the transition of the change in the feature amounts of the person from the input information stored in the input information storage unit 32. Then, the re-registration unit 31 performs control to re-register the person dictionary of the person based on the feature amounts after the change over time.

[0119] Also for example, the time-dependent change processing unit 51-1 leaves a record of updating the person dictionary (speaker vector) for each person, and analyzes the way the speaker vector changes due to the change over time, thereby automatically changing the speaker vector over time.

[0120] Specifically, in the fourth embodiment, when registering the person dictionary, the registration unit 5 stores, in the environment information storage unit 41, environment information including the registration date as the environment information of the person dictionary. Then, the time-dependent change processing unit 51-1 analyzes the way the speaker vector changes due to the change over time, for example, using a method of learning an estimation model that estimates the current speaker vector from past speaker vectors for each user using, for example, LSTM (Long short-term memory). For example, this estimation model is learned as an estimation model that inputs the most recent n past person dictionaries (speaker vectors) with the same environment information into the LSTM and outputs the (n + 1)-th person dictionary (speaker vector).

[0121] FIG. 17 is a diagram showing an example of the processing of the time-dependent change processing unit 51-1 of the fourth embodiment. In the example of FIG. 17, an example of the processing of the change over time of the speaker information (speaker vector) of person A is shown.

[0122] (Modification example of the fourth embodiment) Next, a modification example of the fourth embodiment will be described. In the description of the modification example, descriptions similar to those of the fourth embodiment will be omitted, and differences from the fourth embodiment will be described.

[0123] [Example of Functional Configuration] FIG. 16 is a diagram showing an example of the functional configuration of an information processing apparatus 100-5 according to a modification of the fourth embodiment. The information processing apparatus 100-5 according to the modification includes an input device 1, a signal acquisition unit 2, an extraction unit 3, a person feature model storage unit 4, a registration unit 5, a person dictionary storage unit 6, a specification unit 7, an environmental information acquisition unit 11, an analysis unit 21, a re-registration unit 31, an input information storage unit 32, a generation unit 33, an environmental information storage unit 41, and a time-dependent change processing unit 51-2.

[0124] In the modification, the time-dependent change processing unit 51-1 of the fourth embodiment is changed to the time-dependent change processing unit 51-2 added between the extraction unit 3 and the specification unit 7. In the modification, the time-dependent change processing unit 51-2 is different from the fourth embodiment in that the processing considering the influence of the time-dependent change of a person is performed on the feature amount compared with the person dictionary instead of the person dictionary (speaker vector).

[0125] Even with the configuration of the modification of the fourth embodiment, the same effects as those of the fourth embodiment can be obtained.

[0126] Finally, an example of the hardware configuration of the information processing apparatuses 100 (100-2, 100-3, 100-4) according to the first to fourth embodiments and the information processing apparatus 100-5 according to the modification will be described.

[0127] [Example of Hardware Configuration] FIG. 19 is a diagram showing an example of the hardware configuration of the information processing apparatuses 100 (100-2, 100-3, 100-4) according to the first to fourth embodiments and the information processing apparatus 100-5 according to the modification. The information processing apparatuses 100 (100-2 to 100-5) include a processor 91, a main memory device 92, an auxiliary storage device 93, a display device 94, an input device 95, and a communication device 96. The processor 91, the main memory device 92, the auxiliary storage device 93, the display device 94, the input device 95, and the communication device 96 are connected via a bus 97.

[0128] Note that the information processing apparatus 100 (100-2 to 100-5) may not be provided with some of the above configurations. For example, when the information processing apparatus 100 (100-2 to 100-5) can use the input function and display function of an external device, the information processing apparatus 100 (100-2 to 100-5) may not be provided with the display device 94 and the input device 95.

[0129] The processor 91 executes the program read from the auxiliary storage device 93 into the main storage device 92. The main storage device 92 is a memory such as a ROM and a RAM. The auxiliary storage device 93 is an HDD (Hard Disk Drive), a memory card, or the like.

[0130] The display device 94 is, for example, a liquid crystal display or the like. The input device 95 corresponds to one or more of the above-described input devices 1. The communication device 96 is an interface for communicating with other devices.

[0131] Also, for example, the program executed by the information processing apparatus 100 (100-2 to 100-5) may be stored on a computer connected to a network such as the Internet and provided by being downloaded via the network.

[0132] Also, for example, the program executed by the information processing apparatus 100 (100-2 to 100-5) may be provided via a network such as the Internet without being downloaded. Specifically, it may be configured to execute information processing by a so-called ASP (Application Service Provider) type service in which, from a server computer, only the execution instruction and result acquisition are performed without transferring the program, and the processing function is realized.

[0133] Also, for example, the program of the information processing apparatus 100 (100-2 to 100-5) may be configured to be provided by being pre-embedded in a ROM or the like.

[0134] The program executed by the information processing apparatus 100 (100-2 to 100-5) has a module configuration including functions that can also be realized by the program among the above-described functional configurations. Each of these functions, as actual hardware, is such that the processor 91 reads the program from the storage medium and executes it, whereby each of the above functional blocks is loaded onto the main storage device 92. That is, each of the above functional blocks is generated on the main storage device 92.

[0135] Note that part or all of each of the functions described above may be realized by hardware such as an IC instead of by software.

[0136] Also, each function may be realized using a plurality of processors 91. In that case, each processor 91 may realize one of the functions, or may realize two or more of the functions.

[0137] Although some embodiments of the present invention have been described, these embodiments are presented as examples and are not intended to limit the scope of the invention. These novel embodiments can be implemented in various other forms, and various omissions, replacements, and changes can be made without departing from the gist of the invention. These embodiments and their modifications are included in the scope and gist of the invention, and are included in the invention described in the claims and the equivalent scope thereof.

Description of Reference Numerals

[0138] 1 Input device 2 Signal acquisition unit 3 Extraction unit 4 Person feature model storage unit 5 Registration unit 6 Person dictionary storage unit 7 Identification unit 11 Environment information acquisition unit 12 Acquisition unit 13 Inference unit 21 Analysis unit 31 Re-registration unit 32 Input information storage unit 33 Generation unit 34 Personal Distribution Model Calculation Unit 35 Additional Input Information Generation Unit 36 Embedded Vector Generation Unit 37 Calculation Unit 41 Environment Information Storage Unit 42 Selection Unit 51-1, 51-2 Time-Varying Processing Unit 91 Processor 92 Main Memory Device 93 Auxiliary Memory Device 94 Display Device 95 Input Device 96 Communication Device 97 Bus 100 Information Processing Device 111 Microphone Information Acquisition Unit 112 SNR Acquisition Unit 113 Noise Type Acquisition Unit 114 Spatial Information Acquisition Unit 115 Captured Image Acquisition Unit 116 GPS Information Acquisition Unit 117 Total Number of Moles Acquisition Unit 118 Number of Registered Words Acquisition Unit 119 Age Acquisition Unit 120 Gender Acquisition Unit

Claims

1. One or more input devices that acquire input information, An extraction unit that extracts a feature amount indicating a person's characteristics from the input information using a person feature model, An identification unit that identifies the person by comparing the feature amount with the feature amount indicated by a person dictionary, An environmental information acquisition unit that acquires environmental information from the input information, An analysis unit that analyzes the change in the environmental information and the identification accuracy of the person by the identification unit, A re-registration unit that performs control to re-register the person dictionary when there is a change in the environmental information or when the identification accuracy is less than an accuracy threshold, An information processing apparatus comprising the above.

2. When the environmental information acquisition unit determines that the environmental information is time-series data and some data indicating the environmental information cannot be acquired, the environmental information acquisition unit interpolates the some data from the acquired part of the environmental information to estimate the environmental information. The information processing apparatus according to Claim 1.

3. The analysis unit calculates an environmental change score indicating the magnitude of the change in the environmental information, When the environmental change score is greater than a change threshold, the re-registration unit performs control to re-register the person dictionary. The information processing apparatus according to Claim 1 or 2.

4. The one or more input devices include a microphone, A person dictionary storage unit that stores the person dictionary, An environmental information storage unit that stores the environmental information, An input information storage unit that accumulates the input information, and further includes: The environmental information includes voice recognition environmental information obtained from the input information input by the microphone, When the environmental change score is greater than the change threshold, a generation unit that generates update data by superimposing data including the voice recognition environmental information on the accumulated input information, The re-registration unit executes a first re-registration process for re-registering the person dictionary based on the update data. The information processing apparatus according to Claim 3.

5. The environmental information stored in the environmental information storage unit further includes the total number of moras corresponding to the feature amount indicated by the person dictionary or the number of registered words corresponding to the feature amount indicated by the person dictionary, When re-registering the person dictionary, the re-registration unit executes a second re-registration process for re-registering the person dictionary based on the voice re-input by the microphone when the total number of moras is less than a mora number threshold or the number of registered words is less than a registered word number threshold. The information processing apparatus according to claim 4.

6. When the re-registration unit re-registers the person dictionary, if the environment change score is less than the change threshold, and the total number of moras is greater than or equal to the mora number threshold, and the number of registered words is greater than or equal to the registered word number threshold, the re-registration unit executes a third re-registration process of updating the person feature model by re-learning the person feature model. The information processing apparatus according to claim 5.

7. After the re-registration of the person dictionary is performed, if the specific accuracy is greater than the accuracy threshold, or if the feature amount indicated by the re-registered person dictionary falls within the confidence interval of the distribution of the feature amounts of the person calculated using a plurality of the input information, the re-registration unit ends the re-registration process. The information processing apparatus according to claim 6.

8. The plurality of input information includes additional input information newly generated by superimposing data including the voice recognition environment information on the input information. The information processing apparatus according to claim 7.

9. The plurality of input information includes additional input information newly generated by changing the time-series order of the data included in the input information. The information processing apparatus according to claim 7.

10. When a plurality of the person dictionaries are stored in the person dictionary storage unit for one person, a selection unit that selects the person dictionary according to the environment information. The information processing apparatus according to claim 4, further comprising the selection unit.

11. The information processing apparatus further comprises a time-varying change processing unit that processes the time-varying change of the feature amount indicated by the person dictionary of the person by calculating the transition of the change of the feature amount of the person from the input information stored in the input information storage unit. The re-registration unit performs control to re-register the person dictionary of the person based on the feature amount after the time-varying change. The information processing apparatus according to claim 4.

12. The environment information further includes the frequency characteristics of the microphone. The information processing apparatus according to claim 4.

13. The one or more input devices include a receiver that receives input of GPS (Global Positioning System) information. The environment information further includes the position information of the information processing apparatus based on the GPS information. When some data indicating the environmental information cannot be acquired, the environmental information acquisition unit estimates the position information indicated by the some data from the position information and the moving speed of the information processing apparatus, and estimates the environmental information indicated by the some data that could not be acquired from the estimated position information. The information processing apparatus according to claim 1.

14. The one or more input devices include a camera that acquires a photographed image. The environmental information includes at least one of a photographed image including the person and iris information of the person. The information processing apparatus according to claim 1 or 2.

15. The one or more input devices include a sensor that acquires a fingerprint of the person. The environmental information includes information indicating the fingerprint of the person. The information processing apparatus according to claim 1 or 2.

16. The one or more input devices include a device that receives an operation input indicating at least one of the gender of the person and the age of the person. The environmental information includes at least one of the gender of the person and the age of the person. The information processing apparatus according to claim 1 or 2.

17. A step in which an information processing apparatus acquires input information; A step in which the information processing apparatus extracts a feature amount indicating a feature of a person from the input information using a person feature model; A step in which the information processing apparatus identifies the person by comparing the feature amount with the feature amount indicated by a person dictionary; A step in which the information processing apparatus acquires environmental information from the input information; A step in which the information processing apparatus analyzes the change in the environmental information and the identification accuracy of the person by the identifying step; A step in which the information processing apparatus performs control to re-register the person dictionary when there is a change in the environmental information or when the identification accuracy is less than an accuracy threshold value; An information processing method including the above steps.

18. A computer including one or more input devices that acquire input information, An extraction unit that extracts a feature amount indicating a feature of a person from the input information using a person feature model, An identification unit that identifies the person by comparing the feature amount with the feature amount indicated by a person dictionary, An environmental information acquisition unit that acquires environmental information from the input information, An analysis unit that analyzes the change in the environmental information and the identification accuracy of the person by the identification unit When there is a change in the environmental information or when the specific accuracy is less than the accuracy threshold, a re-registration unit that performs control to re-register the person dictionary. A program for causing it to function as such.

Images