Method and device for data recognition

Abstract

An apparatus and a method for recognizing data are provided. An apparatus for recognizing data according to one embodiment calculates a matching score from input feature data, common component data and distribution component data of a plurality of registered feature data corresponding to a registered user, and can recognize the input data based on the calculated matching score.

Description

Technology Field

[0001] Below, a technology for recognizing data is provided. Background Technology

[0002] Recently, active research has been conducted to apply efficient human pattern recognition methods to actual computers in order to solve the problem of classifying input patterns into specific groups. One such area of ​​research involves artificial neural networks, which model the characteristics of human biological neurons using mathematical expressions. To solve the problem of classifying input patterns into specific groups, artificial neural networks utilize algorithms that mimic the human ability to learn. Through these algorithms, artificial neural networks can generate mappings between input and output patterns, and this ability to generate mappings is referred to as the learning ability of the artificial neural network. Furthermore, artificial neural networks possess generalization capabilities that allow them to generate relatively correct outputs for input patterns that were not used in the learning process, based on learned results. means of solving the problem

[0003] According to one embodiment, a data recognition method may include the steps of: receiving input data and extracting input feature data from the input data; calculating a matching score from the input feature data, common component data and distribution component data of a plurality of registered feature data corresponding to a registered user; and recognizing the input data based on the matching score.

[0004] The above common component data includes an average feature vector having the average value of the elements of the same index among the elements of the plurality of registered feature vectors for each index, and the above distribution component data may include a variance feature vector having the variance value of the elements of the same index among the plurality of registered feature vectors for each index.

[0005] The step of calculating the matching score may include the step of calculating a partial score as the matching score from at least some elements of the input feature data, at least some elements of the common component data, and at least some elements of the distribution component data.

[0006] The step of calculating the partial score may include: a step of calculating an entry score at the target index based on an element corresponding to the target index in the input feature data, an element corresponding to the target index in the common component data, and an element corresponding to the target index in the distribution component data; and a step of calculating the partial score by accumulating the entry score at the target index to the entry score at at least one other index.

[0007] The above input feature data, the above common component data, and the above distribution component data are composed of dimension vectors having the same dimension, and the step of recognizing the input data may include the step of recognizing the input data based on partial scores calculated for some indices of the dimension before scores for all indices of the dimension are calculated.

[0008] The step of recognizing the input data may include: a step of determining whether the recognition based on the input data is successful based on the result of comparing the partial score with a partial success threshold level; and a step of determining whether the recognition based on the input data is unsuccessful based on the result of comparing the partial score with a partial failure threshold level.

[0009] The step of recognizing the input data may include a step of allowing access to the device in response when it is determined that the user indicated by the input data matches the registered user.

[0010] The step of calculating the matching score comprises: loading a representative mean vector and a representative variance vector corresponding to each of the plurality of groups from a registration database in which a plurality of registration vector data is clustered into a plurality of groups and stored; identifying a group among the plurality of groups that the input feature data matches based on the input feature data, the representative mean vector, and the representative variance vector; and determining whether the input data matches the registration vector data belonging to the identified group, wherein each of the plurality of registration vector data may include common component data and distribution component data.

[0011] The step of extracting input feature data includes: receiving an input image from a user as input data; and extracting input feature data from the input image, and the step of recognizing the input data may include determining whether an object appearing in the input image is the registered user based on the matching score.

[0012] The step of extracting input feature data includes: receiving a voice signal from a user as input data; and extracting input feature data from the voice signal, and the step of recognizing the input data may include determining whether the speaker of the voice signal is the registered user based on the matching score.

[0013] A data recognition device according to one embodiment may include an input unit that receives input data; and a processor that extracts input feature data from the input data, calculates a matching score from the input feature data, common component data and distribution component data of a plurality of registered feature data corresponding to a registered user, and recognizes the input data based on the matching score.

[0014] The processor may include a neural processor configured to extract the input feature data; and a main processor configured to calculate the matching score from the input feature data, the common component data, and the distribution component data transmitted from the neural processor.

[0015] The processor may include a digital signal processor (DSP) that detects target data from the input data; and a main processor that extracts the input feature data from the target data. Brief explanation of the drawing

[0016] FIG. 1 is a diagram illustrating a feature extraction model according to one embodiment. FIG. 2 is a flowchart illustrating a data recognition method according to one embodiment. FIG. 3 is a diagram illustrating a data recognition process according to one embodiment. FIG. 4 is a diagram illustrating examples of common component data and distributed component data according to one embodiment. FIGS. 5 and 6 are drawings illustrating the calculation of a matching score and data recognition according to one embodiment. FIG. 7 is a diagram illustrating data recognition according to another embodiment. FIGS. 8a to 8c are drawings illustrating the configuration of a processor according to one embodiment. FIGS. 9 and FIGS. 10 are drawings illustrating the configuration of a data recognition device according to one embodiment. Specific details for implementing the invention

[0017] Hereinafter, embodiments will be described in detail with reference to the attached drawings. However, the scope of the patent application is not limited or restricted by these embodiments. Identical reference numerals in each drawing indicate identical components.

[0018] Various modifications may be made to the embodiments described below. The embodiments described below are not intended to limit the forms of practice and should be understood to include all modifications, equivalents, and substitutions thereof.

[0019] The terms used in the embodiments are used merely to describe specific embodiments and are not intended to limit the embodiments. Singular expressions include plural expressions unless the context clearly indicates otherwise. In this specification, terms such as “comprising” or “having” are intended to indicate the presence of the features, numbers, steps, actions, components, parts, or combinations thereof described in the specification, and should be understood as not precluding the existence or addition of one or more other features, numbers, steps, actions, components, parts, or combinations thereof.

[0020] Unless otherwise defined, all terms used herein, including technical or scientific terms, have the same meaning as generally understood by those skilled in the art to which the embodiments pertain. Terms such as those defined in commonly used dictionaries should be interpreted as having a meaning consistent with their meaning in the context of the relevant technology, and should not be interpreted in an ideal or overly formal sense unless explicitly defined in this application.

[0021] In addition, when describing with reference to the attached drawings, identical components are assigned the same reference numeral regardless of drawing symbols, and redundant descriptions thereof are omitted. When describing the embodiments, if it is determined that a detailed description of related prior art could unnecessarily obscure the essence of the embodiments, such detailed description is omitted.

[0022] FIG. 1 is a diagram illustrating a feature extraction model according to one embodiment.

[0023] A data recognition device according to one embodiment can recognize a user using feature data extracted from an input image. For example, the data recognition device can extract feature data from an input image based on a feature extraction model. The feature data is data in which the image is abstracted, and can be represented, for example, in the form of a vector. The feature extraction model is a model designed to extract feature data from an image, and may, for example, be a machine learning structure and may include a neural network (100).

[0024] A neural network (100) may be an example of a deep neural network (DNN). A DNN may include a fully connected network, a deep convolutional network, and a recurrent neural network. The neural network (100) can perform object classification, object recognition, speech recognition, and image recognition by mapping input data and output data that are in a non-linear relationship to each other based on deep learning. Deep learning is a machine learning technique for solving problems such as image or speech recognition from big data sets, and can map input data and output data to each other through supervised or unsupervised learning.

[0025] In this specification, recognition may include verification of data and / or identification of data. Verification may represent an operation of determining whether input data is true or false. For example, verification may represent a determination operation of determining whether an object indicated by any input image (e.g., a human face) is identical to an object indicated by a reference image. A data recognition device according to one embodiment may verify whether data extracted and obtained from an input image is identical to registered data already registered within the device, and in response to the verification that the two data are identical, determine that the verification of the user corresponding to the input image has been successful. However, this is not limited thereto, and if multiple registered data are stored within the data recognition device, the data recognition device may sequentially verify the data extracted and obtained from the input image for each of the multiple registered data.

[0026] Identification can represent a classification operation that determines the label indicated by the input data among multiple labels, and, for example, each label can indicate a class (e.g., the identity of registered users). For example, through the identification operation, it can be indicated whether the user included in the input data is male or female.

[0027] Referring to FIG. 1, the neural network (100) includes an input layer (110), a hidden layer (120), and an output layer (130). The input layer (110), the hidden layer (120), and the output layer (130) each include a plurality of artificial nodes.

[0028] In FIG. 1, for convenience of explanation, the hidden layer (120) is depicted as including three layers, but the hidden layer (120) may include various numbers of layers. Also, in FIG. 1, the neural network (100) is depicted as including a separate input layer for receiving input data, but input data may be directly input to the hidden layer (120). In the neural network (100), artificial nodes of layers excluding the output layer (130) may be connected to artificial nodes of the next layer through links for transmitting output signals. The number of links may correspond to the number of artificial nodes included in the next layer.

[0029] Each artificial node included in the hidden layer (120) may receive the output of an activation function regarding the weighted inputs of the artificial nodes included in the previous layer. The weighted input is the input of the artificial nodes included in the previous layer multiplied by a weight. The weight may be referred to as a parameter of the neural network (100). The activation function may include sigmoid, hyperbolic tangent (tanh), and rectified linear unit (ReLU), and non-linearity may be formed in the neural network (100) by the activation function. Each artificial node included in the output layer (130) may receive the weighted inputs of the artificial nodes included in the previous layer.

[0030] According to one embodiment, when input data is given, the neural network (100) calculates a function value according to the number of classes to be identified in the output layer (130) after passing through the hidden layer (120), and can identify the input data as the class having the largest value among them. The neural network (100) can identify the input data, but is not limited thereto, and the neural network (100) may also verify the input data against reference data (e.g., registered data). The following description regarding the recognition process is mainly described as a verification process, but it may also be applied to the identification process as long as it is not contrary to the nature.

[0031] If the width and depth of the neural network (100) are sufficiently large, it can have enough capacity to implement any function. If the neural network (100) learns a sufficient amount of training data through an appropriate training process, it can achieve optimal recognition performance.

[0032] In the above, a neural network (100) was described as an example of a feature extraction model, but the feature extraction model is not limited to a neural network (100). Below, verification operations using feature data extracted using a feature extraction model are mainly described.

[0033] FIG. 2 is a flowchart illustrating a data recognition method according to one embodiment.

[0034] First, in step (210), the data recognition device receives input data and can extract input feature data from the input data. According to one embodiment, the data recognition device can extract input feature data from the input data based on a feature extraction model. The feature extraction model may be, for example, a trained neural network described in FIG. 1. If the input feature data is data in the form of a vector, it may be referred to as an input feature vector. The input data may be an image and / or voice signal input by a user.

[0035] And in step (220), the data recognition device can calculate a matching score from input feature data, common component data and distribution component data of multiple registered feature data corresponding to a registered user. The data recognition device can load common component data and distribution component data corresponding to any registered user from a registration database stored in memory. The data recognition device can calculate a matching score from the loaded common component data, distribution component data, and input feature data extracted in step (210).

[0036] In this specification, common component data may be data indicating a common component of multiple registered feature data corresponding to the same registered user. For example, common component data may represent information regarding a common component that remains constant in various environments from the face image and / or voice signal of the registered user. Distribution component data may be data indicating a distribution component of multiple registered feature data corresponding to the same registered user. Distribution component data may represent information regarding a distribution that changes uniquely depending on the characteristics of the registered user, etc. For reference, common component data may be an average feature vector, and distribution component data may be a variance feature vector; the average feature vector and the variance feature vector are explained in Figure 4 below along with the calculation of the matching score.

[0037] Next, in step (230), the data recognition device can recognize input data based on a matching score. For example, the data recognition device can calculate similarity as the matching score and, in response, determine that the object indicated by the input data matches the registered user when the similarity exceeds a threshold similarity level. In response, the data recognition device can determine that the object indicated by the input data does not match the registered user when the similarity is below the threshold similarity level. However, it is not limited to this, and the data recognition device can calculate a value representing the degree of difference (difference level) between two data (e.g., two vectors), such as a Euclidean distance value, as the matching score. In this case, the data recognition device can determine that the object indicated by the input data matches the registered user when the calculated distance value is below the threshold distance level. Conversely, in response, the data recognition device can determine that the object indicated by the input data does not match the registered user when the calculated distance value exceeds the threshold distance level.

[0038] A data recognition device according to one embodiment calculates a matching score for input feature data by considering not only common component data but also distribution component data, thereby achieving a more accurate recognition rate for registered users than when only common components are considered. Furthermore, unlike a method of calculating cosine similarity for two vectors, the data recognition device may calculate a partial matching score (hereinafter referred to as a partial score) for some elements of feature vectors. By recognizing users based on the partial score, the data recognition device can generate recognition results for input users more quickly and without delay.

[0039] FIG. 3 is a diagram illustrating a data recognition process according to one embodiment.

[0040] First, the data recognition device can configure a registration database based on registration data (391) obtained from a user. In this specification, the registration data (391) is data obtained from a user who wishes to register, and may be, for example, data representing an image of a part of the user's body (e.g., face) and / or the voice of the user who wishes to register.

[0041] According to one embodiment, the data recognition device may generate registration vector data (392) from registration data (391) and store it in a registration database. The registration vector data (392) may include an average feature vector (381) and a variance feature vector (382).

[0042] The data recognition device may acquire multiple registration feature vectors to calculate an average feature vector (381) and a variance feature vector (382). For example, the data recognition device may acquire multiple registration data (391) to generate multiple registration feature vectors. During the registration process, the data recognition device may require the user to input multiple registration data (391) or acquire multiple registration data (391) from the user. For example, the data recognition device may acquire multiple voice signals for registration from the same user by requesting the user to speak multiple voices for registration. As another example, the data recognition device may acquire multiple images for registration by photographing parts of the user's body multiple times. The data recognition device may extract a registration feature vector based on a neural network (350) from each of the multiple registration data (391) acquired according to the above description.

[0043] The data recognition device may include an average feature vector (381) and a variance feature vector (382) from a plurality of registered feature vectors. As will be explained in detail in FIG. 4 below, the average feature vector (381) may represent the average value of each element of the plurality of registered feature vectors, and the variance feature vector (382) may represent the variance value of each element. Accordingly, the registered vector data (392) including the average feature vector (381) and the variance feature vector (382) may represent the distribution tendency of the registered feature vectors.

[0044] Here, the number of multiple registered feature vectors used to generate the average feature vector (381) and the variance feature vector (382) may vary depending on the design. For example, the data recognition device is not required to generate the same number of registered feature vectors for every registration, and may adjust the number of registered data (391) obtained from the user during the registration process as needed. Additionally, for convenience of explanation, FIG. 3 describes generating the average feature vector (381) and the variance feature vector (382) from a series of registered data (391) obtained during a single registration process, but this is not limited thereto. If the registered data (391) indicates the same user as the existing registered data (391), even if the registered data (391) is obtained at a different timing than the existing registered data (391), the existing registered data (391) can be used together with the existing registered data (391) to generate the average feature vector (381) and the variance feature vector (382).

[0045] After the registration database is configured, the data recognition device can obtain input data for recognition from the user in step (311). The data recognition device can extract an input feature vector (312) from the input data based on a neural network (350).

[0046] And in step (320), the data recognition device can calculate a matching score from the input feature vector (312) and the registration vector data (392). The matching score may be a score indicating the level to which an object indicated by the input data matches a registered user. Although the matching score is described herein as primarily being a similarity, it is not limited thereto, and the matching score may also be a distance value to improve computational speed. The calculation of the matching score is described in detail in FIG. 4 below.

[0047] Next, in step (330), the data recognition device can determine whether the matching score meets the condition. Based on the result of comparing the matching score to a threshold value, the data recognition device can determine whether the recognition of the input data is successful or failed. In response to the case where the recognition of the input data is successful, the data recognition device may accept access from the user who entered the input data. Conversely, in response to the case where the recognition of the input data fails, the data recognition device may reject access from the user who entered the input data. At this time, if the matching score calculated from the registration vector data (392) and input feature vector (312) corresponding to any registered user does not meet the condition, the data recognition device may attempt to verify another registered user. The data recognition device may calculate the matching score of step (320) from the registration vector data (392) and input feature vector (312) corresponding to each of the multiple registered users stored in the registration database, and determine whether the corresponding matching score meets the condition of step (330). If all matching scores calculated for all registered users do not meet the conditions of step (330), the data recognition device may determine that recognition has failed and restrict the user's access.

[0048] Here, the neural network (350) applied to the input data and the neural network (350) applied to the registration data (391) may be trained to have the same structure and the same parameters.

[0049] FIG. 4 is a diagram illustrating examples of common component data and distributed component data according to one embodiment.

[0050] First, to produce registered vector data (430), the data recognition device may extract registered feature vectors (411, 412, 413) from the registered data. FIG. 4 describes an example in which the data recognition device extracts three registered feature vectors (411, 412, 413) from each of the three registered data. The registered feature vectors (411, 412, 413) may represent an arbitrary distribution (410) for any reference vector. For reference, in FIG. 4, the reference vector is shown as the input feature vector B, but is not limited thereto.

[0051] The data recognition device can normalize the extracted registered feature vectors (411, 412, 413) to generate normalized vectors (421, 422, 423). For example, the data recognition device can normalize each of the registered feature vectors (411, 412, 413) by dividing it by the absolute magnitude of the corresponding vector. The vector size of the normalized feature vectors (421, 422, 423) may be limited to 1. However, the vector size of the normalized feature vectors (421, 422, 423) is not limited to this, and depending on the design, they may be configured to have different sizes by dividing each of the registered feature vectors (411, 412, 413) by the absolute magnitude of the corresponding vector and then multiplying it by an arbitrary constant.

[0052] At this time, a set A of normalized vectors (421, 422, 423) regarding the same object (e.g., the same user) can be represented as in Equation 1 below, and normalized feature vectors (421, 422, 423) constituting set A can be represented as in Equation 2 below.

[0053]

[0054]

[0055] In the above-described mathematical formula 1, m is the number of normalized feature vectors (421, 422, 423) and can be an integer greater than or equal to 1. In the example illustrated in FIG. 4, m can be 3. n can represent the dimension of the normalized feature vector and can be an integer greater than or equal to 1. In the above-described mathematical formula 2 can represent the j-th normalized feature vector, and j can be an integer between 1 and m. represents the value of the i-th entry of the j-th normalized feature vector, and i is an index referring to a specific element of the feature vector and can be an integer from 1 to n. For reference, the above-described mathematical formulas 1 and 2 explain examples where the feature vector is a normalized feature vector, but they are not limited to this and may also be applied to unnormalized feature vectors.

[0056] According to one embodiment, the data recognition device can generate registration vector data from normalized feature vectors (421, 422, 423). The registration vector data may include common component data and distribution component data as described above.

[0057] Common component data is data indicating the common components of multiple registered feature data corresponding to the same object, and for example, for each index, it may include an average feature vector having the average value of the elements of the same index among the elements of multiple registered feature vectors. The average feature vector can be represented as shown in Equation 3 below. Each element of the average feature vector can be represented as shown in Equation 4 below.

[0058]

[0059]

[0060] In the aforementioned mathematical formula 3 can represent the average feature vector of normalized feature vectors (421, 422, 423) constituting a feature distribution (410) regarding the same object. Average feature vector Each element of can be the average value of elements at the same index in the normalized feature vectors (421, 422, 423) as described above. In Equation 4, the average feature vector The i-th element of It can be indicated that is the mean value of the element values ​​corresponding to the i-th index of the normalized feature vectors (421, 422, 423). Mean() can represent a function that calculates the mean.

[0061] Distribution component data is data indicating the distribution components of multiple registered feature data corresponding to the same object. For example, for each index, it may include a variance feature vector having the variance values ​​of the elements of the same index among multiple registered feature vectors as elements. The variance feature vector can be represented as shown in Equation 5 below. Each element of the variance feature vector can be represented as shown in Equation 6 below.

[0062]

[0063]

[0064] In the aforementioned mathematical formula 5 can represent the variance feature vector of normalized feature vectors (421, 422, 423) constituting the feature distribution (410) regarding the same object. Variance feature vector Each element of can be the variance value of elements at the same index in the normalized feature vectors (421, 422, 423) as described above. However, this is not limited thereto, and the data recognition device may extract feature vectors from various data originating from the same speaker or different speakers, pre-calculate the variance value of each element of the extracted feature vectors to generate a variance feature vector, and the same variance feature vector thus generated can be used commonly for the same speaker or different speakers. In Equation 6, the variance feature vector The i-th element of It can be indicated that is the variance value of the elements corresponding to the i-th index of the normalized feature vectors (421, 422, 423). Var() can represent a function that calculates the variance.

[0065] Average feature vector calculated according to the aforementioned mathematical formulas 3 and 5 and dispersion feature vectors The registration vector data (430) containing [the above] is information representing the set A and feature distribution (410) described above. In FIG. 4, the registration vector data (430) is conceptually depicted as a single vector to explain the matching score, but this is only for the sake of understanding; in reality, it is an average feature vector and dispersion feature vectors It can be composed of.

[0066] A data recognition device according to one embodiment uses the above-described average feature vector and dispersion feature vectors For this, a matching score can be calculated using the input feature vector B. The input feature vector B can be expressed as shown in Equation 7 below.

[0067]

[0068] The data recognition device is an average feature vector , dispersion feature vector Similarity can be calculated as a matching score from , and input feature data B, such as in the following mathematical formulas 8 to 13.

[0069]

[0070]

[0071]

[0072]

[0073]

[0074]

[0075] Dist(A,B) in the above-mentioned mathematical formula 8 2 It can represent a function that calculates the distance value between the input feature vector B and the registration vector data (520) corresponding to the set A of registration feature vectors. Is It can represent the result of transposing. is a variance feature vector It can represent the inverse of. In the aforementioned mathematical equation 8, the term expressed as a matrix operation is the average feature vector The i-th element of and the i-th element of the input feature vector B The square of the difference between them is the variance feature vector The i-th element of As the sum of the values ​​divided by It can be expressed as follows. For example, it can be expressed as in Equation 9. For reference, the data recognition device may use the similarity of Equation 8 described above as a matching score, but is not limited thereto, and the distance value Dist(A,B) 2 It can also be used as a matching score. In this case, the data recognition device omits the operation of subtracting from 1, and Since only the result can be calculated, the computation speed can be improved relatively. Additionally, to improve computation speed, matching scores may be used in which the part divisible by 2 is omitted or the absolute value is used instead of the square, as described in Equations 10 to 13. For convenience of explanation, the following description is based mainly on Equation 8, but it is not limited to this.

[0076] A data recognition device according to one embodiment has an average feature vector as shown in Equation 8 described above. and dispersion feature vectors Considering this, the matching score for the set A of registered feature vectors of the input feature vector B can be calculated element by element. Since the data recognition device can calculate partial scores by accumulating the scores calculated element by element, it can recognize the input data using the partial scores before calculating the total matching score. Therefore, the data recognition device can determine at a faster speed whether the object indicated by the input data matches the registered user. In other words, the data recognition device can determine the result in advance before calculating the matching score for the entire index constituting the dimension of the feature vector by determining whether a specific threshold condition set for access permission (e.g., screen unlocking) is satisfied during the calculation, or whether the specific threshold condition cannot be reached even if the calculation is continued.

[0077] The calculation of scores by element and the corresponding calculation of partial scores are explained in Figures 5 and 6 below.

[0078] FIGS. 5 and 6 are drawings illustrating the calculation of a matching score and data recognition according to one embodiment.

[0079] According to one embodiment, the data recognition device can calculate a partial score (531) as a matching score (530) from at least some elements of input feature data (510), at least some elements of common component data (521), and at least some elements of distribution component data (522). For example, the data recognition device can calculate an entry score at a target index based on an element corresponding to a target index (e.g., the i-th index) in the input feature data (510), an element corresponding to the target index in the common component data (521), and an element corresponding to the target index in the distribution component data (522). The data recognition device can calculate the partial score (531) by accumulating the entry score at the target index to the entry score at at least one other index.

[0080] For example, in the aforementioned mathematical formula 8 Since the term is a vector product or matrix product operation, it can be expressed as in Equation 14 below.

[0081]

[0082] In the aforementioned mathematical formula 14 Is or Multiplied by the element corresponding to the i-th index It can represent the elements of. Unlike cosine similarity, the matching similarity according to Equation 8 can have operations for individual indices separated as shown in Equation 15 below.

[0083]

[0084] The above-described mathematical formula 15 can represent the element score at the i-th index. In this way, in the matching score calculation according to mathematical formula 8, the data recognition device can calculate the element score by separating it for each individual index, so that a partial score (531) can be calculated.

[0085] Accordingly, the data recognition device can calculate element scores according to the mathematical formula 15 described above at various indices and calculate partial scores (531) by summing the calculated element scores. For example, as shown in FIG. 5, the data recognition device can calculate and accumulate element scores while sequentially increasing the indices. For example, as shown in FIG. 5, the data recognition device can calculate element scores where i is n by increasing i by 1, starting from an element score where i is 1.

[0086] According to one embodiment, input feature data (510), common component data (521), and distribution component data (522) may be composed of dimension vectors having the same dimension.

[0087] The data recognition device can recognize input data based on partial scores (531) calculated for some indices of a dimension before scores for all indices of a dimension are calculated. For example, in FIG. 5, the data recognition device can calculate partial scores (531) by calculating element scores according to the above-described mathematical formula 15 from the elements of the mean feature vector, the elements of the variance feature vector, and the elements of the input feature vector corresponding to the 1st to 3rd indices. In FIG. 5, the data recognition device can compare partial scores (531) calculated only up to the 3rd index among n indices with partial thresholds.

[0088] For reference, a partial threshold may represent a threshold set for a target index to determine recognition success based on a partial score formed by the accumulation of element scores up to an arbitrary target index. The partial threshold may be set to a value corresponding to the ratio of the target index to the total index within the total threshold distance value set for the entire index. For example, regarding the total threshold distance value calculated for all indices n, Th distance_total If set, the partial threshold distance value for the partial score calculated up to target index i is Th distance_total It can be *i / n. However, this is for the sake of convenience of explanation, and the partial threshold may be set according to a different ratio.

[0089] FIG. 6 illustrates an example of an operation for determining whether there is a match for multiple registered vector data. For example, a data recognition device can determine whether there is a match for each registered vector data based on the result of comparing a partial score calculated for each index from multiple registered vector data with a partial threshold level.

[0090] For reference, since the element score in Equation 15 is a distance value, a smaller value of the element score indicates that the object of the input data is similar to the registered user, and a larger value of the element score indicates that the object of the input data is dissimilar to the registered user. When using the sum of the element scores as the partial score, a smaller value of the partial score indicates that the object of the input data is similar to the registered user, and a larger value of the partial score indicates that the object of the input data is dissimilar to the registered user. Therefore, the data recognition device can compare the partial score calculated in this way with the partial threshold level, and if the partial score exceeds the partial threshold level, it can determine that the object of the input data does not match the registered user. Conversely, if the partial score is below the partial threshold level, the data recognition device can determine that the object of the input data matches the registered user. FIGS. 5 and 6 mainly explain examples where the matching score, element score, and partial score are distance values. However, setting the element score and partial score as distance values ​​in FIGS. 5 and 6 is for convenience of explanation and is not limited thereto, and the element score and partial score may also be set as similarity values. For example, to construct a partial score as a similarity, the partial score may be set as a value obtained by subtracting the sum of the element scores according to the above-described mathematical formula 15 from an arbitrary constant (e.g., 1).

[0091] According to one embodiment, the data recognition device can determine whether recognition based on input data is successful based on the result of comparing a partial score to a partial success threshold level. The data recognition device can determine whether recognition based on input data is unsuccessful based on the result of comparing a partial score to a partial failure threshold level.

[0092] For example, the data recognition device can calculate partial scores by sequentially calculating and summing element scores from index 1 to index n. However, it is not limited to calculating sequentially from index 1 to index n; the data recognition device may also select some indices in a predetermined order and calculate partial scores for the selected indices. In FIG. 6, the data recognition device can calculate the final matching score for the first registration vector data (631), the third registration vector data (633), and the P registration vector data (639). Here, P may be an integer greater than or equal to 1.

[0093] The data recognition device can calculate an element score of 0.019 at index 1 of the second registered vector data (632), an element score of 0.012 at index 2, and an element score of 0.910 at index 3. Accordingly, the data recognition device can calculate a partial score of 0.941 for the target index 3. Assuming that the partial threshold distance value is 0.9 as the partial failure threshold level at the third index, the partial score at the target index 3 exceeds the partial threshold distance value, so the data recognition device can determine that the second registered vector data (632) and the input feature data do not match.

[0094] Similarly, for the fourth registration vector data (634), the data recognition device can calculate a partial score of 0.703 at index 1, and assuming the partial threshold distance value as the partial failure threshold level at the first index is 0.5, the data recognition device can determine that the fourth registration vector data (634) and the input feature data do not match. For the P-1 registration vector data (638), the data recognition device can calculate a partial score of 0.821 at index 2, and assuming the partial threshold distance value as the partial failure threshold level at the second index is 0.7, the data recognition device can determine that the P-1 registration vector data (638) and the input feature data do not match.

[0095] In the example described above, only the partial failure threshold level was explained, but it is not limited thereto. A partial success threshold level may also be set for at least some indices. In this case, if the partial score calculated as a distance value is less than the partial success threshold level, the data recognition device may determine that the input feature data matches the registered vector data.

[0096] Therefore, for multiple registered vector data (P registered vector data in FIG. 6), the data recognition device can determine whether input feature data matches the registered vector data by calculating partial scores up to only some indices, even without calculating matching scores (e.g., similarity or distance values) for all indices constituting the dimension. Even without performing calculations for all indices, the data recognition device can determine more quickly whether the input feature data matches the registered vector data if the input feature data and the registered vector data are sufficiently similar or sufficiently dissimilar during the calculation. As a result, the data recognition device can generate recognition results at high speed with less computation. Furthermore, since the data recognition device also considers the distribution tendencies of the registered feature vectors, it can recognize more accurately than the conventional simple comparison between feature vectors.

[0097] For reference, FIG. 6 mainly describes the case where the partial score is a distance value, but the partial score may be set as a partial similarity equal to the sum of element scores according to Equation 15, which is the value obtained by subtracting from an arbitrary constant (e.g., 1). When the partial score is a partial similarity, the data recognition device may determine that recognition is successful in response when the partial similarity at the target index exceeds the partial success threshold level, and conversely, determine that recognition is failed in response when the partial similarity is below the partial failure threshold level.

[0098] FIG. 7 is a diagram illustrating data recognition according to another embodiment.

[0099] A data recognition device according to one embodiment can load a representative mean vector and a representative variance vector corresponding to each of the plurality of groups from a registration database in which a plurality of registration vector data are clustered into a plurality of groups and stored. The representative mean vector and the representative variance vector corresponding to the target group may be the mean vector and the variance vector of the registration feature vectors belonging to the target group. For example, in FIG. 7, the first group may be the root node, and the second and third groups may be child nodes of the first group. The representative mean vector and the representative variance vector of the second and third groups may be the mean vector and the variance vector of the registration feature vectors of each group. If the first group includes the second and third groups, the representative mean vector and the representative variance vector of the first group may be the mean vector and the variance vector of all registration feature vectors included in the two groups of the second and third groups. Accordingly, the data recognition device can manage the registration feature vectors by grouping them into a tree structure.

[0100] A data recognition device can identify a group among a plurality of groups to which the input feature data matches, based on input feature data, a representative mean vector, and a representative variance vector. For example, taking FIG. 7 as an example, the data recognition device can determine whether the input feature vector (701) satisfies a first cluster condition (710). The first cluster condition (710) may indicate whether a matching score calculated from the input feature vector (701), the representative mean vector (711) of the first group, and the representative variance vector (712) of the first group exceeds a first group threshold similarity. If the calculated matching score exceeds the threshold similarity, the data recognition device may determine that the input feature vector (701) matches the first group. Conversely, in response to the case where the matching score is below the threshold similarity, the data recognition device may determine that the verification of the input feature vector (701) failed in step (792).

[0101] The data recognition device may determine whether an input feature vector (701) identified as a first group belongs to a second group and / or a third group within the first group based on a second cluster condition (720) and / or a third cluster condition (730). The second cluster condition (720) may indicate whether the matching score calculated from the input feature vector (701) and the representative mean vector and representative variance vector of the second group exceeds the second group threshold similarity. The third to seventh cluster conditions (730 to 770) may be set similarly. The data recognition device may calculate the matching similarity with the input feature vector (701) using the representative mean vector and representative variance vector representing each group from the upper layer to the lower layer of the tree structure, and identify the lowest group that is finally matched to the input feature vector (701) based on the result of comparing the matching similarity with the group threshold value of each group.

[0102] The data recognition device can determine whether the input data matches the registration vector data belonging to the identified group. For example, in step (780), the data recognition device can calculate a matching score with the input feature vector (701) using the mean feature vector and the variance feature vector of the registration vector data belonging to the identified lowest group. If the matching score is similarity, the data recognition device can determine that recognition is successful in step (791) by responding that the matching score exceeds the threshold similarity based on the result of comparing the matching similarity to the threshold similarity. Conversely, if the matching score is below the threshold similarity, the data recognition device can determine that recognition is failed in step (792).

[0103] For reference, for the convenience of explanation, it has been described as having one representative mean vector and one representative variance vector for each group, but this is not limited thereto. The groups may have k representative mean vectors and k representative variance vectors for each group, and P registered vector data may be clustered. When N is 2 and P is 1000, instead of measuring similarity for all 1000 vectors, the data recognition device according to one embodiment can find registered vector data having maximum similarity to the input feature data through up to 20 comparison operations (log21000*2). Furthermore, the data recognition device may identify the search direction of the subgroup node through partial scores for some indices instead of operations on the entire index during each operation process, in combination with the method described above in FIGS. 5 and 6.

[0104] FIGS. 8a to 8c are drawings illustrating the configuration of a processor according to one embodiment.

[0105] FIGS. 8a through 8c illustrate processors implemented in various structures. A data recognition device according to one embodiment can recognize images and / or voice through processors implemented in various structures. For example, a processor of the data recognition device can receive an input image from a user as input data. The processor can extract input feature data from the input image. Based on a matching score, the processor can determine whether an object appearing in the input image is a registered user. In another example, the processor can receive a voice signal from a user as input data. The processor can extract input feature data from the voice signal. Based on a matching score, the processor can determine whether the speaker of the voice signal is a registered user. FIGS. 8a through 8c illustrate an example of receiving a voice signal as input data.

[0106] In FIG. 8a, the processor may include a digital signal processor (810) (DSP) and an application processor (820) (AP). The application processor (820) may operate as a main processor. For example, the digital signal processor (810) may detect target data from input data. As illustrated in FIG. 8a, the digital signal processor (810) may detect segment voice signals of a specific pattern (e.g., keywords such as "Hi, Bixby"). The digital signal processor (810) may transmit the detected target data to the application processor (820). Since the digital signal processor (810) operates at low power, it may remain in an active state at all times. Furthermore, the digital signal processor (810) may naively determine whether it is a speaker. The application processor (820) may extract input feature data from the target data. The application processor (820) can calculate a matching score from the registration vector data along with the extracted input feature data to determine whether to accept or reject the user's access indicated by the target data. For reference, the application processor (820) may operate in sleep mode and be activated in response when the target data is received from the digital signal processor (810).

[0107] In FIG. 8b, the processor may include a neural processor (830) and a digital signal processor (810). The digital signal processor (810) may operate as a main processor. The neural processor (830) may be configured to extract input feature data. The neural processor (830) may, for example, receive a voice signal of a keyword segment from the input data from the digital signal processor (810) and extract input feature data from the received segment voice signal. The digital signal processor (810) may monitor the voice signal to monitor whether the keyword voice is input from the user. The digital signal processor (810) may remain in an active state at all times. Additionally, the digital signal processor (810) may be configured to calculate a matching score from the input feature data, common component data, and distribution component data transmitted from the neural processor (830). The digital signal processor (810) can determine whether to accept and / or deny access to the device by determining whether the user of the input data matches a registered user based on the matching score. For reference, the neural processor (830) may be implemented as a dedicated processor configured to perform the operation of the neural network described in FIG. 1.

[0108] In FIG. 8c, the processor may include a digital signal processor (810), an application processor (820), and a neural processor (830). Similar to the description in FIG. 8a, the digital signal processor (810) may detect target data containing voice of a keyword segment from an input voice signal and transmit it to the application processor (820) and the neural processor (830). The neural processor (830) may extract input feature data from the target data corresponding to the segment voice. The application processor (820) may operate in sleep mode and be activated in response when a keyword is detected by the digital signal processor (810), and may calculate a matching score based on the input feature data and registration vector data transmitted from the neural processor (830). The application processor (820) may determine whether to allow user access based on the calculated matching score.

[0109] For reference, in FIGS. 8a to 8c, the application processor (820) may be a chip designed to have higher computing performance and consume relatively more power compared to the digital signal processor (810). However, this is exemplary and is not limited thereto.

[0110] According to one embodiment, the data recognition device may allow access to the device in response when it is determined that the user indicated by the input data matches a registered user. In this specification, allowing access may refer to an action of granting the user permission to at least some functions and / or applications of the device. For example, upon successful recognition, the data recognition device may change the device from a locked state to an unlocked state.

[0111] FIGS. 9 and FIGS. 10 are drawings illustrating the configuration of a data recognition device according to one embodiment.

[0112] The data recognition device (900) illustrated in FIG. 9 may include an input unit (910), a processor (920), and a memory (930).

[0113] The input unit (910) can receive input data. For example, the input unit (910) can receive an input image and / or input voice as input data. The input unit (910) may include a camera for capturing an image. As another example, the input unit (910) may include a microphone for capturing a voice signal.

[0114] The processor (920) can extract input feature data from input data. The processor (920) can calculate a matching score from the input feature data, common component data and distribution component data of a plurality of registered feature data corresponding to a registered user. The processor (920) can recognize input data based on the matching score. However, the operation of the processor (920) is not limited to this, and may be performed chronologically or in parallel with at least one of the operations described above in FIGS. 1 to 8.

[0115] The memory (930) can store a registration database containing multiple registration vector data clustered into multiple groups. Instead of storing multiple registration feature vectors individually, the memory (930) stores registration vector data including the average feature vector and the variance feature vector of multiple registration feature vectors, thereby reducing storage space. However, it is not limited to this, and the memory (930) can temporarily or permanently store data required to perform a data recognition method.

[0116] Referring to FIG. 10, the computing device (1000) is a device that recognizes data using the data recognition method described above. In one embodiment, the computing device (1000) may correspond to the device (900) described in FIG. 9. The computing device (1000) may be, for example, an image processing device, a smartphone, a wearable device, a tablet computer, a netbook, a laptop, a desktop, a PDA (personal digital assistant), or a HMD (head mounted display).

[0117] Referring to FIG. 10, the computing device (1000) may include a processor (1010), a storage device (1020), a camera (1030), an input device (1040), an output device (1050), and a network interface (1060). The processor (1010), the storage device (1020), the camera (1030), the input device (1040), the output device (1050), and the network interface (1060) may communicate with each other through a communication bus (1070).

[0118] The processor (1010) executes functions and instructions to be executed within the computing device (1000). For example, the processor (1010) can process instructions stored in the storage device (1020). The processor (1010) can perform one or more of the operations described above through FIGS. 1 to 9.

[0119] The storage device (1020) stores information or data necessary for the execution of the processor. The storage device (1020) may include a computer-readable storage medium or a computer-readable storage device. The storage device (1020) may store instructions to be executed by the processor (1010) and may store relevant information while software or an application is executed by the computing device (1000).

[0120] The camera (1030) can capture an image composed of multiple image frames. For example, the camera (1030) can generate a frame image.

[0121] The input device (1040) can receive input from a user through tactile, video, audio, or touch input. The input device (1040) may include a keyboard, mouse, touch screen, microphone, or any other device capable of detecting input from a user and transmitting the detected input.

[0122] The output device (1050) may provide the output of the computing device (1000) to the user through a visual, auditory, or haptic channel. The output device (1050) may include, for example, a display, a touch screen, a speaker, a vibration generating device, or any other device capable of providing the output to the user. The network interface (1060) may communicate with an external device through a wired or wireless network. The output device (1050) may provide the result of recognizing input data (e.g., access allowed and / or access denied) to the user using at least one of visual information, auditory information, and haptic information.

[0123] The device described above may be implemented as a hardware component, a software component, and / or a combination of a hardware component and a software component. For example, the device and components described in the embodiments may be implemented using one or more general-purpose or special-purpose computers, such as, for example, a processor, a controller, an arithmetic logic unit (ALU), a digital signal processor, a microcomputer, a field programmable array (FPA), a programmable logic unit (PLU), a microprocessor, or any other device capable of executing and responding to instructions. The processing unit may execute an operating system (OS) and one or more software applications executed on the operating system. Additionally, the processing unit may access, store, manipulate, process, and generate data in response to the execution of the software. For ease of understanding, the processing unit may be described as being used as a single unit, but those skilled in the art will understand that the processing unit may include multiple processing elements and / or multiple types of processing elements. For example, the processing unit may include multiple processors or one processor and one controller. In addition, other processing configurations, such as parallel processors, are also possible.

[0124] Software may include computer programs, code, instructions, or a combination of one or more of these, and may configure a processing unit to operate as desired or command the processing unit independently or collectively. Software and / or data may be permanently or temporarily embodied in any type of machine, component, physical device, virtual equipment, computer storage medium or device, or transmitted signal wave so as to be interpreted by the processing unit or to provide instructions or data to the processing unit. Software may be distributed over networked computer systems and may be stored or executed in a distributed manner. Software and data may be stored on one or more computer-readable recording media.

[0125] The method according to the embodiment may be implemented in the form of program instructions that can be executed through various computer means and recorded on a computer-readable medium. The computer-readable medium may include program instructions, data files, data structures, etc., either alone or in combination. The program instructions recorded on the medium may be those specifically designed and configured for the embodiment, or they may be those known and available to those skilled in the art of computer software. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tapes; optical recording media such as CD-ROMs and DVDs; magneto-optical media such as floptical disks; and hardware devices specifically configured to store and execute program instructions, such as ROM, RAM, and flash memory. Examples of program instructions include machine code, such as that generated by a compiler, as well as high-level language code that can be executed by a computer using an interpreter, etc. The hardware devices described above may be configured to operate as one or more software modules to perform the operation of the embodiment, and vice versa.

[0126] Although the embodiments have been described above with reference to the limited drawings, those skilled in the art can apply various technical modifications and variations based on the above. For example, suitable results may be achieved even if the described techniques are performed in a different order than described, and / or if the components of the described system, structure, device, circuit, etc. are combined or assembled in a form different from described, or replaced or substituted by other components or equivalents.

[0127] Therefore, other implementations, other embodiments, and equivalents to the claims also fall within the scope of the claims set forth below. Explanation of the symbols

[0128] 900: Data recognition device 910: Input section 920: Processor 930: Memory

Claims

Claim 1 A data recognition method performed by a processor, comprising: receiving input data and extracting input feature data from the input data; calculating an element score at the target index based on an element corresponding to the target index in the input feature data, an element corresponding to the target index in common component data of a plurality of registered feature data corresponding to a registered user, and an element corresponding to the target index in distribution component data; calculating a matching score based on calculating a partial score by accumulating the element score at the target index to the element score at at least one other index; and recognizing the input data based on the matching score. Claim 2 A data recognition method according to claim 1, wherein the common component data includes an average feature vector having the average value of elements of the same index among the elements of a plurality of registered feature vectors for each index, and the distribution component data includes a variance feature vector having the variance value of elements of the same index among the plurality of registered feature vectors for each index. Claim 3 delete Claim 4 delete Claim 5 A data recognition method according to claim 1, wherein the input feature data, the common component data, and the distribution component data are composed of dimension vectors having the same dimension, and the step of recognizing the input data includes the step of recognizing the input data based on partial scores calculated for some indices of the dimension before scores for all indices of the dimension are calculated. Claim 6 A data recognition method according to claim 1, wherein the step of recognizing the input data comprises: a step of determining whether the recognition based on the input data is successful based on the result of comparing the partial score with a partial success threshold level; and a step of determining whether the recognition based on the input data is unsuccessful based on the result of comparing the partial score with a partial failure threshold level. Claim 7 A data recognition method according to claim 1, wherein the step of recognizing the input data includes the step of allowing access to a device in response when it is determined that the user indicated by the input data matches the registered user. Claim 8 In claim 1, the step of calculating the matching score comprises: loading a representative mean vector and a representative variance vector corresponding to each of the plurality of groups from a registration database in which a plurality of registration vector data is clustered into a plurality of groups and stored; identifying a group among the plurality of groups that the input feature data matches based on the input feature data, the representative mean vector, and the representative variance vector; and determining whether the input data matches the registration vector data belonging to the identified group, wherein each of the plurality of registration vector data includes common component data and distribution component data. Claim 9 A data recognition method according to claim 1, wherein the step of extracting input feature data comprises: receiving an input image from a user as input data; and extracting input feature data from the input image, and the step of recognizing the input data comprises determining whether an object appearing in the input image is the registered user based on the matching score. Claim 10 A data recognition method according to claim 1, wherein the step of extracting input feature data comprises: receiving a voice signal from a user as input data; and extracting input feature data from the voice signal, and the step of recognizing the input data comprises determining whether the speaker of the voice signal is the registered user based on the matching score. Claim 11 A computer-readable recording medium storing one or more computer programs comprising instructions for performing the method of any one of paragraphs 1, 2, and 5 through 10. Claim 12 A data recognition device comprising: an input unit for receiving input data; and a processor for extracting input feature data from the input data, calculating an element score at the target index based on an element corresponding to the target index in the input feature data, an element corresponding to the target index in common component data of a plurality of registered feature data corresponding to a registered user, and an element corresponding to the target index in distribution component data, and calculating a partial score based on accumulating the element score at the target index to the element score at at least one other index, and recognizing the input data based on the matching score. Claim 13 delete Claim 14 delete Claim 15 In claim 12, the input feature data, the common component data, and the distribution component data are composed of dimension vectors having the same dimension, and the processor recognizes the input data based on partial scores calculated for some indices of the dimension before scores for all indices of the dimension are calculated. Claim 16 A data recognition device according to claim 12, wherein the processor determines whether recognition based on the input data is successful based on the result of comparing the partial score to a partial success threshold level, and determines whether recognition based on the input data is unsuccessful based on the result of comparing the partial score to a partial failure threshold level. Claim 17 In paragraph 12, the processor is a data recognition device that allows access to the device in response to the determination that the user indicated by the input data matches the registered user. Claim 18 A data recognition device according to claim 12, further comprising a memory storing a registration database including multiple registration vector data clustered into multiple groups, wherein each of the multiple registration vector data includes common component data and distribution component data, and wherein the processor loads a representative mean vector and a representative variance vector corresponding to each of the multiple groups from the memory, identifies a group among the multiple groups to which the input feature data matches based on the input feature data, the representative mean vector and the representative variance vector, and determines whether the input data matches a registration vector data belonging to the identified group. Claim 19 In claim 12, the data recognition device comprises: a neural processor configured to extract the input feature data; and a main processor configured to calculate the matching score from the input feature data, the common component data, and the distributed component data transmitted from the neural processor. Claim 20 In claim 12, the data recognition device comprises: a digital signal processor (DSP) that detects target data from the input data; and a main processor that extracts the input feature data from the target data.

Images