Speech recognition device, speech recognition method, and program

Abstract

A speech recognition device 100 according to the present disclosure comprises a conversion unit 121 for converting speech by a speaker into a feature vector, a weighting unit 122 for weighting the feature vector by the degree of importance of the speech based on situation information representing the situation when the speaker spoke, and a recognition unit 123 for recognizing new speech by the speaker on the basis of the weighted feature vector.

Description

Speech recognition device, speech recognition method, and program

[0001] The present disclosure relates to a speech recognition device, a speech recognition method, and a program.

[0002] A speech recognizer is a device that takes input speech and outputs a string of characters corresponding to that speech. There are types of speech recognizers that consist of a feature encoder and a string decoder. A feature encoder takes speech as input and converts it into speech features. A string decoder takes speech features as input and outputs a string of characters.

[0003] An example of a speech recognition technique is disclosed in Patent Document 1. In Patent Document 1, a speech section detection unit detects a speech section and extracts a speech section feature vector Xk, which is a feature of the speech section. Furthermore, a speech recognition unit performs speech recognition based on the speech section feature vector Xk.

[0004] On the other hand, speech recognition rates can be reduced depending on the speaker's speaking conditions, such as their emotions and fatigue level. For this reason, Patent Document 2 proposes a method using a biomonitor as a method for correcting pitch changes due to emotions, etc. Specifically, Patent Document 2 uses a biosignal output from a biomonitor that indicates changes in emotional state to compensate for pitch changes in the speech signal and suppress a decrease in speech recognition rate. Furthermore, Patent Document 3 estimates the speaker's emotions from video and uses speech recognizers individually adjusted for each emotion, thereby suppressing a decrease in speech recognition rate. Thus, Patent Documents 2 and 3 suppress a decrease in speech recognition rate due to changes in speech pitch and speed by estimating the speaker's emotions at the time of utterance.

[0005] International Publication No. 2022 / 049613 Japanese Patent Application Laid-Open No. 07-199986 Japanese Patent Application Laid-Open No. 2020-181022

[0006] However, in the above-mentioned Patent Documents 2 and 3, since speech recognition is performed on the speech at the time of utterance, there is a problem that it is not possible to deal with a decrease in speech recognition rate that takes into account the topic and context, such as recognizing homonyms that should change depending on the topic of utterance. As a specific example of the above problem, when an utterance is made such as "Our company will 'sozo' a better future," there are two kanji candidates for "sozo" (imagination) and "creation" (creation), and either one will be adopted depending on the context, but the appropriate kanji may not be adopted, resulting in a decrease in speech recognition rate.

[0007] Therefore, an object of the present disclosure is to provide a voice recognition device that can solve the above-mentioned problem of a decrease in voice recognition rate.

[0008] A speech recognition device according to one embodiment of the present disclosure includes a conversion unit that converts an utterance by a speaker into a feature vector; a weighting unit that weights the feature vector with an importance of the utterance based on situational information that indicates the situation at the time of the speaker's utterance; and a recognition unit that recognizes a new utterance by the speaker based on the weighted feature vector.

[0009] Furthermore, a speech recognition method according to one embodiment of the present disclosure is configured to convert an utterance by a speaker into a feature vector, weight the feature vector according to the importance of the utterance based on situational information that indicates the situation at the time of the speaker's utterance, and recognize a new utterance by the speaker based on the weighted feature vector.

[0010] Furthermore, a program that is one form of the present disclosure is configured to cause a computer to execute the following processes: converting an utterance by a speaker into a feature vector; weighting the feature vector according to the importance of the utterance based on situational information that represents the situation at the time the speaker spoke; and recognizing a new utterance by the speaker based on the weighted feature vector.

[0011] With the above-described configuration, the present disclosure can improve the speech recognition rate.

[0012] 5 is a block diagram showing the overall configuration of a speech recognition device according to a first embodiment of the present disclosure. FIG. 6 is a block diagram showing a portion of the configuration of the speech recognition device disclosed in FIG. 1. FIG. 7 is a block diagram showing a portion of the configuration of the speech recognition device disclosed in FIG. 1. FIG. 8 is a flowchart showing the operation of the speech recognition device disclosed in FIG. 1. FIG. 9 is a block diagram showing a portion of the configuration of a speech recognition device according to a second embodiment of the present disclosure. FIG. 10 is a flowchart showing the operation of the speech recognition device disclosed in FIG. 5. FIG. 11 is a block diagram showing a portion of the configuration of a speech recognition device according to a third embodiment of the present disclosure. FIG. 12 is a block diagram showing the hardware configuration of a speech recognition device according to a fourth embodiment of the present disclosure. FIG. 13 is a block diagram showing the configuration of a speech recognition device according to the fourth embodiment of the present disclosure.

[0013] <First Embodiment> A first embodiment of the present disclosure will be described with reference to Fig. 1 to Fig. 4. Fig. 1 to Fig. 3 are diagrams for explaining the configuration of a speech recognition device, and Fig. 4 is a diagram for explaining the processing operation of the speech recognition device.

[0014] [Configuration] The speech recognition device 10 in this embodiment is a device that recognizes a character string corresponding to a speech uttered by a speaker from the speech. In particular, the speech recognition device 10 in this embodiment converts the speech into features, recognizes a character string from the features, and outputs the recognized character string.

[0015] The speech recognition device 10 is composed of one or more information processing devices each including a calculation device and a storage device. As shown in FIG. 1 , the speech recognition device 10 includes an utterance acquisition unit 1, an utterance content extraction unit 2, and a speech recognition unit 3. The functions of the utterance acquisition unit 1, the utterance content extraction unit 2, and the speech recognition unit 3 can be realized by the calculation device executing a program for realizing each function stored in the storage device. The speech recognition device 10 also includes an utterance storage unit 4 and a tensor storage unit 5. The utterance storage unit 4 and the tensor storage unit 5 are each composed of a storage device. Each component will be described in detail below.

[0016] The utterance acquisition unit 1 acquires an utterance from a speaker and temporarily stores it in the utterance storage unit 4. At this time, the utterance acquisition unit 1 divides the acquired utterance into a predetermined time width or into a series of utterances, assigns a number i to each divided utterance so that the chronological order can be determined, and stores each utterance i in the utterance storage unit 4 in chronological order. Note that the data format of the utterance may be variable-length character string data or audio data corresponding to each utterance, or even video data including audio data. For example, the character string data may be a character string recognized from the audio of the utterance, and the audio data may be the audio data of the utterance itself or audio data included in the video data. Then, for example, the utterance acquisition unit 1 acquires the first to t-1st past utterances i (i = 1 to t-1) by the speaker as utterances for creating a tensor, which will be described later, and acquires the current tth utterance t by the speaker as an utterance to be subjected to speech recognition. The utterance acquisition unit 1 may assign time information, such as the start time and end time of each utterance, to each utterance, thereby identifying the order of the utterances.

[0017] As described above, the utterance acquisition unit 1 acquires character string data, audio data, and video data as the utterance itself, and also acquires temporal information such as the order of the utterance and time information.The utterance itself and the temporal information become situational information that represents the situation at the time the speaker made the corresponding utterance.

[0018] The speech content extraction unit 2 (conversion unit, weighting unit) has the function of converting past utterances (1 to t-1) by a speaker into feature vectors and weighting the feature vectors with the importance of the utterance based on the situation at the time of the utterance. To achieve this function, the speech content extraction unit 2 includes an utterance content embedding unit 21, an emotion recognition unit 22, an utterance importance inference unit 23, and an utterance content combination unit 24, as shown in FIG.

[0019] The utterance content embedding unit 21 reads out utterance 1 to utterance t-1, which are past utterances i stored in the utterance storage unit 4, and converts the variable-length data of each utterance i into a fixed-length embedding vector space that reflects the content of the corresponding utterance. In other words, the utterance content embedding unit 21 converts the data of each utterance i into a feature vector that represents the features of the content of the utterance.

[0020] The emotion recognition unit 22 reads out utterances 1 to t-1, which are past utterances i stored in the utterance storage unit 4, and estimates the emotion of the speaker at the time of utterance based on the utterance. For example, if the utterance is character string data or audio data, the emotion recognition unit 22 converts the data string into features and estimates the emotion from the features. Also, for example, if the utterance is video data including audio data, the emotion recognition unit 22 converts the speaker's facial expressions, gestures, etc. into features and estimates the emotion from these features. Note that at this time, the emotion recognition unit 22 sets emotion labels indicating the type of emotion and emotion intensities corresponding to each emotion label in advance, corresponding to each feature, and estimates the emotion labels and emotion intensities corresponding to the converted features. However, the emotion recognition unit 22 may estimate emotions using any method. For example, a machine learning model representing the relationship between stored utterances and their emotion labels and intensities may be generated and used to perform estimation.

[0021] As emotion types, labels such as calm, anger, sadness, joy, and excitement are prepared. However, types of emotion labels other than those described above can also be used. When an emotion corresponding to each emotion is estimated, the emotion recognition unit 22 outputs the label. When outputting the magnitude of the emotion label, the emotion recognition unit 22 first estimates the emotion label corresponding to the utterance as described above, and then estimates the magnitude of the emotion based on the extent to which each emotion is contained in the utterance. Note that the magnitude of the estimated emotion may be expressed in the form of an absolute value or as a probability distribution of each emotion. However, since the emotion labels and magnitudes of the emotion labels inferred by the emotion recognition unit 22 are used as inputs to the utterance importance inference unit 23, they only need to be defined in a form that matches the consistency of the connection between them.

[0022] In the following explanation, we will use two emotion labels, "calm" and "other," and assume that the emotion estimation results follow a probability distribution. In other words, we will design the sum of the magnitudes of "calm" and "other" emotions to be "1."

[0023] The utterance importance inference unit 23 calculates the importance of each utterance based on the emotion of each utterance estimated by the emotion recognition unit 22, and weights the embedding vector corresponding to each utterance with this importance. In this embodiment, the utterance importance inference unit 23 further calculates the importance of an utterance based not only on the emotion at the time of the utterance but also on the time information of the utterance. In order to calculate the importance of an utterance and weight the embedding vector corresponding to the utterance with this importance, the utterance importance inference unit 23 includes an inter-utterance distance calculation unit 231, an emotion value calculation unit 232, a forgetting rate calculation unit 233, and an embedding vector importance assignment unit 234, as shown in FIG. 3 .

[0024] For each past utterance i, the inter-utterance distance calculation unit 231 calculates the elapsed time or an amount equivalent to the elapsed time between the occurrence time of the utterance i and the current time, which is the occurrence time of the current utterance t to be recognized, and sets this as the inter-utterance distance δ. For example, if the start times of utterance i and utterance t are stored, the inter-utterance distance calculation unit 231 calculates the inter-utterance distance δ by taking the difference between the two times. Alternatively, the inter-utterance distance calculation unit 231 may calculate the inter-utterance distance δ using another calculation method. For example, the inter-utterance distance calculation unit 231 uses the order i of the utterance as an amount equivalent to time to calculate the inter-utterance distance δ ｉ In the following explanation, the inter-utterance distance δ using the utterance order is ｉ By doing this, the inter-utterance distance δ ｉ The earlier the utterance, the larger the calculated value. ｉ The larger the value, the smaller the importance of the utterance is calculated to be.

[0025] The emotion value calculation unit 232 calculates the emotion value of utterance i by averaging the emotions before and after it. In other words, with utterance i as the starting point, the emergence of utterance i and its associated emotions can be considered to be a response to the emotions contained in the utterances that precede utterance i. On the other hand, the emotional responses of utterances that follow utterance i can be considered to be a response to utterance i. Taking the above into consideration, the emotion value calculation unit 232 calculates the average of each emotion label of the utterances before and after utterance i, or the magnitude of each emotion label. The averaging process can be a simple arithmetic average, as well as a method using a unique weighting or a method of inferring only from utterance i. The range of the averaging process can be adjusted automatically or manually as appropriate. The averaging process also includes a method of adjusting weights depending on the type and magnitude of the emotion label.

[0026] In this embodiment, the backward average is calculated by focusing on the conversational response to utterance i. Furthermore, in this implementation, emotion labels are defined as two types of labels, "calm" and "other," as described above. Therefore, the emotion value is calculated as the average magnitude of the "other" emotion, which is inferred to be a change from "calm." The magnitudes of the k+1 "other" labels, including utterance i, are all averaged with equal weighting. Therefore, the emotion value of the "other" label is calculated using the following formula 1.

[0027] In this embodiment, according to the definition of the emotion recognition unit 22, ｉ In addition, when a past utterance i is close to a current utterance i, specifically when utterance i satisfies i+k>t, <E ｉ >= 1. In other words, the utterance immediately before utterance t is not weighted, but is set to be taken into consideration with a weight of 1. However, the weighting for utterance i immediately before utterance t may be defined differently.

[0028] The forgetting rate calculation unit 233 calculates the importance of an utterance relative to the utterance i based on the inter-utterance distance δ obtained from the inter-utterance distance calculation unit 231 and the emotion value obtained from the emotion value calculation unit 232. Here, the importance is a weight for each utterance i, and the forgetting rate calculation unit 233 calculates the importance based on the elapsed time and the emotion value. In order to reduce the level of attention to an utterance that has been forgotten for a certain amount of time, the forgetting rate calculation unit 233 calculates the importance so that the greater the inter-utterance distance δ, the more the importance decreases.

[0029] In addition, the speech distance constant τ ｉ is set for each emotion label. In this embodiment, the speech distance constant τ ｉ Only the important comment effective time T ｉ To, T ｉ = < E ｉ >×τ ｉ Here, from the definition of the average value of the "Other" label, T ｉ ≦τ ｉ The more important the utterance, the greater the utterance distance constant τ ｉ The importance of utterance i, I ｉ is the effective time of important speech T ｉ As shown in this formula, the greater the predetermined emotion value, i.e., the greater the magnitude of the "other" label, the greater the importance. In the above example, the importance of utterance i is calculated based on the emotion value and the inter-utterance distance, but it may be calculated based only on the emotion value without relying on the inter-utterance distance.

[0030] As described above, the embedding vector importance assigning unit 234 weights the embedded representation of utterance i obtained from the utterance content embedding unit 21 with the importance of utterance i obtained from the forgetting rate calculation unit 233. The embedding vector importance assigning unit 234 can be realized, for example, by multiplying the embedded representation of utterance i by the importance value. In other words, the smaller the importance value, the smaller the weight of the embedding vector corresponding to that utterance i.

[0031] In this embodiment, the embedding vector importance assigning unit 234 assigns the embedding vector importance to the embedded expression Uｉ and the importance I of the above-mentioned utterance i ｉ Using the above, the importance-added embedding vector V ｉ is calculated using equation 3. It should be noted that if the importance falls below a certain level, a process of deleting the embedding vector may be added. However, the weighting process by the embedding vector importance assigning unit 234 is not limited to the above-described process, and other methods may be used.

[0032] The utterance importance inference unit 23 executes the above-mentioned process for all the embedded vectors, and outputs all the embedded vectors with added importance to the utterance content combination unit 24 .

[0033] The utterance content combining unit 24 performs a process of converting each of the embedding vectors weighted by the importance of the utterance by the utterance importance inference unit 23 into one tensor. Then, the utterance content combining unit 24 stores the generated tensor in the tensor storage unit 5.

[0034] The speech recognition unit 3 reads out an utterance t to be subjected to speech recognition, which has been acquired by the speech acquisition unit 1 and stored in the utterance storage unit 4, and also reads out the tensor generated as described above from the tensor storage unit 5. The speech recognition unit 3 then performs speech recognition processing on the utterance t using the tensor. For example, the speech recognition unit 3 converts the utterance t into an embedding vector, extracts the corresponding embedding vector from the tensor, and performs speech recognition based on the extracted embedding vector. The speech recognition unit 3 then outputs a speech recognition result t for the utterance t.

[0035] [Operation] Next, the operation of the speech recognition device 10 described above will be described with reference to the flowchart in FIG. 4. First, the speech recognition device 10 acquires past utterances and converts each of utterances 1 to t-1 into an embedding vector (step S1). Next, the speech recognition device 10 estimates the inter-utterance distance from the present and the emotional value for each of utterances 1 to t-1 (step S2). Next, the speech recognition device 10 estimates the importance of each of utterances 1 to t-1 based on the estimated inter-utterance distance and emotional value, and converts the embedding vector into an embedding vector by taking the importance into account (step S3). Next, the speech recognition device 10 combines the embedding vectors for each of utterances 1 to t-1, converts them into a single embedding tensor, and stores the combined vector (step S4). Thereafter, the speech recognition device 10 acquires a new utterance t to be recognized, performs speech recognition using the embedding tensor, and outputs the speech recognition result t (step S5).

[0036] Note that the above describes a method in which all past utterances 1 to t-1 are processed at once in steps S1 to S3 and then combined into one tensor in step S4. However, it is also possible to perform the processing in steps S1 to S3 one by one for each utterance and then combine them into one tensor in step S4.

[0037] As described above, according to this embodiment, it is possible to generate an embedding vector that represents the characteristics of an utterance, to which the importance of each utterance is assigned according to the situation at the time of each utterance by the speaker, for example, the time of the utterance or the emotion of the speaker. Then, by performing speech recognition using the embedding vector to which the importance is assigned, speech recognition is performed taking into account important information from the past, thereby improving the accuracy of speech recognition.

[0038] Second Embodiment Next, a second embodiment of the present disclosure will be described with reference to Fig. 5 and Fig. 6. Fig. 5 is a diagram for explaining the configuration of a speech recognition device, and Fig. 6 is a diagram for explaining the processing operation of the speech recognition device.

[0039] [Configuration] In addition to the configuration of the first embodiment described above, the speech recognition device 10 of this embodiment further includes an utterance importance inference unit 235 as shown in Fig. 5. The following mainly describes the configuration that differs from the first embodiment.

[0040] The basis conversion unit 235 performs basis conversion on the embedded representation of utterance i, for example, using eigenvalue decomposition. The bases used for eigenvalue decomposition are prepared in advance before the basis conversion unit 235 is operated. For example, a statistically sufficient amount of conversational character strings is prepared in advance, and an embedding vector for the character strings is generated. The basis vectors obtained by principal component analysis of the components of the embedding vector are used as the basis of the basis conversion unit 235. The eigenvalue decomposed embedding vectors are sorted from bases corresponding to general content common to the prepared conversational character strings, i.e., bases with large norms, to bases corresponding to specific content that appears only in individual conversations, i.e., bases with small norms. This allows the ratio of generality to specificity of the content of each utterance to be determined from the size of each component. In this way, by sorting from the most important base, i.e., bases with large norms, the information can be sorted in descending order of importance. Note that the basis conversion unit 235 can be implemented using methods other than eigenvalue decomposition as long as they can extract statistically significant information.

[0041] The basis transformation unit 235 then outputs the embedding vector that has undergone the prescribed transformation to the embedding vector importance assignment unit 234. The embedding vector importance assignment unit 234 efficiently selects the importance by leaving more components for embedding vectors of utterances with high importance and leaving fewer components for embedding vectors of utterances with low importance. There are multiple ways to leave components of each basis according to importance. Specifically, the importance of the conversation I ｉ Since takes a value between 0 and 1, we regard it as a ratio and leave the number of components of the vector with the same ratio. For example, I ｉ In the case of =0.75, 75% of the total components are preserved from the components corresponding to the larger bases of the norm to the smaller bases. ｉ= 0.75, the top 75% of the vector components with the largest absolute values ​​are retained, and the bottom 25% are set to 0. The former evaluates each utterance based on the generality of the topic and reduces the dimension, while the latter retains components with large dimensions even if they are specific to the conversation, thereby reducing the dimension more in line with the conversation content.

[0042] [Operation] Next, the operation of the above-mentioned speech recognition device 10 will be described with reference to the flowchart in Fig. 6. In this embodiment, steps S1 to S5 are almost the same as those in the first embodiment, and step S2' is added between step S2 and step S3.

[0043] First, the speech recognition device 10 converts each of utterances 1 through t-1 into an embedding vector (step S1) and estimates the inter-utterance distance and emotion value (step S2). Next, the speech recognition device 10 performs basis transformation on the embedded representation of utterance i using eigenvalue decomposition (step S2'). By performing this basis transformation, the components of the embedding vector can be rearranged in descending order of importance. Next, as described above, the speech recognition device 10 estimates the importance of each utterance based on the estimated inter-utterance distance and emotion value, and converts the embedding vector into an embedding vector by taking the importance into account (step S3). At this time, the speech recognition device 10 can also reduce the dimension by adjusting the number of components of the embedding vector, for example, by leaving more components for important information. The speech recognition device 10 then combines each embedding vector into a single embedding tensor (step S4). Thereafter, the speech recognition device 10 acquires a new utterance t to be the speech recognition target, performs speech recognition using the embedding tensor, and outputs the speech recognition result t (step S5).

[0044] As described above, this embodiment makes it possible to extract even more important information from speech than in embodiment 1. This makes it possible to focus on important information from speech with greater accuracy. Furthermore, when performing dimensionality compression, the components of each vector are deleted in a predetermined order based on the emotion value, and only the number of remaining components and their respective component values ​​are stored, thereby saving memory.

[0045] Third Embodiment Next, a third embodiment of the present disclosure will be described with reference to Fig. 7. Fig. 7 is a diagram for explaining the configuration of a voice recognition device.

[0046] 7, the speech recognition device 10 of this embodiment differs from the configuration of the first embodiment in that an emotion recognition unit 22' is provided outside the utterance content extraction unit 2 and is configured to recognize the emotion of each utterance. Therefore, for example, when an utterance such as utterance 1 to utterance t-1 is generated, speech recognition is always performed by the speech recognition unit 3 and emotion estimation is always performed by the emotion recognition unit 22'. The speech recognition results and emotion values ​​of each utterance are then input to the utterance importance inference unit 23.

[0047] 7, the speech recognition device 10 of this embodiment includes an embedded vector storage unit 6 within the utterance importance inference unit 23. The utterance content embedding unit 21 within the utterance importance inference unit 23 of this embodiment creates an embedded vector for each utterance in the same manner as described above and stores the vector in the embedded vector storage unit 6. At this time, the utterance content embedding unit 21 also associates and stores the speech recognition result and emotion value for each utterance. As a result, the speech recognition device 10 generates the above-mentioned speech recognition result and emotion value as well as an embedded vector each time utterance 1 to utterance t-1 is generated, and stores these in the embedded vector storage unit 6.

[0048] The utterance importance inference unit 23 in this embodiment then calculates the importance of the embedding vectors corresponding to each of utterances 1 to t-1 stored in the embedding vector storage unit 6 as time passes, based on the emotion values ​​and time information of each utterance also stored in the embedding vector storage unit 6, and weights each embedding vector. This is because the importance of an utterance changes over time. The utterance content combination unit 24 then combines the weighted embedding vectors into a tensor. Note that the weighting and combination of the embedding vectors by the utterance importance inference unit 23 and the utterance content combination unit 24 may be performed at regular time intervals or at a preset timing, or may be performed at the timing when the utterance t to be recognized occurs.

[0049] Then, when the speech recognition unit 3 acquires a new utterance t to be the target of speech recognition, it performs speech recognition using the latest tensors generated over time as described above, and outputs the speech recognition result t.

[0050] As described above, according to this embodiment, speech recognition can be performed using tensors to which appropriate importance has been assigned over time, thereby further improving the accuracy of speech recognition.

[0051] <Fourth Embodiment> Next, a fourth embodiment of the present disclosure will be described with reference to Fig. 8 to Fig. 9. Fig. 8 to Fig. 9 are block diagrams showing the configuration of a speech recognition device in the fourth embodiment. Note that this embodiment shows an outline of the configuration of the speech recognition device described in the above-mentioned embodiments.

[0052] First, the hardware configuration of the speech recognition device 100 of this embodiment will be described with reference to Fig. 8. The speech recognition device 100 is configured as a general information processing device, and is equipped with the following hardware configuration, for example: - CPU (Central Processing Unit) 101 (arithmetic unit) - ROM (Read Only Memory) 102 (storage device) - RAM (Random Access Memory) 103 (storage device) - Programs 104 loaded into RAM 103 - Storage device 105 storing programs 104 - Drive device 106 for reading and writing data from and to a storage medium 110 external to the information processing device - Communication interface 107 for connecting to a communication network 111 external to the information processing device - Input / output interface 108 for inputting and outputting data - Bus 109 for connecting the various components

[0053] 8 shows an example of the hardware configuration of the information processing device that is the speech recognition device 100, and the hardware configuration of the information processing device is not limited to the above-described case. For example, the information processing device may be configured with a part of the above-described configuration, such as excluding the drive device 106. Furthermore, the information processing device may use a GPU (Graphics Processing Unit), a DSP (Digital Signal Processor), an MPU (Micro Processing Unit), an FPU (Floating Point Number Processing Unit), a PPU (Physics Processing Unit), a TPU (Tensor Processing Unit), a quantum processor, a microcontroller, or a combination thereof, instead of the above-described CPU.

[0054] The speech recognition device 100 can be equipped with the conversion unit 121, weighting unit 122, and recognition unit 123 shown in Fig. 9 by having the CPU 101 acquire and execute the program group 104. The program group 104 is stored in advance in the storage device 105 or the ROM 102, for example, and is loaded into the RAM 103 and executed by the CPU 101 as needed. The program group 104 may be supplied to the CPU 101 via the communication network 111, or may be stored in advance in the storage medium 110, and the drive device 106 may read out the program and supply it to the CPU 101. However, the conversion unit 121, weighting unit 122, and recognition unit 123 described above may be constructed using dedicated electronic circuits for realizing such means.

[0055] The conversion unit 121 converts an utterance by a speaker into a feature vector. For example, the conversion unit 121 performs a process of converting variable-length data of each utterance into a fixed-length embedding vector space that reflects the content of the corresponding utterance.

[0056] The weighting unit 122 weights the feature vector with the importance of the utterance based on situation information that indicates the situation at the time of the speaker's utterance. At this time, the situation information is, for example, the speaker's emotion or time information of the utterance. For example, the weighting unit 122 estimates the speaker's emotion from the feature amount of the utterance. Then, the weighting unit 122 calculates the importance of the utterance based on the emotion and time information, and assigns this importance to the feature vector of the utterance.

[0057] The recognition unit 123 recognizes a new utterance from the speaker based on the weighted feature vector.

[0058] With the above-described configuration, the present disclosure can generate feature vectors representing the characteristics of utterances to which importance is assigned according to the circumstances of each utterance by a speaker, such as the time of the utterance or the emotion of the speaker. By performing speech recognition using the feature vectors to which importance is assigned, speech recognition is performed taking into account important information from the past, thereby improving the accuracy of speech recognition.

[0059] The above-described program can be stored and supplied to a computer using various types of non-transitory computer-readable media. Non-transitory computer-readable media include various types of tangible storage media. Examples of non-transitory computer-readable media include magnetic recording media (e.g., flexible disks, magnetic tapes, hard disk drives), magneto-optical recording media (e.g., magneto-optical disks), CD-ROMs (Read Only Memory), CD-Rs, CD-R / Ws, and semiconductor memories (e.g., mask ROMs, PROMs (Programmable ROMs), EPROMs (Erasable PROMs), flash ROMs, and RAMs (Random Access Memory)). The program may also be supplied to a computer by various types of transitory computer-readable media. Examples of transitory computer-readable media include electrical signals, optical signals, and electromagnetic waves. The transitory computer-readable media can be supplied to a computer via wired communication paths such as electric wires and optical fibers, or via wireless communication paths.

[0060] Although the present disclosure has been described above with reference to the above-described embodiments, the present disclosure is not limited to the above-described embodiments. Various modifications that are understandable to those skilled in the art can be made to the configuration and details of the present disclosure within the scope of the present disclosure. Furthermore, at least one or more of the functions of the conversion unit 121, weighting unit 122, and recognition unit 123 described above may be executed by an information processing device installed and connected anywhere on a network, i.e., may be executed by so-called cloud computing.

[0061] <Supplementary Notes> Some or all of the above embodiments can also be described as in the following supplementary notes. Below, an outline of the configurations of the speech recognition method, speech recognition device, and program according to the present disclosure will be described. However, the present disclosure is not limited to the following configurations. (Supplementary Note 1) A speech recognition device comprising: a conversion unit that converts an utterance by a speaker into a feature vector; a weighting unit that weights the feature vector with an importance of the utterance based on situation information that represents a situation at the time of the utterance by the speaker; and a recognition unit that recognizes a new utterance by the speaker based on the weighted feature vector. (Supplementary Note 2) The speech recognition device according to Supplementary Note 1, wherein the weighting unit estimates an emotion of the speaker at the time of speaking based on the situation information, calculates the importance of the utterance based on the emotion, and weights the feature vector with the importance. (Supplementary Note 3) The speech recognition device according to Supplementary Note 2, wherein the weighting unit estimates an emotion of the speaker from the speaker's voice or video, which is the situation information, calculates the importance of the utterance based on the emotion, and weights the feature vector with the importance. (Supplementary Note 4) The speech recognition device according to Supplementary Note 2 or 3, wherein the weighting unit calculates the importance so that the greater the degree of a preset emotion when the speaker speaks, the greater the importance. (Supplementary Note 5) The speech recognition device according to any of Supplements 2 to 4, wherein the weighting unit calculates the importance of speech based on time information when the speaker spoke, which is the situation information, and weights the feature vector with the importance. (Supplementary Note 6) The speech recognition device according to Supplementary Note 5, wherein the weighting unit calculates the importance so that the further in time the speaker speaks, the smaller the importance. (Supplementary Note 7) The speech recognition device according to Supplementary Note 5 or 6, wherein the weighting unit calculates the importance of the utterance based on the time information each time time passes, and weights the feature vector with the importance. (Supplementary Note 8) The speech recognition device according to any of Supplements 1 to 7, wherein the conversion unit converts the utterance of the speaker into the feature vector based on a predetermined basis.(Supplementary Note 9) A speech recognition method comprising: converting an utterance by a speaker into a feature vector; weighting the feature vector with an importance of the utterance based on situation information that represents a situation at the time the speaker spoke; and recognizing a new utterance by the speaker based on the weighted feature vector. (Supplementary Note 10) A speech recognition method according to Supplementary Note 9, comprising: estimating an emotion of the speaker at the time of speaking based on the situation information, calculating the importance of the utterance based on the emotion, and weighting the feature vector with the importance. (Supplementary Note 11) A speech recognition method according to Supplementary Note 10, comprising: estimating an emotion of the speaker from the situation information, which is the speaker's voice or video, calculating the importance of the utterance based on the emotion, and weighting the feature vector with the importance. (Supplementary Note 12) The speech recognition method according to Supplementary Note 10 or 11, wherein the importance is calculated to be greater as the degree of a preset emotion when the speaker speaks is greater. (Supplementary Note 13) The speech recognition method according to any of Supplements 10 to 12, wherein the importance of speech is calculated based on time information when the speaker speaks, which is the situation information, and the feature vector is weighted by the importance. (Supplementary Note 14) The speech recognition method according to Supplementary Note 13, wherein the importance is calculated based on the time information when the speaker speaks, so that the earlier the time, the smaller the importance. (Supplementary Note 15) The speech recognition method according to Supplementary Note 13 or 14, wherein the importance of utterance based on the time information is calculated each time time passes, and the feature vector is weighted by the importance. (Supplementary Note 16) The speech recognition method according to any one of Supplementary Notes 9 to 15, wherein an utterance by the speaker is converted into the feature vector based on a preset basis. (Supplementary Note 17) A program for causing a computer to execute processes of converting an utterance by the speaker into a feature vector, weighting the feature vector with an importance of the utterance based on situation information that indicates a situation at the time of the speaker's utterance, and recognizing a new utterance by the speaker based on the weighted feature vector.

[0062] In addition, the present invention claims the benefit of priority based on the patent application of Japanese Patent Application No. 2022-112878 filed on July 14, 2022 in Japan, and all contents described in the patent application are included in this specification.

[0063] REFERENCE SIGNS LIST 1 Utterance acquisition unit 2 Utterance content extraction unit 3 Speech recognition unit 4 Utterance storage unit 5 Tensor storage unit 6 Embedding vector storage unit 10 Speech recognition device 21 Utterance content embedding unit 22 Emotion recognition unit 23 Utterance importance inference unit 24 Utterance content combination unit 231 Inter-utterance distance calculation unit 232 Emotion value calculation unit 233 Forgetting rate calculation unit 234 Embedding vector importance assignment unit 235 Basis transformation unit 100 Speech recognition device 101 CPU 102 ROM 103 RAM 104 Program group 105 Storage device 106 Drive device 107 Communication interface 108 Input / output interface 109 Bus 110 Storage medium 111 Communication network 121 Conversion unit 122 Weighting unit 123 Recognition unit

Claims

1. A conversion unit that converts an utterance by a speaker into a feature vector; a weighting unit that weights the feature vector with an importance of an utterance based on situation information that indicates a situation at the time of the utterance by the speaker; a recognition unit that recognizes a new utterance of the speaker based on the weighted feature vector; A voice recognition device comprising:

2. 2. The speech recognition device according to claim 1, the weighting unit estimates an emotion of the speaker at the time of speaking based on the situation information, calculates the importance of the utterance based on the emotion, and weights the feature vector with the importance. Speech recognition device.

3. 3. The speech recognition device according to claim 2, the weighting unit estimates an emotion of the speaker from the voice or video of the speaker, which is the situation information, calculates the importance of the utterance based on the emotion, and weights the feature vector with the importance. Speech recognition device.

4. 3. The speech recognition device according to claim 2, The weighting unit calculates the importance so that the importance increases as the degree of a preset emotion at the time of the speaker's speech increases. Speech recognition device.

5. 3. The speech recognition device according to claim 2, the weighting unit calculates the importance of the voice based on time information of the speaker's utterance, which is the situation information, and weights the feature vector with the importance. Speech recognition device.

6. 6. A speech recognition device according to claim 5, The weighting unit calculates the importance based on the time information of the speech of the speaker so that the importance decreases as the time becomes earlier. Speech recognition device.

7. 6. A speech recognition device according to claim 5, the weighting unit calculates the importance of the utterance based on the time information every time time passes, and weights the feature vector with the importance. Speech recognition device.

8. 2. The speech recognition device according to claim 1, The conversion unit converts the utterance of the speaker into the feature vector based on a preset basis. Speech recognition device.

9. Convert the speaker's utterance into a feature vector; weighting the feature vector with a degree of importance of the utterance based on situation information that indicates a situation at the time of the utterance by the speaker; recognizing a new utterance of the speaker based on the weighted feature vector; Speech recognition methods.

10. Convert the speaker's utterance into a feature vector; weighting the feature vector with a degree of importance of the utterance based on situation information that indicates a situation at the time of the utterance by the speaker; recognizing a new utterance of the speaker based on the weighted feature vector; A program that causes a computer to execute a process.