Audio processing method and device

Abstract

To provide a voice processing method and a device capable of determining a voice recognition result of a long voice signal.SOLUTION: A voice processing method includes the steps of: determining a target segment from a voice signal; determining a target character sequence corresponding to the target segment by decoding the target segment; identifying a part to be overlapped between the target character sequence and a preceding character sequence based on an edit distance; and merging the target character sequence and the preceding character sequence based on the overlapping part. Cost applied to the edit distance is determined by at least one among a type of work done at the edit distance, whether the characters being targets of the work are located in overlapping areas, or whether they match or not.SELECTED DRAWING: Figure 8

Description

[Technical field]

[0001] The present invention relates to an audio processing method and device. [Background technology]

[0002] Speech recognition is a technology that recognizes a user's voice. Through speech recognition, the user's speech can be converted into text. With the development of speech recognition technology, the needs of users to command devices or agents by voice are increasing, but it is difficult to guarantee optimal performance due to limited training data for speech recognition models and surrounding noise in the voice signal. Summary of the Invention [Problem to be solved by the invention]

[0003] It is an object of the present invention to provide a method and apparatus for processing audio. [Means for solving the problem]

[0004] In one embodiment, a speech processing method includes the steps of obtaining a target segment from a speech signal, determining a target character sequence corresponding to the target segment by decoding the target segment, determining a first overlapped portion between the target character sequence and a previous character sequence based on an edit distance, and merging the target character sequence and the previous character sequence based on the first overlapped portion, where a cost applied to the edit distance is determined based on any one or a combination of the following: a type of operation to be performed on the edit distance, whether a character to be operated on is located in the overlapped portion, or whether it matches.

[0005] In an embodiment of the speech processing method, the edit distance cost may be applied differently to non-overlapping portions of the target character sequence and the previous character sequence and to the first overlapping portions.

[0006] In an embodiment of the speech processing method, an insertion cost of the edit distance occurring in non-overlapping portions of the target character sequence and the preceding character sequence may be smaller than an insertion cost occurring in the first overlapped portion.

[0007] In an embodiment of the speech processing method, a matching cost of the edit distance may be less than an insertion cost of the edit distance occurring in non-overlapping portions of the target character sequence and the previous character sequence.

[0008] In an embodiment of the speech processing method, the insertion cost, the deletion cost, and the replacement cost of the edit distance occurring in the first overlapped portion may be positive numbers.

[0009] In one embodiment of the speech processing method, a matching cost of the edit distance may be a negative number, and an insertion cost of the edit distance occurring in a non-overlapping portion of the target character sequence and the previous character sequence may be 0.

[0010] In one embodiment of the audio processing method, the step of determining a target segment from the audio signal may comprise determining the target segment from the audio signal based on a length of the segment and a length of overlap between segments.

[0011] In one embodiment of the audio processing method, the length of the overlap between the segments may be determined based on available resources of a device performing the audio processing method.

[0012] In one embodiment of the speech processing method, the step of identifying the overlapped portion may include identifying a second portion that is overlapped between the part sequence extracted from the preceding character sequence and the target character sequence depending on a maximum character length that a user can speak between an overlap length between segments.

[0013] In one embodiment of the speech processing method, the previous character sequence may be determined based on a decoding result of a previous segment of the speech signal that partially overlaps with the target segment.

[0014] In one embodiment of the speech processing method, the step of identifying the overlapped portion may identify a third portion that is overlapped between the decoded result of the preceding segment included in the preceding character sequence and the target character sequence.

[0015] In one embodiment of the audio processing method, the first portion of the target segment that overlaps with the preceding segment may be less than half of the target segment.

[0016] In one embodiment, a speech processing device includes one or more processors, which determine a target segment from a speech signal, determine a target character sequence corresponding to the target segment by decoding the target segment, identify an overlapped portion between the target character sequence and a previous character sequence based on an edit distance, and merge the target character sequence and the previous character sequence based on the overlapped portion, and a cost applied to the edit distance is determined based on any one or combination of a type of operation to be performed on the edit distance, whether a character to be operated on is located in the overlapped portion, or whether it matches.

[0017] In one embodiment, a method includes determining segments from an audio signal, determining character sequences for the segments respectively by decoding the segments, identifying overlapped portions of the character sequences between adjacent character sequences based on corresponding edit distances of the adjacent character sequences, and merging adjacent ones of the character sequences based on corresponding ones of the overlapped portions. Effect of the Invention

[0018] In accordance with the present invention, an audio processing method and apparatus may be provided. [Brief description of the drawings]

[0019] [Figure 1] 1 is a diagram illustrating a neural network-based voice processing according to an embodiment; [Diagram 2] 2 is a diagram illustrating a voice recognition process according to an embodiment; [Diagram 3] 4 is a flowchart illustrating a voice recognition process according to an embodiment. [Figure 4] FIG. 1 illustrates an example of merging character sequences according to one embodiment. [Diagram 5] FIG. 1 illustrates an example of merging character sequences according to one embodiment. [Figure 6] FIG. 1 illustrates an example of merging character sequences according to one embodiment. [Figure 7] 1 is a diagram illustrating an example for explaining an edit distance according to an embodiment; [Figure 8] 1 is a flowchart illustrating an audio processing method according to an embodiment. [Figure 9] FIG. 1 is a diagram illustrating a sound processing device according to an embodiment. [Figure 10] FIG. 1 is a diagram illustrating an example of a sound processing device according to an embodiment. [Figure 11]FIG. 1 is a diagram illustrating an example of a sound processing device according to an embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0020] The specific structural or functional description of the embodiment is disclosed for illustrative purposes only and may be modified in various forms. Therefore, the embodiment is not limited to the specific disclosed form, and the scope of the present specification includes modifications, equivalents, or alternatives within the technical spirit.

[0021] Although terms such as first or second may be used to describe multiple components, such terms should be construed as being solely for the purpose of distinguishing one component from the other components. For example, a first component may be named as a second component, and similarly, a second component may also be named as a first component.

[0022] The singular expression includes the plural expression unless the context clearly indicates otherwise. In this specification, the terms "comprise" or "have" indicate the presence of a feature, shape, step, operation, component, part, or combination thereof described in the specification, and should be understood as not precluding the presence or addition of one or more other features, shape numbers, steps, operations, components, parts, or combinations thereof.

[0023] Unless otherwise defined, all terms used herein, including technical or scientific terms, have the same meaning as commonly understood by a person of ordinary skill in the art to which the present invention belongs. Commonly used predefined terms should be interpreted as having a meaning consistent with the meaning they have in the context of the relevant art, and should not be interpreted as ideal or overly formal unless expressly defined in this specification.

[0024] Hereinafter, the embodiments will be described in detail with reference to the accompanying drawings. In the description of the embodiments, if a detailed description of related prior art is determined to unnecessarily obscure the gist of the present invention, the detailed description will be omitted.

[0025] FIG. 1 is a diagram illustrating a neural network-based voice processing according to an embodiment.

[0026] Referring to FIG. 1, a neural network for speech processing according to an embodiment includes an encoder 110 and a decoder 120. The speech processing device can use the encoder 110 and the decoder 120 to perform speech recognition on a user's speech signal 130 and determine a character sequence 140 as a result. The encoder 110 encodes the given speech signal 130 to generate context information of the speech signal 130, and the decoder 120 can output the progressively recognized character sequence 140 as a recognition result based on decoding based on the context information. In the decoder 120, a word at a particular point in time is provided as an input at the next point in time, and a process of inferring a word expected to come next is repeated until the recognition is completed. The encoder 110 and the decoder 120 are a speech recognition model that has been trained in advance to respond to an input of the user's speech signal 130 and output a corresponding speech recognition result.

[0027] As described above, since the voice processing includes voice recognition based on the encoder 110 and the decoder 120, a phenomenon may occur in which the recognition accuracy is low for a voice signal longer than the training data due to the limited training data of the network and the restriction of the length of the training data due to the training efficiency. To prevent this, the long voice signal can be divided into a plurality of segments, and the voice recognition is performed individually for each divided segment using the encoder 110 and the decoder 120, and the character sequences obtained as the recognition results are integrated to determine the voice recognition result of the long voice signal. This will be described in detail with reference to the drawings.

[0028] 2 and 3 are diagrams illustrating a voice recognition process according to an embodiment.

[0029] 2, the voice processing device according to an embodiment divides a long voice signal 210 into a plurality of segments 221-224 and performs voice recognition on each of the segments to determine a plurality of character sequences 231-234, and merges the plurality of character sequences 231-234 to obtain a voice recognition result 240 of the long voice signal 210. In this specification, a segment may be referred to as a chunk or a window for convenience of explanation. The long voice signal 210 may include a plurality of segments.

[0030] The speech processing device can divide the long speech signal 210 into a number of segments 221-224. For example, the speech processing device can divide the long speech signal 210 into a number of segments 221-224 based on a length A of the segments and a length B of the overlap between the segments.

[0031] In one embodiment, the overlap length B may be determined to be less than half the segment length A. As will be described in detail later, since speech recognition is performed on each of the multiple segments 221-224, the portion of each segment that overlaps with an adjacent segment may be double-decoded. By setting the overlap length B to be less than half the segment length A, it is possible to minimize computational overhead since not all portions of the long speech signal 210 need to be double-decoded. For example, the segment length A is 8 seconds, and the overlap length B between the segments is determined to be 2 seconds, but various other values ​​may be applied.

[0032] In one embodiment, the voice processing device can dynamically determine the overlap length B between segments based on available resources. Since each segment on which voice recognition is performed is divided from the long voice signal 210 based on a specific criterion (e.g., the segment length A and the overlap length B), the accuracy of voice recognition may be lower at the boundary of a segment cut in the middle of an utterance than at the center of the segment. To compensate for this, it is preferable to set the overlap length B between segments to be longer, but the longer the overlap length B, the higher the ratio of the portion of the long voice signal 210 that is decoded twice, making it difficult to avoid an increase in calculation overhead. If the available resources are sufficient and a certain level of performance is guaranteed even if the calculation overhead increases, the voice processing device can set the overlap length B to be longer than the preset threshold length. On the other hand, if the available resources are insufficient, the voice processing device can prevent performance degradation due to an increase in calculation overhead by setting the overlap length B to be shorter than the preset threshold length.

[0033] As an example, when the voice processing device is a server for voice recognition, if the number of voice recognition tasks being performed on the server is small, the overlap length B may be set to be longer than the preset threshold length, and conversely, if the number of voice recognition tasks being performed on the server is large, the overlap length B may be set to be shorter than the preset threshold length. As another example, when the voice processing device is a user terminal similar to a smartphone, if no tasks other than voice recognition tasks are being performed on the user terminal, the overlap length B may be set to be longer than the preset threshold length, and conversely, if tasks other than voice recognition tasks (e.g., mobile games, etc.) are being simultaneously performed on the user terminal, the overlap length B may be set to be shorter than the preset threshold length.

[0034] The voice processing device performs voice recognition based on the encoder-decoder described with reference to Fig. 1 for each of the multiple segments 221-224, and as a result, can determine multiple character sequences 231-234. For example, a first character sequence 231 is determined as the voice recognition result for the first segment 221, and similarly, the remaining character sequences 232-234 are determined as the voice recognition results for the second segment 222-fourth segment 224, respectively.

[0035] The voice processing device can determine a voice recognition result 240 corresponding to a long voice signal 210 by merging a plurality of character sequences 231-234. The voice processing device can identify overlapping portions between adjacent character sequences based on an edit distance and merge the adjacent character sequences based on the overlapping portions. Here, the edit distance is a technique for calculating a minimum cost of operations required to convert one character sequence into another character sequence and quantifying the degree to which two character sequences differ from each other, and the applied cost may be determined based on at least one of the type of operation (e.g., insertion, modification, deletion), the position of the character to be operated on, and whether or not they match. Since the overlapping portions are portions that have been decoded twice, one of the decoding results is selected and included in the voice recognition result 240. The process of identifying overlapping portions and merging character sequences will be described in detail later.

[0036] Referring to FIG. 3, a flowchart illustrating a speech recognition process for a long speech signal according to an embodiment is shown.

[0037] In step S310, when a voice signal is input, the voice processing device may initialize a start time t_start of a target segment for which voice recognition is to be performed to 0, and initialize a history in which voice recognition results are accumulated to an empty value. In this embodiment, the start time t_start of the target segment is initialized to 0, but the embodiment is not limited thereto.

[0038] In step S320, the audio processing device can check whether audio exists in the audio signal from the start time t_start to the length A of the segment, or whether the audio signal ends before that. Through this, the audio processing device can extract the portion from the start time t_start to the length A of the segment in the audio signal and obtain it as the target segment. Alternatively, if the audio signal ends before extracting the portion from the start time t_start to the length A of the segment, the audio processing device can obtain the portion from the start time t_start to the extracted portion as the target segment.

[0039] In step S330, the speech processing device obtains a target character sequence D, which is the result of decoding the extracted target segment.

[0040] In step S340, the voice processing device merges the target character sequence D and HISTORY, and stores the result as HISTORY. In the first case, since an empty value is stored in HISTORY, the target character sequence D obtained in step S330 is stored in HISTORY as it is. In the other case, since the previous character sequence is stored in HISTORY, the voice processing device can identify an overlapped portion between the target character sequence D and HISTORY, merge the target character sequence D and HISTORY based on the identified portion, and store the result as HISTORY. Since the overlapped portion is a portion that has been decoded twice, one of the decoding results can be selected and stored in HISTORY. In this way, the results of speech recognition of the voice signal in segments are sequentially accumulated in HISTORY. The process of merging character sequences will be described later with reference to FIG. 4 to FIG. 6.

[0041] In step S350, the audio processing device checks whether the audio signal has ended. If not, the process continues to step S360; conversely, if the audio signal has ended, the process continues to step S370.

[0042] In step S360, the audio processing device increases the start time t_start of the target segment by a shift time C. The shift time C is set to be greater than half the length A of the segment but less than the length A of the segment, thereby minimizing computation overhead. According to an embodiment, the shift time C may be dynamically determined based on available resources of the audio processing device. For example, the more sufficient the available resources are, the closer the shift time C is set to half the length A of the segment, and conversely, the more scarce the available resources are, the closer the shift time C is set to the length A of the segment.

[0043] In step S370, the speech processing device returns the character sequence accumulated in the HISTORY as the recognition result of the speech signal.

[0044] 4 to 6 are diagrams illustrating examples of merging character sequences according to an embodiment.

[0045] Referring to Figure 4, an example is shown to illustrate a process of merging a target character sequence 420 with a previous character sequence 410 according to one embodiment. Figure 4 exemplarily illustrates a first step of a speech processing method, where the previous character sequence 410 is a result of decoding a first segment extracted from the speech signal, and the target character sequence 420 is a result of decoding a second segment extracted from the speech signal, the second segment being a successor to the first segment.

[0046] The speech processing device may identify an overlapped portion 430 between the preceding character sequence 410 and the target character sequence 420 based on the edit distance. The cost applied to the edit distance may be determined based on at least one of the type of operation (e.g., insertion, modification, deletion) performed on the edit distance, whether the character to be operated on is located in the overlapped portion between the two character sequences, and whether it matches. The cost applied to the edit distance will be described in detail below with reference to FIG. 7.

[0047] As shown in FIG. 4, the preceding character sequence 410 and the target character sequence 420 are aligned at the character level based on the overlapping portion 430 between the preceding character sequence 410 and the target character sequence 420, and at this time, the edit distance between the preceding character sequence 410 and the target character sequence 420 is the least expensive.

[0048] The aligned preceding character sequence 410 and target character sequence 420 may be merged as follows.

[0049] First, since decoding is performed once for the remaining portions of the preceding character sequence 410 and the target character sequence 420, i.e., the non-overlapping portions, excluding the overlapped portion 430, the previously performed decoding result can be used as is. Meanwhile, decoding is performed twice for the overlapped portion 430, and the decoding results are included in both the preceding character sequence 410 and the target character sequence 420. Therefore, one of the decoding results of the overlapped portion 430 included in the preceding character sequence 410 and the target character sequence 420 must be selected.

[0050] For example, the voice processing device may check a word boundary in the overlapped portion 430. The word boundary indicates a space between words, and corresponds to a space between words in the result of decoding. The voice processing device may designate the confirmed word boundary as a transition candidate, and select one of the designated transition candidates located at the most center of the overlapped portion 430 as a transition point. The voice processing device may merge the preceding character sequence 410 and the target character sequence 420 by connecting the words from the beginning to the transition point in the preceding character sequence 410 and the words from the transition point to the end in the target character sequence 420. The boundaries of each segment correspond to a portion where the voice signal is interrupted, and thus have a lower recognition accuracy than the center of the segment. Therefore, the preceding character sequence 410 and the target character sequence 420 may be merged based on the transition point located in the middle of the overlapped portion 430 to obtain a merged result with high recognition accuracy.

[0051] In the example of FIG. 4, the word boundaries in the overlapped portion 430 are determined between "recognition" and "all" in the preceding character sequence 410, and between "recognition" and "algorithm" in the target character sequence 420, and the corresponding portions can be designated as conversion candidates. In this example, the conversion candidate is located in the middle of one overlapped portion 430, so it can be selected as a conversion point. Then, the word "The end-to-end speech recognition" from the beginning to the conversion point in the preceding character sequence 410 and the word "algorithm has been widely studied" from the end of the conversion point in the target character sequence 420 are connected to each other, and the merged result is determined as "The end-to-end speech recognition algorithm has been widely studied". The last word "all" in the preceding character sequence 410 is incorrectly recognized as "algorithm" as "all" due to a speech signal interruption, but based on the above-mentioned method, the incorrectly recognized word can be efficiently prevented from being included in the merged result.

[0052] The merged result determined in Fig. 4 is merged with the decoding result of the next target segment, as will be described below with reference to Fig. 5.

[0053] 5, an example is shown for explaining a process of merging a target character sequence 520 with a previous character sequence 510 according to one embodiment, where the previous character sequence 510 is the result of the merging performed with reference to FIG. 4, and the target character sequence 520 is the result of decoding a third segment extracted from the audio signal.

[0054] The speech processing device can identify the overlapped portion between the preceding character sequence 510 and the target character sequence 520 based on the edit distance. As shown in FIG. 5, depending on the case, there may be no overlapped portion between the preceding character sequence 510 and the target character sequence 520, and as a result, the preceding character sequence 510 and the target character sequence 520 may be aligned so as not to overlap with each other. As described above, the non-overlapping portion may be included in the merged result by utilizing the previously performed decoding result as it is, so that the preceding character sequence 510 and the target character sequence 520 are connected as it is, and the merged result "The end-to-end speech recognition algorithm has been widely studied and showed competitive performance to conventional speech" is determined. The merged result determined in FIG. 5 is merged with the decoding result of the next target segment. This will be described later with reference to FIG. 6.

[0055] 6, an example is shown to explain a process of merging a target character sequence 620 with a previous character sequence 610 according to one embodiment, where the previous character sequence 610 is the result of the merging performed in FIG. 5, and the target character sequence 620 is the result of decoding a fourth segment extracted from the speech signal.

[0056] The speech processing device can identify the overlapped portion 630 between the preceding character sequence 610 and the target character sequence 620 based on the edit distance. As shown in Fig. 6, the preceding character sequence 610 and the target character sequence 620 are aligned at the character level based on the overlapped portion 630, where the edit distance between the overlapped portions 630 has the lowest cost.

[0057] When identifying the overlapped portion 630 based on the edit distance according to an embodiment, the entirety of the previous character sequence 610 may be considered, but only a portion of the previous character sequence 610 may be considered according to an embodiment. As an example, since the length of characters that a user can speak between the overlap lengths between segments is limited, a portion 640 may be extracted from the previous character sequence 610 with a maximum length of characters that a user can speak. By calculating the edit distance between the portion 640 and the target character sequence 620, the overlapped portion 630 may be identified with a small amount of calculations. As another example, the previous character sequence 610 is a combination of the decoded results of the first segment to the third segment, and the decoded result 650 of the third segment, which is the last segment, may overlap with the target character sequence 620. Therefore, by calculating the edit distance between the decoded result 650 of the third segment and the target character sequence 620, the overlapped portion 630 may be identified with a small amount of calculations. In this case, the third segment is a segment immediately before the target segment, and is referred to as the previous segment for convenience of explanation.

[0058] The leading character sequence 610 and the target character sequence 620 aligned based on the overlapped portion 630 are merged in the above-mentioned manner, and the merged result is determined as "The end-to-end speech recognition algorithm has been widely studied and shown competitive performance to conventional speech recognition methods". In the example of Fig. 6, the first words of the overlapped portion 630 of the leading character sequence 610 and the target character sequence 620 are different, such as "conventional" and "convolutional", but "conventional" can be included as the merged result based on the transition points specified in the space between "conventional speech" and the space between "convolutional speech".

[0059] FIG. 7 is a diagram illustrating an example for explaining an edit distance according to an embodiment.

[0060] 7, an example for calculating the edit distance between a target character sequence and a previous character sequence according to one embodiment is shown. In the example table in FIG. 7, "proposed convolution" in the first column indicates the previous character sequence, "conventional algorithm" in the first row is the target character sequence, and " " in the first row and first column indicates a space.

[0061] According to one embodiment, edit distance is a technique for calculating the minimum cost of operations required to convert one character sequence into another character sequence and quantifying the degree to which two character sequences differ from each other. For ease of explanation, the process of calculating the minimum cost of operations may be represented in a table such as that shown in FIG. 7.

[0062] The cost applied to the edit distance is determined based on at least one of the following: the type of operation being performed in the edit distance (e.g., insertion, modification, deletion), whether the characters being operated on are located in an overlapping area, and whether they match.

[0063] The cost of the edit distance is applied differently to the non-overlapped and overlapped parts of the target character sequence and the preceding character sequence. In other words, the insertion cost of the edit distance occurring in the non-overlapped part is smaller than the insertion cost occurring in the overlapped part. For example, the insertion cost of the edit distance occurring in the non-overlapped part is 0, and the insertion cost occurring in the overlapped part is 2. In the example shown in FIG. 7, the front part 710 of the preceding character sequence is maintained the same without increasing the cost toward the bottom of the table (in other words, even if a character of the preceding character sequence is inserted) in the non-overlapped part. Also, the rear part 720 of the target character sequence is maintained the same without increasing the cost toward the right side of the table (in other words, even if a character of the target character sequence is inserted) in the non-overlapped part. On the other hand, in the rear part of the preceding character sequence and the front part of the target character sequence, which are overlapped parts, the cost increases by 2 every time a character is inserted.

[0064] Also, the insertion cost, deletion cost, and edit distance replacement cost occurring in the overlapping part may be positive numbers, and in this case, each cost is determined by tuning for the speech processing method. For example, the insertion cost and / or deletion cost of the edit distance occurring in the overlapping part may be greater than the edit distance replacement cost. For example, the insertion cost and / or deletion cost of the edit distance occurring in the overlapping part may be 2, and the edit distance replacement cost may be 1. This is because the case where a specific character is omitted from the recognition result is recognized as a larger error than the case where a specific character is recognized as a different character due to noise, etc., during the speech recognition process.

[0065] Also, the cost of matching edit distances may be less than the cost of inserting edit distances occurring in non-overlapping portions. In other words, the matching cost is set to a negative number (e.g., -1) and is less than the cost of inserting edit distances occurring in non-overlapping portions, which is 0. In the example shown in FIG. 7, the cost decreases each time a match occurs in overlapping portion 730.

[0066] When the cost according to the above description is applied to calculate the edit distance, the minimum operation cost can be calculated in terms of the overall connection from the point in the upper left section to the end point in the lower right section in the table shown in FIG. 7. In the example shown in FIG. 7, "convolution" is identified as an overlapped portion in the preceding character sequence "proposed convolution", and "conventional" is identified as an overlapped portion in the target character sequence "conventional algorithm". In this way, even if the front part of the target character sequence and the rear part of the preceding character sequence do not completely match, they can be identified as overlapped portions based on the minimum operation cost of the edit distance. As described above, the edit distance is calculated on a character-by-character basis, which can be versatile and applicable to any language.

[0067] FIG. 8 is a flowchart illustrating a voice processing method according to an embodiment.

[0068] Referring to FIG. 8, there is shown a voice processing method performed by a processor included in a voice processing device according to an embodiment.

[0069] In step S810, the audio processing device obtains a target segment from the audio signal. The audio processing device can obtain the target segment from the audio signal based on the length of the segment and the length of the overlap between the segments. The length of the overlap between the segments can be determined based on the available resources of the device performing the audio processing method.

[0070] In step S820, the speech processing device obtains a target character sequence corresponding to the target segment by decoding the target segment.

[0071] In step S830, the speech processing device identifies an overlapped portion between the target character sequence and the preceding character sequence based on the edit distance. The cost applied to the edit distance is determined based on at least one of the type of operation performed on the edit distance, whether the character to be operated on is located in the overlapped portion, and whether it matches. The edit distance cost may be applied differently to non-overlapping and overlapping portions of the target character sequence and the preceding character sequence. The matching cost of the edit distance may be negative, the insertion cost of the edit distance occurring in the non-overlapping portion may be zero, and the replacement cost of the edit distance and the insertion cost occurring in the overlapping portion may be positive.

[0072] In step S840, the speech processor merges the target character sequence and the preceding character sequence based on the overlapped portions.

[0073] The matters described with reference to FIGS. 1 to 7 can be applied to the matters described with reference to FIG. 8, and therefore detailed description thereof will be omitted.

[0074] FIG. 9 is a diagram illustrating a voice processing device according to an embodiment.

[0075] 9, an audio processing device 900 according to an embodiment includes a memory 910, a processor 920, and an input / output interface 930. The memory 910, the processor 920, and the input / output interface 930 can communicate with each other via a bus 940.

[0076] The memory 910 includes computer readable instructions. The processor 920 can perform the above operations by executing the instructions stored in the memory 910. The memory 910 may be a volatile memory or a non-volatile memory.

[0077] The processor 920 is one or more devices that execute instructions or programs and control the audio processing device 900, and may include, for example, a central processing unit (CPU) and a graphic processing unit (GPU). The audio processing device 900 can connect to other devices (for example, a microphone or an external device that acquires a user's voice signal) through an input / output interface 930 to exchange data. In addition, the audio processing device 900 can process the above-mentioned operations.

[0078] User terminals include various computing devices such as smartphones, tablets, laptops, and personal computers, various wearable devices such as smart watches and smart glasses, various home appliances such as smart speakers, smart TVs, and smart refrigerators, smart automobiles, smart kiosks, and Internet of Things (IoT) devices.

[0079] 10 and 11 are diagrams for explaining an example of a sound processing device according to an embodiment.

[0080] Referring to FIG. 10, the audio processing device according to an embodiment can be realized as a server 1000.

[0081] The server 1000 is a separate device separate from a user terminal controlled by a user, and can communicate with the user terminal through a wired and / or wireless network. A voice signal of a user is collected by the user terminal and transmitted to the server 1000 through a network, and the server 1000 can recognize the voice signal using the voice recognition model 1010 based on the above-mentioned voice processing method. At this time, the voice recognition model 1010 may be used. Then, the server 1000 can return the result of the voice recognition to the user terminal. For example, the user terminal includes various computing devices such as smartphones, tablets, laptops, and personal computers, various wearable devices such as smart watches and smart glasses, various home appliances such as smart speakers, smart TVs, and smart refrigerators, smart automobiles, smart kiosks, Internet of Things (IoT) devices, and the like.

[0082] The user terminal may simply provide the recognition result received from server 1000 to the user, or may perform a subsequent action based on the recognition result. For example, the subsequent action may include, without limitation, outputting response information to a user question such as asking about the weather, executing a command in response to a user command such as playing music, inputting text into an input field displayed on the user terminal, translating or interpreting the results of the speech recognition into another language, and the like.

[0083] Referring to FIG. 11, a voice processing device according to an embodiment may be realized as a user terminal 1100. In FIG. 11, the user terminal 1100 is illustrated as a smartphone for convenience of explanation, but the present invention may be applied to any other device controlled by a user without limitation. The user terminal 1100 may directly obtain a voice signal from a user and recognize the voice signal using a voice recognition model 1110 according to the above-mentioned voice processing method. Then, the user terminal 1100 may simply provide the recognition result to the user, or may perform a subsequent operation based on the recognition result.

[0084] The above-described embodiments may be implemented using hardware components, software components, or a combination of hardware and software components. For example, the devices and components described herein may be implemented using one or more general-purpose or special-purpose computers, such as a processor, controller, arithmetic logic unit (ALU), digital signal processor, microcomputer, field programmable array (FPA), programmable logic unit (PLU), microprocessor, or other device that executes and responds to instructions.

[0085] The software may include computer programs, codes, instructions, or any combination of one or more thereof, which may configure or instruct the processing device to operate as desired, either independently or in combination. The software and / or data may be embodied in any type of machine, component, physical device, virtual device, computer storage medium or device, or transmitted signal wave, either permanently or temporarily, to be interpreted by the processing device or to provide instructions or data to the processing device. The software may be distributed across computer systems coupled to a network, and may be stored and executed in a distributed manner. The software and data may be stored on one or more computer readable recording media.

[0086] The method according to the present invention may be embodied in the form of program instructions to be executed by various computer means and recorded on a computer-readable recording medium. The recording medium may include program instructions, data files, data structures, and the like, alone or in combination. The recording medium and the program instructions may be specially designed and constructed for the purposes of the present invention, or may be known and available to those skilled in the art of computer software. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tapes, optical recording media such as CD-ROMs, DVDs, magneto-optical media such as floptical disks, and hardware devices specially configured to store and execute program instructions, such as ROM, RAM, flash memory, and the like. Examples of program instructions include not only machine code, such as produced by a compiler, but also high-level language code executed by a computer using an interpreter, and the like. The hardware devices may be configured to operate as one or more software modules to perform the operations shown in the present invention, and vice versa.

[0087] As described above, even if the embodiments have been described by limited drawings, those having ordinary skill in the art may apply various technical modifications and variations based on the above description. For example, the described techniques may be performed in a different order from the described method, and / or the components of the described system, structure, device, circuit, etc. may be combined or combined in a different form from the described method, or may be replaced or substituted by other components or equivalents to achieve appropriate results. [Explanation of symbols]

[0088] 110 Encoder 120 Decoder 130 Audio Signal 140 character sequence 210 Long voice signal 221, 222, 223, 224 segments 231, 232, 233, 234 character sequences 410 Leading Character Sequence 420 Target Character Sequences 430 Overlapped area

Claims

1. obtaining a target segment from the audio signal; determining a target character sequence corresponding to the target segment by decoding the target segment; determining a first portion of overlap between the target character sequence and a previous character sequence based on an edit distance; merging the target character sequence and the preceding character sequence based on the overlapped first portion; Including, The cost applied to the edit distance is determined based on one or a combination of the following: the type of work performed on the edit distance, whether the character to be worked on is located in the overlapping portion, or whether it matches; Audio processing methods.

2. the edit distance cost is applied differentially to non-overlapping portions of the target character sequence and the previous character sequence and to the first overlapped portions.

2. The audio processing method according to claim 1.

3. an insertion cost of the edit distance occurring in non-overlapping portions of the target character sequence and the preceding character sequence is less than an insertion cost occurring in the first overlapped portion; 2. The audio processing method according to claim 1.

4. a matching cost of the edit distance is less than an insertion cost of the edit distance occurring in non-overlapping portions of the target character sequence and the previous character sequence; 2. The audio processing method according to claim 1.

5. The insertion cost, the deletion cost, and the replacement cost of the edit distance occurring in the overlapped first portion are positive numbers.

2. The audio processing method according to claim 1.

6. The edit distance matching cost is a negative number; the cost of insertion of the edit distance occurring in non-overlapping portions of the target character sequence and the previous character sequence is zero; 2. The audio processing method according to claim 1.

7. determining a target segment from the audio signal based on a length of the segment and a length of overlap between the segments; 2. The audio processing method according to claim 1.

8. the length of the overlap between the segments is determined based on available resources of a device performing the audio processing method; 8. The audio processing method according to claim 7.

9. and identifying the overlapped portion includes identifying a second overlapped portion between the part of the preceding character sequence extracted from the target character sequence and the target character sequence according to a maximum character length that the user can speak between the overlap lengths of the segments.

2. The audio processing method according to claim 1.

10. the preceding character sequence being determined based on a decoding result of a preceding segment of the speech signal that is partially overlapped with the target segment; 2. The audio processing method according to claim 1.

11. and identifying a third portion that is overlapped between the decoding result of the previous segment included in the previous character sequence and the target character sequence. The audio processing method according to claim 10.

12. the first portion of the target segment that overlaps with the preceding segment is less than half of the target segment; The audio processing method according to claim 10.

13. A computer readable storage medium having a program recorded thereon for executing the method according to any one of claims 1 to 12.

14. one or more processors; The one or more processors: determining a target segment from the audio signal; determining a target character sequence corresponding to the target segment by decoding the target segment; identifying an overlap between the target character sequence and a previous character sequence based on an edit distance; merging the target character sequence and the preceding character sequence based on the overlapped portion; The cost applied to the edit distance is determined based on one or a combination of the following: the type of work performed on the edit distance, whether the character to be worked on is located in the overlapping portion, or whether it matches; Audio processing device.

15. the edit distance cost is applied differentially to non-overlapping and overlapping portions of the target character sequence and the previous character sequence; 15. The audio processing device according to claim 14.

16. a matching cost of the edit distance is less than an insertion cost of the edit distance occurring in non-overlapping portions of the target character sequence and the previous character sequence; 15. The audio processing device according to claim 14.

17. The insertion cost, the deletion cost, and the replacement cost of the edit distance occurring in the overlapped portion of the target character sequence and the preceding character sequence are positive numbers.

15. The audio processing device according to claim 14.

18. the one or more processors determine the target segment from the audio signal based on a length of the segment and a length of overlap between the segments.

15. The audio processing device according to claim 14.

19. the length of the overlap between the segments is determined based on available resources of a device performing audio processing; 20. The audio processing device of claim 18.

20. the preceding character sequence being determined based on a decoding result of a preceding segment of the speech signal that is partially overlapped with the target segment; 15. The audio processing device according to claim 14.

21. determining segments from the audio signal; determining character sequences for said segments by decoding said segments respectively; identifying overlapped portions of the character sequences between adjacent character sequences based on corresponding edit distances of the adjacent character sequences; merging adjacent ones of the character sequences based on corresponding ones of the overlapped portions; Including, The cost applied to each of the corresponding edit distances is determined based on one or a combination of the following: a type of operation to be performed at the corresponding distance among the edit distances; whether the character to be operated on is located in an overlapping portion; or whether it matches; method.

22. a cost applied to each of the corresponding edit distances is applied differently to a non-overlapping portion of the character sequence between adjacent character sequences and an overlapping portion of the character sequence; 22. The method of claim 21.

23. The insertion costs for the corresponding edit distances of the non-overlapping portion and the overlapping portion are different from each other.

22. The method of claim 21.

24. The insertion cost, deletion cost, and replacement cost for the edit distance occurring in the overlapped portion are positive numbers.

22. The method of claim 21.

25. determining segments from the audio signal based on a length of each segment and a length of overlap between adjacent segments; 22. The method of claim 21.

26. the overlap length between adjacent segments is determined based on available resources of a device performing the method; 26. The method of claim 25.

Images