Speech context-based decoder correction

Abstract

Disclosed in some examples are methods, systems, and machine-readable media for utilizing contextual information to create decoding feedback information to improve decoder accuracy and / or performance. In some examples, the context information is from a layer of the network stack above a layer in which the decoder is located. The context information may be or based on information about previously received and decoded data and / or information about the sender to provide decoding feedback information to the decoder that is both used to correct previous decoding errors, as well as to inform the decoder about which of the plurality of decoding selections is more likely to be correct. This may improve decoding performance by reducing errors, and in some examples, reduce the complexity of selection and thus improve decoder efficiency by excluding certain decoding possibilities.

Description

Technical Field

[0001] The embodiments relate to improving the performance of a decoder that decodes encoded data. Some embodiments relate to improving decoder performance by using feedback data created based on contextual information about the data being decoded. Background Technology

[0002] Interference, multipath fading, and other effects can alter data during data transmission or storage. To prevent these alterations, computing systems typically employ one or more encoding schemes that apply one or more codes to the data before data transmission. The receiving device uses a corresponding decoding scheme to decode the data and detect and / or correct errors introduced during the data's transmission or storage. Similarly, computing devices can apply one or more codes to data before storing it in a data storage device. When this data is read from the storage device, the device applies a corresponding decoding scheme to decode the data and detect and / or correct errors introduced by storing or reading the data from the storage medium. Example encoding schemes that allow for error detection and / or correction include convolutional codes, turbo codes, etc. Attached Figure Description

[0003] In accompanying drawings that are not necessarily drawn to scale, the same reference numerals may describe similar components in different views. Similar numbers with different letter suffixes may indicate different instances of similar components. The accompanying drawings illustrate, by way of example and not limitation, the various embodiments discussed in this document.

[0004] Figure 1 The system shown is based on some examples of this disclosure.

[0005] Figure 2 Examples of using text context information to determine decoding feedback information to improve decoder performance are shown according to this disclosure.

[0006] Figure 3 Examples of using text context information to determine decoding feedback information to improve decoder performance are shown according to this disclosure.

[0007] Figure 4 A flowchart illustrating a method 400 for improving decoder performance using text context information, according to some examples of this disclosure.

[0008] Figure 5 Examples of using speech context information to determine decoding feedback information to improve decoder performance are shown according to this disclosure.

[0009] Figure 6Examples of using speech context information to determine decoding feedback information to improve decoder performance are shown according to this disclosure.

[0010] Figure 7 Examples of using speech context information to determine decoding feedback information to improve decoder performance are shown according to this disclosure.

[0011] Figure 8 Examples of using speech context information to determine decoding feedback information are shown according to this disclosure to improve the performance of the decoder in environments with multiple speakers.

[0012] Figure 9 A flowchart illustrating a method for improving decoder performance using speech context information, according to some examples of this disclosure.

[0013] Figure 10 Examples of using object identification information to determine decoding feedback information to improve decoder performance are shown according to this disclosure.

[0014] Figure 11 Examples of using object identification information to determine decoding feedback information to improve decoder performance are shown according to this disclosure.

[0015] Figure 12 A flowchart illustrating a method 1200 for improving decoder performance using object recognition, according to some examples of this disclosure.

[0016] Figure 13 The diagram illustrates a multi-rate decoding scenario employing edge buffering, according to some examples of this disclosure.

[0017] Figure 14 The following examples, based on this disclosure, illustrate the use of future user edits of stored data to determine decoding feedback information in order to improve decoder performance.

[0018] Figure 15 Examples of using prediction information as feedback information to the channel decoder are shown according to this disclosure.

[0019] Figure 16 Examples of using correction information to update the decision grid of the decoder are shown, according to some examples of this disclosure.

[0020] Figure 17 An improved decoder system is shown, based on some examples of this disclosure.

[0021] Figure 18 This is a block diagram illustrating an example of a machine on which one or more embodiments can be implemented.

[0022] Figure 19A flowchart illustrating a method for utilizing predictive feedback information according to some examples of this disclosure is shown. Detailed Implementation

[0023] One or more different types of codes can be applied to data before transmission or storage, including source coding and channel coding. Source coding can first use first codes, such as video codecs, audio codecs, compression schemes, etc., to process the raw data to create source-coded data. These source codes may include some error correction and / or detection. The source-coded data is then protected by channel coding applied by the channel encoder. Example channel codes include forward error correction codes, such as convolutional codes, turbo codes, Reed-Solomon codes, etc. At the receiver, the received data is first decoded using a channel decoder corresponding to the channel encoder. Then, the output of the channel decoder is source-decoded, where it is decoded into application data using a source decoder corresponding to the source encoder. The application data may undergo one or more of the following: further processing by the application, output to the user, or storage on a storage device.

[0024] Channel coding typically maps a sequence of n bits or symbols to another sequence of y coded bits or symbols, where y > n. For example, channel-coded data can be created by mapping a four-bit source-coded bit sequence "0101" to a twelve-bit sequence 001111011110 using a codebook. This channel-coded data can then be transmitted over a communication medium or stored on a storage device. If one or more bits change, the original data can be recovered by selecting the code closest to the received data from the codebook. That is, the codeword with the smallest Hamming distance to the received bit sequence is selected. If the code is carefully selected, each possible bit change has a definite most probable codeword. Channel coding is essentially a trade-off between throughput and accuracy. While channel coding is highly effective, correct decoding can be more difficult to accurately determine if the channel has multiple errors. That is, there may be multiple decoding possibilities that are closest to the received data. Furthermore, channel encoders have memory, as a decoding error in one sequence can affect subsequent sequences.

[0025] Traditional codes operate independently of knowledge of what type of data they are encoding. That is, the encoding and decoding schemes only consider the received sequences of ones and / or zeros and the mapping between these sequences and the decoded sequences. For example, some channel decoders make decisions between two or more possible decoded bit sequences based on one or more metrics such as probability or distance. Examples include the state cost and overall path cost of the Viterbi decoder, where any particular state transition in the decoder has a cost, and the total cost of all state transitions traversing a path through the decoder is the path cost. In other decoders (e.g., turbo decoders), two different decoders generate hypotheses about the bit patterns in the payload and derive their likelihood values. These hypothetical bit patterns are compared, and if differences exist, a bit-by-bit comparison is performed, after which each decoder combines the likelihood estimate from the other decoder to form a new hypothesis. This cycle continues until the decoders arrive at the same hypothesis, a process that may take 15 to 18 cycles. In either case, because the decoder's previous decisions influence subsequent decisions, the decoder can be said to have "memory."

[0026] The inventors of this disclosure have recognized that upper layers, such as the application layer, have contextual data that can be utilized in a modified channel decoder to decode data more accurately and / or faster by correcting the decoder's memory. This contextual data can be based on what the data represents and can be used by the decoder to select from multiple potential bitstreams (e.g., update the likelihood of a certain decoder); update path costs; prune one or more erroneous decisions from the decision grid; and so on. Contextual information can be used to improve the initial decoding of data and / or to improve the decoding of subsequent data.

[0027] For example, if the transmitted text message is "tasty sea salt," but is received as "tasty see salt" at the receiver, the application layer can use contextual information—the surrounding words—to correct "see" to "sea." This contextual information is fed back to the decoder as decoding feedback, allowing for mesh trimming, path cost updates, and better decoding of subsequent data. This decoding feedback could be the corrected data (e.g., the word "sea") that should have been received (e.g., the source- and channel-coded word "sea"). As another example, word distribution information about the sender's previous messages can be provided to the decoder as decoding feedback to indicate which of two possible decodings is more likely. For example, a turbo decoder could use this information to adjust the probability of each choice, biasing it towards bit sequences representing words more frequently used by the sender. This word distribution information can also be used to increase or decrease one or more path costs for the Viterbi decoder, making the decoder more inclined to select words more frequently used by the sender.

[0028] Methods, systems, and machine-readable media for leveraging contextual information to create decoding feedback information to improve decoder accuracy and / or performance are disclosed in some examples. In some examples, the contextual information originates from a layer of the network stack above the layer where the decoder is located. The contextual information can be, or is based on, information about previously received and decoded data and / or information about the sender, to provide decoding feedback information to the decoder to correct previous decoding errors or to inform the decoder about which of several decoding options is more likely to be correct. This can improve decoding performance by reducing errors, and in some examples, by reducing the complexity of selection and thus improving decoder efficiency by eliminating certain decoding possibilities.

[0029] Decoding feedback information can be information other than the incoming channel-coded data, which can be used by the channel decoder to assist the decoder in determining the current or future decoding. In some examples, the decoder is the channel decoder, but in other instances, the source decoder may also benefit from such contextual information. The disclosure used herein provides numerous examples of using contextual information to assist channel decoder performance. Those skilled in the art who benefit from this disclosure will understand that the same techniques can be applied to source decoding.

[0030] Decoding feedback information can be created based on contextual information such as previously decoded data, previous historical communication data, etc. Decoding feedback information can be created as a natural result of processing received data for use in an application, for example, and in other examples, it can be specifically created to assist the decoder. In some examples, the decoding feedback information can be directly contextual information, but in others, it can be created based on contextual information such as the contextual information of the source and / or channel coding. As previously mentioned, the feedback information can be an example of an incorrect decoder selection and can provide a corrected selection, or it can be information about which of several possible decoding choices is more likely.

[0031] Examples of contextual information can vary depending on the data being decoded. For text data, such as instant messages or text messages, contextual information can include spell-corrected text from the message or document being transmitted. Spell checking can use context-sensitive spell checking, which determines that a word is misspelled based on a dictionary and the surrounding context, to detect the use of an incorrect version of a word (e.g., "see" instead of "sea"). Other contextual information can include grammar checking. In other examples, the system can determine a probability distribution of the sender's word choices. For example, a user may have a limited vocabulary and may use some words more frequently than others. This probability distribution can be used by a spell or grammar checker to determine the most likely word. In some other examples, contextual information can be predicted text. For example, given an initial seed phrase consisting of one or more words, text messaging applications often use predicted text to predict what the sender wants to say. In this disclosure, predicted text can be used to predict the next possible decoder output. That is, the decoder can decode one or more seed phrases, and the receiver can predict the next word or phrase. In some examples, predicted text can also utilize a specific sender's word or phrase probability distribution. To improve the performance of channel decoding, in some examples, contextual information such as corrected text may be source-coded and / or channel-coded to generate feedback information. This feedback information is then used by the decoder, as described herein.

[0032] For speech data, the speaker possesses unique speech characteristics. In some examples, these characteristics can be pitch, frequency, amplitude, etc. In others, they may include Mel-frequency cepstral coefficients (MFCC) data. For voice communication applications, the speaker can be identified, and the received audio can be compared to those characteristics. The output of the audio codec can be compared to the stored speech characteristics to determine if the output audio falls within the speaker's speech characteristics. If the output audio does not fall within the speaker's speech characteristics, the decoder may have incorrectly decoded one or more bits. In these examples, decoder feedback information can be an indication of which segments are not within the speech characteristics. In these examples, the decoder can use this information to recalculate the path metric by increasing the path metric of these segments to rebalance subsequent decisions. For example, the path metric of the selected decoding path can be increased by a value based on how much the decoded speech deviates from the speaker's speech characteristics. In other examples, the decoded speech can be corrected to fall within the speaker's speech characteristics, and this can be used as feedback information for the decoder to adjust its path cost or probability.

[0033] In other examples, the decoder can output multiple decoding possibilities (e.g., 10 most likely decodes), and each decoding possibility can be further decoded by the source and compared with speech features. The decoder closest to the speech features can be selected. This decoder can be output to the user and / or the selection can be fed back to the decoder for decoding future data. In other examples, the decoded speech data can be automatically transcribed. The transcription can be treated as text input based on the aforementioned context and corrected using spelling and grammar changes. The transcribed and corrected text can then be input into a text-to-speech AI algorithm that uses the user's speech features to generate simulated user speech. The decoder can then use the simulated speech to correct the decoding of real audio, as described above. In other examples, predictive text can be used on the transcription to predict the user's future words. These predicted utterances from the user can then be input into a text-to-speech AI algorithm that uses the user's speech features to generate simulated speech. The decoder can then use the simulated speech to decode future real speech by informing the decoder of the probabilities of the various decoding possibilities.

[0034] For video applications, the system can leverage artificial intelligence to detect one or more objects in video frames. If object detection is unreliable, fails to detect the expected object, or detects an erroneous object, the system can use this information to determine feedback information for the decoder. For example, a deep learning neural network can use information about what errors exist in the image or what was not detected to regenerate the image. This data can then serve as feedback information to the decoder (e.g., after encoding). In other examples, the decoder can output multiple possible decodes, and the selected decoder can be the one that most accurately identifies one or more expected objects. The selected decoder can be output to the user, and feedback on which decoder is correct can be given to the decoder for decoding subsequent data. Object detection of multiple decoding possibilities can be performed in parallel. In some examples, video data can be encoded such that a base coding layer can be sent, and additional coding layers that enhance a first quality, first size, bitrate, or quality can be sent if network conditions and / or processing power are sufficient. In some examples, when object detection is unreliable, the system can request additional coding layers with extra details to assist in encoding.

[0035] In some examples, upper layers may continuously provide decoding feedback, but in other instances, the channel decoder may set one or more flags indicating that the channel is noisy (e.g., based on the number of bit errors exceeding a threshold). In these instances, the flags can trigger higher layers to provide decoding feedback information, such as a comparison of received audio with a speech profile. In other instances, the flags may allow the media codec to decide to use interpolated data instead of the raw bits transmitted by the channel decoder. For example, by passing digital samples through a low-pass filter that matches the user's speech.

[0036] As described above, the channel decoder can utilize decoding feedback information to assist decoding. For example, in the Viterbi decoder, the path metrics used to select various decoding possibilities can be adjusted based on the decoding feedback information. Penalties can be added to decoding paths that do not correspond to the decoding corresponding to the feedback information or to past paths that do not correspond to the feedback information. Alternatively, or instead of adding penalties to paths that do not correspond to the decoding feedback information, the path cost of the decoding path corresponding to the decoding feedback information can be reduced. For turbo codes, the probability metrics for various decoding possibilities can be adjusted similarly. In some examples, the amount of penalty or path reduction, or the amount of probability metric adjustment, can be based on the feedback information. For example, for word frequency distribution feedback, the amount of path adjustment can be based on the frequency with which the sender uses a particular word. Therefore, very frequently used words can have a greater reduction in the path cost corresponding to that word compared to less frequently used words.

[0037] As mentioned above, decoding feedback can correct the current state—for example, by changing or re-performing state transitions that led the decoder to an incorrect state, or the decoder can use the decoding feedback to prune one or more incorrect paths. For example, the Viterbi decoder utilizes a grid that includes valid state transitions based on the current state. This allows for faster and more accurate decoding. In some examples, the decoder may have a buffer. That is, the decoder's state can be cached, and when decoding feedback indicates an error, the state can be replayed back to the point where the error occurred, allowing the decoder to select the correct path (e.g., traversing the grid) in a second iteration.

[0038] As will be understood, this disclosure solves the technical problem of accurate data decoding by providing decoding feedback information from a higher level to the decoder. This data may contain errors due to transmission through unreliable communication channels or storage in unreliable storage devices, enabling the decoder to make more accurate decisions and, in some cases, skip one or more states. This has the technical effect of improving the performance of the computing system by reducing errors and increasing decoder efficiency. By decoding data more accurately, the performance of the computer system is improved. This can reduce network usage and congestion by reducing the retransmission of erroneously received data packets; reduce the processing time required to decode data (thereby reducing power consumption); and improve decoding accuracy.

[0039] Figure 1 System 100 is illustrated according to some examples of this disclosure. Sender 110 has a data source 112. Data source 112 can be any application that generates data for storage or transmission. An example data source 112 can be a communication application that provides one or more network-based communication modalities (e.g., voice, video, text, screen sharing, file sharing, etc.). Example communication applications include Microsoft Teams®, Skype®, Edge®, etc. Example sender 110 can include computing devices such as desktop computing devices, laptop computing devices, tablet computers, telephones, etc. Data from data source 112 can be sent to source encoder 114. Source encoder 114 can apply one or more codecs, such as voice codecs, video codecs, etc. In other instances, source encoder 114 can apply compression schemes, etc. In some examples, source encoder 114 may not be used. In some examples, source encoder 114 may be part of data source 112.

[0040] The source-coded data is then passed to channel encoder 116. Channel encoder 116 may apply one or more codes, such as forward error correction codes. Example forward error correction codes may include block codes, convolutional codes, and turbo codes. Example block codes may include Reed-Solomon codes, Gory codes, BCH codes, Hamming codes, Doviziosmotic parity codes, etc. Exemplary turbo codes include parallel concatenated convolutional codes, serial concatenated convolutional codes, repeating cumulative codes, etc. The channel-coded data is then modulated and transmitted through transmission medium 118. Transmission medium 118 may be a radio frequency medium, such as an RF channel, optical fiber, etc. In some examples, the channel-coded data may be stored on a storage medium such as magnetic storage, optical storage, or solid-state storage.

[0041] The signal transmitted by sender 110 is received and demodulated by receiver 120. The demodulated data is then processed by channel decoder 122, which decodes the channel code. Channel decoder 122 can be an iterative channel decoder, such as a Viterbi decoder. Channel decoder 122 can pass the decoded information to source decoder 124, which can perform the reverse of source coding 114. The decoded data is then passed to destination application 126, such as a communications application. The data can then be displayed, played, or otherwise conveyed to the user.

[0042] In some examples, the application may have a context-based error detection and correction component 128. The context-based error detection and correction component 128 may provide decoding feedback information 130 back to the channel decoder 122 based on the context of the data. The channel decoder may utilize the decoding feedback information 130 to improve decoding performance by: ensuring the channel decoder is in the correct state, pruning the grid, adjusting path or state weights, adjusting probabilities (e.g., adjusting the derived bit likelihood of the hypothesis), etc. In other examples, such as when the destination application 126 is storage on disk, the channel decoder 122 may re-decode the corrected information to ensure the remaining decoded data is correct and to ensure the decoder state is correct. In still other examples, the source decoder may output multiple decoding guesses, and the context-based error detection and correction component 128 may select the most probable decoding possibility and feed that selection back to the channel decoder.

[0043] In some examples, the context-based error detection and correction component 128 can provide contextual information such as word distribution frequencies, which can be source-encoded and / or channel-encoded to create decoding feedback information 130. In other instances, the context-based error detection and correction component 128 can provide corrected data as decoding feedback information 130—which can be source-encoded and / or channel-encoded. For example, the context-based error detection and correction component can utilize a spell checker (which may or may not use surrounding words and / or dialogue context), a grammar checker, the sender's word probability distribution, speech profile comparison, speech transcription checking, video object detection, etc., to create decoding feedback information. In some examples, the context-based error detection and correction component 128 can correct data, for example, by changing text, changing image pixels to match detected objects or previous frames, updating audio data to match speech characteristics, etc. This corrected data can be re-encoded using source-encoded and / or channel-encoded before being used by the decoder.

[0044] Text-based decoding Figure 2The following examples illustrate the use of textual context information to determine decoding feedback information to improve decoder performance, according to some examples of this disclosure. Received encoded data 224 is input to channel decoder 220. Encoded data 224 can be received from a storage device or from a transmission source. Binary data 210 is then output, processed by source decoder 221 to produce the phrase “Tasty see salt” 211, which is then passed to higher layers, including a context-based error detection and correction component 212. The context-based error detection and correction component 212 can correct the word “see” to “sea” because it recognizes that the word “see” is inappropriate in the given context of discussing salt. The context-based error detection and correction component 212 can utilize spelling and / or grammar correction. In some examples, spelling and / or grammar correction can also utilize the sender's word frequency distribution to more accurately correct the phrase or grammar.

[0045] In some examples, the corrected phrase "tasty sea salt" 214 can then be source-encoded by source encoder 230 to produce source-encoded data 232. The source encoding can be the same as the source encoding applied to the data decoded by source decoder 221. The source-encoded data 232 can then be passed directly to the channel decoder as feedback information 218, or it can be channel-encoded first by channel encoder 216. Channel encoder 216 can apply the same channel encoding as that being decoded by channel decoder 220.

[0046] The channel decoder 220 can utilize feedback information to process subsequently received encoded data 224. For example, the channel decoder 220 can use the feedback information to prune the decision grid; update the state or path cost, etc. The channel decoder 220 can use the decoding feedback as an indication of what the previously decoded data should have been, and use this indication to correct the decoder state. The channel decoder 220 can use the channel-coded feedback information to recalculate the state and path cost, modify the decoding probability, and / or trim the grid. In addition to the feedback information now informing the channel decoder 220 what should have been decoded, the channel decoder 220 can utilize non-channel-coded feedback information in a similar manner. This can also be used to update path and state costs and / or trim the grid. By trimming the grid, the decision space can be reduced, thereby improving the decoder's computational performance and reducing the error rate. By updating the state and path cost, the error rate can be reduced.

[0047] Figure 3 Examples of using textual context information to determine decoding feedback information are shown according to this disclosure to improve decoder performance. Figure 2 By correcting the previously decoded information, Figure 3The predicted text information is used to predict what the next data might be. The received encoded data 324 is input to the channel decoder 320. The encoded data 324 can be received from a storage device or from the transmission source. Binary data 310 is then output, which is processed by the source decoder 321 to produce the phrase "Tasty sea salt" 311. The phrase "Tasty sea salt" 311 is then passed to a higher layer, which includes a context-based error detection and correction component 312. Given the phrase "Tasty Sea Salt" and / or other contextual information such as previous messages in the dialogue, the sender's word frequency history database, etc., the context-based error detection and correction component 312 can infer that the next phrase could be "is good" 314.

[0048] In some examples, the predicted next phrase "is good" 314 can then be source-coded by source encoder 330 to produce source-coded data 332. The source coding can be the same as the source coding applied to the data decoded by source decoder 321. The source-coded data 332 can then be passed directly to channel decoder as feedback information 318, or it can be channel-coded first by channel encoder 316. Channel encoder 316 can apply the same channel coding that channel decoder 320 is currently decoding.

[0049] The channel decoder 320 can utilize feedback information to process subsequently received encoded data 324. For example, the channel decoder 320 can use the feedback information to prune the decision grid of subsequent data; prune the state or path cost weights corresponding to the predicted next phrase, etc. Utilizing decoding feedback information in this way improves the speed and accuracy of the decoder using data prediction techniques. By reducing path costs (or subsequently increasing the path costs of probabilities that do not correspond to the predicted text), the decoding probability corresponding to the predicted text can be increased, which can improve the decoder's speed and / or accuracy. Although a single prediction result is shown, in some other examples, a list of predicted next phrases can be utilized. Each of these predicted phrases can be processed by the decoder in the same manner.

[0050] As mentioned above, context-based error detection and correction can leverage spelling and / or grammar correction. Spelling and / or grammar correction can be based on dictionaries or other sets of rules that identify spelling and / or grammar errors and provide corrections. In some examples, spelling and / or grammar correction can employ morphological analysis to consider different forms of the same word and handle incorrectly spelled words (e.g., "see" instead of "sea"). Spell checkers can use approximate string matching algorithms, such as Levenstein distance, to find the correct spelling of incorrectly spelled words; clustering algorithms; n-gram-based methods, etc. Spell checkers and / or grammar checkers can be context-sensitive. For example, machine learning-based Winnow-based spelling correction algorithms. Grammar checking algorithms can include natural language processing (NLP) algorithms, part-of-speech tagging rules utilizing regular expression-based rules, etc.

[0051] Figure 4 A flowchart illustrating method 400 for improving decoder performance using text context information according to some examples of this disclosure is provided. At operation 410, a computing device receives encoded text. For example, the encoded text may be received via a communication channel of a transmission medium (e.g., as part of a communication session) or may be loaded from storage. The transmission medium may be wired, wireless, or other media, such as via RF, optical, electrical, or other media. In some examples, the transmission medium may include an internal system bus and may receive data from storage.

[0052] At operation 412, at least a portion of the encoded text is decoded to produce a decoded text portion. The decoder may be a channel decoder and / or a source decoder, such as a Viterbi decoder and / or a turbo decoder.

[0053] At operation 414, the computing device applies correction or prediction techniques, such as generating a corrected decoded text portion by performing language checks (e.g., spell checking and / or grammar checking) on ​​the decoded text portion, or running a predictive text generator that generates subsequent text to produce the predicted next text portion of the decoded text. In some examples, spell checking, grammar checking, or text prediction utilizes contextual information to correct spelling, grammar, or predict the next text portion. Contextual information may include previous text sent by the sender in the current communication session, text sent by the sender in other communication sessions, the sender's word frequency distribution, etc. Communication sessions may include network-based communication sessions, such as network-based meetings, chat, email, etc.

[0054] At operation 416, the computing device can encode the corrected decoded text portion or the predicted next decoded text portion (source coding or channel coding or both) to generate coded feedback information.

[0055] At operation 418, the computing device uses the encoded feedback information to decode the second portion of the encoded text. For example, by utilizing the corrected text, the decoder can update the state and / or the encoded path cost. In other examples, one or more branches of the decoding grid can be updated in addition to or instead of updating the encoded path cost. In other examples, in the case of a turbo decoder, the probability of one or more of the multiple outputs can be biased by a weighting factor indicating whether the output corresponds to a correct or incorrect previous output. That is, if there are two outputs for decoding at operation 412, namely a first output corresponding to the original decoding and a second output corresponding to the corrected decoding, the probability of the first output associated with the output of the decoding of the second portion can be negatively modified (e.g., decreased), while the probability of the second output associated with the output of the decoding of the second portion can be positively modified (e.g., increased). This reflects the fact that channel conditions can change over time, but more slowly than the data rate. That is, the first portion of the data can be modified in the same way as the second portion of the data transmitted immediately following the first portion. At operation 420, the system can output the second decoded text portion, for example, to a display.

[0056] Speech-based decoding If the data being transmitted is speech data, the system can use one or more speech characteristics of one or more speakers to determine that the speaker's speech pattern does not match or otherwise does not correspond to its expected characteristics. For example, speech characteristics such as frequency, amplitude, pitch, timbre, etc., may be too high or too low beyond a threshold. In some examples, the speech characteristics may be MFCC, which is a linear cosine transform of the logarithmic power spectrum based on a nonlinear Mel frequency scale, representing the speaker's short-time power spectrum. In response to determining one or more mismatches between the speaker's speech data and its expected characteristics, the system can generate decoding cues in various ways.

[0057] For example, the system can apply one or more correction algorithms to the speech data to correct it so that the speech data accurately matches the speaker's characteristics. Examples include pitch correction and rhythm correction. The corrected data can then be used as decoding feedback information to assist the decoder. The corrected data can be encoded in the source and / or channel decoders and then used by the channel decoder as described herein.

[0058] In another example, the decoder can be modified to output multiple candidate decodes. That is, the top N possible decode possibilities can be output. For example, the N decode possibilities with the lowest path scores; the N decode possibilities with the highest probabilities, etc. Each decode possibility can be further decoded by the source and then checked against one or more speech features. These operations can be performed in parallel. The decoder that best matches one or more speech features can be selected and output to the user. Feedback information can be which decoder was selected. The decoder can then use this information as decoding feedback to update the decoder state, as described herein.

[0059] In other examples, the system can utilize transcription techniques to transcribe speech into text. The previously described text-based correction data can then be applied. A speech synthesis algorithm can be employed to generate speech data that sounds like the speaker's voice. This synthesized speech data can then be encoded (e.g., source and / or channel coding) and used as feedback information to update the decoder state, as described herein.

[0060] Figure 5 This disclosure illustrates the use of speech context information to determine decoding feedback information to improve decoder performance, according to some examples. Channel decoder 507 performs channel decoding on encoded speech data 505. For example, a Viterbi decoder and / or a turbo decoder may be used. Source decoder 509 can then decode the output of channel decoder 507. An application may output (e.g., to a speaker) the output of source decoder 509. Additionally, a context-based error detection and correction component 512, which may be part of the application, can utilize this output. Specifically, a comparison component 514 of the context-based error detection and correction component 512 may compare the decoded speech data with one or more stored speech characteristics 530 of the speaker. If the comparison indicates that the decoded speech matches one or more stored speech characteristics 530 of the speaker, corrected feedback information can be sent to channel decoder 507. If the comparison indicates that the decoded speech does not match one or more of the speaker's stored speech characteristics 530, the speech data may be corrected by correction component 516 to create corrected speech data. In some examples, the corrected voice data may then be encoded by the source encoder 518 and, alternatively, by the channel encoder 520. This data can then be passed as feedback information to the channel decoder 507. Example channel codes may include convolutional codes, cyclic redundancy check (CRC) codes, turbo codes, etc. Example source codes may include media codecs such as Scalable Video Coding (SVC), AAC, H.265, MPEG encoding, H.264, etc.

[0061] In some examples, one or more stored voice features 530 may be transmitted over a network by a network-based voice feature service 522, such as network 524 (which may be the same network or part of the same network through which encoded voice data 505 is received). In other examples, one or more stored voice features 530 may be transmitted by the speaker's computing device (e.g., at the start of a communication session—e.g., before the transmission of voice data). In other examples, previous voice data from the current communication session may be used to compute the stored voice features 530 using the feature generator component 532.

[0062] The network-based voice feature service 522 can collect voice samples from various applications. For example, whenever a user is part of a network-based communication session. Therefore, for example, Figure 5 The decoded voice data shown can also be provided to a web-based voice profiling service 522. The web-based voice profiling service 522 can use voice samples to create, update, and maintain a user's voice profile. The service can do this with the permission of the person whose voice is being profiled. For example, a user can register to participate in the service, or can otherwise set preferences to enable this feature.

[0063] Figure 6 Examples of using speech context information to determine decoding feedback information to improve decoder performance are shown according to this disclosure. Figure 6 In this example, channel decoder 607 decodes the encoded speech data 605. Channel decoder 607 outputs N most likely decoding possibilities. That is, the N decoding possibilities are determined by channel decoder 607 to be the most likely decoding possibilities. These can correspond to the N decoding possibilities with the lowest path cost in the grid, the N decoding possibilities with the highest probability, etc. N can be pre-specified or determined based on a threshold (e.g., all decoding possibilities above a certain probability threshold, or all decoding possibilities with path costs below a threshold). Source decoder 609 can process each of the N most likely decoding possibilities serially or in parallel. The N most likely decoding possibilities of the source decoding can then be tested by context-based error detection and correction component 612. Context-based error detection and correction component 612 can compare the N most likely decoding possibilities of each source decoding with one or more stored speech features 630 and select the decoding possibility that best matches one or more stored speech features (e.g., the one with the smallest error). It can output the N most likely decoders of the selected source to the user, and can pass the selected decoder (indicating which decoder was selected, or indicating the decoder itself) as decoding feedback to the channel decoder 607.

[0064] Figure 7 This disclosure illustrates the use of speech context information to determine decoding feedback information to improve decoder performance, according to some examples. A channel decoder 707 decodes encoded speech data 705. A source decoder 709 then decodes the output of the channel decoder 707. The decoded speech data is then sent to an application for output to a user, and may also be sent to a context-based error detection and correction 712. In some examples, a transcription component 713 of the context-based error detection and correction 712 may transcribe the speech data. A text correction component 714 utilizes transcription and previously described techniques (which may include predicting the next word or sentence) to correct previously uttered phrases or predict the next uttered phrase. The speech synthesis component 715 then uses this data to create synthesized speech in the speaker's voice. The source encoder 718 and channel encoder 720 may then perform source and channel encoding on the synthesized speech, respectively. This encoded synthesized speech can then be passed as feedback information to the channel decoder 707 for decoding future received encoded speech data 705. The transcription component 713 and / or the speech synthesis component 715 can utilize one or more stored speech features 730 of the speaker to transcribe and / or synthesize speech data.

[0065] Figure 8 This disclosure illustrates the use of speech context information to determine decoding feedback information to improve decoder performance in environments with multiple speakers, according to some examples. Channel decoder 807 decodes encoded speech data 805. Source decoder 809 then decodes the output of channel decoder 807. The decoded speech data is then sent to an application for output to a user. Demultiplexer component 810 then separates each speech stream into multiple individual speech tracks. Then, as previously described, context-based error detection and correction component 812 can individually correct each speech track and send each individually corrected speech track to multiplexer 814. Multiplexer 814 can remultiplex the streams. Source encoder 818 and channel encoder 820 can then perform source and channel coding on the remultiplexed audio data, which can then be passed as feedback information to channel decoder 807 for decoding of subsequently received encoded speech data 805. One or more stored voice features 830 can be used to demultiplex the stream and / or to generate corrections for the context-based error detection and correction component 812.

[0066] Figure 9A flowchart illustrating method 900 for improving decoder performance using speech context information according to some examples of this disclosure is provided. At operation 910, the system receives one or more characteristics of a speaker's speech in a communication session over a network. For example, the one or more characteristics may be sent by the speaker (e.g., prior to the communication session), determined using previously received speech from the speaker, or obtained from a network-based service. The network-based service may receive user speech data samples from various communication applications and may utilize this data to create and maintain one or more characteristics. Example characteristics may include MFCC data. Speech characteristics may be received as part of the communication session.

[0067] At operation 912, the system receives, for example, encoded voice data from a speaker during a communication session. The encoded voice data can be received from the speaker's computing device, a server device, etc. In other instances, the encoded voice data can be received from a storage device.

[0068] At operation 914, the system can use a decoder to decode the first portion of the speaker's encoded voice data to produce decoded voice data. Example decoders may include a channel decoder, a source decoder, etc.

[0069] At operation 916, the system can compare the decoded speech data with one or more received characteristics of the speaker's speech. This can be done, for example, by utilizing speaker recognition algorithms such as Support Vector Machines (SVM), neural networks such as Convolutional Neural Networks (CNN), and using characteristics (e.g., MFCC data). In some examples, if the audio data is determined to originate from a speaker identified according to the characteristics, the decoded speech data is considered to match the speech characteristics. If the audio data is determined by the algorithm to be uncertain or not originating from a speaker identified according to the characteristics, the decoded speech data is considered not to match the speech characteristics. In cases where the speech characteristics are simple, such as pitch or frequency, the audio data matches the speech characteristics when the pitch and / or frequency of the decoded audio matches the pitch and / or frequency of the speech characteristics.

[0070] At operation 918, the system can determine a mismatch between the decoded voice data and the characteristics of the received speaker's speech based on a comparison. If there is no difference, the method can terminate. If a difference exists, in response to determining a mismatch between the decoded voice data and the characteristics of the received speaker, the method can perform operations 920 and 922.

[0071] At operation 920, the system can correct the decoded speech data to match the characteristics of the speaker's voice, thereby generating corrected speech data. The system can utilize one or more machine learning algorithms, such as generative adversarial networks (GANs), neural networks such as CNNs, etc.

[0072] At operation 922, the system can be configured by the decoder to decode a second portion of the encoded voice data using the corrected voice data to create second decoded voice data, thereby eliminating at least one erroneous decoding of the second portion of the encoded voice data. In some examples, the corrected voice data may first be source- and / or channel-coded for use by the decoder. The decoder can utilize the corrected voice data by removing or pruning paths in the decoding grid, updating or modifying state and / or path values, biasing the probabilities of one or more possible decodings, etc.

[0073] Object recognition If the data being decoded is image or video data, the receiver can use object recognition algorithms to create decoding feedback. For example, the system can provide various forms of feedback to the decoder by determining that expected objects in the image were not detected, that detected objects do not have the expected characteristics, or that object detection is impossible given the quality of the decoded image.

[0074] In some examples, object recognition can be blind recognition, where recognition is based on absolute patterns without comparison to reference images. In these examples, image portfolios can be developed in real time as the audio or video stream progresses. Initially, there will be detection of general patterns, and then in future frames, these patterns are constructed into familiar clusters; in even more future frames, the image background (e.g.) can form a familiar background, which can be used to allow background detection to improve the performance of future source-coded / channel-coded frames. In other examples, object recognition can be based on comparing patterns with geometric configurations stored on a web-based service. Object recognition can look up specific, pre-specified objects in a library of the user's surrounding environment stored locally on a computing device, on a web-based service, or on an edge device. In some other examples, object recognition can be learned, where recognition is based on real-time learning from the current image sequence. For example, objects and their characteristics detected in previous frames can be searched in subsequent frames. This can be well-suited for videos with static backgrounds. In other examples, the sender and / or a web-based intermediary can perform object recognition and send the detected objects and their characteristics to the receiver. In other examples, the object can be a human face identified using facial recognition and identification technologies.

[0075] Video or frames can be corrected based on object recognition, so that the identified objects match the characteristics of the expected objects. For example, images can be corrected using machine learning techniques such as deep learning oversampling. Then, as described, the corrected image can be encoded by a decoder and used to improve decoding performance for future frames.

[0076] In other examples, deep learning supersampling frame generation techniques can be used as feedback information to determine one or more decoding steps for one or more future frames based on the current frame. For example, prediction information can be used to predict the next frame. This predicted frame can then be encoded and used by the encoder, as described herein, to improve decoding performance.

[0077] In other examples, the decoder can output multiple decoding possibilities, and object recognition can be used to select the optimal encoding from these possibilities. In other examples, the system can utilize facial recognition algorithms to provide decoding information.

[0078] Figure 10 Examples of object recognition information used according to this disclosure are shown to determine decoding feedback information to improve decoder performance. Channel decoder 1007 and source decoder 1009 can perform channel and source decoding on encoded image data 1005, respectively. The encoded image data 1005 can then be output to a user. In some examples, object detection component 1013 of context-based error detection and correction component 1012 can analyze uncoded image data. Object detection component 1013 can search for objects in the image.

[0079] The objects searched may be unknown—that is, object detection searches for any and all objects. In some examples, the library of objects to be searched is not defined, but in other examples, the library may be a predefined library of objects given by object property 1030. If no object is found, the feedback to the decoder is that the decoding is incorrect. In this example, the decoder may use this feedback to trim the mesh of the selected decode, improve the path cost of the selected decode, etc.

[0080] In other examples, object detection component 1013 searches for objects specified by a sending communication device—for example, the sending device searches for objects in an image and then sends an identifier, along with the object's characteristics as object characteristics 1030, to the receiver. The list of objects and / or their characteristics can be sent via the same communication channel as the image, or via an alternative communication channel to improve redundancy. In other examples, an intermediate server—for example, a communication server—searches for objects in an image and then sends an identifier, along with the object's characteristics, to the receiver. In other examples, the expected objects and their characteristics can be created from a previous image (e.g., a previous video frame).

[0081] In some examples, the image correction component 1014 may attempt to correct the image rather than simply providing feedback to the decoder about whether certain objects have been identified. For example, it may utilize object detection results from the object detection component 1013 by leveraging convolutional neural network (CNN) algorithms, deep learning oversampling (DLSS) algorithms, etc. In other examples, the image correction component 1014 may iteratively apply one or more denoising, anti-aliasing, or other image correction algorithms until the object detection component 1013 detects the expected object in the image. The source encoder 1018 and channel encoder 1020 can then perform source and channel coding on the data, respectively, and pass the decoding feedback information to the channel decoder 1007 for processing future data.

[0082] In some examples, for video, the frame generation algorithm can provide decoder feedback information about what the next frame in the sequence might be. That is, the context-based error detection and correction component 1012 can feed a first sequence of one or more frames to the frame generation algorithm (e.g., the DLSS 3.0 algorithm) to generate one or more predicted subsequent frames. The channel decoder 1007 can then use the predicted frames to adjust one or more path costs, trim the grid, or update one or more decoding probabilities to bias the decoding probability of predicted frames that are close to or match future data.

[0083] Figure 11 Examples of using object identification information to determine decoding feedback information to improve decoder performance are shown according to this disclosure. Figure 11In this example, channel decoder 1107 decodes the encoded image data 1105. Channel decoder 1107 outputs N most likely decoding possibilities. That is, these N decoding possibilities are determined by channel decoder 1107 to be the most likely decoding possibilities. These may correspond to the N decoding possibilities with the lowest path cost in the grid, the N decoding possibilities with the highest probability, etc. N can be pre-specified or determined based on a threshold (e.g., all decoding possibilities above a certain probability threshold, or all decoding possibilities with path costs below a threshold). Source decoder 1109 can process each of the N most likely decoding possibilities serially or in parallel. The N most likely decoding possibilities of the source decoding can then be tested by context-based error detection and correction component 1112. Context-based error detection and correction component 1112 can utilize the aforementioned object recognition process and select the object to be decoded, wherein one or more of the detected objects most closely match the expected object and / or its object characteristics 1130 (e.g., an object with the smallest error). It can output the N most likely decoders of the selected source to the user, and can pass the selected decoder (a number indicating which decoder was selected, or the decoder itself) as decoding feedback to the channel decoder 1107.

[0084] Figure 12 A flowchart illustrating method 1200 for improving decoder performance using object recognition according to some examples of this disclosure is shown. At operation 1210, the system receives encoded image data, such as video associated with a communication session, from a sender. The encoded image data can be received from a speaker's computing device, a server device, an edge device, etc. In other instances, the encoded image data can be received from a storage device.

[0085] At operation 1212, the system can decode a first portion of the first coded image to create a first decoded image portion. Example decoders may include a channel decoder, a source decoder, or both a channel decoder and a source decoder.

[0086] At operation 1214, the system can search for objects in the first decoded image portion. This can be done, for example, by searching for one or more geometric shapes, searching for objects detected in previous frames of the video stream corresponding to the sender, etc. In some examples, the objects searched may be provided by a communication server or the sender. Object recognition and search algorithms can include non-neural network-based methods combined with classifiers such as Support Vector Machines (SVMs). Examples of non-neural network-based methods include the Viola-Jones object detection framework, Scale Invariant Feature Transform (SIFT), Histogram of Oriented Gradients (HOG) features, etc. Neural network methods include Region-Based Convolutional Neural Networks (R-CNN), Single-Shot Multi-Box Detector (SSD), You Only Need to See Once (YOLO), Single-Shot Optimized Object Detection Neural Networks, etc.

[0087] At operation 1216, the system determines whether the object search at operation 1214 was successful. If the object search is successful, the operation terminates (e.g., until the next frame or image); otherwise, the operation proceeds to operations 1218-1222. In some examples, whether the object search is successful can be whether a threshold number or percentage of objects expected to be in that frame is identified, and in some examples, where the expected feature is identified. In other examples, success can be whether any object is identified. In other examples, the object can be the sender's face, and if the sender's face is identified and identified as the sender, the object search can be considered successful.

[0088] At operation 1218, if the object search fails or the object is not as expected, the system can correct the first decoded image portion. For example, an image correction algorithm based on the expected object characteristics can be used, such as by employing DLSS, XeSS, FSR, etc. In some examples, the corrected image undergoes source and / or channel decoding.

[0089] At operation 1220, the system can use the corrected coded image to decode a second portion of the coded image (e.g., a second portion of a video, etc.) or a second coded image to produce a second corrected decoded image. For example, by utilizing the corrected coded image as decoding feedback. At operation 1222, the system can output the second corrected decoded image, for example, on a display device at the receiver.

[0090] Multi-rate decoding Image, video, and audio source coding techniques are based on multi-rate transmission techniques. In this case, the CODEC rate is negotiated end-to-end. Once the data rate R is determined, the transmission between network elements (cloud-user or cloud-edge-user) is fixed for the sequence of packets. In some examples, the coding rate can be adjusted after the fixed sequence. In some examples, these source codes can be improved using multi-rate coding techniques, where a basic video or image is transmitted at a low rate, and then details are added sequentially based on one or more of the following: user input (e.g., user selection of enhanced image options), physical layer and link layer characteristics (which may include available historical or real-time bandwidth and channel noise characteristics), and / or congestion in network nodes.

[0091] In some examples, the base layer stream can be sent automatically, and one or more enhancement layers or enhancement streams can be sent based on the factors mentioned above. In some examples, the enhancement layer or enhancement stream can be cached at one or more edge computing devices, including end-user devices, for example, based on a cost function (e.g., the cost associated with the three points mentioned above).

[0092] In some examples, the source encoder uses wavelet transform to decompose the signal into a fundamental signal R and details {λ1, λ2, ..., λs}. A continuous version of the wavelet transform with index s and spectral spread τ is given by the following formula: Use a familiar discrete version of F(τ,s). The quantity s is determined by the multi-rate depth provided by the system administrator (e.g., an IT administrator). In this case, the complete data rate satisfies R = R0 U λ1 U λ2…U λs , R1 = R0 U λ1, … Rn = R(n-1) U λn. It is understandable that, just as details can be added, they can also be removed based on user feedback, device capabilities, or network conditions.

[0093] In some examples, the object recognition method described above can be used in systems such as multi-rate decoding. For instance, when the object recognition algorithm fails to detect one or more objects (e.g., the number of recognized objects is less than a specified threshold T), the system can request additional layers from the sender, server, or edge device. These additional layers can then be used by the decoder as decoding feedback to reduce the decision region, enabling higher-probability decisions and better decoding. Thus, by adding details, the decoder and / or object recognition process described above can operate more efficiently.

[0094] Figure 13The illustration shows an intent of a multi-rate decoding scenario employing edge buffering, according to some examples of this disclosure. Transmitting device 1305 sends an image, such as a video stream, to receiving device 1325. In some examples, transmitting device 1305 transmits the image via communication server 1310. In some examples, transmitting device 1305 encodes the image using multiple layers, as described above. For example, a base layer (BL) and an enhancement layer (EL). The enhancement layer adds additional details, such as increasing the first quality, first size, resolution, bit rate, frame rate, etc. of the BL. Communication server 1310 may process the image, for example, during a communication session (e.g., an online conference), by synthesizing multiple images from multiple transmitting devices. In other examples, the actual encoding performed on transmitting device 1305 may be a simpler encoding utilizing a single layer, and communication server 1310 may re-encode the image or the synthesized image using multi-rate encoding.

[0095] Communication server 1310 can send BL streams to receiving device 1325. Additionally, communication server 1310 can send one or more EL streams based on one or more of the following: user input (e.g., user selection of enhanced image options), physical layer and link layer characteristics (which may include historical or real-time available bandwidth and channel noise characteristics), and / or congestion in network nodes. In some examples, communication server 1310 determines whether to send an EL stream, but in other examples, receiving device 1325 can request an EL stream based on the factors described above.

[0096] In some examples, one or more of the BL stream and / or EL stream can be cached at one or more edge devices, such as edge device 1320. Receiving device 1325 can obtain one or more of the BL stream and / or EL stream from edge device 1320. This can allow receiving device 1325 to receive BL streams and / or EL streams more quickly, or avoid network problems that might hinder receiving EL streams or BL streams from communication server 1310 or sending device 1305.

[0097] File storage and future editing As previously mentioned, the methods, systems, and machine-readable media described herein can also be applied to reading data from storage devices. For example, some storage devices may not be entirely reliable. That is, the media stored statically can be unintentionally modified through multiple physical processes. For instance, solar flares can induce bit flips; read interference errors from solid-state storage; and magnetic interference on hard disk drives (HDDs) can all lead to data read errors. In some examples, to store the data more securely, it can be encoded before storage. Furthermore, sometimes the receiving computing device can simply stream the received data to the storage device instead of immediately decoding the transmission. Therefore, data can be stored in an encoded state, and the data source being decoded can be the storage device.

[0098] Additionally, in some examples, decoding feedback may include one or more edits made to the document. For example, if the document is content, the user can edit one or more sections of the document. This editing can be provided to the decoder as decoding feedback for other parts of the document, feedback for re-decoding the current document, or feedback for another document.

[0099] Figure 14 The following examples illustrate the use of future user edits to stored data, according to this disclosure, to determine decoding feedback information to improve decoder performance. Source encoder 1412 and channel encoder 1414 can perform source encoding and channel encoding on input data 1410, respectively. This data can then be stored in storage device 1424 or transmitted via a communication medium. Channel decoder 1418 then decodes the data received from the memory or from the communication medium. Source decoder 1422 then decodes the channel-decoded data. This data can then be stored in storage device 1424 or output to a user. Context-based error detection and correction component 1420 can utilize one or more user edits 1426 to create decoder feedback information, for example, by performing source encoding and channel encoding on a corrected file.

[0100] Predictive feedback information As previously mentioned, the decoder can use various feedback information to correct previously incorrect decoding, and thus correct the decoder's memory to achieve correct decoding of future data. In some examples, prediction information can be used to prevent incorrect decoding in the first place, rather than using corrected data to fix the decoder state. That is, the decoder can utilize prediction feedback information to predict one or more decoding possibilities. This can make the decoding possibility corresponding to the predicted data more likely to be selected by the decoder. For example, by reducing the path cost corresponding to the decoding corresponding to the predicted data, and increasing the cost of other paths that do not correspond to the decoding corresponding to the predicted data, and / or both. In other examples, this can be achieved by increasing or decreasing the probability in the turbo decoder, making it more likely to select the decoding possibility corresponding to the predicted data.

[0101] In other instances, multiple possible decoding probabilities can be output. For example, instead of selecting a single decoder, the decoder can choose a specified number of K most likely decoding probabilities. For a lattice decoder, the most likely decoding probability might correspond to the decoding probability with the lowest total path cost. For a turbo decoder, the most likely decoding probability might correspond to the decoding probability with the highest probability. Prediction information can then assist the decoder in selecting one of the K possible decoding probabilities. This can be done by biasing the path score or probability of each decoder based on the prediction information. For example, the path score can be biased based on the frequency of word usage by the sending user. Thus, the decoding probability corresponding to words frequently used by the sender can reduce its path cost, and the decoding probability corresponding to words infrequently used by the sender can reduce its path cost less, remain unchanged, or even increase it. That is, the bias applied to the path score can be based on the sender's word frequency usage. In other examples, the system can predict the next word or phrase, and the decoding probability corresponding to the next word or phrase can be biased because the path cost of those words and / or phrases can be reduced or the probability of those words and / or phrases can be increased—depending on the type of decoder.

[0102] The same prediction techniques can be used for speech data, where the frequency of selected words or the predicted next utterance can be similarly used. Therefore, the system can predict a user's next utterance based on their past speech. For example, by transcribing the user's speech and subsequently applying the same text prediction algorithm. Additionally, a word history frequency database can be used to determine which of several decoding possibilities are more probable. For example, captured real-time speech samples or synthetically generated speech samples, as described above, can be used along with frequency information as feedback.

[0103] In the case of video data, the system can predict the next frame of the video based on other video frames. For example, this can be achieved using deep learning supersampling, XE supersampling, fidelity super-resolution, etc. In some examples, decoded frames that are within a predetermined threshold variance from the predicted frame can be positively biased (e.g., more likely to be selected for decoding), and other frames can be unbiased or negatively biased. The variance can be measured based on multiple different pixel values.

[0104] Figure 15 The following examples illustrate the use of prediction information as feedback information to the channel decoder according to some examples of this disclosure. Channel decoder 1507 performs channel decoding on encoded data 1505. Encoded data 1505 may include text data (e.g., chat or other dialogue), voice data, image data, video data, file data, etc. Source decoder 1509 then decodes the channel-decoded data and outputs the result. Source decoder 1509 may utilize feedback information, such as feedback information based on word-frequency distribution. Prediction component 1513 of context-based error detection and correction component 1512 may utilize the decoded data to provide a prediction of the next data likely to be received from the transmitting device. For example, the next word, phrase, or other text item; utterance, etc. In some examples, the prediction may be the next frame of a video stream. In some examples, prediction component 1513 may utilize model data 1530 to predict the next data. Source encoder 1518 and channel encoder 1520 may perform source and channel coding on the predicted data, respectively.

[0105] Figure 19 A flowchart illustrating a method 1900 utilizing predictive feedback information according to some examples of this disclosure is shown. At operation 1910, the system receives encoded communication data encoded with error correction codes, the encoded communication data being communication from a communication user. The communication data can be voice, images (e.g., video), text, etc. The data can be received via a computing network or other transmission medium, or can be read from storage.

[0106] At operation 1912, the system can identify the predictive feedback information of the communicating user based on the user's previous communications. For example, this could be achieved using word frequency distributions from previous communication sessions, the current communication session, etc. In other examples, other types of predictive feedback could include the next video frame predicted by, for example, DLSS 3.0 frame generation technology; the next text predicted from a sentence completion algorithm; and so on. The system can identify this information by calculating it, receiving it (e.g., from the communicating user or a third party), etc.

[0107] At operation 1914, the system can use prediction feedback information to create decoding feedback information, which includes predictions of encoded data from communications sent by the communication user. The decoding feedback information can be prediction information of the source and / or channel coded versions, such as predictions of the next text from the source and / or channel coded versions; predictions of the next frame, etc.

[0108] At operation 1916, the system can decode the encoded communication data using a decoder and decoding feedback information to produce decoded communication data. This decoding uses the decoding feedback information to select one or more of several possible decoders. For example, by changing path cost, probability, etc.

[0109] At operation 1918, the system can cause the output of decoded communication data. For example, by displaying it or playing it through a speaker.

[0110] Use of decoding feedback information As previously mentioned, feedback information can be used by the decoder to improve its speed and / or accuracy. This can be done in various ways. One example is increasing the path cost of certain decoding possibilities; decreasing the path cost of other decoding possibilities; or both. For example, the feedback information could be coded word frequency distribution information describing the distribution of the sending user's language usage. For example, the channel-coded message 1011 1001 0011 can be received. This message may have a first decoded word that translates to the word "fax" and a second decoded word that translates to the word "fix". The decoder may understand that the first word ("fax") appears far less frequently than the word "fix" in the sender's communication mode (e.g., spoken, written, or both), and therefore the decoded word representing "fix" is more likely to be the correct decoder. In some examples, the decoder may decrease the path cost of decoding the word representing "fix". In some examples, in addition to or instead of decreasing the path cost of decoding the word representing "fix", the decoder may increase the path cost of decoding the word representing "fax". The encoder can know which decoding probability represents the words "fax" and "fix" because each word can be channel-coded and source-coded, and can include the corresponding usage frequency.

[0111] In another example, probabilities can be biased. For instance, in the word frequency distribution example above, if the decoder produces a probability of 45% for the word "fix" and 55% for the word "fax," then based on the word frequency distribution, the decoder can change the probabilities so that "fix" is given a 60% probability and "fax" is given a 40% probability. Similar to path cost, probabilities can be modified to favor certain decoding possibilities by increasing the probability of a more likely decoding possibility, decreasing the probability of other less likely decoding possibilities, or both. By increasing the probability of more likely decoding possibilities or decreasing the probability of other less likely decoding possibilities, this increases the decoder's chance of selecting a given decoding based on decoding feedback.

[0112] The amount by which path costs and / or probabilities change can be a predetermined amount; or it can depend on decoding feedback information. For example, decoding feedback information can include a confidence factor or level. In the case of the word frequency distribution example, this could correspond to the frequency of use of a particular term. The amount can be set based on a value calculated using a formula and that confidence factor or level. The higher the confidence factor or level, the greater the change in path costs and / or probabilities. In some examples, for the word usage frequency example, each possible decoding can be adjusted based on the usage frequency. This establishes a relative change for each term. That is, if we have three decoding possibilities: “fix,” “fax,” and “wax,” and the word frequency distribution is that fix is ​​used more frequently than fax and fax is used more frequently than wax; then the path cost of fix decreases more than that of fax, and the path cost of fax decreases more than that of wax. As a result, the decoder is more likely to decode the more frequently used word. Since the decoder's decision is not forced by feedback (only aided), the decoder may still decode less frequently used words.

[0113] In other examples where the feedback information is about previously decoded data, the system can "rewind" the decoder state to the state it was in before receiving its feedback data. Thus, the decoder can have memory storing information about its previous state. The decoder can then replay the decoding, but instead of the feedback information (e.g., what the data should have been) during the correction of the decoded state. In yet another example, the system can easily reconstruct the state using the feedback information by adjusting paths, probabilities, etc.

[0114] In other examples, the system can trim one or more mesh paths based on feedback information. Figure 16This illustration shows an example of using corrected information to update the decision grid 1600 of the decoder according to some examples of this disclosure. An example of the received sequence 1602 (01, 10, 11, 11, 01, 00, 01) and decoder state 1604 are shown in columns at the top. At each path, valid state transitions are shown. For example, at S0, valid transitions are 00 and 11. Arrows indicate the next state based on this transition. For example, from state S0, bit pattern 11 transitions the decoder to state S2. For each path, the decoder determines a branch metric and a path metric. In some examples, the branch metric may be the cost of moving from one state to the next. For example, for the received sequence 01, the cost of moving from state S0 to S2 is 1 because there is a difference of 1 between 01 and 11. The path metric may be calculated as follows: first, the branch metric is added to the path metric of the old state; then the sum of the paths leading to the new state is compared (in...). Figure 3 (In some examples, there are only two paths); and the path with the minimum value is selected. In some examples, a tie can be arbitrarily broken. Feedback information may indicate that some paths marked "XXX" are unlikely to be correct. These paths can be removed, given a high path cost, or treated similarly to avoid being considered.

[0115] Figure 17 A system 1700 of an improved decoder according to some examples of this disclosure is shown. A transmitting communication device 1726 transmits channel and / or source-coded data to a receiving computing device 1710 via a network such as network 1730. In some examples, the transmission is from the transmitting communication device 1726 to the receiving computing device 1710 via network 1730, but in other examples, the data stream passes through a communication server 1724. For example, the communication server 1724 can compose video or other files based on data transmitted from the transmitting communication device 1726 and one or more other devices (including, in some examples, data transmitted by the receiving computing device 1710). For example, if the data transmitted by the transmitting communication device 1726 is associated with a communication session such as an online conference, the communication server 1724 can combine video, audio, and other data to form a conference. In some examples, the stream from the transmitting communication device 1726 can be split into a BL and one or more ELs, such as... Figure 13 As shown. Similarly, edge device 1722 may store one or more of BL and EL. Details of transmitting communication device 1726 are not shown for clarity, but in some examples, it may be or include the same components as receiving computing device 1710.

[0116] The communication device, including receiving computing device 1710 and, in some examples, transmitting communication device 1726, may include network interface 1720, such as an Ethernet interface, a Wi-Fi interface, etc. Network interface 1720 may include one or more physical protocol layers. Channel decoder 1718 can decode one or more channel codes for the received data. Source decoder 1716 can perform source decoding on the channel-decoded data output from channel decoder 1718. For data being transmitted, source encoder 1717 performs source encoding on the data, and before transmitting data 1719 through network interface 1720, channel encoder performs channel encoding on data 1719.

[0117] The source-decoded data is then passed to an application layer application, such as a communication application 1714. In some examples, the communication application 1714 may also include a source encoder / decoder and / or a channel encoder / decoder. Context-based error detection and correction component 1715 is an example of a context-based error detection and correction component described herein. Graphical user interface component 1712 can provide one or more GUIs that can display the received data. For example, displaying video, playing audio, displaying text, etc.

[0118] As used throughout this disclosure, the specific arrangement and composition of the various components are exemplary, and those skilled in the art who benefit from this disclosure will understand that other configurations are possible.

[0119] Figure 18 A block diagram of an example machine 1800 on which any one or more of the techniques (e.g., methods) discussed herein can be performed is shown. In alternative embodiments, machine 1800 may operate as a standalone device or may be connected (e.g., networked) to other machines. In a networked deployment, machine 1800 may operate as a server machine, a client machine, or both in a server-client network environment. In the example, machine 1800 may act as a peer-to-peer (P2P) (or other distributed) network environment. Machine 1800 may be a personal computer (PC), tablet PC, set-top box (STB), personal digital assistant (PDA), server computer, mobile phone, smartphone, network device, network router, switch, or bridge, or any machine capable of executing instructions (sequentially or otherwise) specifying actions to be taken by the machine. Furthermore, although only a single machine is shown, the term "machine" should also be considered to include any collection of machines that individually or jointly execute a set (or more) of instructions to perform any one or more methods discussed herein, such as cloud computing, Software as a Service (SaaS), and other computer cluster configurations. In some examples, Machine 1800 can be configured to implement Figure 2 , Figure 3, Figure 5-8 , Figure 10 , Figure 11 , Figure 14 and Figure 15 Sender 110, receiver 120, decoding process; Figure 4 , Figure 9 , Figure 12 and Figure 19 Methods; Figure 13 Edge device 1320, communication server 1310, transmitting device 1305, receiving device 1325; implemented (e.g., in a data structure). Figure 16 The grid; Figure 17 The receiving computing device 1710, the transmitting communication device 1726, the communication server 1724, the edge device 1722; and any other network-based service or computing device described herein.

[0120] The examples described herein may include one or more logical units, components, or mechanisms (hereinafter referred to as "components"), or those that can be operated upon. A component is a tangible entity (e.g., hardware) capable of performing a specified operation and can be configured or arranged in a certain way. In the examples, circuitry may be arranged as a component in a specified manner (e.g., internally or relative to an external entity such as other circuitry). In the examples, all or part of one or more computer systems (e.g., standalone, client, or server computer systems) or one or more hardware processors may be configured by firmware or software (e.g., instructions, application portions, or applications) to operate to perform the specified operation. In the examples, the software may reside on a machine-readable medium. In the examples, when executed by the underlying hardware of the component, the software causes the hardware to perform the specified operation of the component.

[0121] Therefore, the term "component" is understood to encompass tangible entities that are physically constructed, concretely configured (e.g., hardwired), or temporarily (e.g., provisionally) configured (e.g., programmed) to operate or perform any of the operations described herein in a specified manner. Consider an example where components are provisionally configured, and each component does not need to be instantiated at any given time. For example, in the case where components include a general-purpose hardware processor configured using software, the general-purpose hardware processor can be configured as different components at different times. The software can accordingly configure the hardware processor, for example, to constitute a specific module at one time and different components at different times.

[0122] Machine (e.g., computer system) 1800 may include one or more hardware processors, such as processor 1802. Processor 1802 may be a central processing unit (CPU), a graphics processing unit (GPU), a hardware processor core, or any combination thereof. Machine 1800 may include main memory 1804 and static memory 1806, some or all of which may communicate with each other via interconnect (e.g., bus) 1808. Instances of main memory 1804 may include synchronous dynamic random access memory (SDRAM), such as double data rate memory, such as DDR4 or DDR5. Interconnect 1808 may be one or more interconnect links of different types, such that one or more components can be connected using a first type of interconnect link, and one or more components can be connected using a second type of interconnect link. Example interconnects may include a memory bus, peripheral component interconnect (PCI), peripheral component interconnect high speed (PCIe) bus, universal serial bus (USB), etc.

[0123] Machine 1800 may also include a display unit 1810, an alphanumeric input device 1812 (e.g., a keyboard), and a user interface (UI) navigation device 1814 (e.g., a mouse). In this example, the display unit 1810, input device 1812, and UI navigation device 1814 may be a touchscreen display. Machine 1800 may additionally include a storage device (e.g., a drive unit) 1816, a signal generation device 1818 (e.g., a speaker), a network interface device 1820, and one or more sensors 1821, such as a Global Positioning System (GPS) sensor, a compass, an accelerometer, or other sensors. Machine 1800 may include an output controller 1828, such as a serial (e.g., Universal Serial Bus (USB), parallel, or other wired or wireless (e.g., infrared (IR), near field communication (NFC), etc.) connection, to communicate with or control one or more peripheral devices (e.g., a printer, a card reader, etc.).

[0124] Storage device 1816 may include machine-readable medium 1822 thereon storing one or more sets of data structures or instructions 1824 (e.g., software) embodying or utilized by any one or more technologies or functions described herein. Instructions 1824 may also reside wholly or at least partially within main memory 1804, static memory 1806, or hardware processor 1802 during execution by machine 1800. In this example, one or any combination of hardware processor 1802, main memory 1804, static memory 1806, or storage device 1816 may constitute a machine-readable medium.

[0125] Although machine-readable medium 1822 is shown as a single medium, the term "machine-readable medium" can include a single medium or multiple media (e.g., a centralized or distributed database and / or associated caches and servers) configured to store one or more instructions 1824.

[0126] The term "machine-readable medium" can include any medium capable of storing, encoding, or carrying instructions for execution by machine 1800 and causing machine 1800 to perform any one or more of the techniques disclosed herein, or any medium capable of storing, encoding, or carrying data structures used by or associated with such instructions. Examples of non-limiting machine-readable media can include solid-state memory as well as optical and magnetic media. Specific examples of machine-readable media can include: non-volatile memory, such as semiconductor memory devices (e.g., electrically programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM)) and flash memory devices; disks, such as internal hard disks and removable disks; magneto-optical disks; random access memory (RAM); solid-state drives (SSDs); and CD-ROM and DVD-ROM disks. In some examples, machine-readable media can include non-transitory machine-readable media. In some examples, machine-readable media can include machine-readable media that are not transient propagating signals.

[0127] Instruction 1824 can also be sent or received over communication network 1826 via network interface device 1820 using a transmission medium. Machine 1800 can communicate wired or wirelessly with one or more other machines using any of a variety of transmission protocols, such as Frame Relay, Internet Protocol (IP), Transmission Control Protocol (TCP), User Datagram Protocol (UDP), Hypertext Transfer Protocol (HTTP), etc. Example communication networks may include local area networks (LANs), wide area networks (WANs), packet data networks (e.g., the Internet), mobile phone networks (e.g., cellular networks), common old-style telephone (POTS) networks, and wireless data networks such as the IEEE 802.11 standard family, the IEEE 802.15.4 standard family, the 5G New Radio (NR) standard family, the Long Term Evolution (LTE) standard family, the Universal Mobile Telecommunications System (UMTS) standard family, peer-to-peer (P2P) networks, etc. In the example, network interface device 1820 may include one or more physical jacks (e.g., Ethernet, coaxial, or telephone jacks) or one or more antennas for connection to communication network 1826. In the example, network interface device 1820 may include multiple antennas to perform wireless communication using at least one of single-input multiple-output (SIMO), multiple-input multiple-output (MIMO), or multiple-input single-output (MISO) technologies. In some examples, network interface device 1820 may use multi-user MIMO technology for wireless communication.

[0128] Other notes and examples Example 1 is a method for improving decoding performance using language checking or predicted text, the method comprising: receiving encoded text encoded with error correction codes via a transmission medium; decoding at least a portion of the encoded text by a decoder to produce a decoded text portion; performing language checking on the decoded text portion to produce a corrected decoded text portion, or inputting the decoded text portion into a predicted text generator, the predicted text generator generating subsequent text to produce a predicted next decoded text portion, the language checking including spell checking or grammar checking; encoding the corrected decoded text portion or the predicted next decoded text portion using error correction codes to produce encoding feedback information; eliminating at least one potential decoding possibility by utilizing the encoding feedback information when the decoder decodes a second portion of the encoded text to create a second decoded text portion; and displaying the second decoded text portion of the encoded text on a display.

[0129] In Example 2, the subject of Example 1 includes eliminating at least one potential decoder by trimming the mesh by eliminating choices that are inconsistent with the encoded feedback information.

[0130] In Example 3, the topics of Examples 1-2 include a spell check decoding text section that utilizes context-sensitive spell check to identify incorrect spellings, where the context-sensitive spell check scans multiple words.

[0131] In Example 4, the subject of Example 3 includes a context-sensitive spell check that leverages the sending user's historical usage.

[0132] In Example 5, the topics of Examples 1-4 include a predictive text generator that utilizes the historical word selections of the sending user.

[0133] In Example 6, the topics of Examples 1-5 are included, where the decoder is a mesh decoder.

[0134] In Example 7, the topics of Examples 1-6 include language checking, which involves determining the most likely words based on the word frequency usage of the sending user.

[0135] Example 8 is a computing device for improving decoding performance using language checking or predictive text, the computing device comprising: a hardware processor; and a memory storing instructions that, when executed by the hardware processor, cause the computing device to perform operations including: receiving encoded text encoded with error correction codes via a transmission medium; decoding at least a portion of the encoded text by a decoder to produce a decoded text portion; performing language checking on the decoded text portion to produce a corrected decoded text portion, or inputting the decoded text portion into a predictive text generator that generates subsequent text to produce a predicted next decoded text portion, the language checking including spell checking or grammar checking; encoding the corrected decoded text portion or the predicted next decoded text portion using the error correction codes to produce encoding feedback information; eliminating at least one potential decoding possibility by utilizing the encoding feedback information when the decoder decodes a second portion of the encoded text to create a second decoded text portion; and displaying the second decoded text portion of the encoded text on a display.

[0136] In Example 9, the subject of Example 8 includes an operation to eliminate at least one potential decoder, which involves trimming the mesh by eliminating choices that are inconsistent with the feedback information of the encoding.

[0137] In Example 10, the topics of Examples 8-9 include operations where the spell-checking decoding of text involves using context-sensitive spell checking to determine incorrect spellings, which scans multiple words.

[0138] In Example 11, the subject of Example 10 includes a context-sensitive spell check that leverages the sending user's historical usage.

[0139] In Example 12, the topics of Examples 8-11 include a predictive text generator that utilizes the historical word selections of the sending user.

[0140] In Example 13, the subject of Examples 8-12 includes, where the decoder is a mesh decoder.

[0141] In Example 14, the topics of Examples 8-13 include language checking, which involves determining the most likely words based on the word frequency usage of the sending user.

[0142] Example 15 is a machine-readable medium storing instructions for improving decoding performance using language checking or predictive text, the instructions causing a machine to perform operations when executed, the operations including: receiving encoded text encoded with error-correcting codes via a transmission medium; decoding at least a portion of the encoded text by a decoder to produce a decoded text portion; performing language checking on the decoded text portion to produce a corrected decoded text portion, or inputting the decoded text portion into a predictive text generator that generates subsequent text to produce a predicted next decoded text portion, the language checking including spell checking or grammar checking; encoding the corrected decoded text portion or the predicted next decoded text portion using the error-correcting codes to produce encoded feedback information; eliminating at least one potential decoding possibility by utilizing the encoded feedback information when the decoder decodes a second portion of the encoded text to create a second decoded text portion; and displaying the second decoded text portion of the encoded text on a display.

[0143] In Example 16, the subject of Example 15 includes an operation in which eliminating at least one potential decoder involves trimming the mesh by eliminating choices that are inconsistent with the feedback information of the encoding.

[0144] In Example 17, the topics of Examples 15-16 include operations in which the spell-checking decoding of text includes using context-sensitive spell checking to determine incorrect spellings, where context-sensitive spell checking scans multiple words.

[0145] In Example 18, the subject of Example 17 includes a context-sensitive spell check that leverages the sending user's historical usage.

[0146] In Example 19, the topics of Examples 15-18 include a predictive text generator that utilizes the historical word selections of the sending user.

[0147] In Example 20, the subject of Examples 15-19 includes, where the decoder is a mesh decoder.

[0148] In Example 21, the topics of Examples 15 to 20 include the language check that includes determining the most likely word based on the word frequency usage of the sending user.

[0149] Example 22 is an apparatus for improving decoding performance using language checking or predictive text, the apparatus comprising: a unit for receiving encoded text encoded with error correction codes via a transmission medium; a unit for decoding at least a portion of the encoded text by a decoder to produce a decoded text portion; a unit for performing language checking on the decoded text portion to produce a corrected decoded text portion or inputting the decoded text portion into a predictive text generator, the predictive text generator generating subsequent text to produce a predicted next decoded text portion, the language checking including spell checking or grammar checking; a unit for encoding the corrected decoded text portion or the predicted next decoded text portion using error correction codes to produce encoded feedback information; a unit for eliminating at least one potential decoding possibility by utilizing the encoded feedback information when a second portion of the encoded text is decoded by the decoder to create a second decoded text portion; and a unit for displaying the second decoded text portion of the encoded text on a display.

[0150] In Example 23, the subject of Example 22 includes a unit for eliminating at least one potential decoder, which includes a unit for trimming the grid by eliminating choices inconsistent with the encoded feedback information.

[0151] In Example 24, the topics of Examples 22-23 include a spell check decoding text section that utilizes context-sensitive spell checking to identify incorrect spellings, where the context-sensitive spell check scans multiple words.

[0152] In Example 25, the subject of Example 24 includes a context-sensitive spell check that leverages the sending user's historical usage.

[0153] In Example 26, the topics of Examples 22-25 include a predictive text generator that utilizes the historical word selections of the sending user.

[0154] In Example 27, the topics of Examples 22 to 26 are included, wherein the decoder is a mesh decoder.

[0155] In Example 28, the topics of Examples 22 to 27 include the language check, which includes a unit for determining the most likely word based on the word frequency usage of the sending user.

[0156] Example 29 is a method for improving decoder performance using characteristics of a speaker's speech, the method comprising: receiving characteristics of a speaker's speech over a network; receiving encoded speech data of the speaker; decoding a first portion of the encoded speech data of the speaker using a decoder to generate decoded speech data; comparing the decoded speech data with the characteristics of the received speaker's speech; determining, based on the result of comparing the decoded speech data with the characteristics of the received speaker's speech, that the decoded speech data does not correspond to the characteristics of the received speaker; and, in response to determining that the decoded speech data does not correspond to the characteristics of the received speaker: correcting the decoded speech data based on the characteristics of the speaker's speech to create corrected speech data; and, when the decoder decodes a second portion of the encoded speech data to create second decoded speech data, eliminating at least one potential decoding possibility by utilizing the corrected speech data.

[0157] In Example 30, the subject of Example 29 includes the use of deep learning neural networks to correct decoded speech data based on the characteristics of the speaker's voice to create corrected decoded speech data.

[0158] In Example 31, the subject matter of Examples 29 to 30 includes, wherein the characteristic includes Mel frequency cepstral coefficients.

[0159] In Example 32, the subject matter of Examples 29 to 31 includes features where the characteristic is one of the following: sent by the speaker as part of a communication session or received from a network-based service.

[0160] In Example 33, the subject of Example 32 includes the following: features are sent by the speaker as part of the speech during a communication session.

[0161] In Example 34, the subject matter as described in Examples 29 to 33 includes: encoding the corrected voice data to create encoded corrected voice data; and wherein, when the second portion of the encoded voice data is decoded by the decoder to create the second decoded voice data, eliminating the at least one potential decoding possibility by utilizing the corrected voice data includes utilizing the encoded corrected voice data.

[0162] In Example 35, the subject matter of Examples 29 to 34 includes determining that decoded speech data does not correspond to speaker characteristics based on the result of a comparison between decoded speech data and the characteristics of the speaker's speech, which includes using decoded speech data and speaker characteristics as input to a deep learning network.

[0163] In Example 36, the subject matter of Examples 29 to 35 includes, wherein determining that the decoded voice data does not correspond to the speaker's characteristics based on a comparison of the characteristics of the decoded voice data with those of the speaker's speech includes: using the speaker's characteristics to create a transcript of the decoded voice data; identifying that the frequency of a first word in the transcript is less than a specified frequency in the speaker's word frequency distribution data; in response to identifying the first word, changing the first word in the transcript to a second word to create an updated transcript, the frequency of the second word being greater than a second specified frequency; and wherein correcting the decoded voice data to correspond to the characteristics of the speaker's speech to create corrected voice data includes applying a deep learning neural network to the updated transcript to create corrected voice data.

[0164] In Example 37, the subject matter of Examples 29 to 36 includes, wherein, when a second portion of encoded voice data is decoded by a decoder to create second decoded voice data, eliminating at least one potential decoding possibility by utilizing corrected voice data includes one of the following: trimming the decoder grid, updating path costs, or re-decoding the first portion before decoding the second portion.

[0165] In Example 38, the subject matter of Examples 29 to 37 includes, wherein the characteristics of a speaker’s speech received over a network include calculating the characteristics using previously received speaker speech data.

[0166] In Example 39, the subject matter of Examples 29 to 38 includes, wherein the characteristics of a speaker’s speech received over a network include received characteristics without calculating characteristics from previously received speaker speech data.

[0167] In Example 40, the subject matter of Examples 29 to 39 includes determining that the decoded speech data does not correspond to the characteristics of the received speaker based on the result of comparing the decoded speech data with the characteristics of the received speaker's speech, which includes determining that one or more of the frequency, amplitude, rhythm, pitch, or one or more Mel frequency cepstral coefficients of the decoded speech data are higher or lower than a pre-specified threshold for the corresponding characteristics of the speaker.

[0168] In Example 41, the subject matter as described in Examples 29 to 40 includes: wherein, when the second portion of the encoded voice data is decoded by the decoder to create the second decoded voice data, eliminating the at least one potential decoding possibility by utilizing the corrected voice data includes one of the following: removing a path through a grid data structure, increasing the path cost of the potential decoding possibility, or reducing the probability metric of the potential decoding possibility.

[0169] Example 42 is a computing device for improving decoder performance using characteristics of a speaker's speech, the computing device comprising: a hardware processor; and a memory storing instructions that, when executed by the hardware processor, cause the computing device to perform operations including: receiving characteristics of a speaker's speech over a network; receiving encoded speech data of the speaker; decoding a first portion of the encoded speech data of the speaker using a decoder to generate decoded speech data; comparing the decoded speech data with the received speech characteristics of the speaker; determining, based on the comparison result, that the decoded speech data does not correspond to the received speech characteristics of the speaker; and in response to determining that the decoded speech data does not correspond to the received speaker characteristics: correcting the decoded speech data based on the characteristics of the speaker's speech to create corrected speech data; and creating second decoded speech data by using the corrected speech data during decoding to eliminate at least one potential decoding possibility, by the decoder, a second portion of the encoded speech data.

[0170] In Example 43, the subject of Example 42 includes the operation of correcting decoded speech data based on the characteristics of the speaker's speech to create corrected speech data, which includes the use of deep learning neural networks.

[0171] In Example 44, the subject matter of Examples 42 to 43 includes, wherein the characteristic includes Mel frequency cepstral coefficients.

[0172] In Example 45, the subject of Examples 42-44 includes features where the characteristic is one of the following: sent by the speaker as part of a communication session or received from a network-based service.

[0173] In Example 46, the subject of Example 45 includes the following: features are transmitted by the speaker as part of the speech during a communication session.

[0174] In Example 47, the subject matter of Examples 42-46 includes an operation that further includes: encoding the corrected speech data to create encoded corrected speech data; and wherein, when the second portion of the encoded speech data is decoded by the decoder to create second decoded speech data, eliminating at least one potential decoding possibility by utilizing the corrected speech data includes: utilizing the encoded corrected speech data.

[0175] In Example 48, the subject matter of Examples 42-47 includes an operation in which determining that the decoded speech data does not correspond to the speaker's characteristics based on the result of a comparison between the decoded speech data and the characteristics of the speaker's speech includes using the decoded speech data and the speaker's characteristics as input to a deep learning network.

[0176] In Example 49, the subject matter of Examples 42-48 includes an operation in which determining that the decoded voice data does not correspond to the characteristics of the speaker's speech based on a comparison of the characteristics of the decoded voice data with those of the speaker's speech includes: using the characteristics of the speaker to create a transcript of the decoded voice data; identifying that the frequency of a first word in the transcript is less than a specified frequency in the speaker's word frequency distribution data; in response to identifying the first word, changing the first word in the transcript to a second word to create an updated transcript, the frequency of the second word being greater than a second specified frequency; and wherein correcting the decoded voice data to correspond to the characteristics of the speaker's speech to create corrected voice data includes: applying a deep learning neural network to the updated transcript to create corrected voice data.

[0177] In Example 50, the subject of Examples 42-49 includes an operation in which, by using corrected voice data to eliminate at least one potential decoding possibility when the second part of the encoded voice data is decoded by the decoder to create the second decoded voice data, one of the following is included: trimming the decoder grid, updating the path cost, or re-decoding the first part before decoding the second part.

[0178] In Example 51, the subject of Examples 42-50 includes an operation in which the characteristics of a speaker’s speech received over a network are calculated using previously received speech data of the speaker.

[0179] In Example 52, the subject of Examples 42-51 includes an operation in which the characteristics of a speaker’s speech are received over a network, which includes receiving the characteristics without calculating the characteristics from previously received speaker speech data.

[0180] In Example 53, the subject matter of Examples 42 to 52 includes an operation in which determining that the decoded speech data does not correspond to the characteristics of the received speaker, based on the result of comparing the decoded speech data with the characteristics of the received speaker's speech, includes determining that one or more of the frequency, amplitude, rhythm, pitch, or one or more Mel frequency cepstral coefficients of the decoded speech data are higher or lower than a pre-specified threshold for the corresponding characteristics of the speaker.

[0181] In Example 54, the subject of Examples 42-53 includes the following, wherein the operation of eliminating at least one potential decoding possibility by utilizing corrected voice data when the second part of the encoded voice data is decoded by the decoder to create second decoded voice data includes one of the following: removing a path through the grid data structure, increasing the path cost of the potential decoding possibility, or reducing the probability metric of the potential decoding possibility.

[0182] Example 55 is a machine-readable medium storing instructions for improving decoder performance using characteristics of a speaker's speech, the instructions causing the machine, when executed by a machine, to perform operations including: receiving the characteristics of the speaker's speech via a network; receiving coded speech data of the speaker; decoding a first portion of the coded speech data of the speaker using a decoder to generate decoded speech data; comparing the decoded speech data with received characteristics of the speaker's speech; determining, based on the result of comparing the decoded speech data with the received characteristics of the speaker's speech, that the decoded speech data does not correspond to the received characteristics of the speaker; and, in response to determining that the decoded speech data does not correspond to the received characteristics of the speaker: correcting the decoded speech data based on the characteristics of the speaker's speech to create corrected speech data; and, when the decoder decodes a second portion of the coded speech data to create second decoded speech data, eliminating at least one potential decoding possibility by utilizing the corrected speech data.

[0183] In Example 56, the subject of Example 55 includes the operation of correcting decoded speech data based on the characteristics of the speaker's speech to create corrected decoded speech data, which includes the use of deep learning neural networks.

[0184] In Example 57, the subject matter of Examples 55 to 56 includes, wherein the characteristic described includes Mel frequency cepstral coefficients.

[0185] In Example 58, the subject of Examples 55 to 57 includes features where the characteristic is one of the following: sent by the speaker as part of a communication session or received from a network-based service.

[0186] In Example 59, the subject of Example 58 includes the following: features are transmitted by the speaker as part of the speech during a communication session.

[0187] In Example 60, the subject matter of Examples 55 to 59 includes an operation that further includes: encoding the corrected speech data to create encoded corrected speech data; and wherein eliminating at least one potential decoding possibility by utilizing the corrected speech data includes utilizing the encoded corrected speech data when the second portion of the encoded speech data is decoded by the decoder to create second decoded speech data.

[0188] In Example 61, the subject matter of Examples 55 to 60 includes an operation in which determining that the decoded speech data does not correspond to the speaker's characteristics based on a comparison between the decoded speech data and the characteristics of the speaker's speech includes using the decoded speech data and the speaker's characteristics as input to a deep learning network.

[0189] In Example 62, the subject matter of Examples 55 to 61 includes an operation in which determining that the decoded speech data does not correspond to the characteristics of the speaker based on a comparison between the decoded speech data and the characteristics of the speaker's speech includes: using the characteristics of the speaker to create a transcript of the decoded speech data; identifying that the frequency of a first word in the transcript is less than a specified frequency in the speaker's word frequency distribution data; in response to identifying the first word, changing the first word in the transcript to a second word to create an updated transcript, wherein the frequency of the second word is greater than the frequency of a second specified frequency; and wherein correcting the decoded speech data to correspond to the characteristics of the speaker's speech to create corrected speech data includes: applying a deep learning neural network to the updated transcript to create corrected speech data.

[0190] In Example 63, the subject matter of Examples 55 to 62 includes the operation of eliminating at least one potential decoding possibility by utilizing corrected voice data when the second part of the encoded voice data is decoded by the decoder to create the second decoded voice data, which includes one of the following: trimming the decoder grid, updating the path cost, or re-decoding the first part before decoding the second part.

[0191] In Example 64, the subject matter of Examples 55 to 63 includes an operation in which the characteristics of a speaker’s speech received over a network are calculated using previously received speech data of the speaker.

[0192] In Example 65, the subject matter of Examples 55 to 64 includes an operation in which the characteristics of a speaker’s speech are received over a network, which includes receiving the characteristics without calculating the characteristics from previously received speaker speech data.

[0193] In Example 66, the subject matter of Examples 55 to 65 includes the operation of determining that the decoded speech data does not correspond to the characteristics of the received speaker based on the result of comparing the decoded speech data with the characteristics of the received speaker's speech, which includes determining that one or more of the frequency, amplitude, rhythm, pitch, or one or more Mel frequency cepstral coefficients of the decoded speech data are higher or lower than a pre-specified threshold for the corresponding characteristics of the speaker.

[0194] In Example 67, the subject matter of Examples 55 to 66 includes the following, wherein, when a second portion of encoded voice data is decoded by a decoder to create second decoded voice data, the operation of eliminating at least one potential decoding possibility by utilizing corrected voice data includes one of the following: removing a path through a grid data structure, increasing the path cost of a potential decoding possibility, or reducing the probability metric of a potential decoding possibility.

[0195] Example 68 is an apparatus for improving decoder performance using characteristics of a speaker's speech. The apparatus includes: a unit for receiving characteristics of a speaker's speech over a network; a unit for receiving coded speech data of the speaker; a unit for decoding a first portion of the coded speech data of the speaker using a decoder to generate decoded speech data; a unit for comparing the decoded speech data with the characteristics of the received speaker's speech; a unit for determining, based on the result of comparing the decoded speech data with the characteristics of the received speaker's speech, that the decoded speech data does not correspond to the characteristics of the received speaker; and, in response to determining that the decoded speech data does not correspond to the characteristics of the received speaker, a unit for correcting the decoded speech data based on the characteristics of the speaker's speech to create corrected speech data; and a unit for eliminating at least one potential decoding possibility by utilizing the corrected speech data when the second portion of the coded speech data is decoded by the decoder to create second decoded speech data.

[0196] In Example 69, the subject of Example 68 includes a unit for correcting decoded speech data based on the characteristics of the speaker's speech to create corrected decoded speech data, which includes the use of a deep learning neural network.

[0197] In Example 70, the subject matter of Examples 68 to 69 includes, wherein the characteristic said includes Mel frequency cepstral coefficients.

[0198] In Example 71, the subject of Examples 68-70 includes features where the characteristic is one of the following: sent by the speaker as part of a communication session or received from a network-based service.

[0199] In Example 72, the subject of Example 71 includes the following: features are transmitted by the speaker as part of the speech during a communication session.

[0200] In Example 73, the subject matter of Examples 68 to 72 includes a unit for encoding corrected speech data to create encoded corrected speech data; and wherein the unit for eliminating at least one potential decoding possibility by utilizing the corrected speech data when the second part of the encoded speech data is decoded by the decoder to create second decoded speech data includes a unit for utilizing the encoded corrected speech data.

[0201] In Example 74, the subject matter of Examples 68-73 includes a unit for determining whether the decoded speech data does not correspond to the characteristics of the speaker based on the result of a comparison between the decoded speech data and the characteristics of the speaker's speech, which includes a unit for using the decoded speech data and the characteristics of the speaker as input to a deep learning network.

[0202] In Example 75, the subject matter of Examples 68-74 includes a unit for determining that decoded speech data does not correspond to speaker characteristics based on a comparison of decoded speech data with characteristics of the speaker's speech, comprising: a unit for creating a transcription of the decoded speech data using the speaker's characteristics; a unit for identifying a first word in the transcription whose frequency of use is less than a specified frequency in the speaker's word frequency distribution data; and a unit for creating an updated transcription by changing the first word in the transcription to a second word in response to identifying the first word, the second word having a frequency of use greater than a second specified frequency; and wherein the unit for correcting the decoded speech data to correspond to characteristics of the speaker's speech to create corrected speech data comprises a unit for applying a deep learning neural network to the updated transcription to create corrected speech data.

[0203] In Example 76, the subject of Examples 68-75 includes a unit for eliminating at least one potential decoding possibility by utilizing corrected voice data when the second part of the encoded voice data is decoded by the decoder to create the second decoded voice data, which includes one of the following: trimming the decoder grid, updating the path cost, or re-decoding the first part before decoding the second part.

[0204] In Example 77, the subject matter of Examples 68 to 76 includes a unit for receiving characteristics of a speaker’s speech over a network, which includes a unit for calculating characteristics using previously received speaker speech data.

[0205] In Example 78, the subject matter of Examples 68 to 77 includes a unit for receiving characteristics of a speaker’s speech over a network, which includes a unit for receiving characteristics without calculating characteristics from previously received speaker speech data.

[0206] In Example 79, the subject matter of Examples 68 to 78 includes a unit for determining that the decoded speech data does not correspond to the characteristics of the received speaker based on the result of comparing the decoded speech data with the characteristics of the received speaker's speech, including a unit for determining that one or more of the frequency, amplitude, rhythm, pitch, or one or more Mel frequency cepstral coefficients of the decoded speech data are higher or lower than a pre-specified threshold for the corresponding characteristics of the speaker.

[0207] In Example 80, the subject of Examples 68-79 includes a unit for eliminating at least one potential decoding possibility by utilizing corrected voice data when the second part of the encoded voice data is decoded by the decoder to create second decoded voice data, which includes one of the following: removing a path through the grid data structure, increasing the path cost of the potential decoding possibility, or reducing the probability metric of the potential decoding possibility.

[0208] Example 81 is a method for improving the decoding performance of an image using object detection, the method comprising: receiving an encoded image; decoding a first portion of the encoded image to create a first decoded image portion; using an automatic object recognition process to search for at least one object in the first decoded image portion; determining, based on the automatic object recognition process, that an error exists in the first decoded image portion, and in response: correcting the first decoded image portion based on the automatic object recognition process to produce a corrected decoded image; encoding the corrected decoded image to create a corrected encoded image; using the corrected encoded image to decode a second portion of the encoded image to create a second corrected decoded image; and outputting the second corrected decoded image to a display device.

[0209] In Example 82, the subject of Example 81 includes a method where the encoded image comprises only a base layer encoding that encodes a first quality, a first size, or a first bit rate, wherein an additional layer encoding provides additional information for improving the quality, size, quality, or bit rate of the decoded video relative to the first quality, the first size, or the first bit rate, respectively, and wherein the method further includes: determining that an automatic object recognition process does not meet a specified reliability metric; and in response to determining that the automatic object recognition process does not meet the specified reliability metric: requesting additional layer encoding from a network device; and searching for at least one object in a first decoded image portion using the additional layer encoding and the automatic object recognition process.

[0210] In Example 83, the subject of Examples 81 to 82 includes the use of a deep learning neural network to perform an automatic object recognition process.

[0211] In Example 84, the subject matter of Examples 81 to 83 includes determining that an error exists in a first decoded image portion based on an automatic object recognition process, which includes: receiving an object identifier from a communication server; and determining that the object cannot be recognized in the first decoded image portion.

[0212] In Example 85, the subject matter of Examples 81 to 84 includes determining that an error exists in a first decoded image portion based on an automatic object recognition process, which includes: receiving object information from a communication server; detecting an object in the first decoded image portion; and determining that the object does not match the object information.

[0213] In Example 86, the subject of Examples 81 to 85 includes the use of a channel decoder, wherein decoding a first portion of an encoded image to create a first decoded image portion.

[0214] In Example 87, the subject of Example 86 includes a channel decoder that is one of the following: a Viterbi decoder or a turbo decoder.

[0215] Example 88 is a computing device for improving the decoding performance of an image using object detection, the computing device comprising: a hardware processor; and a memory storing instructions that, when executed by the hardware processor, cause the computing device to perform operations including: receiving an encoded image; decoding a first portion of the encoded image to create a first decoded image portion; using an automatic object recognition process to search for at least one object in the first decoded image portion; determining, based on the automatic object recognition process, that an error exists in the first decoded image portion, and in response: correcting the first decoded image portion based on the automatic object recognition process to produce a corrected decoded image; encoding the corrected decoded image to create a corrected encoded image; using the corrected encoded image to decode a second portion of the encoded image to create a second corrected decoded image; and outputting the second corrected decoded image to a display device.

[0216] In Example 89, the subject of Example 88 includes a base layer encoding that encodes only a first quality, a first size, or a first bit rate, wherein an additional layer encoding provides additional information for improving the quality, size, quality, or bit rate of the decoded video relative to the first quality, the first size, or the first bit rate, respectively, and wherein the operation further includes: determining that the automatic object recognition process does not meet a specified reliability metric; and in response to determining that the automatic object recognition process does not meet the specified reliability metric: requesting additional layer encoding from a network device; and searching for at least one object in a first decoded image portion using the additional layer encoding and the automatic object recognition process.

[0217] In Example 90, the subject of Examples 88-89 includes operations in which the automatic object recognition process involves the use of a deep learning neural network.

[0218] In Example 91, the subject matter of Examples 88 to 90 includes an operation in which determining that an error exists in a first decoded image portion based on an automatic object recognition process includes: receiving an identifier of an object from a communication server; and determining that the object cannot be recognized in the first decoded image portion.

[0219] In Example 92, the subject of Examples 88-91 includes an operation in which determining that an error exists in a first decoded image portion based on an automatic object recognition process includes: receiving object information from a communication server; detecting an object in the first decoded image portion; and determining that the object does not match the object information.

[0220] In Example 93, the subject of Examples 88-92 includes the operation of decoding a first portion of an encoded image to create a first decoded image portion, which includes using a channel decoder.

[0221] In Example 94, the subject of Example 93 includes a channel decoder that is one of the following: a Viterbi decoder or a turbo decoder.

[0222] Example 95 is a machine-readable medium storing instructions for improving the decoding performance of an image using object detection, the instructions causing the machine to perform operations when executed by a machine, the operations including: receiving an encoded image; decoding a first portion of the encoded image to create a first decoded image portion; using an automatic object recognition process to search for at least one object in the first decoded image portion; determining, based on the automatic object recognition process, that an error exists in the first decoded image portion, and in response: correcting the first decoded image portion based on the automatic object recognition process to produce a corrected decoded image; encoding the corrected decoded image to create a corrected encoded image; using the corrected encoded image to decode a second portion of the encoded image to create a second corrected decoded image; and outputting the second corrected decoded image to a display device.

[0223] In Example 96, the subject of Example 95 includes a base layer encoding that encodes only a first quality, a first size, or a first bit rate, wherein an additional layer encoding provides additional information for improving the size, quality, or bit rate of the decoded video relative to the first quality, the first size, or the first bit rate, respectively, and wherein the operation further includes: determining that the automatic object recognition process does not meet a specified reliability metric; and in response to determining that the automatic object recognition process does not meet the specified reliability metric: requesting additional layer encoding from a network device; and searching for at least one object in a first decoded image portion using the additional layer encoding and the automatic object recognition process.

[0224] In Example 97, the subjects of Examples 95 and 96 include operations in which the automatic object recognition process includes the use of a deep learning neural network.

[0225] In Example 98, the subject matter of Examples 95 to 97 includes an operation in which an error is determined to exist in a first decoded image portion based on an automatic object recognition process, comprising: receiving an identifier of an object from a communication server; and determining that the object cannot be recognized in the first decoded image portion.

[0226] In Example 99, the subject matter of Examples 95 to 98 includes an operation in which an error is determined to exist in a first decoded image portion based on an automatic object recognition process, comprising: receiving object information from a communication server; detecting an object in the first decoded image portion; and determining that the object does not match the object information.

[0227] In Example 100, the subject of Examples 95 to 99 includes an operation in which decoding a first portion of an encoded image to create a first decoded image portion includes using a channel decoder.

[0228] In Example 101, the subject of Example 100 includes, where the channel decoder is one of the following: a Viterbi decoder or a turbo decoder.

[0229] Example 102 is an apparatus for improving the decoding performance of an image using object detection. The apparatus includes: a unit for receiving an encoded image; a unit for decoding a first portion of the encoded image to create a first decoded image portion; a unit for searching for at least one object in the first decoded image portion using an automatic object recognition process; a unit for determining, based on the automatic object recognition process, that an error exists in the first decoded image portion, and in response: a unit for correcting the first decoded image portion based on the automatic object recognition process to generate a corrected decoded image; a unit for encoding the corrected decoded image to create a corrected encoded image; a unit for using the corrected encoded image to decode a second portion of the encoded image to create a second corrected decoded image; and a unit for outputting the second corrected decoded image to a display device.

[0230] In Example 103, the subject of Example 102 includes a base layer encoding that encodes only a first quality, a first size, or a first bit rate, wherein an additional layer encoding provides additional information for improving the size, quality, or bit rate of the decoded video relative to the first quality, the first size, or the first bit rate, respectively, and wherein the device further includes: a unit for determining that an automatic object recognition process does not meet a specified reliability metric; and, in response to determining that the automatic object recognition process does not meet the specified reliability metric: a unit for requesting additional layer encoding from a network device; and a unit for searching for at least one object in a first decoded image portion using the additional layer encoding and the automatic object recognition process.

[0231] In Example 104, the subject matter of Examples 102-103 includes a unit for searching for at least one object in a first decoded image portion using an automatic object recognition process, which includes a unit for using a deep learning neural network.

[0232] In Example 105, the subject matter of Examples 102-104 includes a unit for determining that an error exists in a first decoded image portion based on an automatic object recognition process, comprising: a unit for receiving an identifier of an object from a communication server; and a unit for determining that the object cannot be recognized in the first decoded image portion.

[0233] In Example 106, the subject matter of Examples 102-105 includes a unit for determining that an error exists in a first decoded image portion based on an automatic object recognition process, comprising: a unit for receiving object information from a communication server; a unit for detecting objects in the first decoded image portion; and a unit for determining that the object does not match the object information.

[0234] In Example 107, the subject matter of Examples 102-106 includes, wherein the unit for decoding a first portion of an encoded image to create a first decoded image portion includes a unit for using a channel decoder.

[0235] In Example 108, the subject of Example 107 includes a channel decoder that is one of the following: a Viterbi decoder or a turbo decoder.

[0236] Example 109 is a method for improving decoding performance using predictive feedback information, the method comprising: receiving encoded communication data encoded with error-correcting codes, the encoded communication data being communication from a communication user; identifying predictive feedback information of the communication user based on previous communication of the communication user; using the predictive feedback information to create decoding feedback information, the decoding feedback information including predicted encoded data of communication sent by the communication user; decoding the encoded communication data by a decoder using the decoding feedback information to produce decoded communication data, the decoding using the decoding feedback information to select one or more of a plurality of possible decodes; and outputting the decoded communication data.

[0237] In Example 110, the subject of Example 109 includes a prediction feedback information comprising a word frequency database describing word frequency usage from past communications of a communicating user, and wherein creating the decoding feedback information comprises: identifying a first word and a second word in the word frequency database, the first word having a higher usage frequency than the second word; encoding the first word and the second word to create a first encoded word and a second encoded word; and using the first encoded word and the second encoded word, along with information indicating the usage frequency of the first encoded word and the second encoded word, to create the decoding feedback information.

[0238] In Example 111, the subject of Example 110 includes, wherein identifying the first word and the second word includes: identifying the first word and the second word within a threshold similarity between them, and: the difference in the frequency of use of the first word and the second word exceeds a first threshold, or the frequency of use of one of the first word or the second word is lower than a second threshold, and the frequency of use of the other of the first word or the second word is higher than a third threshold.

[0239] In Example 112, the subject of Example 111 includes, wherein identifying the first word includes identifying the first word in a portion of the decoded communication data.

[0240] In Example 113, the subject of Examples 111-112 includes decoding communication data, which includes one of the following: decoding chat data; decoding instant message data; decoding file data; or decoding voice communication.

[0241] In Example 114, the subject of Examples 109-113 includes the following: the decoded communication data is a document, and the sender of the document is a person different from the communication user.

[0242] In Example 115, the subject of Examples 109-114 includes the following: encoded communication data is one of the following: received through a communication medium or retrieved from a storage device.

[0243] In Example 116, the subject of Examples 109-115 includes, wherein, decoding encoded communication data by a decoder and using predictive feedback information to produce decoded communication data, and decoding using decoding feedback information to select one or more possible decodes from a plurality of possible decodes, includes: adjusting the probability of the decoding result in the turbo decoder based on the decoding feedback information.

[0244] In Example 117, the subject of Examples 109-116 includes the following: decoding encoded communication data by a decoder and using predictive feedback information to produce decoded communication data, and decoding using decoding feedback information to select one or more possible decodes from a plurality of possible decodes, including: adjusting the path cost of the grid-based decoder based on the decoding feedback information.

[0245] Example 118 is a computing device for improving decoding performance using predictive feedback information, the computing device comprising: a hardware processor; a memory storing instructions that, when executed by the hardware processor, cause the hardware processor to perform operations including: receiving encoded communication data encoded with error-correcting codes, the encoded communication data being communication from a communication user; identifying predictive feedback information of the communication user based on previous communication of the communication user; using the predictive feedback information to create decoding feedback information, the decoding feedback information including predicted encoded data of communication sent by the communication user; decoding the encoded communication data by a decoder using the decoding feedback information to produce decoded communication data, the decoding using the decoding feedback information to select one or more of a plurality of possible decodes; and outputting the decoded communication data.

[0246] In Example 119, the subject of Example 118 includes a predictive feedback information comprising a word frequency database describing word frequency usage from past communications of a communicating user, and wherein the operation of creating the decoding feedback information comprises: identifying a first word and a second word in the word frequency database, the first word having a higher usage frequency than the second word; encoding the first word and the second word to create a first encoded word and a second encoded word; and using the first encoded word and the second encoded word, along with information indicating the usage frequency of the first encoded word and the second encoded word, to create the decoding feedback information.

[0247] In Example 120, the subject of Example 119 includes the operation of identifying the first word and the second word, which includes: identifying the first word and the second word as being within a threshold similarity to each other, and: the difference in the frequency of use of the first word and the second word exceeds a first threshold, or the frequency of use of one of the first words or the second word is lower than a second threshold, and the frequency of use of the other of the first words or the second word is higher than a third threshold.

[0248] In Example 121, the subject of Example 120 includes an operation in which the identification of the first word includes identifying the first word in a portion of the decoded communication data.

[0249] In Example 122, the subject of Examples 120-121 includes decoding communication data, which includes one of the following: decoding chat data; decoding instant message data; decoding file data; or decoding voice communication.

[0250] In Example 123, the subject of Examples 118-122 includes the following: the decoded communication data is a document, and the sender of the document is a person different from the communication user.

[0251] In Example 124, the subject of Examples 118-123 includes the following: encoded communication data is one of the following: received through a communication medium or retrieved from a storage device.

[0252] In Example 125, the subject of Examples 118-124 includes the operation of decoding encoded communication data by a decoder and using predictive feedback information to produce decoded communication data, and decoding using decoding feedback information to select one or more possible decodes from a plurality of possible decodes including: adjusting the probability of the decoding result in the turbo decoder based on the decoding feedback information.

[0253] In Example 126, the subject matter of Examples 118-125 includes the operation of decoding encoded communication data by a decoder and using predictive feedback information to produce decoded communication data, and using decoding feedback information to select one or more possible decodes from a plurality of possible decodes including: adjusting the path cost of the grid-based decoder based on the decoding feedback information.

[0254] Example 127 is a machine-readable medium storing instructions for improving decoding performance using predictive feedback information, the instructions causing the machine to perform operations when executed by a machine, the operations including: receiving encoded communication data encoded with error-correcting codes, the encoded communication data being communication from a communication user; identifying predictive feedback information of the communication user based on previous communication of the communication user; using the predictive feedback information to create decoding feedback information, the decoding feedback information including encoded data predicting communication sent by the communication user; decoding the encoded communication data by a decoder and using the decoding feedback information to produce decoded communication data, the decoding using the decoding feedback information to select one or more of a plurality of possible decodes; and outputting the decoded communication data.

[0255] In Example 128, the subject of Example 127 includes a prediction feedback information comprising a word frequency database describing word frequency usage from past communications of a communication user, and wherein the operation of creating the decoding feedback information comprises: identifying a first word and a second word in the word frequency database, the first word having a higher usage frequency than the second word; encoding the first word and the second word to create a first encoded word and a second encoded word; and using the first encoded word and the second encoded word, along with information indicating the usage frequency of the first encoded word and the second encoded word, to create the decoding feedback information.

[0256] In Example 129, the subject of Example 128 includes the operation of identifying the first word and the second word, which includes: identifying the first word and the second word as being within a threshold similarity to each other, and: the difference in the frequency of use of the first word and the second word exceeds a first threshold, or the frequency of use of one of the first word or the second word is lower than a second threshold, and the frequency of use of the other of the first word or the second word is higher than a third threshold.

[0257] In Example 130, the subject of Example 129 includes an operation in which the identification of the first word includes identifying the first word in a portion of the decoded communication data.

[0258] In Example 131, the subject of Examples 129-130 includes decoding communication data, which includes one of the following: decoding chat data; decoding instant message data; decoding file data; or decoding voice communication.

[0259] In Example 132, the subject of Examples 127-131 includes the following: the decoded communication data is a document, and the sender of the document is a person different from the communication user.

[0260] In Example 133, the subject of Examples 127-132 includes the following: encoded communication data is one of the following: received through a communication medium or retrieved from a storage device.

[0261] In Example 134, the subject of Examples 127-133 includes the operation of decoding encoded communication data by a decoder and using predictive feedback information to produce decoded communication data, and decoding using decoding feedback information to select one or more possible decodes from a plurality of possible decodes including: adjusting the probability of the decoding result in the turbo decoder based on the decoding feedback information.

[0262] In Example 135, the subject matter of Examples 127-134 includes the operation of decoding encoded communication data by a decoder and using predictive feedback information to produce decoded communication data, and decoding using decoding feedback information to select one or more possible decodes from a plurality of possible decodes including: adjusting the path cost of the grid-based decoder based on the decoding feedback information.

[0263] Example 136 is an apparatus for improving decoding performance using predictive feedback information, the apparatus comprising: a unit for receiving encoded communication data encoded with error-correcting codes, the encoded communication data being communication from a communication user; a unit for identifying predictive feedback information of the communication user based on previous communication of the communication user; a unit for creating decoding feedback information using the predictive feedback information, the decoding feedback information including predicted encoded data of communication sent by the communication user; a unit for decoding the encoded communication data by a decoder using the decoding feedback information to produce decoded communication data, the decoding using the decoding feedback information to select one or more of a plurality of possible decodes; and a unit for outputting the decoded communication data.

[0264] In Example 137, the subject of Example 136 includes, wherein the predicted feedback information includes a word frequency database describing word frequency usage from past communications of a communication user, and wherein the unit for creating the decoded feedback information includes: a unit for identifying a first word and a second word in the word frequency database, the first word having a higher usage frequency than the second word; a unit for encoding the first word and the second word to create a first encoded word and a second encoded word; and a unit for using the first encoded word and the second encoded word, as well as information indicating the usage frequency of the first encoded word and the second encoded word, to create the decoded feedback information.

[0265] In Example 138, the subject of Example 137 includes, wherein the unit for identifying the first word and the second word includes: a unit for identifying the first word and the second word as being within a threshold similarity to each other and the following: the difference in the frequency of use of the first word and the second word exceeds a first threshold, or the frequency of use of one of the first word or the second word is lower than a second threshold, and the frequency of use of the other of the first word or the second word is higher than a third threshold.

[0266] In Example 139, the subject of Example 138 includes a unit for identifying the first character, wherein the unit for identifying the first character is a unit for identifying the first character in a portion of the decoded communication data.

[0267] In Example 140, the subject of Examples 138-139 includes decoding communication data including one of the following: decoding chat data; decoding instant message data; decoding file data; or decoding voice communication.

[0268] In Example 141, the subject of Examples 136-140 includes the following: the decoded communication data is a document, and the sender of the document is a person different from the communication user.

[0269] In Example 142, the subject of Examples 136-141 includes the following: encoded communication data is one of the following: received through a communication medium or retrieved from a storage device.

[0270] In Example 143, the subject matter of Examples 136-142 includes a unit for decoding encoded communication data by a decoder and using predictive feedback information to produce decoded communication data, wherein decoding using decoding feedback information to select one or more possible decodes from a plurality of possible decodes includes a unit for adjusting the probability of the decoding result in the turbo decoder based on the decoding feedback information.

[0271] In Example 144, the subject matter of Examples 136-143 includes a unit for decoding encoded communication data by a decoder and using predictive feedback information to produce decoded communication data, wherein decoding using decoding feedback information to select one or more possible decodes from a plurality of possible decodes includes a unit for adjusting the path cost of the grid-based decoder based on the decoding feedback information.

[0272] Example 145 is at least one machine-readable medium including instructions that, when executed by processing circuitry, cause the processing circuitry to perform operations to implement any one of Examples 1-144.

[0273] Example 146 is a device that includes units for implementing any one of Examples 1-144.

[0274] Example 147 is a system for implementing any one of Examples 1-144.

[0275] Example 148 is a method for implementing any one of Examples 1-144.

Claims

1. A method for improving decoder performance using speaker speech characteristics, the method comprising: The speaker's voice characteristics are received via a network; Receive the coded voice data of the speaker; The first portion of the speaker's encoded voice data is decoded using a decoder to generate decoded voice data; The decoded voice data is compared with the received voice characteristics of the speaker; Based on the result of comparing the decoded voice data with the received speech characteristics of the speaker, it is determined that the decoded voice data does not correspond to the characteristics of the received speaker; as well as In response to determining that the decoded voice data does not correspond to the characteristics of the received speaker: The decoded speech data is corrected based on the speaker's speech characteristics to create corrected speech data; as well as When the second portion of the encoded voice data is decoded by the decoder to create second decoded voice data, at least one potential decoding possibility is eliminated by utilizing the corrected voice data.

2. The method according to claim 1, wherein, Correcting the decoded speech data based on the speaker's speech characteristics to create corrected speech data includes using a deep learning neural network.

3. The method according to claim 1, wherein, The characteristic includes Mel frequency cepstral coefficients.

4. The method according to claim 1, wherein, The characteristic is one of the following: sent by the speaker as part of a communication session or received from a network-based service.

5. The method according to claim 4, wherein, The feature is transmitted by the speaker as part of the speech during the communication session.

6. The method according to claim 1, further comprising: The corrected voice data is encoded to create encoded corrected voice data; and Wherein, when the decoder decodes the second portion of the encoded voice data to create the second decoded voice data, eliminating the at least one potential decoding possibility by utilizing the corrected voice data includes: utilizing the encoded corrected voice data.

7. The method according to claim 1, wherein, Determining that the decoded voice data does not correspond to the speaker's characteristics based on the comparison between the decoded voice data and the speaker's voice characteristics includes using the decoded voice data and the speaker's characteristics as input to a deep learning network.

8. The method according to claim 1, wherein, Determining that the decoded voice data does not correspond to the speaker's characteristics based on the comparison between the decoded voice data and the speaker's speech characteristics includes: The characteristics of the speaker are used to create a transcription of the decoded speech data; The frequency of the first word in the transcription is less than a specified frequency in the speaker's word frequency distribution data; In response to recognizing the first word, the first word in the transcription is changed to a second word to create an updated transcription, wherein the second word is used at a frequency greater than a second specified frequency; and The step of correcting the decoded speech data to correspond to the speaker's speech characteristics to create corrected speech data includes applying a deep learning neural network to the updated transcript to create corrected speech data.

9. The method according to claim 1, wherein, When the second portion of the encoded voice data is decoded by the decoder to create the second decoded voice data, eliminating the at least one potential decoding possibility by utilizing the corrected voice data includes one of the following: trimming the decoder grid, updating the path cost, or re-decoding the first portion before decoding the second portion.

10. The method according to claim 1, wherein, Receiving the speaker's voice characteristics via the network includes calculating the characteristics using previously received speech data of the speaker.

11. The method according to claim 1, wherein, Receiving the speaker's voice characteristics via the network includes receiving the characteristics without calculating them from previously received speech data of the speaker.

12. The method according to claim 1, wherein, Determining that the decoded voice data does not correspond to the characteristics of the received speaker based on the result of comparing the decoded voice data with the speech characteristics of the received speaker includes: determining that one or more of the frequency, amplitude, rhythm, pitch, or one or more Mel frequency cepstral coefficients of the decoded voice data are higher or lower than a pre-specified threshold for the corresponding characteristics of the speaker.

13. The method according to claim 1, wherein, When the second portion of the encoded voice data is decoded by the decoder to create the second decoded voice data, eliminating the at least one potential decoding possibility by utilizing the corrected voice data includes one of the following: removing a path through the grid data structure, increasing the path cost of the potential decoding possibility, or reducing the probability metric of the potential decoding possibility.

Images