Semiautomated relay method and apparatus

Abstract

A relay for captioning a hearing user's (HU's) voice signal during a phone call between an HU and a hearing assisted user (AU), the HU using an HU device and the AU using an AU device where the HU voice signal is transmitted from the HU device to the AU device, the relay comprising a display screen, a processor linked to the display and programmed to perform the steps of receiving the HU voice signal from the AU device, transmitting the HU voice signal to a remote automatic speech recognition (ASR) server running ASR software that converts the HU voice signal to ASR generated text, the remote ASR server located at a remote location from the relay, receiving the ASR generated text from the ASR server, present the ASR generated text for viewing by a call assistant (CA) via the display and transmitting the ASR generated text to the AU device.

Description

CROSS-REFERENCE TO RELATED APPLICATIONS[0001]This application is a continuation-in-part of U.S. patent application Ser. No. 15 / 729,069 which was filed on Oct. 10, 2017, and which is titled “SEMIAUTOMATED RELAY METHOD AND APPARATUS” which is a continuation in part of U.S. patent application Ser. No. 15 / 171,720, filed on Jun. 2, 2017, and titled “SEMIAUTOMATED RELAY METHOD AND APPARATUS”, which is a continuation-in-part of U.S. patent application Ser. No. 14 / 953,631, filed on Nov. 30, 2015, and titled “SEMIAUTOMATED RELAY METHOD AND APPARATUS”, which is a continuation-in-part of U.S. patent application Ser. No. 14 / 632,257, filed on Feb. 26, 2015, and titled “SEMIAUTOMATED RELAY METHOD AND APPARATUS”, which claims priority to U.S. provisional patent application Ser. No. 61 / 946,072 filed on Feb. 28, 2014, and titled “SEMIAUTOMATED RELAY METHOD AND APPARATUS”, and claims priority to each of the above applications, each of which is incorporated herein in its entirety by reference.STATEMEN...

AI Technical Summary

Benefits of technology

[0017]It has been recognized that a hybrid semi-automated system can be provided where, when acceptable accuracy can be achieved using automated transcription software, the system can automatically use the transcription software to transcribe HU voice messages to text and when accuracy is unacceptable, the system can patch in a human CA to transcribe voice messages to text. Here, it is believed that the number of CAs required at a large relay facility may be reduced appreciably (e.g., 30% or more) where software can accomplish a large portion of transcription to text. In this regard, not only is the automated transcription software getting better over time, in at least some cases the software may train to an HU's voice and the vagaries associated with voice messages received over a phone line (e.g., the limited 300 to 3000 Hz range) during a first portion of a call so that during a later portion of the call accuracy is particularly good. Training may occur while and in parallel with a CA manually (e.g., via typing, revoicing, etc.) transcribing voice-to-text and, once accuracy is at an acceptable threshold level, the system may automatically delink from the CA and use the text generated by the software to drive the AU display device.

Problems solved by technology

When it comes to remotely communicating using a telephone, unfortunately, there is no way for a hearing impaired person (e.g., an assisted user (AU)) to use sight to compensate for hearing loss as conventional telephones do not enable an AU to see a person on the other end of the line (e.g., no lip reading or sign viewing).

For others with more severe hearing loss conventional telephones cannot compensate for their loss and telephone communication is a poor option.

Erroneous words provide inaccurate messages and also can cause confusion for an AU reading transcribed text.

In this regard, text that trails a voice message by several seconds can cause confusion.

However, at times a CA can fall behind when captioning so that longer delays (e.g., 10-15 seconds) occur.

CA jobs are also relatively high pressure jobs and therefore turnover is relatively high when compared jobs in many other industries which further increases the costs associated with operating a relay.

Professional typists are relatively expensive.

In addition, despite being able to keep up with a conversation most of the time, at other times (e.g., during long conversations or during particularly high speed conversations) even professional typists fall behind transcribing real time text and more substantial delays can occur.

In the past there have been several problems with this solution which have resulted in no one implementing a workable system.

First, most voice messages (e.g., an HU's voice message) delivered over most telephone lines to a relay are not suitable for direct voice-to-text transcription software.

Thus, automated transcription software does not work well with voice signals delivered over a telephone line and large numbers of errors occur.

Accuracy further suffers where noise exists on a telephone line which is a common occurrence.

Second, many automated transcription software programs have to be trained to the voice of a speaker to be accurate.

When a new HU calls an AU's device, there is no way for a relay to have previously trained software to the HU voice and therefore the software cannot accurately generate text using the HU voice messages.

Third, many automated transcription software packages use context in order to generate text from a voice message.

Systems that hold off on presenting text to correct using subsequent context cause delay in text presentation which is inconsistent with the relay system need for real time or close to real time text delivery.

Images