Multilingual dialog tool

CN115867916BActive Publication Date: 2026-10-09CABO APPL LLC
View PDF 4 Cites 0 Cited by

Patent Information

Application Number
CN202180047076.8
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Priority Date
2020-07-02
Filing Date
2021-06-29
Publication Date
2026-10-09
Estimated Expiration
2041-06-29

AI Technical Summary

Technical Problem

存在手持式电子翻译器,其中输入语句(通过文本或语音输入)被翻译成其它语言,但当用户不知道用户希望与之沟通的人的语言时(因为用户不知道要选择哪种输出语言),这些翻译器会造成困难

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN115867916B_ABST
    Figure CN115867916B_ABST
Patent Text Reader

Abstract

A user's computing device (e.g., a smartphone) runs an application through which the user can connect with a nearby bystander who can speak a different language. After a connection is established with the bystander, the user can enter a message in the user's primary language. The message is translated into the bystander's primary language, and each such translated message is also back-translated into the user's primary language for review and editing. The back-translated message allows the user to identify problems in the original message that can cause the bystander to misunderstand. Subsequent editing of the original message and / or back-translated message causes updates to all of the original message, translated message, and back-translated message, and, upon the user's approval, the translated message is sent to the bystander, who can respond in the same manner via his or her device.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This document relates to an invention generally relating to devices and methods for interpersonal communication, and more specifically, to devices and methods for enabling multilingual dialogue. Background Technology

[0002] Travelers and others often benefit from the ability to communicate easily with people who don't speak their language. Handheld electronic translators exist, where input (text or voice) is translated into other languages, but these translators can be difficult when the user doesn't know the language of the person they wish to communicate with (because the user doesn't know which output language to choose). Furthermore, these translators often suffer from translation flaws due to context, idioms, multiple meanings, and other challenges. The difficulties posed by language barriers are compounded by the awkwardness of introducing oneself to people who may have different cultural standards and / or who may simply be unwilling to communicate with non-speakers (whether out of shyness, a need to avoid potential frustration, or other reasons). Many existing handheld translators require close proximity between the parties involved in the communication and / or require the sharing and transmission of potentially sensitive personal information, such as phone numbers, email addresses, usernames, device information, etc., raising privacy and information security concerns. Summary of the Invention

[0003] This invention is defined by the claims appended to this document and relates to methods and apparatus for at least partially alleviating the aforementioned problems. A basic understanding of some features of a preferred version of the invention can be obtained from a review of the following brief summary of the invention, with further details provided elsewhere in this document. To aid the reader's understanding, the following summary refers to the accompanying drawings (which are briefly summarized in the "Description of Drawings" section following the Summary of the Invention section of this document).

[0004] This invention allows a user to communicate with one or more bystanders who may not know the user's language using a computing device (preferably a mobile computing device such as a smartphone). The invention can then take the form of a program (“application”) running on both the user's smartphone and the bystanders' smartphones, during which the processes described below occur. It should be understood that, for clarity, the term “user” is used to refer to any particular user, while the term “bystander” refers to other nearby users. Thus, from the perspective of each bystander, each bystander is also a “user,” and from the perspective of other users, each user is also a “bystander.”

[0005] Users create profiles stored by the application, which contain information about the user's identity and preferred language for communication. For example... Figure 1As shown at point 104, when the application is running, the user can indicate his / her communication status as "available" (i.e., seeking or willing to communicate with others) or "unavailable." When the user's communication status is set to available, an introductory data packet is transmitted from the user's device (at point 106) to a nearby bystander via a suitable short-range wireless transmission protocol (e.g., Bluetooth) that the user's device can accommodate. The introductory data packet contains a packet header (a unique identifier corresponding to the user's device and therefore to the user), and a payload containing at least some brief information about the user's identity (preferably, at least an image of the user and the user's preferred language). The introductory data packet is received by the device of the nearby bystander running the application and is at least partially displayed on said device (at point 108). Thus, for example, at an airport where an "available" user is running the application, a bystander within the user's transmission range may see the user's image and preferred language (e.g., Bluetooth) on their device. Figure 4 (As illustrated). This allows each bystander to visually identify nearby users and, if the bystander knows the user's preferred language, to communicate verbally with the user. Otherwise, the bystander can communicate with the user via an application on the bystander's device, as described below. The user also receives introductory data packets from any nearby "available" bystanders (in... Figure 1 (at location 110), his / her device at least partially displays the contents of an introductory data packet for the bystanders, instructing them to communicate when they have time.

[0006] Once a user sees nearby bystanders on their device, the user can select one or more bystanders with whom they wish to communicate, for example, by pressing the image / icon / list of the selected bystander on the screen of the user's device. Such a selection sends a connection request from the user's device to each selected bystander (at 112), i.e., a request for further private communication with the selected bystander, where both the connection request and further communication are sent via a short-range wireless transmission protocol. The selected bystander's device then displays the user's connection request (e.g., by highlighting or otherwise altering the appearance of the user's image / icon / list) and allows the selected bystander to accept the connection request (at 114, such as by pressing the user's image / icon / list on the screen of the bystander's device). As shown at 116, any bystander can similarly select the user (or another bystander), thereby sending a connection request to the selected party for possible acceptance by the receiver.

[0007] The acceptance of the connection request establishes a connection between the observer and the user, enabling further communication between the connected parties (the "dialogue" at point 118). Either the user or the observer can initiate a dialogue; the following discussion assumes the user initiates it, but the same steps are performed when the observer initiates a dialogue. Reference Figure 2The user enters a user message on his / her device (at 202), which is displayed on the screen of the user's device (at 206). The user message is then translated (at 208) into one or more translated user messages using a translation engine (discussed in more detail below), each translated user message presented in the preferred language of the corresponding bystander among the connected bystanders (and preferably also displayed on the screen of the user's device, at 210). Each translated user message is then translated into a corresponding back-translated message in the user's preferred language (at 212, via a different or the same translation engine), which is displayed on the screen of the user's device (at 214). The display of the user message and the back-translated message provides a profound understanding of the meaning that the bystanders will glean from the translated user message. If the intended meaning of the user message displayed from the back-translated message is not correctly stated, the user can edit one or more of the user message and the back-translated message (at 216). The edited message is then translated and back-translated again, with at least the back-translated message being displayed to the user. The user can then edit one or more messages and back-translate them, with the steps of translation, back-translation, and editing performed until the user is satisfied with the message. The user can then indicate their approval of the user message (e.g., by pressing the "Send" button on the user's display), at which point each translated user message is transmitted to its corresponding observer's device and displayed in each observer's preferred language (at 220).

[0008] Other potential advantages, features and objects of the present invention will be apparent from the remainder of this document in conjunction with the accompanying drawings. Attached Figure Description

[0009] Figure 1 It is a flowchart that schematically illustrates the process of introduction, connection and dialogue between a user and an observer.

[0010] Figure 2 This is a flowchart illustrating the process of transforming a message from a user into a message for an observer, where optional steps are defined by dashed lines (broken lines).

[0011] Figure 3 This is a schematic diagram illustrating the process of presentation of the computing devices of the user (at 302) and the observer (at 304).

[0012] Figure 4 An exemplary device screen 402 depicts a user, showing nearby bystanders with whom the user can establish connections, and an exemplary device screen 404 depicts other bystanders with whom the user and bystanders can establish connections.

[0013] Figure 5An exemplary message 506 that a user can input on a user's device is shown, along with corresponding translated messages 510 and back-translated messages 514.

[0014] Figures 6a-6c The steps for a conversation between users who speak Spanish, English, and Hindi are shown. Detailed Implementation

[0015] This invention relates to a method for facilitating introductions and two-way dialogue between people who speak different languages, preferably executed by an application running on a smartphone, laptop computer, or other computing device. The invention also relates to apparatus that allows such introductions and dialogues, i.e., computing devices that execute the aforementioned application or otherwise implement the aforementioned method. Users can connect with nearby users (bystanders) whom they are not previously acquainted with. Users who are geographically close (e.g., within 100 meters) and each running an application on their devices can connect and engage in dialogue (i.e., share one or more messages) without sharing their names, phone numbers, or other sensitive personal information.

[0016] When a user is running his / her application and has set his / her status to "available" for conversation within the application, the application uses any known wireless communication technology that supports peer-to-peer connections and transmissions (e.g., Bluetooth Low Energy, Zigbee, Ultra Wideband, or another Wireless Personal Area Network (WPAN) protocol) to have the user's device send an introduction (introductory data packet). The transmitted introductory data packet contains a packet header (a unique identifier corresponding to the user's device and therefore to the transmitting user) and a payload. The payload preferably contains an encrypted / encoded user profile, and preferably at least includes the user's preferred language for communication and an image of the user. If the user's application does not have an image available, and / or if the user's image exceeds a certain predefined age, the user's application may prompt the user to capture a new image of the user. This new image can be verified to correspond to the user via facial recognition, such as by comparing it with the user image captured when the user initially created his / her account. Here, the purpose is to verify that the image presented in the introductory data packet actually corresponds to the user and to prevent the user from transmitting introductory data packets with images of other people (e.g., images of other people nearby). User profiles may optionally include other data such as the age of the image (e.g., the date the image was created), username, spoken language, gender, age, occupation (e.g., "student," "IT consultant," etc.), a brief description chosen by the user (e.g., "professional traveler," "vacationer," "local," "art lover," "skier," etc.), and / or the preferred format (text, audio, or both) of messages the user receives from other users. In addition to or instead of text, some of this information may be in audio format (e.g., recorded by the user).

[0017] Accompanying Figure 3 An example is shown. Devices 302 and 304 of both the first and second users are running an application and are transmitting their introductory data packets (introductions) at frequent intervals. Nearby devices within range are listening (i.e., receiving) the transmitted introductory data packets, and thus the first user's device 302 is listening for the transmitted introductory data packets from the second user's device 304, and similarly, the second user's device 304 is listening for the transmitted introductory data packets from the first user's device 302. Another user's device 306 also receives introductory data packets from devices 302 and 304, as does the unknown device 308. When each user's introductory data packet is received by another user's device, the application verifies (decodes / decrypts) the received packet and displays some or all of the user profiles contained in the received introductory data packet in the "Nearby Users" section of the application (see [link to application]). Figure 4The displayed user profile preferably includes at least an image of another user and a preferred language for communication, but it may also include other details from another user's profile. Thus, each user can see at least a portion of the profiles of other nearby users, potentially allowing the user to visually identify other users (e.g., a user sees someone nearby whose appearance corresponds to the image in the displayed profile).

[0018] The user can then use their device to initiate a one-on-one or group conversation with other nearby users (spectators) by selecting one or more of the displayed nearby spectators, thereby sending a connection request (i.e., a Bluetooth or other short-range wireless signal indicating the user's desire to participate in further communication) to each selected spectator. A connection is established when a spectator accepts the connection request, and a conversation can begin (as described below). This process... Figure 4 The diagram, partially shown, depicts the device screens of an English-speaking user and a Spanish-speaking bystander at points 402 and 404, respectively. Their screens display images and profile information of other bystanders (including each other), with audio icons visible next to those bystanders who provide brief audio introductions as part of their user profiles (which the user can play by clicking the icons). When the English-speaking user clicks on the icon / list 406 of a Spanish-speaking bystander, a pop-up window 408 appears on the user's screen 402 in the lower left corner, asking the user to confirm whether they wish to send a connection request to the Spanish-speaking bystander. Once confirmed, a pop-up window 410 appears on the bystander's screen 404 in the lower right corner, asking the bystander to indicate whether they wish to accept the connection request. Once accepted, either the user or the bystander can begin a conversation by sending a message to the other in text and / or audio format. The receiving bystander can receive messages in text and / or audio format (based on their chosen configuration).

[0019] Users can choose to update their status from "Available" to "Do Not Disturb" or "Not Available." The "Do Not Disturb" status prevents others from joining the conversation (i.e., rejects newly received connection requests) but allows users to send requests to start a new conversation (i.e., sends a new connection request). The "Not Available" status stops transmitting user availability (i.e., stops transmitting the user's introductory data packet, thereby stopping the display of the user's profile on nearby devices and preventing other users from sending connection requests to the user). Users can also selectively set their display status to "Available" only for certain selected users based on information in their user profile (e.g., male or female user, user in a specific age group, etc.). Here, the payload of the introductory data packet transmitted by the user contains availability data instructing bystanders whether their devices should display the user's profile to them.

[0020] The dialogue can contain messages in audio input mode and / or text input mode. In audio input mode, the user can speak a message to his / her device, and a bystander can hear and / or read the message in the bystander's preferred language (i.e., the user's message is translated into the bystander's language if needed). Preferably, voice cloning is used to transmit the user's audio message to the bystander in the user's voice. This process is handled by... Figure 2 The selected steps shown indicate:

[0021] (1) At 202, the user’s device captures audio messages (words spoken by the user).

[0022] (2) At 204, the message is sent to a speech-to-text transcription engine, which converts the user's message into text in the primary language set in the user's profile. The transcription engine can be one or more of the following, or any other suitable speech-to-text application:

[0023] Google Speech API (Google Inc., Mountain View, California, USA)

[0024] IBM Watson API (International Business Machines Corporation, Armonk, NY, USA)

[0025] Microsoft Azure (Microsoft Corporation, Redmond, Washington, USA)

[0026] Amazon Speech API (Amazon, Seattle, Washington, USA)

[0027] Nuance Speech API (Nuance Inc., Burlington, Massachusetts, USA)

[0028] Temi Transcription API (Temi Corporation, San Francisco, California, USA)

[0029] FF Transcription (Wayne Company, Pennsylvania, USA)

[0030] Rev.AI (Rev Inc., San Francisco, California, USA)

[0031] TranscribeMe (Oakland, California, USA)

[0032] TakeNote (London, England, UK)

[0033] Wit.AI Speech API (Wit.AI, Palo Alto, California, USA)

[0034] (3) At 208, the transcribed text of the user's audio message is sent to the translation engine. The translation engine converts the transcribed text of the user's message into text in the preferred language set in the observer's user profile. The translation engine can be one or more of the following, or any other suitable translation application:

[0035] Google Translate (Google Inc., Mountain View, California, USA)

[0036] Microsoft Translate (Microsoft Corporation, Redmond, Washington, USA)

[0037] IBM Watson (International Business Machines Corporation, Armonk, NY, USA)

[0038] Yandex Translate (Yandex Corporation, Moscow, Russia)

[0039] Unabel (Lisbon, Portugal)

[0040] SysTran.io Translation API (Systran, Paris, France)

[0041] Stepes (San Francisco, California, USA)

[0042] SmartLing (New York, USA)

[0043] Tilde (Riga, Latvia)

[0044] Straker Translations (Auckland, New Zealand)

[0045] Amazon Translate (Amazon, Inc., Seattle, Washington, USA)

[0046] (4) At 218, the text containing the user's message is sent to a text-to-speech engine in an observer's language, thereby converting the text into audio. The audio is preferably also processed by one or more of an accent translator engine (which accents the translated message to make it sound less machine-like and more human) and / or a voice cloning engine (which adjusts pitch, tone, accent, etc., to better match the user's pitch, tone, accent, so that the user's translated message sounds as if it were actually spoken by the user). Many of the speech-to-text engines mentioned above also provide a text-to-speech engine and / or have a built-in accent translator engine. Exemplary voice cloning engines that can be used include:

[0047] CereVoice Me (CereProc, Edinburgh, Scotland, UK)

[0048] ISpeech.org Voice Cloning (Speech Inc., Newark, New Jersey, USA)

[0049] Lyrebird (Lyrebird AI, Montreal, Quebec, Canada)

[0050] Resemble.AI (Resemble AI, Toronto, Ontario, Canada)

[0051] Baidu Deep Voice (Baidu Corporation, Beijing, China)

[0052] (5) At 220, the generated audio message is sent to the bystander, thereby presenting the user’s audible message (which is generated in the user’s primary language) as an audible message in the bystander’s language (and preferably in the user’s voice).

[0053] The transcription engine, translation engine, text-to-speech engine, accent translator engine, and voice cloning engine can be on the user's device, but are preferably network-based or remotely accessible by an application (preferably by the user's application, but possibly at least partially by a bystander's application). The user may choose to send the final audio message in real time (i.e., immediately after the user's message has been fully processed), or alternatively, may choose to deliver the final audio message to a bystander after the user has had the opportunity to review and approve the final audio message (e.g., by using steps 206, 210, 212, 214, and 216 as described below).

[0054] In text input mode, the user can type a text message to send to the bystander (at 202). The text message is then provided to a translation engine (at 208) to generate text in the bystander's preferred language, which can then be delivered to the bystander via the application (at 220). The translated message can be received by the bystander in text and / or audio form (at 218, by using the aforementioned text-to-speech engine) according to preferences set in the bystander's user profile.

[0055] Regardless of whether the user enters a message using audio or text input mode, the text of the message typed or spoken by the user should ideally be displayed to the user along with the message that has been converted into the observer's language (at 214) and then back-translated into the user's preferred language (at 206). Translation engines do not always translate in the expected or anticipated context, and therefore, displaying the message the user initially entered, along with the back-translated message, can help reduce errors and misunderstandings. Figure 5An example is shown. Here, a user (whose primary language is English) attempts to send the message "May I borrow your book?" to a bystander whose preferred language is Italian. The user types this phrase, or otherwise speaks it (where the spoken phrase is converted to text), causing step 506 ( Figure 2 At point 206 in the code, the user sees this phrase; in step 508 ( Figure 2 At step 208), the English phrase is translated into Italian via a translation engine (for practical or suggestive communication to an observer); in step 512 ( Figure 2 At point 212, the Italian is translated using the same translation engine or preferably one or more different translation engines (at point 510 or...). Figure 2 210) is back-translated into English; and in step 514 ( Figure 2 At point 214 in section B, the translated English phrase is displayed to the user along with the initially entered English phrase. For example... Figure 5 As seen in the example, the user intended to send the message "May I borrow your book?", but the translated message in Italian reads as "I can borrow your book" because the initial input message lacked a question mark, causing the translation engine to misunderstand the context during translation. Figure 2 As seen in step 216, providing the user with a back-translated message before transmitting the translated message allows the user to edit his / her message (e.g., add a question mark), thereby updating and potentially correcting the message before its transmission. When the user edits the message he / she initially entered, both the translated and back-translated messages are updated, and the back-translated information is displayed to the user before the translated message is passed to a bystander (in...). Figure 2 Step 214 and Figure 5 (At step 514) to obtain approval. Then, the user can preferably send the approved message to the bystander in text and / or (preferably, voice-cloned) audio form via the short-range communication protocol used for the above-described transmission between the two parties.

[0056] Figures 6a-6c This demonstrates group conversations between users / bystanders speaking English, Spanish, and Hindi. Figure 6a In the example, at point 602, a Spanish-speaking user initiates a conversation, displaying the user's Spanish-language message along with its translated version. Also shown are the messages received by an English-speaking bystander at point 604 (upon being acknowledged and passed on), and by a Hindi-speaking bystander at point 606. (As discussed above, messages can be entered and / or received in text and / or audio format, with the microphone icon pressed for voice input and the speaker icon pressed for audio output.) Figure 6bIn section 608, an English-speaking bystander enters an English reply containing a typo ("restuarant" instead of "restaurant"), and a back-translated message is returned. The Hindi back-translated message contains an incorrect translation, which can explain to the English-speaking bystander that the reply he / she entered might benefit from editing. Then, Figure 6c The edited reply message from the English-speaking bystander at point 610 and its translated message are shown. The edited reply message (when acknowledged and transmitted) is received by the Spanish-speaking bystander at point 612 and by the Hindi-speaking bystander at point 614.

[0057] Figure 1 A simplified diagram of a preferred version of the dialogue process is provided. At 102, the user activates the application. Once the user sets his / her status to "available" at 104, the application is ready to introduce itself at 106, begins sending user introductions (introductory data packets) at 108, and begins receiving bystander introductions (introductory data packets) from "available" bystanders at 110. (Alternatively, the user's application may begin receiving bystander introductions from "available" bystanders after the user's application is activated, but will only begin sending user introductions when the user's status is set to "available.") Users and nearby bystanders within range of each other's devices are then visible to each other on their devices, preferably at least through their displayed images (such as...). Figure 4 As shown in the diagram. Then, the application prepares connection request 112, through which the user can select one or more bystanders to talk to (where the selected bystander receives the connection request from the user).

[0058] Alternatively or additionally, the user can be selected by one or more bystanders for the conversation, in which case the user receives connection requests from each of such bystanders.

[0059] Then, conversation 118 may begin after an observer accepts a connection request from user 114, or after a user accepts a connection request from observer 116. The user may send a message to any or all observers who accept the user's connection request, wherein the user's message (if in audio format, transcribed into text) is displayed to the messaging user as: (1) the text of the message entered by the user; (2) optionally, the text translated into the primary language of each connected observer; and (3) the translated text translated back into the user's preferred language. The user may then edit any one or more of (1)-(3), wherein the editing causes updates to the other items in (1)-(3). Upon the messaging user's approval, the message is provided to other connected users in their preferred language (in text and / or audio format) (at 120). At 122, observers may similarly send messages to the user (and to any or all other observers who accept the observer's connection request) in the same manner.

[0060] The exemplary versions of the present invention described above may optionally include any one or more of the following features.

[0061] As a response Figure 2 Enhancements to the dialogue steps shown and described above, this invention can display user messages "red-lined" instead of translated messages. For example, text present in the original user message but not in the translated message is shown as underlined in the user message, and text present in the translated message but not in the original user message is shown as deleted. In this case, the translated message may not be shown separately, but may only be shown as embedded in the user message (via clicked text and text shared with the user message).

[0062] As another modification to the above dialogue steps, and as Figure 2 As shown, a user can enter their user message in their preferred language and can approve the message (after any necessary editing). At least for those bystanders whose approved user message matches the translated message, if the approved user message matches the translated message (which may not be displayed to the user), the translated user message (which may also not be displayed to the user) can then be automatically transmitted to the corresponding bystander. However, for any translated message that does not match the approved user message, the corresponding translated message (and possibly the corresponding translated user message) can be displayed to the user for review, editing, and approval / sending (where the translated message is displayed completely independently of the user message, or embedded within the user message highlighted in red).

[0063] This invention can accommodate visually impaired users by specifying in the user profile that all messages delivered to this user are delivered via audio. If a connection request is sent from an observer to such a user, the observer can be prompted to record a brief audio description to be included as part of the connection request (if such a description has not yet been recorded). The user can then hear this description upon receiving the connection request, which is preferably translated into the user's language as described above.

[0064] Conversations (text and / or audio messages) can be saved by the user (preferably transcribed by a bystander), tagged, searched by keywords, and shared / distributed to other media such as email, social media, team spaces such as Slack, etc. (preferably with any additional content specified by the user, such as photos / images, text, etc.).

[0065] As mentioned above, introductions, connection requests, and conversations (messages) are preferably sent via short-range wireless transmission protocols (which, for the purposes of this document, should be considered protocols with a range of 15 meters or less). This effectively limits communication with nearby users / bystanders. However, in addition to restricting the use of the application to short-range communication, the application can also, or alternatively, allow long-range communication. Here, the user can select bystanders from his / her list of SMS (Short Message Service / Text Messages), phone, and / or email contacts, and use SMS, cellular, and / or Internet protocols to deliver introductions, connections, and conversations instead of via short-range transmission protocols. The invention can also be implemented on social media platforms or other platforms where users talk to each other. Short-range and long-range communication modes may also be used; for example, introductions and connection requests can be sent via short protocols, and then conversations can be conducted via long protocols.

[0066] This invention can also be used as a learning tool by generating translated audio and / or text messages in a selected language without initiating a conversation with another user. The generated messages can be stored for review and / or later use (e.g., sent to the recipient during a subsequent conversation) or deleted.

[0067] In addition to the types of stored messages mentioned above, applications can also generate and / or store "standard" messages, such as:

[0068] How much will that cost?

[0069] I have a food allergy

[0070] Where is the nearest hospital?

[0071] I feel uncomfortable

[0072] Are there any restaurants nearby?

[0073] Are there any ATMs nearby?

[0074] These "standard" messages can be selected to be sent to the recipient during the conversation.

[0075] The present invention preferably enables access to and configuration of various digital wallets (e.g., digital wallets for cryptocurrencies) to transfer funds to another user during a conversation.

[0076] This invention can be incorporated into a location-based currency converter, thereby automatically converting any currency amount recorded in a message (and can be assumed to be the user's "national currency" as described in the user's profile) into the local currency. The local currency can be manually set by the user or automatically updated based on location provided by any GPS (Global Positioning System) or other location services incorporated into the user's device.

[0077] It should be understood that the versions of the invention described above are merely exemplary, and the invention is not intended to be limited to these versions. Specifically, the scope of the invention is defined only by the listed claims, and the invention covers all different versions that literally or equivalently fall within the scope of these claims.

Claims

1. A method for enabling multilingual communication between a user's computing device and the computing devices of one of a plurality of bystanders, the method comprising the steps of: a. Receive information about the observer's preferred language from the observer's device on the user's device; b. Perform the following message input steps on the user's device: (1) Accepting user input of a user message, which is displayed on the screen of the user's device. (2) Translate the user message into a translated user message in the observer's preferred language. (3) The translated user message is translated into a back-translated message in the user's preferred language, and the back-translated message is displayed on the screen of the user's device. (4) Accept any user edits to the translated user message, such user edits will cause a repetition of the message input steps that begin in step (2); c. After the user's device receives the user's approval of the user message, the translated user message is transmitted from the user's device to the bystander's device.

2. The method of claim 1, further comprising the step of displaying the translated user message adjacent to the user message and the translated user message on the screen of the user's device.

3. The method of claim 1, further comprising the step of transmitting a user introduction data packet from the user's device, the user introduction data packet comprising: a. A user packet header that uniquely identifies the device of the user, and b. User profile, which encodes information about the user's identity and preferred language.

4. The method of claim 3, further comprising the following steps: a. Receiving a bystander introductory data packet on the user's device, the bystander introductory data packet comprising: (1) A bystander label, which uniquely identifies the bystander, and (2) Bystander profile, which encodes information about the bystander's identity and preferred language. b. Display at least a portion of the information regarding the identity of the bystander on the user's device.

5. The method of claim 4, wherein the information displayed regarding the identity of the bystander includes an image of the bystander.

6. The method of claim 5, wherein the displayed information regarding the identity of the bystander further includes the age of the bystander in the image.

7. The method according to claim 4, a. It further includes the following steps after displaying at least a portion of the information about the bystander's identity on the user's device: (1) On the user's device, the user is allowed to select the bystander corresponding to the displayed information. (2) After the user selects the bystander, the connection request from the user's device is transmitted to the bystander's device. b. Wherein, after the user's device receives acceptance of the connection request from the bystander, the message input step is performed on the user's device.

8. A method for enabling multilingual communication between a user's computing device and the computing devices of one of a plurality of bystanders, the method comprising the steps of: a. Transmitting a user-introductory data packet from the user's device to the bystander's device, the introductory data packet comprising: (1) A bystander label, which uniquely identifies the bystander, and (2) Bystander profile, which encodes information about the preferred language of the bystander. b. After the user accepts the connection request from the bystander, the following message input steps are performed on the user's device: (1) Accepting user input of a user message, which is displayed on the screen of the user's device. (2) Translate the user message into a translated user message in the observer's preferred language. (3) Translate the translated user message into a back-translated message in the user's preferred language. (4) The user message is displayed on the screen of the user's device adjacent to the translated user message. (5) Accept any user edits to the translated user message, such user edits will cause the message input steps to be repeated from step (2).

9. The method of claim 8, further comprising the following steps: a. Any editing of at least one of the following via the user's device: (a) the user message, and (b) the translated user message, b. Translate any edited message from the user's preferred language to the observer's preferred language, thereby generating an updated translated user message. c. After the user's device receives the user's approval of the translated user message, the translated message is transmitted to the bystander's device.

10. The method of claim 8, wherein the translated user message is displayed on the screen of the user's device adjacent to the user message and the translated user message.

11. The method of claim 8, further comprising the following steps: a. Receiving a bystander introductory data packet on the user's device, the bystander introductory data packet comprising: (1) A bystander label, which uniquely identifies the bystander, and (2) Bystander profile, which encodes information about the bystander's identity and preferred language. b. Display at least a portion of the information regarding the bystander's identity on the user's device. c. On the user's device, the user is permitted to accept the connection request from the bystander.

12. The method of claim 11, wherein the bystander introductory data packet corresponds to a bystander whose device receives the user introductory data packet.

13. The method of claim 11, wherein the information displayed regarding the identity of the bystander includes an image of the bystander.

14. The method of claim 13, wherein the displayed information regarding the identity of the bystander further includes the age of the bystander in the image.

15. A method for enabling multilingual communication between a user's computing device and the computing devices of one of a plurality of bystanders, the method comprising the steps of: a. Receive one or more bystander introductory data packets on the user's device, each bystander introductory data packet: (1) Corresponding to the sole bystander, and (2) Includes a bystander profile, which encodes information about the bystander's identity and preferred language. b. For each bystander introductory data packet received on the user's device, display at least a portion of the corresponding information about the bystander's identity on the user's device. c. On the user's device, the user is allowed to select one or more bystanders corresponding to the information displayed regarding the bystander's identity. d. After the user selects one or more bystanders, a connection request from the user's device is transmitted to each selected bystander. e. After the device receives acceptance of the connection request from one or more of the bystanders, the following message input step is performed on the user's device: (1) Accepting user input of a user message, which is displayed on the screen of the user's device. (2) Translate the user message into one or more translated user messages, each translated user message being in the preferred language of the corresponding observer among the selected observers. (3) Each of the translated user messages is translated into a corresponding back-translated message in the user's preferred language, and the back-translated message is displayed on the screen of the user's device. (4) Accept any user edits to the translated user message, such user edits will cause a repetition of the message input steps that begin in step (2); f. After the device receives acceptance of the user message from the user, it transmits each of the translated user messages from the device to its corresponding observer.

16. The method of claim 15, further comprising the step of displaying the translated user message on the screen of the user's device adjacent to the user message and the translated user message.

17. The method of claim 15, further comprising the step of: transmitting a user-introductory data packet from the user's device before allowing the user to select one or more bystanders corresponding to the displayed information regarding the bystander's identity, the user-introductory data packet comprising: a. A user packet header that uniquely identifies the device of the user, and b. User profile, which encodes information about the user's identity and preferred language.

18. The method of claim 17, wherein the displayed information regarding the identity of the bystander includes an image of the bystander.

19. The method of claim 18, wherein the displayed information regarding the identity of the bystander further includes the age of the bystander in the image.

20. The method of claim 15, wherein: a. Receive the bystander introductory data packet. b. Transmit the connection request, and c. The translated user message is transmitted via a short-range wireless communication protocol.

Citation Information

Patent Citations

  • Method and apparatus for cross-lingual communication

    US20090204386A1

  • Usage Based Content Search Results

    US20170053189A1

  • Interaction tracking and organizing system

    US20180070208A1

  • Communication device for multiple language translation system

    US9183199B2