Transferring visual representations of speech between devices

Abstract

Methods and apparatus are provided for allowing transmission of a display of a visual representation between a head-mounted device and a computing device during display of a video. In one embodiment, a video is displayed on a computing device display of a computing device, a visual representation of speech for audio components of the video is received, the visual representation is displayed on the computing device display, and upon determining that the head-mounted device is in use, a display of the visual representation is transmitted to the head-mounted device for display on the head-mounted device display.

Description

Technical Field

[0001] The present specification relates to providing a visual representation of speech from a video. Background Art

[0002] The present specification generally relates to methods and apparatus for displaying a visual representation of a speech component of a video while the video is playing. Visual representations including translations, transliterations, summaries, subtitles, narration, emoticons, or graphics of human speech can make the video more convenient and understandable to users. Summary of the invention

[0003] The present application solves the problem of providing a visual representation of speech when a user is watching a video across multiple devices. In an example, a video can be displayed on a personal computing device (such as a handheld device, a laptop, a smart monitor, a desktop, a tablet computer, etc.) and viewed by the user through the lenses of a head-mounted device worn by the user. In an example, the user can watch a video and a visual representation that are both displayed on the computing device. However, when the head-mounted device is turned on and / or the head-mounted device is placed on the user's head, the display of the visual representation can be seamlessly transmitted to the head-mounted device display. The user can then see the visual representation displayed on the head-mounted device display while also viewing the video on the computing device display through the lenses of the head-mounted device. In some examples, the visual representation of the speech can be first displayed on the head-mounted device, and when it is determined that the visual representation is subsequently displayed on the computing device, the display of the visual representation can stop being displayed on the head-mounted device and / or be seamlessly transmitted to the computing device display.

[0004] Certain conditions may trigger the transmission of the visual representation back to the computing device display for display with the video. Other conditions may trigger the transmission of the visual representation to the head mounted device for display.

[0005] In some aspects, the technology described herein relates to a computer-implemented method comprising: displaying a video on a computing device display of a computing device; receiving a visual representation of speech for an audio component of the video; displaying the visual representation on the computing device display; and upon determining that a head mounted device is in use, transmitting the display of the visual representation to the head mounted device for display on the head mounted device display.

[0006] In some aspects, the technology described herein relates to a computing device comprising: a computing device display; and a processor configured with instructions to: display a video on the computing device display of the computing device; receive a visual representation of speech for an audio component of the video; display the visual representation on the computing device display; and upon determining that a head mounted device is in use, transmit the display of the visual representation to the head mounted device for display on the head mounted device display.

[0007] In some aspects, the technology described herein relates to a computer-implemented method comprising: determining that a video and a visual representation of a speech component of the video are being displayed on a computing device display of a computing device; receiving an indication to transmit the display of the visual representation to a head-mounted device display of a head-mounted device; receiving a visual representation of speech from an audio component of the video from the computing device; and displaying the visual representation on the head-mounted device display.

[0008] In some aspects, the technology described herein relates to a head-mounted device comprising: a head-mounted device display; and a processor configured with instructions to: determine that a visual representation of a video and a speech component of the video are being displayed on a computing device display of a computing device; receive an indication to transmit the display of the visual representation to the head-mounted device display of the head-mounted device; receive a visual representation of speech from an audio component of the video from the computing device; and display the visual representation on the head-mounted device display.

[0009] In some aspects, the techniques described herein involve receiving a visual representation of speech from an audio component of a video, displaying the visual representation on a head mounted device display, receiving an indication to transmit display of the visual representation to a computing device display of a computing device, and ceasing to display the visual representation on the head mounted device display.

[0010] In some aspects, the technology described herein relates to a head-mounted device comprising: a head-mounted device display; and a processor configured with instructions to: receive a visual representation of speech from an audio component of a video, display the visual representation on the head-mounted device display, receive an indication to transmit the display of the visual representation to a computing device display of a computing device, and stop displaying the visual representation on the head-mounted device display. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] Figure 1 An example of a user viewing a video with a visual representation provided by a system using devices and software is depicted according to examples described throughout this disclosure.

[0012] Figure 2A and Figure 2BDepicted are examples of a user viewing a video in different visual representation display modes according to an example.

[0013] Figure 3 Depicted is a perspective view of a head mounted device according to an example.

[0014] Figure 4A An example system of apparatus operable to perform the methods of the present disclosure is depicted.

[0015] Figure 4B A block diagram of a head mounted device according to an example is depicted.

[0016] Figure 4C A block diagram of a computing device according to an example is depicted.

[0017] Figure 5A Depicted are example methods that may be performed to transfer a display of a visual representation between a head mounted device and a computing device.

[0018] Figure 5B Depicted are example methods that may be performed to transfer visual representations between a head mounted device and a computing device.

[0019] Figure 5C Describes the Figure 1 An example method for transmitting a display of a video between a head mounted device and a computing device.

[0020] Figure 5D Describes the Figure 1 An example method for transmitting a display of a video between a head mounted device and a computing device. DETAILED DESCRIPTION

[0021] The present disclosure describes methods and apparatus for seamlessly transmitting a visual representation of the speech component of a video between devices while the video is playing on one of a head-mounted device and a computing device. In an example, a user can watch a video on a computing device such as a mobile phone or a laptop. The computing device can also display a visual representation of the speech component of the video along with the video. However, if the user wears a head-mounted device that also displays a visual representation of the speech, the user can see the visual representation twice, which can be confusing. In an example, when the user turns on the head-mounted device and / or places the head-mounted device on their head, the display of the visual representation can be transmitted from the computing device to the head-mounted device. In other words, the visual representation can stop being displayed on the computing device and instead be displayed on the head-mounted device.

[0022] In an example, a user may wear a head mounted device that displays a visual representation of speech from a video. The video may be viewed on a computing device or on a head mounted device display through transparent or semi-transparent lenses of the head mounted device. However, the user may then see the computing device display that displays the visual representation along with the video. At this point, the visual representation may be displayed twice, which may confuse the user. In an example, when the visual representation is displayed on both the head mounted device display and the computing device display, the display of the visual representation may be transmitted from the head mounted device to the computing device.

[0023] In an example, various events may cause the display of a visual representation to be transmitted between the head mounted device and the user's computing device, as further described below.

[0024] The user can watch the video using a computing device with a display. The computing device may include, for example, handheld devices (smart phones, etc.), head-mounted devices (smart glasses, goggles, head-mounted devices, etc.), neck-mounted lanyard devices, other mobile devices (tablet computing devices, etc.), desktop and laptop computing devices, smart televisions, and / or other such devices.

[0025] In an example, the video may include a movie, a live event, or a web conference. An example web conference may include two or more people connected via their own respective computing devices, each respective computing device generating its own respective video and / or audio feed to be sent to each of the other computing devices associated with the other conference participants. Each respective user computing device may be operable to display the video and / or audio feeds of the other conference participants.

[0026] In an example, a user watching a video, live event, or web conference may wish to display a visual representation of the conversation from the video while the user is watching. The visual representation may include a translation, transliteration, summary, subtitles, transcript, emoticons, or graphics of human speech. Providing a visual representation service seamlessly across devices may increase the accessibility of speech from video.

[0027] In some examples, a user can use a combination of computing devices and a combination of displays to watch a video simultaneously. Figure 1 A user is depicted in conjunction with an example device system 100, which may be used to watch a video, live stream an event, or participate in a web conference. Figure 1In the example shown, for the purpose of discussion and illustration, the user is wearing a head mounted device 110 (e.g., an augmented reality viewing device, a virtual reality device, or smart glasses) and using a laptop device 140 to watch a video. In the example, the system 100 may include other computing and / or electronic devices that the user can use to watch videos and view visual representations of voices from those videos. Example computing devices may include a handheld device 120 such as, for example, a smart phone, a display screen 130 (which may include a television monitor or a monitor connected to any computing device), a laptop device 140, a tablet device 150, and a desktop device 160. The device may communicate with one or more servers 170 via a network 180. The server 170 may include, for example, a configuration server that provides coordination between viewing devices. In the example, the computing device may communicate through the network 180 and / or through an alternative network. The network 180 may include a local area network or a personal area network using any combination of wired or wireless communications, including but not limited to Wi-Fi, Ethernet, Bluetooth, IrDA, etc.

[0028] Figure 2A and Figure 2B Depicted are examples of a user watching a video in different visual representation display modes. Figure 2A depicts mode 200A, and Figure 2B Mode 200B is depicted. Figure 2A and Figure 2B In both, the user is watching video 210 on video display screen 130. This is not intended to be limiting. In the example, the user can watch the video on any combination of handheld device 120, video display screen 130, laptop device 140, tablet device 150, tablet device 150, or desktop device 160.

[0029] In the example mode 200A, the user is watching a video 210 on the video display screen 130. A visual representation 220 is displayed on the video display screen 130. In the example, the visual representation 220 can be displayed at any location on the video display screen 130. In the example, the visual representation 220 can cover or partially cover the video 210. In the example Figure 2A It can be further seen in FIG. 1 that the user is holding the head mounted device 110 but has not yet worn or used it.

[0030] In example mode 200B, the user is wearing head mounted device 110. Video 210 is being displayed on video display screen 130, however, visual representation 220 is being displayed in a different manner. Visual representation 220 is displayed on head mounted device display 230. Thus, the user in mode 200B is viewing video 210 and visual representation 220 using a combination of head mounted device 110 and video display screen 130.

[0031] Figure 3 A perspective view of a head mounted device 110 according to an example is depicted. As shown, the head mounted device 110 can be implemented as smart glasses (e.g., augmented reality, virtual reality, simulated reality, mixed reality, see-through reality, fused reality, or alternative reality glasses) configured to be worn on the head of a user. The head mounted device 110 includes a left lens and a right lens coupled to the user's ears via left and right temples, respectively. The user can view the world through the left lens and the right lens, which are coupled together by a nose bridge configured to rest on the wearer's nose.

[0032] The head mounted device 110 includes a head mounted device display 340 operable to present a display to a user wearing the head mounted device 110. In an example, the head mounted device display 340 can be configured to display information (e.g., text, graphics, images) in one or both lenses. The head mounted device display 340 can include all or part of the lenses of the head mounted device 110 and can be visually clear or translucent so that the user can see through the display area when it is not in use.

[0033] In an example, the head mounted device 110 may include a sensing device configured to help determine where the user's focus is directed. For example, the head mounted device 110 may include at least one front camera 330. The front camera 330 may be directed toward a front field of view (i.e., a front field of view 335), or may include an optical device for directing light from the front field of view 335 to an image sensor. The front field of view 335 may include all (or part) of the user's field of view, so that an image or video of the world from the user's perspective may be captured by the front camera 330.

[0034] In an example, the head mounted device 110 may further include at least one eye tracking camera. The eye tracking camera 320 may be directed toward an eye field of view (i.e., eye field of view 325), or may include an optical device for directing light from the eye field of view 325 to an eye image sensor. For example, the eye tracking camera 320 may be directed toward the user's eye and include at least one lens for creating an image of the eye field of view 325 on the eye image sensor. The eye field of view 325 may include all (or part) of the field of view of the eye. The image of the eye may be analyzed by a processor (not shown) of the head mounted device to determine where the user is looking. For example, the relative position of the pupil in the image of the eye may correspond to the user's gaze direction.

[0035] In an example, the head mounted device 110 may further include at least one orientation sensor 350. The orientation sensor 350 may be implemented as any combination of an accelerometer, a gyroscope, and a magnetometer that are combined to form an inertial measurement unit (i.e., IMU) for determining the orientation of the head mounted device. The IMU may be configured to provide multiple measurements describing the orientation and movement of the head mounted display. For example, the IMU may have 6 degrees of freedom (6-DOF) that may describe three translational movements along the axes of the world coordinate system 360 (i.e., x-direction, y-direction, or z-direction) and may describe three rotational movements around the axes of the world coordinate system 360 (i.e., pitch, yaw, roll). Sensor fusion may be used to combine data from the IMU with information about the earth's magnetic field to determine the orientation of the head mounted device coordinate system 370 relative to the world coordinate system 360. The information from the front camera 330, the eye field of view 325, and the IMU 350 can be combined to determine where the user's focus is directed, which can enable augmented reality applications. The head mounted display can further include interface devices for these applications.

[0036] Figure 4A An example system 400 operable to perform the methods of the present disclosure is depicted. The system 400 includes a head mounted device 110 and a computing device 410. The head mounted device 110 can communicate directly with the computing device 410. In an example, the system 400 can further include a server 170. The server 170 can communicate with the computing device 410. In an example, the server 170 can further communicate with the head mounted device 110. The components of the system 400 can communicate with each other via any wireless or wired communication method. In an example, the head mounted device 110 and the computing device 410 can communicate over a local area network. The server 170 can be operable to communicate with the head mounted device 110 and the computing device 410 over the Internet.

[0037] Figure 4B A block diagram of a head mounted device 110 is depicted, and Figure 4C A block diagram of a computing device 410 is depicted. For the sake of brevity and clarity, Figure 4B The block diagram of the head mounted device 110 in FIG. Figure 3 However, the head mounted device 110 may include Figure 3 and Figure 4B Any combination of the components depicted in . Figure 4B, the example head mounted device 110 is depicted as including a position sensor 402, an eye tracking camera 320, a front facing camera 330, a head mounted device display 340, at least one orientation sensor 350, a processor 420, a memory 430, a communication interface 450, a video and visual representation display determination module 460, a visual representation receiving module 462, a visual representation display module 466, a visual representation transmitting module 468, a video display module 470, and a battery determination module 472.

[0038] The head mounted device 110 includes a processor 420 and a memory 430. In an example, the processor 420 may include multiple processors and the memory 430 may include multiple memories. The processor 420 may communicate with any cameras, sensors, and other modules and electronics of the head mounted device 110. The processor 420 is configured by instructions (e.g., software, applications, modules, etc.) to display a visual representation of speech from a video or facilitate transmission of the display to another user device. The instructions may include non-transitory computer-readable instructions stored in and called from the memory 430. In an example, the instructions may be passed from a computing device (e.g., computing device 410) to the processor 420 from the network 180 via the communication interface 450.

[0039] The processor 420 of the head mounted device 110 is in communication with the head mounted device display 340. The processor 420 can be configured by instructions to transmit text, graphics, video, images, etc. to the head mounted device display 340. For example, the processor 420 can be configured to display one or more of the video 210 or the visual representation 220.

[0040] The communication interface 450 of the head mounted device 110 may be operable to facilitate communication between the head mounted device 110 and the computing device 410. In an example, the communication interface 450 may utilize Bluetooth, Wi-Fi, Zigbee, or any other wireless or wired communication method.

[0041] In an example, the processor 420 of the head mounted device 110 may be configured with instructions to execute the video and visual representation display determination module 460. The module 460 may provide an indication, message, or signal to the other modules indicating whether the computing device 410 is displaying the video and visual representation. In an example, the module 460 may determine that an indication, message, or signal indicating whether the computing device 410 is displaying the video and visual representation has been received from the computing device 410 via the communication interface 450. In an example, the module 460 may determine that an indication has been received from the server 170 whether the computing device 410 is displaying the video 210 and the visual representation 220. In an example, the module 460 may use the eye tracking camera 320 to determine whether the user is watching the video 210 on the computing device 410 by observing the reflection of the computing device display in the user's eyes. In an example, the module 460 may use the front camera 330 to identify whether the computing device display playing the video 210 is within the field of view of the front camera 330 of the head mounted device 110.

[0042] In an example, the processor 420 of the head mounted device 110 may be configured with instructions to execute the visual representation receiving module 462. In an example, the visual representation receiving module 462 may be operable to receive the visual representation 220 from the computing device 410 or the server 170 via the communication interface 450. In an example, the processor 420 may generate the visual representation 220 directly from the video 210.

[0043] In an example, the processor 420 of the head mounted device 110 can be configured with instructions to execute the visual representation display module 466. The visual representation display module 466 can receive the visual representation 220 from the visual representation receiving module 462 and display the visual representation 220 on the head mounted device display 340.

[0044] In an example, the processor 420 may be configured with instructions to execute the visual representation transmission module 468. Upon receiving the indication, the visual representation transmission module 468 may be operable to start or stop display of the visual representation 220 on the head mounted device display 340. In an example, the indication may include a signal or command received from the computing device 410 or one or more servers 170 instructing the head mounted device 110 to start or stop displaying the visual representation 220. In an example, the indication may include a signal received from another module executing on the processor 420, the other module being operable to determine that the computing device 410 is being used with the head mounted device 110.

[0045] In an example, the processor 420 can be configured with instructions to execute the video display module 470. The video display module 470 can facilitate the display of the video 210 on the head mounted device display 340.

[0046] In some examples, the video display module 470 may also facilitate the display of the video 210 when it is determined that the user has moved away from the computing device display 443 that is displaying the video 210. When it is determined that the computing device display 443 is not within the field of view of the head mounted camera 330, the video 21 and the visual representation 220 may be displayed on the head mounted device display 340. For example, if the user decides to take a break from watching the video 210 on the computing device 410, this may be helpful. Possibly, for example, the video 210 may include a web conference in which the user is participating, and the user may wish to take a break to have a cup of tea. With the video 210 and the visual representation 220 continuing to be displayed on the head mounted device display 340, the user may continue to observe the web conference or video even when away from the computing device 410.

[0047] In an example, the processor 420 of the head mounted device 110 may be configured with instructions to execute a battery determination module 472. The battery determination module 472 may determine whether the head mounted device battery charge level is low or whether the battery is otherwise in danger of losing charge. In an example, the battery determination module 472 may compare the battery charge level of the head mounted device 110 to a battery threshold level.

[0048] The system 400 further includes a computing device 410 . Figure 4C A block diagram of a computing device 410 is depicted. In an example, the computing device 410 may include any of the handheld device 120, the video display screen 130, the laptop device 140, the tablet device 150, the desktop device 160, or any other similar computing device. The computing device 410 includes a processor 423, a memory 433, and a communication interface 453. In an example, the processor 423, the memory 433, and the communication interface 453 may include features similar to the processor 420, the memory 430, and the communication interface 450 described above with reference to the head mounted device 110.

[0049] The computing device 410 includes a computing device display 443. In examples, the computing device display 443 may include an internal or external display component. In examples where the computing device 410 is a handheld device 120, the computing device display 443 may include a mobile phone display. However, in examples where the computing device 410 is a desktop device 160, the computing device display 443 may include an external monitor, etc.

[0050] The processor 423 of the computing device 410 may be configured to execute a visual representation receiving module 480 , a visual representation display module 482 , a video display module 484 , a head mounted device in use determination module 486 , a battery determination module 488 , and a visual representation transmitting module 490 .

[0051] In an example, the visual representation receiving module 480 may receive the visual representation 220 from any other module executing on the processor 423. In an example, the visual representation receiving module 480 may receive the visual representation 220 from the server 170 or any other device depicted in the device system 100.

[0052] Visual representation display module 482 may receive visual representation 220 from visual representation receiving module 480 and display visual representation 220 on computing device display 443 .

[0053] The video display module 484 may facilitate displaying the video 210 on the computing device display 443. In an example, the video 210 may be stored on the memory 433, or streamed from the server 170. In an example, the video 210 may include a video conference.

[0054] The head mounted device in use determination module 486 can be operable to determine whether the head mounted device 110 is being used with the computing device 410. In an example, the use determination module 486 can determine whether the head mounted device 110 and the computing device 410 are being used simultaneously, or whether they are being used together to view the video 210 on the computing device display 443.

[0055] In an example, the head mounted device in-use determination module 486 may determine that the same user is logged into both the head mounted device 110 and the computing device 410. In an example, the usage determination module 486 may determine that an indication has been received from the server 170 that the head mounted device 110 is in use. In an example, the head mounted device in-use determination module 486 may receive an indication that the head mounted device 110 determined that the user is watching the video 210 on the computing device 410 by observing the reflection of the computing device display in the user's eyes using the eye tracking camera 320. In an example, the video and visual representation display determination module 460 may use the front facing camera 330 of the head mounted device 110 to identify that the computing device display is displaying the video 210.

[0056] In an example, the video and visual representation display determination module 460 can determine whether the position and orientation of the head mounted device indicates whether the user is viewing the computing device display through the head mounted device. For example, one or more position sensors 402 of the head mounted device 110 can be configured to determine the position of the head mounted device (i.e., the user). The position sensor 402 may include an ultra-wideband (UWB) sensor. The position sensor 402 can communicate with the computing device 410 through the communication interface 450. For example, the head mounted device 110 and the communication interface 450 can exchange information packets through the UWB communication link to determine the relative position of the device. For example, the position sensor 402 can be configured to determine the round trip time (RTT) of the packet to be transmitted between the devices. In an example, the position sensor 402 may include multiple receivers configured to receive packets transmitted from the computing device 410. The position sensor 402 can be configured to determine the time when the packet arrives at the receiver to determine the angle between the computing device 410 and the position sensor 402. The position sensor may further include a global positioning system (GPS) sensor that can be used to determine the geographic location of the head mounted device (i.e., the user). The geographic location may be further determined through sensor fusion methods, where information from a local area network (e.g., a Wi-Fi network) and / or a cellular network may further refine the geographic location.

[0057] In an example, the video and visual representation display determination module 460 can determine whether the user is watching a video 210 with or without a visual representation 220 on the computing device 410 by predicting whether the head mounted device is used with the computing device by at least one prediction model. In an example, the prediction model can include a machine learning model. For example, a mapping database can be constructed or trained over time via machine learning to connect a fingerprint ID group to a source group with a matching context (e.g., the same account, Bluetooth, ultra-wideband signal, etc.): Mappingt = mapping of timestamp t, ID1, ID2, …, IDn = n IDs obtained from one or a combination of identification technologies or sensors, P1, P2, …, Pn = n probabilities for a given ID, Source1, Source2, …, Sourcem = m sources that are target candidates for n IDs, and Q1, Q2, …, Qm × n = m × n probabilities for each of the IDs mapped to each user's ID. In an example, the ID may include audio captured via a microphone, video captured via a camera, motion captured via an IMU, schedule information captured via a calendar or email, and / or location information captured via GPS, Bluetooth positioning, Wi-Fi positioning ultra-wideband, ultrasonic positioning, and mobile positioning. Various inputs may be used to extract features and determine voice, facial, or gait fingerprints, or to detect activities, events, or locations, thereby determining when a user uses the head mounted device 110 with the computing device 410. In an example, the database may be local to the computing device 410, or stored on the server 170. In an example, the database may be trained via a module executed on the computing device 410 or via a module executed on the server 170.

[0058] In an example, the head mounted device in use determination module 486 can determine that the user is viewing the computing device display 443 via a computing device camera in communication with the computing device 410. For example, a combination of image processing, predictive analysis, and / or machine learning can be used to identify the user on an image or video frame captured with the computing device camera. The head mounted device can be further identified as being worn on the user's head via further image processing, predictive analysis, and / or machine learning.

[0059] The battery determination module 488 can be operable to determine that the battery level of the head mounted device 110 is below a threshold level. In some examples, 488 / / can receive an indication from 110 / / that the head mounted device battery level is low.

[0060] Upon receiving the indication, visual representation transmission module 490 may be operable to start or stop display of visual representation 220 on computing device display 443 and / or transmit the display of visual representation 220 to head mounted device 110. In an example, the indication may include a signal or command instructing computing device 410 to start or stop displaying visual representation 220. In an example, the indication may include a signal received from head mounted device 110, server 170, or another module executing on processor 423.

[0061] In an example, an instruction to transmit the visual representation to the head mounted device may be received from the visual representation transmission module 490 to determine that the head mounted device battery charge is below a threshold. In an example, an instruction to transmit the visual representation may be received when it is determined via the head mounted device in use determination module 486 that the user is no longer using the head mounted device 110.

[0062] In an example, upon determining that the head mounted device is in use, an instruction to transmit a display of the visual representation to the head mounted device may be sent. At least one predictive model may be used to predict whether the head mounted device is being used with the computing device, as described above.

[0063] In some examples, the video 210 may include a video conference that a group of users are viewing on a single computing device display. An indication may be sent upon determining that at least one user in a group of users viewing the video on the computing device display is not displaying a visual representation using a corresponding head mounted device display. In an example, the head mounted device in use determination module 486 may be operable to determine whether each corresponding user is wearing / using a corresponding head mounted device via any combination of the above methods. For example, when one user stops displaying a visual representation 220 with their head mounted device 110, this may allow the visual representation 220 to be seamlessly transferred from each user's corresponding head mounted device 110 to the computing device display 443. In this way, interruptions in the visual representation service to users in a video conference may be avoided.

[0064] In an example, any module described with respect to computing device 410 can be executed on a web browser extension, add-on, or plug-in. In an example, the web browser extension, add-on, or plug-in can be constructed with HTML, JavaScript, or CSS scripts, or can include executable code. By executing the modules described herein with respect to computing device 410 in a web browser extension, add-on, or plug-in, seamless visual representation display functionality between devices can be achieved, regardless of the service used to display the video or web conference.

[0065] Processor 423 may be configured with instructions for visual representation transmission module 490. Upon receiving the indication, visual representation transmission module 490 may be operable to stop display of visual representation 220 on computing device display 443 so that the visual representation may be transmitted to head mounted device 110. In an example, the indication may include a signal or command received from head mounted device 110, one or more servers 170, or another module instructing computing device 410 to stop displaying visual representation 220. In some examples, visual representation transmission module 490 may send an indication to head mounted device 110 to display visual representation 220.

[0066] The system 400 includes a server 170. In an example, the server 170 may include one or more servers. Like the head mounted device 110 and the computing device 410, the server 170 also includes a processor 425, a memory 435, and a communication interface 445. In an example, the server 170 may further include its own corresponding visual representation transmission module 495. The server's own corresponding visual representation transmission module 495 may help manage the transmission of the display of the visual representation 220 between the head mounted device 110 and the computing device 410.

[0067] Figure 5AAn example method 500A is depicted that may be performed to transfer a display of a visual representation 220 between a head mounted device 110 and a computing device 410. In an example, the method 500A may be performed on the computing device 410. In an example, the method 500A may include steps 502, 504, 506, 508, and 510. However, in further examples, the method 500A may include any combination of steps 502-514.

[0068] Method 500A begins at step 502. In step 502, video 210 can be displayed on computing device display 443 of computing device 410. For example, video 210 can be displayed via video display module 484, as described above.

[0069] Method 500A continues with step 504. In step 504, visual representation 220 of speech can be received for the audio component of video 210. For example, visual representation 220 can be received via visual representation receiving module 480, as described above.

[0070] Method 500A continues with step 506. In step 506, visual representation 220 can be displayed on computing device display 443. For example, visual representation 220 can be displayed via visual representation display module 482.

[0071] The method 500A proceeds to step 508. In step 508, it may be determined whether the head mounted display is in use. For example, step 508 may execute the head mounted device in use determination module 486, as described above. If the determination is yes, i.e., the head mounted device is being used by the user, the method 500A proceeds to step 510. However, if the determination is no, i.e., the head mounted device is not in use, the method 500A remains at step 508.

[0072] The method 500A continues with step 510. In step 510, the display of the visual representation is transmitted to the head mounted device for display on the head mounted device display. For example, the visual representation 220 can be transmitted to the head mounted device 110 via the visual representation transmission module 490, as described above.

[0073] The method 500A proceeds to step 512. In step 512, it is determined whether an indication has been received for transmitting a display to a computing device display. If an indication has been received, the method 500A proceeds to step 514. However, if an indication has not been received, the method 500A may remain at step 512.

[0074] The method 500A continues with step 514. At step 514, the visual representation 220 is displayed on the computing device display 320. For example, the visual representation 220 may be displayed via the visual representation display module 482, as described above.

[0075] Figure 5B An example method 500B is depicted that may be performed to seamlessly transfer a visual representation 220 between a handheld device 120 and a computing device 410. In an example, the method 500B may be performed on an example head mounted device 110.

[0076] Method 500B begins at step 550. At step 550, it can be determined that video 210 and visual representation 220 are being displayed on computing device display 443 of computing device 410. For example, head mounted device 110 can execute video and visual representation display determination module 460, as described above.

[0077] Method 500B may continue with step 552. At step 552, an instruction may be received to transmit the display of visual representation 220 to head mounted device display 340. For example, computing device 410 may execute visual representation transmission module 490, as described above.

[0078] Method 500B may proceed to step 554. At step 554, visual representation 220 of speech of the audio component of video 210 may be received from computing device 410. For example, head mounted device 110 may execute visual representation receiving module 462, as described above.

[0079] The method 500B may proceed to step 556. In step 556, the visual representation 220 may be displayed on the head mounted device display 340. For example, the head mounted device 110 may execute the visual representation display module 466, as described above.

[0080] The method 500B may proceed to step 558. At step 558, an instruction may be received to transmit the display of the visual representation 220 to the computing device 410. For example, the head mounted device 110 may execute the visual representation transmission module 468, as described above.

[0081] If an indication is received at step 558, method 500B may proceed to step 560. In step 560, head mounted device 110 may stop displaying visual representation 220. In an example, step 560 may also include sending an indication for transmitting the display of visual representation 220 to computing device 410.

[0082] Figure 5CMethod 500C is depicted. In an example, method 500B may further include steps of method 500C. In an example, method 500C may be performed after step 556 of method 500B.

[0083] Method 500C may begin at step 562. At step 562, it may be determined whether the user is viewing computing device display 443. In an example, front-facing camera 330 or position sensor 402 may be used to determine whether the user is viewing computing device display 443, as described above. In an example, a camera connected to computing device 410 may be used to determine whether the user is viewing computing device display 443.

[0084] If the answer to step 562 is no, then method 500B proceeds to step 564. In step 564, video 210 may be displayed on head mounted display 230. For example, head mounted device 110 may execute video display module 470. Displaying video 210 on head mounted device display 340 when computing device display 443 is not in view may allow the user to continue viewing video content while moving away from computing device display 443. For example, if the user must leave computing device display 443 to receive a package, the user may continue to follow along.

[0085] If the answer to step 562 is true, then the method 500C proceeds to step 566 . In step 566 , the head mounted display 110 may stop displaying the video 210 on the head mounted device display 230 .

[0086] Figure 5D A method 500D is depicted. In an example, the method 500D includes steps 570 through 576. However, in further examples, the method 500D may include any combination of steps 570-580.

[0087] Method 500D may begin at step 570. In step 570, a visual representation of speech from an audio component of a video may be received. For example, head mounted device 110 may execute visual representation receiving module 462, as described above.

[0088] The method 500D may proceed to step 572. In step 572, the visual representation may be displayed on the head mounted device display. For example, the head mounted device 110 may execute the visual representation display module 466, as described above.

[0089] The method 500D may proceed to step 574. In step 574, an instruction may be received to transmit the display of the visual representation to a computing device display of the computing device. For example, the head mounted device 110 may execute the visual representation transmission module 468, as described above.

[0090] In an example, upon receiving an indication that a computing device display is displaying a visual representation, an indication for transmitting the display of the visual representation to the computing device display may be sent. In an example, upon determining that a video displayed on the computing device display is within a field of view of a head mounted display camera connected to the head mounted display, an indication for transmitting the display of the visual representation to the computing device display may be sent. In an example, upon determining that a head mounted device battery is below a charge threshold, an indication for transmitting the display of the visual representation to the computing device display may be sent.

[0091] The method 500D may proceed to step 576. In step 576, display of the visual representation on the head mounted device display may be stopped. In an example, step 576 may further include executing the visual representation transmission module 468, as described above.

[0092] The method 500D may proceed to step 578. In step 578, it may be determined whether the user is viewing the computing device display. For example, after executing the video and visual representation display determination module 460, an indication that the user is not viewing the computing device display may be received, as described above. If the answer to step 578 is yes, the method 500D may again perform step 578. However, if the answer to step 578 is no, the method 500D may proceed to step 580.

[0093] In step 580, both the video and the visual representation can be displayed on the head mounted device display. For example, the visual representation display module 466 and the video display module 470 can be executed by the head mounted device 110.

[0094] The methods and apparatus of the present specification allow for seamless transfer of visual representations of speech from video between user devices, thereby increasing user accessibility to speech and video content.

[0095] Various examples of the systems and techniques described herein can be implemented in digital electronic circuit systems, integrated circuit systems, specially designed ASICs (application-specific integrated circuits), computer hardware, firmware, software, and / or combinations thereof. These various examples may include examples in one or more computer programs executable and / or interpretable on a programmable system, the programmable system including at least one programmable processor, the at least one programmable processor may be dedicated or general, may be coupled to receive data and instructions from a storage system, at least one input device, and at least one output device and transmit data and instructions to the storage system, at least one input device, and at least one output device. Various examples of the systems and techniques described herein may be implemented as and / or are generally referred to herein as circuits, modules, blocks, or systems that can combine software and hardware aspects. For example, a module may include functions / actions / computer program instructions executed on a processor or some other programmable data processing device.

[0096] Some of the examples above are described as processes or methods depicted as flow charts. Although the flow charts describe the operations as sequential processes, many operations may be performed in parallel, concurrently, or simultaneously. In addition, the order of the operations may be rearranged. Processes may terminate when their operations are completed, but may also have additional steps not included in the figure. A process may correspond to a method, function, procedure, subroutine, subprogram, etc.

[0097] The methods discussed above (some of which are illustrated by flowcharts) may be implemented by hardware, software, firmware, middleware, microcode, hardware description languages, or any combination thereof. When implemented by software, firmware, middleware, or microcode, the program code or code segments for performing the necessary tasks may be stored in a machine or computer readable medium, such as a storage medium. A processor may perform the necessary tasks.

[0098] The specific structural and functional details disclosed herein are merely representative for describing the examples. However, the examples may be embodied in many alternate forms and should not be construed as limited to only the embodiments set forth herein.

[0099] It should be understood that although the terms first, second, etc. may be used herein to describe various elements, these elements should not be limited by these terms. These terms are only used to distinguish one element from another element. For example, without departing from the scope of the exemplary embodiment, a first element may be referred to as a second element, and similarly, a second element may be referred to as a first element. As used herein, the term "and / or" includes any and all combinations of one or more items in the associated listed items.

[0100] The terms used herein are only used for the purpose of describing specific embodiments and are not intended to limit the exemplary embodiments. As used herein, unless the context clearly indicates otherwise, the singular forms "a", "an", and "the / said" are also intended to include the plural forms. It should be further understood that the terms "includes", "comprising", "includes" and / or "including" when used herein specify the presence of stated features, integers, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or groups thereof.

[0101] It should also be noted that in some alternative examples, the functions / actions noted may not occur in the order noted in the figures. For example, depending on the functions / actions involved, two figures shown in succession may actually be executed simultaneously or may sometimes be executed in the reverse order.

[0102] Unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art to which the example embodiments belong. It should be further understood that terms such as those defined in commonly used dictionaries should be interpreted as having a meaning consistent with their meaning in the context of the relevant art, and should not be interpreted in an idealized or overly formal sense unless explicitly defined as such herein.

[0103] Portions of the above example embodiments and corresponding detailed descriptions are presented in terms of software or algorithms and symbolic representations of operations on data bits in computer memory. These descriptions and representations are descriptions and representations used by those of ordinary skill in the art to effectively convey the substance of their work to other persons of ordinary skill in the art. As the term is used herein, and as generally used, an algorithm is considered to be a self-consistent sequence of steps that produces a desired result. The step is a step that requires physical manipulation of physical quantities. Although not necessary, these quantities are typically in the form of optical, electrical or magnetic signals that can be stored, transferred, combined, compared and otherwise manipulated. It has been demonstrated that, primarily for common usage reasons, these signals can sometimes be conveniently referred to as bits, values, elements, symbols, characters, terms, numbers, etc.

[0104] In the above illustrative embodiments, references to symbolic representations of actions and operations that can be implemented as program modules or functional processes (e.g., in the form of flowcharts) include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types, and can be described and / or implemented using existing hardware at existing structural elements. Such existing hardware may include one or more central processing units (CPUs), digital signal processors (DSPs), application specific integrated circuits, field programmable gate arrays (FPGAs), computers, etc.

[0105] It should be borne in mind, however, that all of these and similar terms are to be associated with the appropriate physical quantities and are merely convenient labels applied to these quantities. Unless otherwise specifically noted or as is apparent from the discussion, terms such as processing or computing or calculating or determining display refer to the actions and processes of a computer system or similar electronic computing device that manipulates data represented as physical electronic quantities within the computer system's registers and memories and transforms that data into other data similarly represented as physical quantities within the computer system's memories or registers or other such information storage, transmission or display devices.

[0106] It is also noted that the software-implemented aspects of the example embodiments are typically encoded on some form of non-transitory program storage medium or implemented on some type of transmission medium. The program storage medium may be magnetic (e.g., a floppy disk or hard disk) or optical (e.g., a compact disk read-only memory or CD ROM), and may be read-only or random access. Similarly, the transmission medium may be a twisted pair, a coaxial cable, an optical fiber, or some other suitable transmission medium known in the art. The examples are not limited by these aspects of any given example.

[0107] Finally, it should also be noted that although the appended claims set forth specific combinations of features described herein, the scope of the present disclosure is not limited to the specific combinations claimed below, but extends to cover any combination of features or examples disclosed herein, regardless of whether the specific combination is specifically listed in the appended claims.

[0108] In some aspects, the technology described herein relates to a computer-implemented method in which determining that a head-mounted device is in use further includes: receiving an indication that a user is logged into both the head-mounted device and a computing device; receiving an indication that the user is looking through the head-mounted device at a computing device display; receiving an indication that a position and orientation of the head-mounted device indicates that the user is viewing the computing device display through the head-mounted device; or determining that the user is viewing the computing device display via a computing device camera that communicates with the computing device.

[0109] In some aspects, the techniques described herein relate to a computer-implemented method that further includes displaying the visual representation on a computing device display upon receiving an indication to transmit the display of the visual representation back to the computing device.

[0110] In some aspects, the technology described herein relates to a computer-implemented method in which an indication is sent when it is determined that a head mounted device battery charge is below a threshold or a user is no longer using a head mounted display.

[0111] In some aspects, the techniques described herein relate to a computer-implemented method in which an indication is sent when a head mounted device is determined to be in further use by predicting, by at least one predictive model, whether the head mounted device is being used with a computing device.

[0112] In some aspects, the technology described herein relates to a computer-implemented method in which a video includes a video conference and an indication is sent when it is determined that at least one user of a plurality of users viewing the video on a computing device display is not displaying a visual representation using a corresponding head mounted device display of a corresponding head mounted device.

[0113] In some aspects, the technology described herein relates to a computer-implemented method that is performed on a web browser extension, add-on, or plug-in.

[0114] In some aspects, the technology described herein relates to a computer-implemented method wherein the visual representation includes at least one of: a transcription of the speech, a translation, a summary, a caption, an emoticon, or a graphic of the speech.

[0115] In some aspects, the techniques described herein relate to a computing device wherein determining that a head mounted device is in use further comprises: receiving an indication that a user is logged into both the head mounted device and the computing device; receiving an indication that the user is looking through the head mounted device at a computing device display; receiving an indication that a position and orientation of the head mounted device indicates that the user is viewing the computing device display through the head mounted device; or determining that the user is viewing the computing device display via a computing device camera in communication with the computing device.

[0116] In some aspects, the techniques described herein relate to a computing device in which the processor is further configured with instructions to: display the visual representation on a computing device display upon receiving an indication to transmit the display of the visual representation back to the computing device.

[0117] In some aspects, the techniques described herein relate to a computing device in which an indication is sent upon determining that a head mounted device battery charge is below a threshold or a user is no longer using a head mounted display.

[0118] In some aspects, the techniques described herein relate to a computing device in which an indication is sent when it is determined that a user is no longer using a head mounted display by predicting, by at least one predictive model, whether the head mounted device is being used with the computing device.

[0119] In some aspects, the technology described herein relates to a computing device in which a video includes a video conference and an indication is sent when it is determined that at least one user of a plurality of users viewing the video on a computing device display is not displaying a visual representation using a corresponding head mounted device display of a corresponding head mounted device.

[0120] In some aspects, the techniques described herein relate to a computing device in which instructions are executed on a web browser extension, add-on, or plug-in.

[0121] In some aspects, the technology described herein relates to a computing device wherein the visual representation includes at least one of: a transcription of the speech, a translation, a summary, an emoticon, or a graphic of the speech.

[0122] In some aspects, the techniques described herein relate to a computer-implemented method that further includes, upon receiving an indication to transmit display of the visual representation to a computing device, ceasing to display the visual representation on a head mounted device display.

[0123] In some aspects, the techniques described herein relate to a computer-implemented method that further includes displaying both the video and the visual representation on a head mounted device display when it is determined that the user is not viewing the computing device display.

[0124] In some aspects, the technology described herein relates to a computer-implemented method in which determining that a video is being displayed on a computing device display of a computing device further includes: receiving an indication that the computing device display is displaying a video; or determining that the video displayed on the computing device display is within a field of view 335 of a head mounted display camera connected to a head mounted display.

[0125] In some aspects, the technology described herein relates to a computer-implemented method that further includes: upon determining that a head mounted device battery is below a charge threshold, transmitting a display of a visual representation to a computing device for display on a computing device display.

[0126] In some aspects, the technology described herein relates to a computer-implemented method wherein the visual representation is at least one of: a transcription of the speech, a translation, a summary, an emoticon, or a graphic of the speech.

[0127] In some aspects, the technology described herein relates to a head mounted device, wherein the processor is further configured with instructions to: upon receiving an indication to transmit the display of the visual representation to the computing device, cease displaying the visual representation on the head mounted device display.

[0128] In some aspects, the technology described herein relates to a head mounted device, wherein the processor is further configured with instructions to: display both the video and the visual representation on the head mounted device display when it is determined that the user is not viewing the computing device display.

[0129] In some aspects, the technology described herein relates to a head-mounted device, wherein determining that a video is being displayed on a computing device display of the computing device further includes: receiving an indication that the computing device display is displaying the video; or determining that the video displayed on the computing device display is within a field of view of a head-mounted display camera connected to the head-mounted display.

[0130] In some aspects, the technology described herein relates to a head-mounted device, wherein the processor is further configured with instructions to: upon determining that a battery of the head-mounted device is below a charge threshold, transmit a display of a visual representation to a computing device for display on a computing device display.

[0131] In some aspects, the technology described herein relates to a head-mounted device wherein the visual representation is at least one of: a transcription of the speech, a translation, a summary, an emoticon, or a graphic of the speech.

[0132] In some aspects, the techniques described herein relate to a computer-implemented method that further includes displaying both the video and the visual representation on a head mounted device display upon receiving an indication that a user is not viewing the computing device display.

[0133] In some aspects, the technology described herein relates to a computer-implemented method in which an indication for transmitting a display of a visual representation to a computing device display is received when: an indication is received that the computing device display is displaying the visual representation; and / or a determination is made that a video displayed on the computing device display is within a field of view of a head-mounted display camera connected to a head-mounted display; and / or a determination is made that a head-mounted device battery is below a charge threshold.

[0134] In some aspects, the technology described herein relates to a computer-implemented method wherein the visual representation is at least one of: a transcription of the speech, a translation, a summary, an emoticon, or a graphic of the speech.

[0135] In some aspects, the technology described herein relates to a head mounted device, wherein the processor is further configured with instructions to: upon receiving an indication that a user is not viewing the computing device display, display both the video and the visual representation on the head mounted device display.

[0136] In some aspects, the technology described herein relates to a head-mounted device in which, upon receiving an indication that the computing device display is displaying a visual representation or upon determining that a video displayed on the computing device display is within a field of view of a head-mounted display camera connected to the head-mounted device, an indication is received to transmit display of a visual representation to a computing device display.

[0137] In some aspects, the techniques described herein relate to a head mounted device in which, upon determining that a battery of the head mounted device is below a charge threshold, an indication is sent to transmit a display of a visual representation to a computing device for display on a computing device display.

[0138] In some aspects, the technology described herein relates to a head-mounted device wherein the visual representation is at least one of: a transcription of the speech, a translation, a summary, an emoticon, or a graphic of the speech.

Claims

1. A computer-implemented method comprising: displaying the video on a computing device display of the computing device; receiving a visual representation of speech for an audio component of the video; displaying the visual representation on a display of the computing device; as well as Upon determining that the head mounted device is in use, a display of the visual representation is transmitted to the head mounted device for display on a head mounted device display.

2. The computer-implemented method of claim 1 , wherein: Determining that the head mounted device is in use further includes: receiving an indication that a user is logged into both the head mounted device and the computing device; and / or receiving an indication that the user is looking at the computing device display through the head mounted device; and / or receiving an indication that the position and orientation of the head mounted device indicates that the user is viewing the computing device display through the head mounted device; and / or It is determined via a computing device camera in communication with the computing device that the user is viewing the computing device display.

3. The computer-implemented method of claim 1 or claim 2, further comprising: Upon receiving an indication to transmit a display of the visual representation back to the computing device, the visual representation is displayed on the computing device display.

4. The computer-implemented method of claim 3, wherein: The indication is sent when it is determined that the battery level of the head mounted device is below a threshold or the user is no longer using the head mounted display.

5. The computer-implemented method of claim 3 or 4, wherein: The indication is sent upon determining that the head mounted device is in further use by predicting, by at least one prediction model, whether the head mounted device is being used with the computing device.

6. A computer-implemented method according to any one of claims 3 to 5, wherein: The video includes a video conference, and the indication is sent upon determining that at least one user of a plurality of users viewing the video on the computing device display is not displaying the visual representation using a corresponding head mounted device display of a corresponding head mounted device.

7. A computer-implemented method according to any one of the preceding claims, wherein: The computer-implemented method is performed on a web browser extension, add-on, or plug-in.

8. A computer-implemented method according to any one of the preceding claims, wherein: The visual representation includes at least one of: a transcription of the speech, a translation, a summary, an emoticon, or a graphic of the speech.

9. A computing device comprising: computing device display; as well as A processor configured with instructions to: displaying the video on a computing device display of the computing device; receiving a visual representation of speech for an audio component of the video; displaying the visual representation on a display of the computing device; as well as Upon determining that the head mounted device is in use, a display of the visual representation is transmitted to the head mounted device for display on a head mounted device display.

10. The computing device according to claim 9, wherein: Determining that the head mounted device is in use further includes: receiving an indication that a user is logged into both the head mounted device and the computing device, and / or receiving an indication that the user is looking through the head mounted device at a display of the computing device, and / or receiving an indication that the position and orientation of the head mounted device indicates that the user is viewing the computing device display through the head mounted device, and / or It is determined via a computing device camera in communication with the computing device that the user is viewing the computing device display.

11. The computing device according to claim 9 or 10, wherein: The processor is further configured with instructions to: display the visual representation on the computing device display upon receiving an indication to transmit the display of the visual representation back to the computing device.

12. The computing device of claim 11, wherein: The indication is sent when it is determined that the battery level of the head mounted device is below a threshold or the user is no longer using the head mounted display.

13. The computing device according to claim 11 or 12, wherein: The indication is sent when it is determined that the user is no longer using the head mounted display by predicting, by at least one predictive model, whether the head mounted device is used with the computing device.

14. The computing device according to any one of claims 11 to 13, wherein: The video includes a video conference, and the indication is sent upon determining that at least one user of a plurality of users viewing the video on the computing device display is not displaying the visual representation using a corresponding head mounted device display of a corresponding head mounted device.

15. The computing device according to any one of claims 9 to 14, wherein: The instructions are executed on a web browser extension, add-on or plug-in.

16. The computing device according to any one of claims 9 to 15, wherein: The visual representation includes at least one of: a transcription of the speech, a translation, a summary, an emoticon, or a graphic of the speech.

17. A computer-implemented method comprising: determining that a video and a visual representation of a speech component of the video are being displayed on a computing device display of the computing device; receiving an instruction to transmit a display of the visual representation to a head mounted device display of the head mounted device; receiving from the computing device the visual representation of speech from an audio component of the video; as well as The visual representation is displayed on the head mounted device display.

18. The computer-implemented method of claim 17, further comprising: Upon receiving an indication to transmit display of the visual representation to the computing device, display of the visual representation on the head mounted device display is ceased.

19. The computer-implemented method of claim 17 or 18, further comprising: Upon determining that the user is not viewing the computing device display, displaying both the video and the visual representation on the head mounted device display.

20. The computer-implemented method of any one of claims 17 to 19, wherein: Determining that the video is being displayed on the computing device display of the computing device further comprises: receiving an indication that the computing device display is displaying the video; and / or A determination is made that the video displayed on the computing device display is within a field of view of a head mounted display camera connected to the head mounted display.

21. The computer-implemented method of any one of claims 17 to 20, further comprising: Upon determining that the head mounted device battery is below a charge threshold, transmitting a display of the visual representation to the computing device for display on the computing device display.

22. A computer-implemented method according to any one of claims 17 to 21, wherein: The visual representation is at least one of: a transcription of the speech, a translation, a summary, an emoticon, or a graphic of the speech.

23. A head mounted device comprising: Head mounted device displays; as well as A processor configured with instructions to: determining that a video and a visual representation of a speech component of the video are being displayed on a computing device display of the computing device, receiving an instruction to transmit a display of the visual representation to a head mounted device display of the head mounted device, receiving from the computing device the visual representation of speech from the audio component of the video, and The visual representation is displayed on the head mounted device display.

24. The head mounted device of claim 23, wherein: The processor is further configured with instructions to, upon receiving an indication to transmit display of the visual representation to the computing device, cease displaying the visual representation on the head mounted device display.

25. The head mounted device of claim 23, wherein: The processor is further configured with instructions to: display both the video and the visual representation on the head mounted device display when it is determined that the user is not viewing the computing device display.

26. The head mounted device of claim 23, wherein: Determining that the video is being displayed on the computing device display of the computing device further includes: receiving an indication that the computing device display is displaying the video; or determining that the video displayed on the computing device display is within a field of view of a head-mounted display camera connected to the head-mounted display.

27. The head mounted device of claim 23, wherein: The processor is further configured with instructions to: upon determining that the head mounted device battery is below a charge threshold, transmit a display of the visual representation to the computing device for display on the computing device display.

28. The head mounted device of claim 23, wherein: The visual representation is at least one of: a transcription of the speech, a translation, a summary, an emoticon, or a graphic of the speech.

29. A computer-implemented method comprising: receiving a visual representation of speech from the audio component of the video, displaying the visual representation on the head mounted device display, receiving an instruction to transmit a display of the visual representation to a computing device display of the computing device, and Ceasing to display the visual representation on the head mounted device display.

30. The computer-implemented method of claim 29, further comprising: Upon receiving an indication that a user is not viewing the computing device display, displaying both the video and the visual representation on the head mounted device display.

31. A computer-implemented method according to any one of claims 29 or 30, wherein: The instruction to transmit the display of the visual representation to the computing device display is received when: receiving an indication that the computing device display is displaying the visual representation; and / or determining that the video displayed on the computing device display is within a field of view of a head mounted display camera coupled to the head mounted display; and / or Determines if the headset battery is below a charge threshold.

32. A computer-implemented method according to any one of claims 29 to 31, wherein: The visual representation is at least one of: a transcription of the speech, a translation, a summary, an emoticon, or a graphic of the speech.

33. A head mounted device comprising: Head mounted device displays; as well as A processor configured with instructions to: receiving a visual representation of speech from the audio component of the video, displaying the visual representation on the head mounted device display, receiving an instruction to transmit a display of the visual representation to a computing device display of the computing device, and Ceasing to display the visual representation on the head mounted device display.

34. The head mounted device of claim 33, wherein: The processor is further configured with instructions to: display both the video and the visual representation on the head mounted device display upon receiving an indication that a user is not viewing the computing device display.

35. A head mounted device according to claim 33 or 34, wherein: Upon receiving an indication that the computing device display is displaying the visual representation or upon determining that the video displayed on the computing device display is within the field of view of a head mounted display camera connected to the head mounted device, the indication for transmitting display of the visual representation to the computing device display is received.

36. A head mounted device according to any one of claims 33 to 35, wherein: Upon determining that the head mounted device battery is below a charge threshold, the indication is sent for transmitting a display of the visual representation to the computing device for display on the computing device display.

37. A head mounted device according to any one of claims 33 to 36, wherein: The visual representation is at least one of: a transcription of the speech, a translation, a summary, an emoticon, or a graphic of the speech.

Images