In-vehicle meeting with user tracking and speaker prioritization
Through the intelligent controller system in the vehicle, multiple cameras and microphones are used to identify speakers and optimize video and audio signals, the problem of difficult and insufficient audio quality of speakers in conferences in vehicles is solved, and high-quality video and audio is achieved Feeding, improving user experience.
Patent Information
- Application Number
- CN202410699237.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Priority Date
- 2023-06-06
- Filing Date
- 2024-05-31
- Publication Date
- 2024-12-06
AI Technical Summary
It is difficult to determine who is talking when conducting digital meetings within a vehicle, and the use of a single camera and microphone leads to insufficient video and audio quality, especially for occupants located at the rear of the vehicle, especially in large vehicles, due to physical features occlusion and Noise problem.
Using a controller system within a smart vehicle, video and audio data acquisition is carried out through multiple cameras and microphones, processors and memory are used to identify speakers, determine the best image and audio sources, crop and process video and audio signals, and provide clear Video and audio feeds, isolate background noise and ensure meeting quality.
Effectively identify and prioritize speakers, provide clear video and audio signals, improve user experience, and ensure the quality of meetings for remote meeting participants, especially in multi-person participation scenarios within the vehicle.
Smart Images

Figure CN119094688A_ABST
Abstract
Description
Technical Field
[0001] The present disclosure relates to systems and methods for smart in-vehicle conferencing with user tracking and speaker prioritization. Background Art
[0002] There are many challenges associated with providing digital conferencing solutions in vehicles. For example, a typical in-vehicle conferencing system may have only one camera to serve multiple occupants located throughout the cabin of the vehicle. In addition, it may be difficult to be able to determine who is speaking, which may be complicated by the fact that the video feeds of certain occupants are obscured by physical features of the vehicle, such as headrests or other items in the cabin. Additionally, microphones located at the front of the vehicle may provide reduced functionality for occupants in the rear of the vehicle, especially for larger vehicles with two or three rows of seats. Conversely, attempts to use multiple microphones may increase the noise level for participants. Summary of the invention
[0003] The techniques described herein may be used to allow intelligent prioritization and processing of video and / or audio feed information, which may be used in the context of digital conferencing software and solutions. The techniques described herein should only be implemented in accordance with the owner's manual and safety guidelines. The driver of the vehicle should only participate in a meeting when the vehicle is in autonomous mode or stopped.
[0004] Providing digital or virtual conferencing solutions in the context of a vehicle presents unique challenges. While a laptop or tablet may provide a front-facing camera that is ideally positioned for capturing video and / or audio from conference participants, a vehicle may have only a single camera to serve multiple occupants of the vehicle. A typical in-vehicle video conferencing system may provide an aggregated video and audio output for all participants of the vehicle. For example, a camera may be located in the center dashboard or near the front of the vehicle. An aggregate feed may be provided for all vehicle occupants within the camera's field of view, as well as capturing sound from within the vehicle cabin. That is, a single video and audio collector is used to present a group of individuals in the vehicle as a single participant to the video conferencing platform or software.
[0005] As mentioned above, one of the challenges to be addressed relates to the difficulty in determining who is speaking within the cabin of a vehicle. Consider a hypothetical example where multiple occupants of a vehicle are conducting a video conference using a single camera, and when viewed by other participants of the video conference, the video feed of the vehicle may be presented using a wide angle lens that is suitable for capturing all of the vehicle occupants but may be lacking because the image size provided to the other participants of the video conference is not sufficient to determine who is speaking in the vehicle and see their facial expressions.
[0006] Additionally, the placement of the camera in the cabin of the vehicle may be limited or constrained. For most vehicles, when a single camera is used, it may be positioned in a way that some occupants are obscured. While the view for the front row passengers may be optimal, passengers in the second or even third row may be obscured by headrests or other items in the vehicle.
[0007] Furthermore, when using microphones at the front of the vehicle to capture audio signals, the audio may be poor for speakers farthest from the microphone, and noise captured from other microphones may overwhelm the sound detected from the person speaking at a distance. If multiple microphones are used, this may increase the noise level for the participants, as the aggregate audio from multiple microphones around the vehicle cabin may generate noisy audio due to the pickup of unwanted sounds, making it difficult to understand the speech of the person who is actually speaking.
[0008] Some or all of the above challenges can be addressed using technology for conferencing in a smart vehicle with user tracking and speaker prioritization. As described in more detail below, various embodiments of the present disclosure can be used to implement technology for conferencing in a smart vehicle with user tracking and speaker prioritization. In at least one embodiment, a vehicle includes a vehicle controller system having an electronic circuit system. The electronic circuit system may include one or more processors (e.g., processing units) and a memory that stores executable instructions so that the executable instructions implement the various techniques described herein when executed by one or more processors. In various embodiments, the vehicle has a controller system that performs localization of vehicle occupants speaking by using various sensors in the vehicle (such as cameras and / or microphones) to capture visual and / or audio data to identify speakers in the vehicle. The controller system can identify speakers by detecting mouth movement, audio signal strength, audio signal arrival time (indicating the distance the sound travels), evaluation of hand movement, or a combination thereof.
[0009] In at least one embodiment, a vehicle includes one or more cameras, one or more microphones, and a controller system, the controller system also including one or more processors and a memory storing executable instructions, the executable instructions, due to being executed by the one or more processors, causing the controller system to implement various techniques described in more detail below in this document. These techniques cause the vehicle's controller system to: determine a plurality of candidate best image sources based at least in part on the one or more cameras; determine a best image source for a speaker in the vehicle from the plurality of candidate best image sources; determine a video feed based at least in part on the best image source; determine a plurality of candidate best audio sources based at least in part on the one or more microphones; determine a best audio source for the speaker from the plurality of candidate best audio sources; determine an audio feed based at least in part on the best audio source; and provide the video feed and the audio feed to a conference call line.
[0010] In various embodiments, the controller system of the vehicle is further configured to process the best image source to generate a video feed, wherein the best image source is processed by at least the following operations: performing localization of a speaker in the vehicle; determining an area of one or more images of the best image source associated with the speaker; and cropping the best image source to remove one or more other occupants of the vehicle from the video feed.
[0011] In various embodiments, the positioning of the speaker is based at least in part on audio information collected by one or more microphones to determine where the speaker is. For example, microphones located throughout the cabin of the vehicle can be used to triangulate the position of the speaker based on the speech. In various embodiments, the controller system of the vehicle is configured to: determine that the speaker is obscured from the field of view of one or more cameras; and select a virtual avatar as the best image source for the speaker.
[0012] In various embodiments, the controller system of the vehicle is configured to determine the identity of the speaker, determine a display name based on the identity, and provide the display name to the conference call line. For example, when the vehicle is connected to a conference call, it may have a display name that identifies the participant as "Joe's Car", which will be sent to and viewed as a display name by remote participants in the call. Continuing with this example, when the first occupant of the car begins speaking, the name of the speaker can be identified (e.g., using voice recognition technology) and used to update the display name. The updated display name can be sent to the remote participants, and they can view additional information about the speaker as the display name, such as: "Joe's Car (Kelly speaking)". If another occupant of the vehicle then speaks, the display name can be changed to the name of the other speaker, such as: "Joe's Car (Lance speaking)".
[0013] In various embodiments, the controller system of the vehicle is configured to: determine a second speaker; determine a second best image source for the second speaker from the plurality of candidate best image sources, wherein the second best image source is different from the best image source; and use the second best image source in place of the best image source for the video feed.
[0014] In various embodiments, the plurality of candidate optimal audio sources includes one or more external devices in the vehicle connected to a controller system.
[0015] In various embodiments, the best image of the identified speaker is selected or cropped, and the audio from the speaker's location is enhanced or isolated. These enhancements may include, but are not limited to, post-processing of the video, cropping, splicing, or replacing archived images of known speakers, and post-processing of the audio to isolate, amplify, balance, bandpass filter, or noise cancel audio from background noise, thereby providing an improved user experience for all participants in the video conference.
[0016] The vehicle's controller system may act as a multiplexer that selects from a plurality of possible audio and / or video feeds from the vehicle to be provided for a video conference. In various embodiments, the vehicle's controller system is configured to select the best image and best audio of the vehicle. The best image may be selected from a cropped view or archived view of the nearest camera or speaker. The best audio may be selected from the processed / filtered audio of the nearest microphone or available microphone audio. These best video and best audio signals will be injected into a single conference call line, and as different participants in the vehicle speak, the best video and best audio of the person speaking can be isolated and injected into the telephone line. Thus, from a technical perspective, a single continuous video / audio feed is provided for the conference call, which is spliced together by the vehicle from multiple image and / or audio sources prior to transmission. BRIEF DESCRIPTION OF THE DRAWINGS
[0017] Detailed description is set forth with respect to the accompanying drawings. The use of the same reference numerals may indicate similar or identical items. Various embodiments may utilize elements and / or components other than those shown in the accompanying drawings, and some elements and / or components may not be present in various embodiments. The elements and / or components in the accompanying drawings are not necessarily drawn to scale. Throughout this disclosure, singular and plural terms may be used interchangeably, depending on the context.
[0018] Figure 1 A diagram is shown depicting an example of a smart in-vehicle conferencing with user tracking and speaker prioritization in accordance with at least one embodiment of the present disclosure.
[0019] Figure 2 A diagram is shown depicting an example of in-vehicle video processing with user tracking and speaker prioritization in accordance with at least one embodiment of the present disclosure.
[0020] Figure 3 A diagram is shown depicting an example of in-vehicle audio processing with user tracking and speaker prioritization in accordance with at least one embodiment of the present disclosure.
[0021] Figure 4 An illustrative example of a process for a smart in-vehicle conferencing with user tracking and speaker prioritization according to one or more example embodiments of the present disclosure is shown.
[0022] Figure 5 A block diagram illustrating an example of a machine or system upon which any one or more of the techniques (eg, methodologies) discussed herein may be performed. DETAILED DESCRIPTION
[0023] Figure 1 Illustration 100 of an example vehicle 102 with multiple onboard sensors is shown. In various embodiments, the vehicle 102 may include one or more interior cameras 104 and 106 positioned to capture images of occupants in the vehicle. For example, camera 104 may be a camera with a wide angle lens that captures images of occupants in the front and / or rear rows of the vehicle. The field of view of some occupants may be obstructed by headrests or other physical features of the vehicle. In some embodiments, multiple interior cameras 104, 106 are used to capture images of occupants in all seating positions within the vehicle (e.g., front seats and rear seats) and facing in any direction (e.g., facing the front, rear, or to the side of the vehicle). As Figure 1 As shown in FIG, a vehicle 102 may have cameras 104 and 106, which may be mounted to the roof, doors, or other vehicle components of the vehicle. Microphones 108 and 110 are positioned so that each microphone can capture audio of a vehicle occupant. Microphones may be located throughout the vehicle's cabin and independently capture audio of speakers.
[0024] Although Figure 1 Example configurations of microphones and cameras are depicted, but they should be considered merely as illustrative, non-limiting examples of where such audio and image collection devices may be placed. In various embodiments, each row of the vehicle has a dedicated camera and / or microphone. In some embodiments, each occupant (or only some occupants) has a dedicated camera and / or microphone.
[0025] Figure 2 A diagram 200 is shown depicting an example of in-vehicle video processing with user tracking and speaker prioritization in accordance with at least one embodiment of the present disclosure.
[0026] In at least one embodiment, the method is implemented in the context of a vehicle cabin including multiple cameras. Figure 2 For example, a first camera may be used to provide camera feed 202-A, a second camera may be used to provide camera feed 202-B, a third camera may be used to provide camera feed 202-C, a fourth camera may be used to provide camera feed 202-D, and so on. In various embodiments, the camera feeds are candidate sources for optimal image 204. Candidate sources may include other visual sources, such as a software module that may be used to generate graphics for a virtual avatar.
[0027] The best image 204 may be selected from a plurality of candidate best image sources. As discussed above, each camera in the vehicle may be used as a candidate best image source. In some embodiments, the best image 204 is selected based on a ranking of image clarity, and the first, second, etc. ranked candidate best image sources may be selected based on various factors such as image quality, bit rate, etc.
[0028] In various embodiments, the best image 204 is selected from a plurality of candidate best image sources (e.g., from one of the camera feeds 202-A to 202-D or other sources such as a virtual avatar) and processed. In some embodiments, the best image 204 is selected or cropped. These enhancements may include, but are not limited to, post-processing of the video, cropping, splicing, or replacing archived or virtual images of known speakers, thus providing an improved user experience for all participants.
[0029] In some embodiments, the best image 204 is selected from the secondary image sources and uses a file photo or avatar of the occupant who may be in the outboard seat and not in the field of view of any of the cameras in the vehicle or who may be partially obscured. The teleconferencing (CC) image may be one of a character or animation that they picked from a catalog of pre-recorded image files. In some embodiments, the speaker is obscured, but their identity is known, for example, based on the speaker's voice or from previous visual imagery. Rather than providing a partially obscured or low-resolution view of the speaker, a virtual avatar (e.g., a photorealistic emoji or cartoon character) may be generated based at least in part on the audio to lip-sync the words being recorded.
[0030] In various embodiments, the best image 204 is selected based at least in part on audio information. For example, microphones located throughout the vehicle cabin can be used to detect the general direction of where a speaker is located. Multiple microphones can detect speech and triangulate the source of the speech, and in the video, the speaker can be determined based on where the audio originates.
[0031] In various embodiments, the best image 204 is processed to produce a video feed 206. In some embodiments, the camera is able to locate or zoom in on an identified speaker that may be provided as the best image. However, in some cases, the camera may be in a fixed position and the image captured by the camera may include multiple occupants. The camera feed may be processed to crop to only the face of the person speaking. In some embodiments, where cropped, the full video image is used, but occupants not participating in the call or not speaking have their faces "muted" or obscured via blurring or high pixelation.
[0032] In various embodiments, video and / or audio from a connected device in the cabin of the vehicle may be plugged in as part of the video feed 206. In various embodiments, video from a smartphone in the cabin may be used as a candidate video source. This external video source may be preferable to using the onboard camera in situations where the speaker is far from the nearest camera or is obscured by items or other individuals in the cabin of the vehicle. The external video source may be connected via an out-of-band connection such as Wi-Fi (Wireless Fidelity), BLE ( Low power), UWB (ultra-wideband), or other types of wired or wireless technologies) connected to the vehicle's controller system.
[0033] Video feed 206 may be a composite video feed that includes video from multiple sources, such as some or all of camera feed 202-A, camera feed 202-B, camera feed 202-C, and / or camera feed 202-D. Video feed 206 may provide images to network 208, such as to a video conferencing platform that provides video feed 206 to participants 210 of a video conference call.
[0034] The video feed may be transmitted over the communication network 208 using a transmission medium via a network interface device / transceiver that utilizes any of a number of transmission protocols (e.g., frame relay, Internet Protocol (IP), Transmission Control Protocol (TCP), User Datagram Protocol (UDP), Hypertext Transfer Protocol (HTTP), etc.). In an example, the network interface device / transceiver may include one or more antennas to connect to the communication network. In an example, the network interface device / transceiver may include multiple antennas to communicate wirelessly using at least one of: single input multiple output (SIMO) technology, multiple input multiple output (MIMO) technology, or multiple input single output (MISO) technology. The term "transmission medium" shall be deemed to include any intangible medium that is capable of storing, encoding, or carrying instructions for machine execution and includes digital or analog communication signals, or other intangible media for facilitating the communication of such software.
[0035] In various embodiments, the vehicle connects to a conference call (e.g., via a network 208 with participants 210) by extracting a conference link, conference ID, password, dial-in number, or other means for specifying a conference environment, which conference link may be an HTTP-based link. The vehicle's controller system may run a script to open a web browser and access the URL of the conference call, and run a script to join the call by simulating a button click, a cursor click, a key press, etc. (e.g., using a user account associated with one of the participants in the vehicle or a user account associated with the vehicle). A script may be run to open the join conference and extract a frame or real-time video stream of the call and present it to a display screen of the vehicle, for example, on a human-machine interface (HMI) or center console of the vehicle. Audio for the call may be played on the vehicle's speakers, a connected device (e.g., via a connected to the vehicle) or the like. In various embodiments, according to the following Figure 3 Techniques are described to multiplex and process audio for conference calls.
[0036] In various embodiments, the vehicle's HMI or other display presents information to the vehicle's occupants regarding the remaining time to reach a destination. For example, if a conference call is ongoing, or is scheduled to last beyond the estimated arrival time at the vehicle's destination, a timer may be displayed to indicate the remaining time to reach the destination and alert the driver to notify participants as the vehicle approaches the destination. In some embodiments, the vehicle will provide the ability to pass the conference call line to one or more connected devices (e.g., smartphones or tablets) so that attendees of the conference call can continue to participate in the conference call when the vehicle arrives at the destination.
[0037] Figure 3 A diagram 300 is shown depicting an example of in-vehicle video processing with user tracking and speaker prioritization in accordance with at least one embodiment of the present disclosure.
[0038] In at least one embodiment, the present invention is implemented in the context of a vehicle cabin including a plurality of microphones. Figure 3 For example, the first microphone may be used to provide microphone feed 302-A, the camera microphone may be used to provide microphone feed 302-B, the third microphone may be used to provide microphone feed 302-C, the camera microphone may be used to provide microphone feed 302-D, and so on. In various embodiments, the microphone feeds are candidate sources for optimal audio 304. Candidate audio sources may include other audio sources, such as from connected devices in the vehicle cabin (such as via a audio from a smartphone paired with the vehicle's controller system).
[0039] The best audio 304 can be selected from a plurality of candidate best audio sources. As discussed above, each microphone in the vehicle can be used as a candidate best audio source. In some embodiments, the best audio 304 is selected based on a ranking of sound quality, and the first, second, etc. ranked candidate best audio sources can be selected based on various factors such as audio quality, bit rate, etc.
[0040] In various embodiments, the best audio 304 is selected from a plurality of candidate best audio sources (e.g., from one of the microphone feeds 302-A to 302-D or other sources) and processed. In some embodiments, the vehicle's controller system isolates, amplifies, balances, bandpass filters, or noise cancels the audio of the background noise to produce the audio feed 306.
[0041] In some embodiments, the best audio is selected based at least in part on processing the video feed 304. For example, the video feed may be analyzed to detect mouth movements or gestures that may be used to select an occupant who may be speaking.
[0042] In various embodiments, the best audio 304 is processed to produce an audio feed 306. In some embodiments, microphones of non-speakers are muted to reduce the amount of background noise captured in the vehicle. In some embodiments, unrecognized speech is muted. If there are multiple cabin microphones, the microphone closest to a speaker who is not on a call or is determined to be unauthorized to make a call can be muted. In some embodiments, techniques such as machine learning can be used to determine the identity of the speaker in the vehicle. If the speaker is not included in the attendee list of the video conference, the microphone closest to the speaker can be muted. This may be helpful in situations where, for example, an adult is participating in a conference call from his or her vehicle but there are children in the vehicle who are not authorized to participate in the call.
[0043] In various embodiments, two or more speakers will speak simultaneously, in which case their corresponding images can be presented in a double frame image. The corresponding audio of multiple speakers can be captured by one microphone or by the microphone closest to the corresponding speaker. In such cases, the best audio 304 may include microphone feeds from the microphone closest to each speaker and may be processed to create a composite audio feed, such as by at least equalizing the volume of multiple speakers and combining them into a composite audio feed. The composite audio feed can be provided as an audio feed 306 transmitted to the remote participant 310 of the conference call.
[0044] In various embodiments, video and / or audio from a connected device in the cabin of the vehicle can be plugged in as part of the audio feed 306. In various embodiments, audio from a smart phone in the cabin can be used as a candidate audio source. In the case where the speaker is far away from the nearest microphone, this external audio source may be preferable to using the onboard microphone. The external audio source can be connected to the vehicle's controller system via an out-of-band connection (such as Wi-Fi, BLE, UWB or other types of wired or wireless technologies). In addition, if the audio is delivered via a cellular connection rather than a direct path RF connection, latency compensation can be used to adjust the audio from the mobile device.
[0045] Audio feed 306 may be a composite audio feed that includes audio from multiple sources, such as some or all of microphone feed 302-A, microphone feed 302-B, microphone feed 302-C, and / or microphone feed 302-D. Audio feed 306 may provide audio packets to network 308, such as to a video conferencing platform that provides audio feed 306 to participants 310 of a video conference call. In various embodiments, the audio feed 306 may be provided to a network 308 in accordance with the above combined audio feed 302-A, microphone feed 302-B, microphone feed 302-C, and / or microphone feed 302-D. Figure 2 The present invention relates to a method for multiplexing and processing video for a conference call using the described techniques. In various embodiments, the same network is used to provide both audio and video to the video conferencing platform. In other embodiments, separate communication channels may be used to provide audio and video, for example, a smartphone may use a data connection (e.g., Wi-Fi) to provide video and a phone call to a conference number to provide audio.
[0046] The audio feed 306 may be transmitted over the communication network 308 via a network interface device / transceiver using a transmission medium that utilizes any of a number of transmission protocols (e.g., frame relay, Internet Protocol (IP), Transmission Control Protocol (TCP), User Datagram Protocol (UDP), Hypertext Transfer Protocol (HTTP), etc.). In an example, the network interface device / transceiver may include one or more antennas to connect to the communication network. In an example, the network interface device / transceiver may include multiple antennas to communicate wirelessly using at least one of: single input multiple output (SIMO) technology, multiple input multiple output (MIMO) technology, or multiple input single output (MISO) technology. The term "transmission medium" shall be deemed to include any intangible medium that is capable of storing, encoding, or carrying instructions for machine execution and includes digital or analog communication signals, or other intangible media for facilitating the communication of such software.
[0047] In some embodiments, the identity of the speaker is determined and used to provide metadata to other participants 310 of the conference call. For example, when a vehicle is connected to a conference call, it may have a display name that identifies the participant as "Joe's Sedan," which will be sent to and viewed as the display name by other participants 310 on the call. Continuing with this example, when the first occupant of the car begins speaking, the speaker's name may be identified (e.g., using voice recognition technology) and used to update the display name. The updated display name may be sent to the other participants 310, and they may view additional information about the speaker as the display name, such as: "Joe's Sedan (Kelly speaking)." If another occupant of the vehicle then speaks, the display name may change to the name of the other speaker, such as: "Joe's Sedan (Lance speaking)." In this way, the overall conference call experience for the other participants 310 is improved. In some embodiments, an enlarged or cropped version of the speaker's avatar or camera feed provides a view of the speaker. In this way, other participants 310 of the video conference call have an improved experience because they are able to more clearly identify which of the several occupants of the vehicle is speaking in the conference call at any given moment.
[0048] Figure 4 An illustrative example of a process 400 for conferencing in a smart vehicle with user tracking and speaker prioritization according to one or more example embodiments of the present disclosure is shown. In at least one embodiment, some or all of the process 400 (or any other process described herein, or variations and / or combinations thereof) is executed under the control of one or more computer systems storing computer executable instructions, and may be implemented as code (e.g., computer executable instructions, one or more computer programs, or one or more applications) that is executed collectively on one or more processors, implemented by hardware, software, or a combination thereof. In at least one embodiment, the code is stored on a computer-readable storage medium in the form of a computer program storing a plurality of computer-readable instructions executable by one or more processors. In at least one embodiment, the computer-readable storage medium is a non-transitory computer-readable medium. In at least one embodiment, at least some of the computer-readable instructions that can be used to perform the process 400 are not stored using only transient signals (e.g., propagated transient electrical or electromagnetic transmissions). The non-transitory computer-readable medium does not necessarily include non-transitory data storage circuits (e.g., buffers, caches, and queues) within the transceiver of the transient signal. Process 400 may be implemented in the context of various systems and methods described elsewhere in this disclosure, such as in conjunction with Figures 1 to 3In at least one embodiment, process 400 or a portion thereof is co-implemented by a controller system of a vehicle. In various embodiments, a vehicle controller is used to perform some or all of the steps of process 400 and / or steps of other processes described throughout this disclosure.
[0049] In various embodiments, process 400 Figure 4 4 and begins at step 402, where a system (e.g., a controller system of a vehicle) performing process 400 determines a plurality of candidate optimal image sources based at least in part on one or more cameras within a cabin of the vehicle. For example, cameras may be located throughout the cabin of the vehicle. In some embodiments, the vehicle has a camera located at the front of the vehicle that is ideally positioned to capture video of occupants in the front row of the vehicle and another camera that is ideally positioned to capture video of occupants in the second or third row of the vehicle.
[0050] More generally, the vehicle cabin described in the context of process 400 is a non-limiting example of one type of enclosed space in which the techniques described herein may be practiced. Figures 1 to 4 The described technology can be applicable to a variety of enclosed spaces, such as conference rooms, airplanes, trains, or other automotive environments. Other automotive environments may include shuttles, taxis, or buses, including unmanned autonomous vehicles with seats facing each other, all facing one direction (forward or backward), facing left or right, or a combination of seat orientations.
[0051] In various embodiments, process 400 includes a step 404 of selecting or otherwise determining an optimal image source of a speaker in the vehicle from a plurality of candidate optimal image sources. In some embodiments, the optimal image is selected from a secondary image source and uses a file photo or avatar of the occupant who may be in an outboard seat and not in view of any of the cameras in the vehicle or who may be partially obscured. The teleconferencing (CC) image may be one of a character or animation that they picked from a catalog of pre-recorded image files. In some embodiments, the speaker is obscured, but their identity is known, for example, based on the speaker's voice or from previous visual imagery. Rather than providing a partially obscured or low-resolution view of the speaker, a virtual avatar (e.g., a photorealistic emoji or cartoon character) may be generated based at least in part on the audio to lip-sync the words being recorded.
[0052] In various embodiments, process 400 includes a step 406 of determining a video feed based at least in part on the best image source. In some embodiments, the best image will receive a video feed from the best candidate image source and perform cropping, splicing, or replacement of an archived or virtual image of a known speaker, thereby providing an improved user experience for all participants. For example, in some embodiments, the best candidate image source is an avatar of the speaker. In some embodiments, a virtual avatar of the speaker will be presented as the best image. The virtual avatar can be a static image, or can be an animated virtual avatar that has lip-sync animations that mimic speech detected from an onboard microphone.
[0053] In various embodiments, process 400 includes step 408 of determining a plurality of candidate optimal audio sources based at least in part on one or more microphones. The plurality of candidate optimal audio sources may include one candidate optimal audio source for each microphone. In some embodiments, additional candidate optimal audio sources may include, for example, UWB or Wi-Fi connects other devices connected to the vehicle's controller system, such as a smartphone in the vehicle's cabin.
[0054] In various embodiments, process 400 includes a step 410 of determining an optimal audio source for a speaker from a plurality of candidate optimal audio sources. The optimal audio source may be determined based on video information. For example, facial detection technology may be used to determine when an occupant of the vehicle is speaking, and the microphone closest to the speaker may be selected as the optimal audio source.
[0055] In various embodiments, process 400 includes step 412 of determining an audio feed based at least in part on the optimal audio source. In various embodiments, the audio from the optimal audio source is processed to produce the audio feed. In some embodiments, the controller system of the vehicle isolates, amplifies, balances, bandpass filters, or noise cancels the audio of the background noise to produce the audio feed.
[0056] In various embodiments, two or more speakers will speak simultaneously, in which case their respective images may be presented in a dual-frame image. The respective audio of the multiple speakers may be captured by one microphone or by the microphone closest to the respective speakers. In such cases, the best audio may include microphone feeds from the microphone closest to each speaker and may be processed to create a composite audio feed, such as by at least equalizing the volume of the multiple speakers and combining them into a composite audio feed. The composite audio feed may be provided as an audio feed for transmission to the remote participants of the conference call.
[0057] In various embodiments, process 400 includes step 414 of providing a video feed and an audio feed of a teleconference platform to the teleconference line, which can receive a continuous stream of video and / or audio content from the system performing process 400. For example, even when a second speaker participates in the teleconference and the best image and / or best audio source shifts to the camera and / or microphone closest to the second speaker, it will appear as a seamless video and audio feed to another remote participant of the teleconference.
[0058] In various embodiments, the best image is selected based on audio information. For example, a microphone may be used to detect a speaker in the vehicle, and the camera closest to the audio may be selected as the best image. In various embodiments, microphones located at different areas in the vehicle cabin may capture audio and triangulation may be used to determine the location of the speaker. The location of the speaker may be used to select a candidate video source as the best image.
[0059] In various embodiments, to improve the teleconference experience for those not in the vehicle, the best image of the speaker (nearest camera or cropped view or archived view) and the best audio capture of the speaker (e.g., nearest microphone or processed / filtered audio of available microphone audio) will be selected. The best video and best audio signals will be injected into a single teleconference line, and as different participants in the vehicle cabin speak, the best video and best audio of the speaker will be isolated and injected into the telephone line.
[0060] In various embodiments, the vehicle's controller system collects video feeds from all cabin cameras to decide which cabin camera has the best video stream of one or more speakers, and the vehicle then merges the feeds into one outgoing video stream that is sent as a seamless video feed to the remote conference call participants.
[0061] If there are multiple cabin microphones, if all authorized participants have trained their voices, the microphone closest to the speaker who is clearly not on the call or is determined to be unauthorized to talk can be muted. Unrecognized speech from a single microphone system can also be muted, but is more difficult.
[0062] In various embodiments, the system can choose not to use the camera feed for people who are not authorized to talk or are not actively speaking by not selecting that camera. If the person cannot be removed from the camera field, the system can crop to only the faces of the person who is speaking and / or pixelate the faces of those who are not speaking.
[0063] In various embodiments, microphones and / or cameras may be used to detect if a participant has dozed off, and the system may buzz their headrest (while silencing their local microphone) to discreetly wake them up.
[0064] In various embodiments, the best image and / or audio can be selected from an external video or audio source. For example, video and audio from a second or third phone can be piped into the vehicle cabin for sending / receiving a different call (e.g., a main conference call). This would allow third row participants to be better heard and seen, for example. This could allow for better audio and noise cancellation for the entire system. Phone audio for third row users can be bound by an environment using an out-of-band connection (BLE, Wi-Fi, UWB, etc.) as a supplemental microphone, allowing for single call connectivity and management (no unmanaged open microphones).
[0065] In various embodiments, process 400 includes using a facial recognition engine in the vehicle to align the user's face with the camera (e.g., so they appear to be looking at the camera, rather than being viewed from above or from an off angle). The system can determine the angle of the user's face relative to the camera and, assuming they are outside a threshold, correct the image to be looking at the camera.
[0066] Figure 5A block diagram of an example of a machine 500 or system on which any one or more of the techniques (e.g., methodologies) discussed herein may be performed is shown. The machine (e.g., computer system) 500 may include any combination of the components shown. For example, the machine 500 may include a hardware processor 502 (e.g., a central processing unit (CPU), a graphics processing unit (GPU), a tensor processing unit (TPU) including an artificial intelligence application specific integrated circuit (ASIC), a hardware processor core, or any combination thereof), a main memory 504, and a static memory 506, some or all of which may communicate with each other via an interconnect (e.g., a bus) 508. The machine 500 may also include a power management device 532, a graphics display device 510, an alphanumeric input device 512 (e.g., a keyboard), and a user interface (UI) navigation device 514 (e.g., a mouse). In an example, the graphics display device 510, the alphanumeric input device 512, and the UI navigation device 514 may be a touch screen display. The machine 500 may additionally include a storage device (i.e., a drive unit) 516, a signal generating device 518 (e.g., a data signal), a network interface device / transceiver 520 coupled to an antenna 530, and one or more sensors 528, such as a sound detection sensor (e.g., a microphone), an accelerometer, a magnetometer, a position sensor, etc. The machine 500 may include an output controller 534, such as a serial (e.g., Universal Serial Bus (USB)), parallel or other wired or wireless (e.g., infrared (IR), near field communication (NFC), etc.) connection to communicate with or control one or more peripheral devices (e.g., a printer, a card reader, other sensors, etc.).
[0067] The storage device 516 may include a machine-readable medium 522 on which is stored one or more sets of data structures or instructions 524 (e.g., software) embodied or utilized by any one or more of the techniques or functionalities described herein. The instructions 524 may also reside, completely or at least partially, within the main memory 504, within the static memory 506, or within the hardware processor 502 during execution of the instructions by the machine 500. In an example, one or any combination of the hardware processor 502, the main memory 504, the static memory 506, or the storage device 516 may constitute a machine-readable medium. In various embodiments, the controller system of the vehicle is implemented using one or more machines (e.g., the machine 500).
[0068] Although machine-readable medium 522 is illustrated as a single medium, the term “machine-readable medium” may include a single medium or multiple media (eg, a centralized or distributed database, and / or associated caches and servers) configured to store one or more instructions 524 .
[0069] Various embodiments may be implemented in whole or in part in software and / or firmware. This software and / or firmware may take the form of instructions contained in or on a non-transitory computer-readable storage medium. Those instructions may then be read and executed by one or more processors to enable the execution of the operations described herein. The instructions may be in any suitable form, such as, but not limited to, source code, compiled code, interpreted code, executable code, static code, dynamic code, and the like. Such computer-readable media may include any tangible, non-transitory medium for storing information in a form readable by one or more computers, such as, but not limited to, read-only memory (ROM); random access memory (RAM); magnetic disk storage media; optical storage media; flash memory, and the like.
[0070] The term "machine-readable medium" may include any medium capable of storing, encoding, or carrying instructions for execution by the machine 500; and causing the machine 500 to perform any one or more of the techniques disclosed herein; or capable of storing, encoding, or carrying data structures used by or associated with such instructions. Non-limiting examples of machine-readable media may include solid-state memory and optical and magnetic media. In an example, a large-capacity machine-readable medium includes a machine-readable medium having a plurality of particles having a rest mass. Specific examples of large-scale machine-readable media may include non-volatile memory, such as semiconductor memory devices (e.g., electrically programmable read-only memory (EPROM), or electrically erasable programmable read-only memory (EEPROM)) and flash memory devices; magnetic disks, such as internal hard disks and removable disks; magneto-optical disks; and CD-ROM and DVD-ROM disks.
[0071] The instructions 524 may also be transmitted or received over the communication network 526 using a transmission medium via the network interface device / transceiver 520 using any of a number of transmission protocols (e.g., frame relay, Internet Protocol (IP), transmission control protocol (TCP), user datagram protocol (UDP), hypertext transfer protocol (HTTP), etc.). Example communication networks may include DOCSIS, fiber optics, a local area network (LAN), a wide area network (WAN), a packet data network (e.g., the Internet), a mobile telephone network (e.g., a cellular network), a plain old telephone (POTS) network, a wireless data network (e.g., a wireless data network called The Institute of Electrical and Electronics Engineers (IEEE) 802.11 series of standards, known as ), IEEE 802.16 series of standards), IEEE 802.15.4 series of standards, and peer-to-peer (P2P) networks, etc. In an example, the network interface device / transceiver 520 may include one or more physical sockets (e.g., Ethernet sockets, coaxial sockets, or telephone sockets) or one or more antennas to connect to the communication network 526. In an example, the network interface device / transceiver 520 may include multiple antennas to communicate wirelessly using at least one of the following: single-input multiple-output (SIMO) technology, multiple-input multiple-output (MIMO) technology, or multiple-input single-output (MISO) technology. The term "transmission medium" should be deemed to include any intangible medium that can store, encode, or carry instructions for execution by the machine 500 and includes digital or analog communication signals, or other intangible media for facilitating the communication of such software.
[0072] The operations and processes described and shown above can be implemented or performed in any suitable order when necessary in various embodiments. In addition, in some embodiments, at least a portion of the operations can be implemented in parallel. In addition, in some embodiments, less or more operations than the described operations can be performed.
[0073] The word "exemplary" is used herein to mean "serving as an example, instance, or illustration". Any embodiment described herein as "exemplary" is not necessarily to be construed as preferred or advantageous over other embodiments. The terms "computing device", "user device", "communication station", "station", "handheld device", "mobile device", "wireless device" and "user equipment" (UE) as used herein refer to a wireless communication device such as a cable box, a wearable smart device, a cellular phone, a smart phone, a tablet, a netbook, a wireless terminal, a laptop computer, a femtocell, a high data rate (HDR) subscriber station, an access point, a printer, a point of sale device, an access terminal or other personal communication system (PCS) device. The device may be mobile or fixed.
[0074] As used in this document, the term "communication" is intended to include transmission or reception, or both transmission and reception. This may be particularly useful in a claim when describing the organization of data transmitted by one device and received by another device, but requiring the functionality of only one of those devices violates the claim. Similarly, a two-way data exchange between two devices (during which both devices transmit and receive) may be described as "communication" when only the functionality of one of those devices is claimed. The term "communication" as used herein with respect to wireless communication signals includes transmitting wireless communication signals and / or receiving wireless communication signals. For example, a wireless communication unit capable of transmitting wireless communication signals may include: a wireless transmitter for transmitting wireless communication signals to at least one other wireless communication unit; and / or a wireless communication receiver for receiving wireless communication signals from at least one other wireless communication unit.
[0075] As used herein, unless otherwise indicated, the use of ordinal adjectives "first," "second," "third," etc. to describe common objects indicates merely that different instances of similar objects are mentioned and is not intended to imply that the objects so described must be in a given sequence in time, space, ranking, or in any other manner.
[0076] Some embodiments may be used in conjunction with various devices and systems, such as wearable smart devices, personal computers (PCs), desktop computers, mobile computers, laptop computers, notebook computers, tablet computers, server computers, handheld computers, handheld devices, personal digital assistant (PDA) devices, handheld PDA devices, in-vehicle devices, out-of-vehicle devices, hybrid devices, vehicle devices, non-vehicle devices, mobile or portable devices, consumer devices, non-mobile or non-portable devices, wireless communication stations, wireless communication devices, wireless access points (APs), wired or wireless routers, wired or wireless modems, video devices, audio devices, audio-video (A / V) devices, wired or wireless networks, wireless area networks, wireless video area networks (WVANs), local area networks (LANs), wireless LANs (WLANs), personal area networks (PANs), wireless PANs (WPANs), and the like.
[0077] Some embodiments may be used in conjunction with: one-way and / or two-way radio communication systems, cellular radio-telephone communication systems, mobile phones, cellular phones, cordless phones, personal communication system (PCS) devices, PDA devices including wireless communication devices, mobile or portable global positioning system (GPS) devices, devices including GPS receivers or transceivers or chips, devices including RFID elements or chips, multiple-input multiple-output (MIMO) transceivers or devices, single-input multiple-output (SIMO) transceivers or devices, multiple-input single-output (MISO) transceivers or devices, devices with one or more internal antennas and / or external antennas, digital video broadcasting (DVB) devices or systems, multi-standard radio devices or systems, wired or wireless handheld devices (e.g., smart phones), wireless application protocol (WAP) devices, etc.
[0078] Some embodiments may be used in conjunction with one or more types of wireless communication signals and / or systems that comply with one or more wireless communication protocols, such as DOCSIS, radio frequency (RF), infrared (IR), frequency division multiplexing (FDM), orthogonal FDM (OFDM), time division multiplexing (TDM), time division multiple access (TDMA), extended TDMA (E-TDMA), general packet radio service (GPRS), extended GPRS, code division multiple access (CDMA), wideband CDMA (WCDMA), CDMA 2000, single carrier CDMA, multi-carrier CDMA, multi-carrier modulation (MDM), discrete multi-tone (DMT), Global Positioning System (GPS), Wi-Fi, Wi-Max, ZigBee, Ultra-Wideband (UWB), Global System for Mobile Communications (GSM), 2G, 2.5G, 3G, 3.5G, 4G, fifth generation (5G) mobile networks, 3GPP, Long Term Evolution (LTE), LTE Advanced, Enhanced Data Rates for GSM Evolution (EDGE), etc. Other embodiments may be used in various other devices, systems and / or networks.
[0079] Embodiments according to the present disclosure are particularly disclosed in the attached claims, which relate to methods, storage media, devices, and computer program products, wherein any feature mentioned in one claim category (e.g., method) may also be claimed in another claim category (e.g., system). The dependencies or references in the attached claims are selected for formal reasons only. However, any subject matter resulting from deliberate reference to any previous claim (especially multiple dependencies) may also be claimed, so that any combination of claims and their features is disclosed and may be claimed, regardless of the dependencies selected in the attached claims. The subject matter that may be claimed includes not only the combination of features set forth in the attached claims, but also any other combination of features in the claims, wherein each feature mentioned in the claims may be combined with any other feature in the claims or with a combination of other features. In addition, any of the embodiments and features described or depicted herein may be claimed in a separate claim and / or in any combination with any embodiment or feature described or depicted herein or with any feature of the attached claims.
[0080] The above description of one or more embodiments provides illustration and description, but is not intended to be exhaustive or to limit the scope of the embodiments to the precise forms disclosed. Modifications and variations are possible in light of the above teachings or may be acquired from practice of the various embodiments.
[0081] Certain aspects of the present disclosure are described above with reference to block diagrams and flow charts of systems, methods, devices and / or computer program products according to various embodiments. It should be understood that one or more blocks of the block diagrams and flow charts and combinations of blocks in the block diagrams and flow charts may be implemented by computer executable program instructions, respectively. Likewise, according to some embodiments, some blocks of the block diagrams and flow charts may not necessarily need to be executed in the order presented, or may not necessarily need to be executed at all.
[0082] These computer executable program instructions can be loaded onto a special purpose computer or other specific machine, processor, or other programmable data processing device to produce a specific machine, so that the instructions executed on the computer, processor, or other programmable data processing device create a device for implementing one or more functions specified in one or more flowchart boxes. These computer program instructions can also be stored in a computer-readable storage medium or memory, which can guide the computer or other programmable data processing device to act in a specific manner when executed, so that the instructions stored in the computer-readable storage medium produce a product including an instruction device for implementing one or more functions specified in one or more flowchart boxes. As an example, some embodiments may provide a computer program product, which includes a computer-readable storage medium having a computer-readable program code or program instructions implemented therein, and the computer-readable program code is suitable for being executed to implement one or more functions specified in one or more flowchart boxes. Computer program instructions can also be loaded onto a computer or other programmable data processing device to cause a series of operating elements or steps to be executed on a computer or other programmable device to produce a computer-implemented process, so that the instructions executed on a computer or other programmable device provide elements or steps for implementing the functions specified in one or more flowchart boxes.
[0083] Therefore, the blocks of the block diagrams and flow charts support a combination of means for performing the specified functions, a combination of elements or steps for performing the specified functions, and program instruction means for performing the specified functions. It will also be understood that each block of the block diagrams and flow charts and the combination of blocks in the block diagrams and flow charts can be implemented by a dedicated hardware-based computer system or a combination of dedicated hardware and computer instructions that performs the specified functions, elements or steps.
[0084] Embodiments of the systems, devices, apparatuses and methods disclosed herein may include or utilize a special or general-purpose computer, which includes computer hardware, such as one or more processors and system memory as discussed herein. Computer executable instructions include, for example, instructions and data, which, when executed at a processor, cause a general-purpose computer, a special-purpose computer or a special-purpose processing device to perform a specific function or function group. Embodiments of the apparatus, systems and methods disclosed herein may communicate via a computer network. A "network" is defined as one or more data links that support the transmission of electronic data between a computer system and / or a module and / or other electronic devices.
[0085] Although the subject matter has been described in language specific to structural features and / or methodological acts, it should be understood that the subject matter defined in the appended claims may not necessarily be limited to the features or acts described above. Rather, the described features and acts are disclosed as exemplary forms of implementing the claims.
[0086] Although various embodiments of the present disclosure have been described above, it should be understood that various embodiments of the present disclosure are presented only by way of example and not by way of limitation. It will be understood by those skilled in the relevant art that various changes in form and detail may be made without departing from the spirit and scope of the present disclosure. Therefore, the breadth and scope of the present disclosure should not be limited by any of the above exemplary embodiments, but should only be limited according to the attached claims and their equivalents. The foregoing description has been presented for illustration and description purposes. The foregoing description is not intended to be exhaustive or to limit the present disclosure to the precise form disclosed. In view of the above teachings, many modifications and variations are possible. In addition, it should be noted that any or all of the aforementioned optional embodiments may be used in any desired combination to form additional hybrid embodiments of the present disclosure. For example, any of the functionality described with respect to a particular device or component may be performed by another device or component. Unless otherwise specifically stated or otherwise understood within the context when used, conditional language, such as, in particular, "can," "might," "may," or "may" is generally intended to convey that certain embodiments may include certain features, elements, and / or steps, while other embodiments may not. Thus, such conditional language is generally not intended to imply that one or more embodiments require each feature, element, and / or step in any way.
[0087] In one aspect of the invention, the positioning of the speaker is based at least in part on audio information collected by one or more microphones to determine where the speaker is.
[0088] In one aspect of the invention, the method includes: determining, by the one or more processors, that the speaker is obscured from the field of view of the one or more cameras; and selecting, by the one or more processors, a virtual avatar as the optimal image source for the speaker.
[0089] In one aspect of the present invention, the method includes: determining, by the one or more processors, an identity of the speaker; determining, by the one or more processors, a display name based on the identity; and providing, by the one or more processors, the display name to the conference call line.
[0090] In one aspect of the present invention, the method includes: determining, by the one or more processors, a second speaker; determining, by the one or more processors, a second best image source for the second speaker from the plurality of candidate best image sources, wherein the second best image source is different from the best image source; and using, by the one or more processors, the second best image source in place of the best image source for the video feed.
[0091] In one aspect of the invention, the plurality of candidate optimal audio sources include one or more smartphone devices in the enclosure.
Claims
1. A vehicle comprising: one or more cameras; one or more microphones; as well as A controller system, the controller system further comprising one or more processors and a memory storing executable instructions, wherein the executable instructions are executed by the one or more processors to cause the controller system to: determining a plurality of candidate optimal image sources based at least in part on the one or more cameras; determining a best image source for a speaker in the vehicle from among the plurality of candidate best image sources; determining a video feed based at least in part on the optimal image source; determining a plurality of candidate optimal audio sources based at least in part on the one or more microphones; Determining the best audio source for the speaker from among the plurality of candidate best audio sources; determining an audio feed based at least in part on the optimal audio source; as well as The video feed and the audio feed are provided to a telephone conference line.
2. The vehicle of claim 1, wherein the executable instructions include further instructions that, upon execution by the one or more processors, further cause the controller system to: Processing the best image source to generate the video feed, wherein the best image source is processed by at least the following operations: performing localization of the speaker in the vehicle; determining a region of one or more images of the optimal image source associated with the speaker; and The best image source is cropped to remove one or more other occupants of the vehicle from the video feed. 3 . The vehicle of claim 2 , wherein the positioning of the speaker is based at least in part on audio information collected by the one or more microphones to determine where the speaker is.
4. The vehicle of claim 1 , wherein the executable instructions include additional instructions that, upon execution by the one or more processors, further cause the controller system to: determining that the speaker is obscured from a field of view of the one or more cameras; and An avatar is selected as the optimal image source for the speaker.
5. The vehicle of claim 1 , wherein the executable instructions include additional instructions that, upon execution by the one or more processors, further cause the controller system to: determining the identity of said speaker; determining a display name based on the identity; and The display name is provided to the conference call line.
6. The vehicle of claim 1 , wherein the executable instructions include additional instructions that, upon execution by the one or more processors, further cause the controller system to: Identify the second speaker; determining a second best image source for the second speaker from the plurality of candidate best image sources, wherein the second best image source is different from the best image source; and The second best image source is used in place of the best image source for the video feed.
7. The vehicle of claim 6, wherein the plurality of candidate optimal audio sources comprises one or more external devices in the vehicle connected to the controller system.
8. A non-transitory computer-readable storage medium storing executable instructions that, when executed by one or more processors of a controller system of a vehicle, cause the controller system to at least: determining a plurality of candidate optimal image sources based at least in part on one or more cameras in a cabin of the vehicle; determining a best image source for a speaker in the vehicle from among the plurality of candidate best image sources; determining a video feed based at least in part on the optimal image source; determining a plurality of candidate optimal audio sources based at least in part on one or more microphones in the cabin of the vehicle; Determining the best audio source for the speaker from among the plurality of candidate best audio sources; determining an audio feed based at least in part on the optimal audio source; as well as The video feed and the audio feed are provided to a telephone conference line.
9. The non-transitory computer-readable storage medium of claim 8, wherein the executable instructions, as a result of being executed by the one or more processors, further cause the controller system to: Processing the best image source to generate the video feed, wherein the best image source is processed by at least the following operations: performing localization of the speaker in the vehicle; determining a region of one or more images of the optimal image source associated with the speaker; and The best image source is cropped to remove one or more other occupants of the vehicle from the video feed.
10. The non-transitory computer-readable storage medium of claim 9, wherein the positioning of the speaker is based at least in part on audio information collected by the one or more microphones to determine where the speaker is.
11. The non-transitory computer-readable storage medium of claim 8, wherein the executable instructions, as a result of being executed by the one or more processors, further cause the controller system to: determining that the speaker is obscured from a field of view of the one or more cameras; and An avatar is selected as the optimal image source for the speaker.
12. The non-transitory computer-readable storage medium of claim 8, wherein the executable instructions, as a result of being executed by the one or more processors, further cause the controller system to: determining the identity of said speaker; determining a display name based on the identity; and The display name is provided to the conference call line.
13. The non-transitory computer-readable storage medium of claim 8, wherein the executable instructions, as a result of being executed by the one or more processors, further cause the controller system to: Identify the second speaker; determining a second best image source for the second speaker from the plurality of candidate best image sources, wherein the second best image source is different from the best image source; and The second best image source is used in place of the best image source for the video feed.
14. A method comprising: determining, by one or more processors, a plurality of candidate optimal image sources based at least in part on one or more cameras of the enclosure; determining, by the one or more processors, an optimal image source for a speaker in the closed space from among the plurality of candidate optimal image sources; determining, by the one or more processors, a video feed based at least in part on the optimal image source; determining, by the one or more processors, a plurality of candidate optimal audio sources based at least in part on the one or more microphones in the enclosure; determining, by the one or more processors, an optimal audio source for the speaker from among the plurality of candidate optimal audio sources; determining, by the one or more processors, an audio feed based at least in part on the optimal audio source; as well as The video feed and the audio feed are provided by the one or more processors to a teleconference circuit.
15. The method of claim 14, further comprising: The best image source is processed by the one or more processors to generate the video feed, wherein the best image source is processed by at least the following operations: performing localization of a speaker in the enclosed space; determining a region of one or more images of the optimal image source associated with the speaker; as well as The best image source is cropped to remove one or more other occupants of the enclosure from the video feed.
