Conference equipment with multiple video stream control

Abstract

A conferencing device with multi-video stream control includes: a first image sensor for providing first image data; a first image processor configured to provide a first main video stream and a first secondary video stream based on the first image data; an intermediate image processor communicating with the first image processor and configured to provide a field of view video stream and an area of interest video stream, wherein the field of view video stream is based on the first main video stream, and the area of interest video stream is based on the first secondary video stream; and an output processor configured to determine first image control data based on the field of view video stream and send the first image control data to the first image processor; wherein the first image processor is configured to provide the first secondary video stream based on the first image control data.

Description

Technical Field

[0001] The present disclosure relates to a conference device and related methods for multi-video stream control. Background Art

[0002] In video conferencing, it may be advantageous to have multiple video streams (e.g., dual video streams or trial video streams) simultaneously. However, there are difficulties in achieving high resolution and / or high quality for all video streams, which limits the scalability of video streaming. Summary of the Invention

[0003] Therefore, there is a need for improved conferencing equipment, such as conferencing systems, solutions, components, and methods, which can expand multi-video stream capabilities.

[0004] One or more exemplary conferencing devices are disclosed herein. The conferencing device may include a first image sensor. The first image sensor may be configured to provide first image data. The conferencing device may include a second image sensor. The second image sensor may be configured to provide second image data. The conferencing device may include a first image processor. The first image processor may be configured to provide a first main video stream and a first secondary video stream based on the first image data. The conferencing device may include a second image processor. The second image processor may be configured to provide a second main video stream and a second secondary video stream based on the second image data. The conferencing device may include an intermediate image processor. The intermediate image processor may communicate with the first image processor and the second image processor. The intermediate image processor may be configured to provide a field of view video stream. The intermediate image processor may be configured to provide an area of interest video stream. The field of view video stream may be based on the first main video stream and the second main video stream. The area of interest video stream may be based on one or more of the first video stream and the second video stream.

[0005] One or more exemplary conferencing devices are disclosed herein. The conferencing device may include a first image sensor. The first image sensor may be configured to provide first image data. The conferencing device may include a first image processor. The first image processor may be configured to provide a first main video stream and a first secondary video stream based on the first image data. The conferencing device may include an intermediate image processor. The intermediate image processor may communicate with the first image processor. The intermediate image processor may be configured to provide a field of view video stream. The intermediate image processor may be configured to provide a region of interest video stream. The field of view video stream may be based on the first main video stream. The region of interest video stream may be based on the first secondary video stream. The conferencing device may include an output processor. The output processor may be configured to determine first image control data. The first image control data may be based on the field of view video stream. The output processor may be configured to send the first image control data to the first image processor. The first image processor may be configured to provide the first secondary video stream based on the first image control data.

[0006] The present disclosure provides a device (eg, a conferencing device) and method for implementing multi-video stream capabilities, as well as control of the multi-video streams. Specifically, the conferencing device can utilize parallel video streams.

[0007] Specifically, the present disclosure allows for providing a field-of-view video stream along with a region-of-interest video stream to a user. The region-of-interest video stream can change as different areas of the original image data come into focus. The focus of the region-of-interest video stream can be controlled by user input or via a machine learning engine. Furthermore, the present disclosure can utilize the field-of-view video stream to determine control data for influencing the region-of-interest video stream.

[0008] Advantageously, the present disclosure may allow for a modular, plug-and-play system or conferencing device for implementing multiple video streams. Thus, instead of a complex setup, a user may, for example, plug in a single connector to implement multiple video streams, thereby greatly simplifying the user's requirements for operating the conferencing device.

[0009] Furthermore, the conferencing device and method can be expanded for high-quality video streams. Because the field-of-view video stream and the area-of-interest video stream are associated with the raw image data from the image sensor, the conferencing device does not have to perform extensive high-quality video processing. Consequently, both the field-of-view and area-of-interest video streams can be high-quality, improving user experience and viewing options while reducing costs and time. Consequently, when providing multiple video streams to a user or host device, the present invention can utilize the raw image data without any loss of quality.

[0010] An important advantage of the present disclosure is that separate and independent field of view and area of interest video streams are provided to allow for flexible and high quality viewing options.

[0011] Furthermore, the present disclosure allows for real-time adjustment of the region of interest video stream based on the field of view video stream while maintaining high image quality of the region of interest video stream. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] The above and other features and advantages will become apparent to those skilled in the art from the following detailed description of exemplary embodiments of the present disclosure with reference to the accompanying drawings, in which:

[0013] Figure 1 shows an exemplary conference device of the present disclosure,

[0014] Figure 2 A detailed view showing an exemplary conferencing device of the present disclosure, and

[0015] Figure 3 A partial view of an exemplary conferencing device of the present disclosure is shown. DETAILED DESCRIPTION

[0016] Various exemplary embodiments and details will be described below with reference to the relevant drawings. It should be noted that the drawings may or may not be drawn to scale, and elements of similar structure or function are represented by similar reference numerals throughout the figures. It should also be noted that the drawings are intended only to facilitate the description of the embodiments. They are not intended to serve as an exhaustive description of the invention or to limit the scope of the invention. In addition, the illustrated embodiments do not necessarily have all the aspects or advantages shown. Aspects or advantages described in conjunction with a particular embodiment are not necessarily limited to that embodiment, and even if not shown or explicitly described as such, they may be practiced in any other embodiment.

[0017] This document discloses conferencing devices (e.g., devices for conferences), conferencing systems, solutions, and components with multi-video stream capabilities (e.g., multiple video stream capabilities). This document also discloses methods for implementing multi-video stream capabilities in conferencing devices. For example, conferencing devices can provide dual video streams. Conferencing devices can provide live and / or real-time video streams.

[0018] In one or more exemplary conference devices, the conference device can be set up in a room. For example, the conference device can be configured for use in an office, house, factory, or building. In addition, the conference device can be configured for outdoor use. The specific location of the conference device is not limited.

[0019] In one or more exemplary embodiments, a conferencing device may be configured to provide multiple video streams as output to a user. For example, the conferencing device may provide a field of view. This allows the user to see everything or a significant portion of what the conferencing device's camera or image sensor sees. This allows the user to roughly understand what the camera sees in a specific area where the conferencing device is located.

[0020] Furthermore, conferencing devices can provide separate video streams. These separate video streams can be based on a region of interest. The region of interest can be a user-desired view that can be received alongside the field of view. Thus, the user can receive a broad and more general field of view while also receiving a more customized and / or focused region of interest. During a video conference, the region of interest may change, while the field of view may remain static or substantially static.

[0021] As examples, the focus area can be a speaker, a specific location, or a specific person. There are no limitations on the specific focus area. Furthermore, the focus area may change over time. For example, the focus area may change from a first person to a second person, or from a first area to a second area. Alternatively, if two people are speaking simultaneously, the focus area may change from focusing on one person to a broader image of multiple people speaking simultaneously. The focus area may be determined by the conference device. The focus area may be determined by the user and / or the host device. The focus area may be determined by a machine learning and / or artificial intelligence engine.

[0022] Thus, in one or more exemplary conferencing devices, the conferencing device can provide two or more video streams to a user so that they can see a field of view and an area of interest. The field of view and the area of interest can be real-time, typically real-time, and / or real-time. The two or more video streams can be interleaved and provided via a single connector.

[0023] As disclosed herein, certain components of the conferencing device may communicate with each other. They may communicate directly with each other. They may communicate indirectly, such as through intermediate components that may or may not be discussed herein.

[0024] The connection may be wired or wireless. The wireless communications discussed herein may be, for example, cellular, Bluetooth, LTE, radio, PAN, LAN, WLAN, satellite, microwave, WAN, etc., and the specific wireless communications are not limiting.

[0025] The conference device may include one or more memory components configured to store data. Each component of the conference device may include its own memory component for storing data. Each component of the conference device may share a common memory for storing data.

[0026] In one or more exemplary conference devices, the conference device may include one or more sensors. In one or more exemplary conference devices, the conference device may include one or more image sensors, such as a camera, an imaging device, a digital camera, an optical sensor, an electro-optical sensor, a scanner. In one or more exemplary conference devices, the conference device may be integrated with one or more sensors (e.g., one or more image sensors). One or more sensors (e.g., one or more image sensors) may also provide audio information to the conference device. Alternatively or additionally, a separate audio device (e.g., a microphone) may be used to provide the audio information.

[0027] In one or more exemplary conferencing devices, the conferencing device may communicate with one or more sensors. In one or more exemplary conferencing devices, the conferencing device may communicate with one or more image sensors. Thus, one or more image sensors may not be part of the conferencing device but may be in communication with the conferencing device. The image sensors in the one or more image sensors may be configured to communicate with each other.

[0028] In one or more exemplary conferencing devices, the conferencing device may include a first image sensor. The conferencing device may include the first image sensor for providing first image data. In one or more exemplary conferencing devices, the conferencing device may include a second image sensor. The conferencing device may include the second image sensor for providing second image data. In one or more exemplary conferencing devices, the conferencing device may include a third image sensor. The conferencing device may include the third image sensor for providing third image data. The conferencing device may further include additional image sensors.

[0029] The number of image sensors is not limited, and a conference device may include and / or be associated with 1, 2, 3, 4, 5, 6, 7, 8, 9, or 10 image sensors. A conference device may include more than 1, 2, 3, 4, 5, 6, 7, 8, 9, or 10 image sensors and / or be associated with more than 1, 2, 3, 4, 5, 6, 7, 8, 9, or 10 image sensors. A conference device may include less than 2, 3, 4, 5, 6, 7, 8, 9, or 10 image sensors and / or be associated with less than 2, 3, 4, 5, 6, 7, 8, 9, or 10 image sensors. Each image sensor may provide its own image data.

[0030] One or more image sensors may be configured for wireless transmission of image data. One or more image sensors may be configured for wired transmission of image data.

[0031] One or more image sensors may be charge-coupled devices (CCDs). One or more image sensors may be active pixel sensors (APSs). One or more image sensors may be CMOS image sensors. One or more sensors may be digital sensors. One or more image sensors may be part of or all of a camera. One or more image sensors may be configured to capture visible images. The specific type of image sensor is not limited.

[0032] One or more image sensors may also be configured to provide additional data. For example, one or more image sensors may be configured to provide audio data. Thus, one or more image sensors may include audio capture technology, such as a microphone.

[0033] The one or more image sensors may have a specific resolution. Each of the one or more image sensors may have the same resolution. Some of the one or more image sensors may have a different resolution than other image sensors.

[0034] The resolution of one or more image sensors may be 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.0, 8.0, 9.0, 10, 13, 15, 20, 25, 30, 35, 40, 50, 60, 70, 80, 90, 100, 150, 200, 250, 300, 300, 350, 400, 450, or 500 megapixels (MP). The resolution of one or more image sensors may be greater than 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.0, 8.0, 9.0, 10, 13, 15, 20, 25, 30, 35, 40, 50, 60, 70, 80, 90, 100, 150, 200, 250, 300, 300, 350, 400, 450, or 500 megapixels (MP). The resolution of one or more image sensors can be less than 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.0, 8.0, 9.0, 10, 13, 15, 20, 25, 30, 35, 40, 50, 60, 70, 80, 90, 100, 150, 200, 250, 300, 300, 350, 400, 450, or 500 megapixels (MP). In one or more exemplary conferencing devices, the image sensor can provide image data with a resolution of 4208x3120 pixels.

[0035] Therefore, the resolution of image data such as the first image data, the second image data, the third image data, etc. can be 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.0, 8.0, 9.0, 10, 13, 15, 20, 25, 30, 35, 40, 50, 60, 70, 80, 90, 100, 150, 200, 250, 300, 300, 350, 400, 450 or 500 million pixels (MP). The resolution of image data, such as the first image data, the second image data, the third image data, etc., may be greater than 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.0, 8.0, 9.0, 10, 13, 15, 20, 25, 30, 35, 40, 50, 60, 70, 80, 90, 100, 150, 200, 250, 300, 300, 350, 400, 450, or 500 megapixels (MP). The resolution of image data such as the first image data, the second image data, the third image data, etc. can be less than 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.0, 8.0, 9.0, 10, 13, 15, 20, 25, 30, 35, 40, 50, 60, 70, 80, 90, 100, 150, 200, 250, 300, 300, 350, 400, 450, or 500 megapixels (MP). In one or more exemplary conferencing devices, image data such as the first image data, the second image data, the third image data, etc. can have a resolution of 4208x3120 pixels.

[0036] The one or more image sensors can be mountable. For example, the one or more image sensors can be mounted on a wall and / or ceiling. The one or more image sensors can include accessories or other components, such as a bracket.

[0037] One or more image sensors may store image data, such as in a memory. For example, a first image sensor may store first image data. A second image sensor may store second image data. A third image sensor may store third image data. In alternative conferencing devices, the image sensors may not store image data.

[0038] The image data such as the first image data, the second image data and the third image data may be in any format, and the specific format is not limited. Example formats include but are not limited to mp4, mov, avi, mkv, TS.

[0039] In one or more exemplary conferencing devices, the conferencing device may include one or more image processors, such as one or more image sensor processors. For example, the image processor may be an image processing unit, an image signal processor, a processing engine, an encoder, an image encoder, etc.

[0040] The image processor may be integrated on a chip. The image processor may be integrated on a circuit. The image processor may be integrated on an integrated circuit. The image processor may be included in the conferencing device.

[0041] The image processor may store data. For example, the image processor may include a memory for data storage.

[0042] One or more image processors can communicate with one or more image sensors. One or more image processors can interface with one or more image sensors. For example, the Mobile Industry Processor Interface (MIPI) can be used to communicate between one or more image sensors and one or more image processors. However, the specific interface / standard is not limited. CSI-1, CSI-2, and CSI-3 standards can be used. A display serial interface can be used. A serial bus can be used. eDP and HDMI can be used.

[0043] The communication between the one or more image sensors and the one or more image processors may be wireless.The communication between the one or more image sensors and the one or more image processors may be wired.

[0044] In one or more exemplary conferencing devices, the conferencing device may include a 1:1 ratio of image sensors to image processors. Thus, the conferencing device may include a first image processor, a second image processor, and a third image processor. The first image processor may communicate with the first image sensor, the second image processor may communicate with the second image sensor, and the third image processor may communicate with the third image sensor.

[0045] Alternatively, the conferencing device may not have a 1:1 ratio of image sensors to image processors. Thus, more than one image sensor may communicate with a single image processor. Furthermore, a single image sensor may communicate with more than one image processor. In one or more exemplary conferencing devices, the ratio of image sensors to image processors may be 2:1, 3:1, 4:1, 5:1, 6:1, etc. In one or more exemplary conferencing devices, the ratio of image processors to image sensors may be 2:1, 3:1, 4:1, 5:1, 6:1, etc.

[0046] The number of image processors is not limited, and a conference device may include and / or be associated with 1, 2, 3, 4, 5, 6, 7, 8, 9, or 10 image processors. A conference device may include and / or be associated with more than 1, 2, 3, 4, 5, 6, 7, 8, 9, or 10 image processors. A conference device may include and / or be associated with less than 2, 3, 4, 5, 6, 7, 8, 9, or 10 image processors.

[0047] Each image processor can be configured to provide at least one video stream. For example, each image processor can be configured to provide two video streams. Each image processor can be configured to provide more than two video streams, such as three, four, five, six, seven, eight, nine, or ten video streams. For example, the video streams can be primary, secondary, tertiary, quaternary, etc. The specific labeling of the video streams may or may not be indicative of the quality and / or importance of a particular video stream.

[0048] In one or more exemplary conferencing devices, the image processor can be configured to provide a video stream based on received image data. Thus, if the image processor is in communication with a specific image sensor, the video stream can be based on image data from the specific image sensor. Consequently, multiple video streams can be provided for each image data.

[0049] Thus, for example, the first image processor may be configured to provide a first primary video stream and a first secondary video stream based on the first image data. The second image processor may be configured to provide a second primary video stream and a second secondary video stream based on the second image data. The third image processor may be configured to provide a third primary video stream and a third tertiary video stream based on the third image data. In one or more exemplary conferencing devices, the image processor may be configured to provide two, three, four, etc., times as many video streams as the image data received by the image processor.

[0050] Thus, the image processor can be used to separate two or more different video streams from the image data. The image processor can select two or more video streams from the image data. The image processor can separate two or more video streams from the image data.

[0051] In one or more exemplary conferencing devices, the corresponding primary video streams and secondary video streams can be independent, e.g., separate from each other. Thus, a first primary video stream can be independent from a first primary video stream. In other words, the first primary video stream and the first primary video stream can be independent and / or separate video streams. A second primary video stream can be independent from a second primary video stream. In other words, the second primary video stream and the second secondary video stream can be independent and / or separate video streams. A third primary video stream can be independent from a third primary video stream. In other words, the third primary video stream and the third secondary video stream can be independent and / or separate video streams.

[0052] Therefore, in one or more exemplary conferencing devices, the corresponding secondary video stream is not a subset of the corresponding primary video stream. In one or more exemplary conferencing devices, the first video stream is not a subset of the first primary video stream. In one or more exemplary conferencing devices, the second video stream is not a subset of the second primary video stream. In one or more exemplary conferencing devices, the third video stream is not a subset of the third primary video stream.

[0053] The secondary video stream may overlap with the primary video stream. The secondary video stream may be completely contained within the primary video stream. In one or more exemplary conferencing devices, the resolution of the secondary video stream and / or the primary video stream may be less than the resolution of the raw data image received / obtained from the image sensor. In one or more exemplary conferencing devices, the resolution of the secondary video stream and the primary video stream may be the same as the resolution of the received raw data image.

[0054] Therefore, the image processor can be used to modify the original image data into a primary video stream and a secondary video stream. The first image processor can modify, encode, or process the first image data into a first primary video stream and a first secondary video stream. The second image processor can modify, encode, or process the second image data into a second primary video stream and a second secondary video stream. The third image processor can modify, encode, or process the third image data into a third primary video stream and a third tertiary video stream.

[0055] For example, the primary resolution of the primary video stream may be smaller than the resolution of the image data. The first primary resolution of the first primary video stream may be smaller than the resolution of the first image data. The second primary resolution of the second primary video stream may be smaller than the resolution of the second image data. The third primary resolution of the third primary video stream may be smaller than the resolution of the third image data. The secondary resolution of the secondary video stream may be smaller than the resolution of the image data. The first resolution of the first primary video stream may be smaller than the resolution of the first image data. The second resolution of the second secondary video stream may be smaller than the resolution of the second image data. The third resolution of the third secondary video stream may be smaller than the resolution of the third image data.

[0056] Thus, the image processor can typically prepare the views of both streams that the user will ultimately see, for example, after any subsequent processing of the video streams. Thus, the image processor can use image control data, such as first image control data and / or second image control data and / or third image control data, to provide the first video stream and / or the second video stream and / or the third video stream. For example, the first image control data can provide instructions to the first image processor to select a portion or region of the first image data for the first video stream. For example, this can be one or more of the first image data containing a person, the first image data containing a person's face, and the first image data containing an object of particular interest. For example, the second image control data can provide instructions to the second image processor to select a portion or region of the second image data for the second video stream. For example, this can be one or more of the second image data containing a person, the second image data containing a person's face, and the second image data containing an object of particular interest. For example, the third image control data can provide instructions to the third image processor to select a portion or region of the third image data for the third video stream. For example, this may be one or more of an area of the third image data containing a person, an area of the third image containing a person's face, and an area of the third image data containing a thing of particular interest.

[0057] In one or more exemplary conferencing devices, a single image control data may be sent to all image processors, such as the first image processor, the second image processor, and / or the third image processor.

[0058] The first main video stream may be part of the first image data. The first main video stream may be the entire first image data. The first secondary video stream may be part of the first image data. The first secondary video stream may be the entire first image data. The first main video stream and the first secondary video stream may overlap on the first image data. The first main video stream and the first secondary video stream may not overlap on the first image data.

[0059] The second primary video stream may be a portion of the second image data. The second primary video stream may be the entire second image data. The secondary video stream may be a portion of the second image data. The secondary video stream may be the entire second image data. The second primary video stream and the secondary video stream may overlap on the second image data. The second primary video stream and the secondary video stream may not overlap on the second image data.

[0060] The third main video stream may be a portion of the third image data. The third main video stream may be the entire third image data. The third tertiary video stream may be a portion of the third image data. The third tertiary video stream may be the entire third image data. The third main video stream and the third tertiary video stream may overlap on the third image data. The third main video stream and the third tertiary video stream may not overlap on the third image data.

[0061] The resolution of the main video stream (for example, the first main video stream and / or the second main video stream and / or the third main video stream, etc.) can be 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.0, 8.0, 9.0, 10, 15, 20, 25, 30, 35, 40, 50, 60, 70, 80, 90, 100, 150, 200, 250, 300, 300, 350, 400, 450 or 500 million pixels (MP). The resolution of the main video stream (for example, the first main video stream and / or the second main video stream and / or the third main video stream, etc.) can be greater than 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.0, 8.0, 9.0, 10, 15, 20, 25, 30, 35, 40, 50, 60, 70, 80, 90, 100, 150, 200, 250, 300, 300, 350, 400, 450 or 500 megapixels (MP). The resolution of the primary video stream (e.g., the first primary video stream and / or the second primary video stream and / or the third primary video stream, etc.) can be less than 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.0, 8.0, 9.0, 10, 15, 20, 25, 30, 35, 40, 50, 60, 70, 80, 90, 100, 150, 200, 250, 300, 300, 350, 400, 450, or 500 megapixels (MP). In one or more exemplary conferencing devices, the primary video stream, such as the first primary video stream and / or the second primary video stream and / or the third primary video stream, can have a resolution of 4208x3120 pixels. In one or more exemplary conferencing devices, the primary video streams, such as the first primary video stream and / or the second primary video stream and / or the third primary video stream, may have a resolution of 1600x1200 pixels.

[0062] The resolution of the secondary video stream (e.g., the first video stream and / or the second video stream and / or the third video stream, etc.) can be 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.0, 8.0, 9.0, 10, 15, 20, 25, 30, 35, 40, 50, 60, 70, 80, 90, 100, 150, 200, 250, 300, 300, 350, 400, 450 or 500 megapixels (MP). The resolution of the secondary video stream (e.g., the first video stream and / or the second video stream and / or the third video stream, etc.) can be greater than 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.0, 8.0, 9.0, 10, 15, 20, 25, 30, 35, 40, 50, 60, 70, 80, 90, 100, 150, 200, 250, 300, 300, 350, 400, 450 or 500 megapixels (MP). The resolution of the secondary video stream (e.g., the first video stream and / or the second video stream and / or the third video stream, etc.) can be less than 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.0, 8.0, 9.0, 10, 15, 20, 25, 30, 35, 40, 50, 60, 70, 80, 90, 100, 150, 200, 250, 300, 300, 350, 400, 450, or 500 megapixels (MP). In one or more exemplary conferencing devices, the resolution of the secondary video stream, such as the first video stream and / or the second video stream and / or the third video stream, etc., can be 4208x3120 pixels. In one or more exemplary conferencing devices, the resolution of the secondary video streams, such as the first video stream and / or the second video stream and / or the third video stream, can be 1920x1080 pixels. In one or more exemplary conferencing devices, the resolution of the secondary video streams, such as the first video stream and / or the second video stream and / or the third video stream, can reach the resolution of the image sensor, the first image data, the second image data, and the third image data, respectively.

[0063] The image processor may be configured to provide any of a number of functions to the image data. For example, the image processor may be configured to perform one or more of Bayer conversion, demosaicing, noise reduction, image sharpening, etc. The specific processing functions performed are not limited.

[0064] In one or more exemplary conferencing devices, the conferencing device may include one or more intermediate processors, such as one or more intermediate image processors. For example, the intermediate image processor may be a processing engine, an encoder, an image encoder, and the like.

[0065] The intermediate image processor may store data. For example, the intermediate image processor may include a memory for data storage.

[0066] The intermediate image processor may be integrated on a chip. The intermediate image processor may be integrated on a circuit. The intermediate image processor may be integrated on an integrated circuit. The intermediate image processor may be integrated on a field programmable gate array.

[0067] The intermediate image processor may be on the same chip and / or circuit and / or integrated circuit as the image processor. The intermediate image processor may be on a different chip and / or circuit and / or integrated circuit than the image processor.

[0068] One or more intermediate image processors can communicate with one or more image processors. One or more intermediate image processors can interface with one or more image processors. For example, the Mobile Industry Processor Interface (MIPI) can be used for communication between one or more image processors and one or more intermediate image processors. However, the specific interface / standard is not limited. CSI-1, CSI-2, CSI-3 standards can be used. Display Serial Interface can be used. Serial bus can be used. eDP and HDMI can be used.

[0069] The communication between the one or more image processors and the one or more intermediate image processors may be wireless.The communication between the one or more image processors and the one or more intermediate image processors may be wired.

[0070] In one or more exemplary conferencing devices, the conferencing device may include a single intermediate image processor. The intermediate image processor may be configured to receive a primary video stream and a secondary video stream, as discussed above. Thus, the intermediate image processor may be configured to receive a first primary video stream and / or a second primary video stream and / or a third primary video stream and / or a first video stream and / or a second video stream and / or a third video stream.

[0071] Alternatively, in one or more exemplary conferencing devices, the conferencing device may include a 1:1 ratio of intermediate image processors to image processors. Thus, the conferencing device may include a first intermediate image processor, a second intermediate image processor, or a third intermediate image processor. The first image processor may communicate with the first intermediate image processor, the second image processor may communicate with the second intermediate image processor, and the third image processor may communicate with the third intermediate image processor.

[0072] The number of intermediate image processors is not limited, and a conference device may include and / or be associated with 1, 2, 3, 4, 5, 6, 7, 8, 9, or 10 intermediate image processors. A conference device may include more than 1, 2, 3, 4, 5, 6, 7, 8, 9, or 10 intermediate image processors and / or be associated with more than 1, 2, 3, 4, 5, 6, 7, 8, 9, or 10 intermediate image processors. A conference device may include less than 2, 3, 4, 5, 6, 7, 8, 9, or 10 intermediate image processors and / or be associated with less than 2, 3, 4, 5, 6, 7, 8, 9, or 10 intermediate image processors.

[0073] Thus, the intermediate image processor can communicate with the first image processor. The intermediate image processor can communicate with the second image processor. The intermediate image processor can communicate with the third image processor. The intermediate image processor can communicate with the first image processor and / or the second image processor and / or the third image processor.

[0074] The intermediate image processor may be configured to provide a field of view video stream, such as a room video stream, a region of interest video stream, or a field of view video stream and a region of interest video stream.

[0075] The field of view video stream may have a resolution of 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.0, 8.0, 9.0, 10, 13, 15, 20, 25, 30, 35, 40, 50, 60, 70, 80, 90, 100, 150, 200, 250, 300, 300, 350, 400, 450, or 500 megapixels (MP). The resolution of the field of view video stream can be greater than 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.0, 8.0, 9.0, 10, 13, 15, 20, 25, 30, 35, 40, 50, 60, 70, 80, 90, 100, 150, 200, 250, 300, 300, 350, 400, 450, or 500 megapixels (MP). The resolution of the field of view video stream can be less than 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.0, 8.0, 9.0, 10, 13, 15, 20, 25, 30, 35, 40, 50, 60, 70, 80, 90, 100, 150, 200, 250, 300, 300, 350, 400, 450, or 500 megapixels (MP). In one or more exemplary conferencing devices, the resolution of the field of view video stream can be 4208x3120 pixels.

[0076] The resolution of the region of interest video stream can be 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.0, 8.0, 9.0, 10, 13, 15, 20, 25, 30, 35, 40, 50, 60, 70, 80, 90, 100, 150, 200, 250, 300, 300, 350, 400, 450 or 500 megapixels (MP). The resolution of the area of interest video stream can be greater than 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.0, 8.0, 9.0, 10, 13, 15, 20, 25, 30, 35, 40, 50, 60, 70, 80, 90, 100, 150, 200, 250, 300, 300, 350, 400, 450 or 500 megapixels (MP). The resolution of the region of interest video stream can be less than 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.0, 8.0, 9.0, 10, 13, 15, 20, 25, 30, 35, 40, 50, 60, 70, 80, 90, 100, 150, 200, 250, 300, 300, 350, 400, 450, or 500 megapixels (MP). In one or more exemplary conferencing devices, the resolution of the region of interest video stream can be 4208x3120 pixels.

[0077] The field of view video stream may be based on the first primary video stream. The field of view video stream may be based on the second primary video stream. The field of view video stream may be based on the third primary video stream. The field of view video stream may be based on the first primary video stream and the second primary video stream. The field of view video stream may be based on the first primary video stream and the third primary video stream. The field of view video stream may be based on the third primary video stream and the second primary video stream. The field of view video stream may be based on the first primary video stream, the second primary video stream, and the third primary video stream.

[0078] In one or more exemplary conferencing devices, a field of view video stream can be combined (e.g., spliced) from multiple video streams. An intermediate image processor can perform the combination and / or splicing. One or more splicing engines can be used. The field of view video stream can be spliced from a first primary video stream, a second primary video stream, and a third primary video stream. The field of view video stream can be spliced from a first primary video stream and a second primary video stream. The field of view video stream can be spliced from a first primary video stream and a third primary video stream. The field of view video stream can be spliced from a third primary video stream and a second primary video stream. The field of view video stream can be combined, for example, by splicing from any number of video streams.

[0079] The area of interest video stream may be based on one or more of the first video stream and the second video stream. The area of interest video stream may be based on one or more of the first video stream, the second video stream, and the third video stream. The area of interest video stream may be based on the first video stream. The area of interest video stream may be based on the second video stream. The area of interest video stream may be based on the third video stream. The area of interest video stream may be based on the first video stream and the second video stream. The area of interest video stream may be based on the first video stream and the third video stream. The area of interest video stream may be based on the third video stream and the second video stream. The area of interest video stream may be based on the first video stream, the second video stream, and the third video stream. The area of interest video stream may be based on the first video stream and / or the second video stream and / or the third video stream.

[0080] The intermediate image processor can be configured to select from the secondary video streams, for example, the first, second, and third video streams. Thus, only one secondary video stream may be used. For example, the intermediate image processor can be configured to select one of the first, second, and third video streams for the region of interest video stream. In other words, the intermediate image processor can be configured to splice the first, second, and third primary video streams together to provide the field of view video stream.

[0081] In one or more exemplary conferencing devices, the intermediate image processor can receive region of interest selection control data, which will be discussed in detail below. Thus, the intermediate image processor can use the region of interest selection control data to select a specific one of the first video stream, the second video stream, and the third video stream for providing the region of interest video stream.

[0082] As an example, the region of interest selection control data may indicate that the region of interest may fall within the secondary video stream. Therefore, the secondary video stream may be selected by the intermediate image processor for the region of interest video stream.

[0083] Furthermore, the region of interest selection control data can vary across the video stream. Thus, the region of interest selection control data may initially indicate selection of the second video stream. The region of interest selection control data may then be adjusted to select the first video stream. The region of interest selection control data may then be adjusted again to select the second video stream. Each region of interest selection control data provided may be considered the same region of interest selection control data, or may be designated as first, second, third, etc., e.g., first region of interest selection control data.

[0084] Alternatively, the intermediate image processor may be configured to combine (e.g., splice) the secondary video streams. For example, the intermediate image processor may be configured to combine (e.g., splice together) the first video stream and the second video stream. The intermediate image processor may be configured to combine (e.g., splice together) the first video stream and the third video stream. The intermediate image processor may be configured to combine (e.g., splice together) the second video stream and the third video stream. The intermediate image processor may be configured to combine (e.g., splice together) the first video stream, the second video stream, and the third video stream. Thus, the region of interest video stream may be combined, e.g., spliced, from the first video stream, the second video stream, and the third video stream. If all secondary video streams are combined (e.g., spliced), the intermediate image processor may not receive the region of interest selection control data.

[0085] The intermediate image processor may perform image processing on the primary video stream and / or the secondary video stream, such as one or more of the first primary video stream, the second primary video stream, the third primary video stream, the first video stream, the second video stream, and the third video stream. The intermediate image processor may perform image processing on the field of view video stream and / or the area of interest video stream. The intermediate image processor may perform signal processing on the primary video stream and / or the secondary video stream, such as one or more of the first primary video stream, the second primary video stream, the third primary video stream, the first video stream, the second video stream, and the third video stream. The intermediate image processor may perform signal processing on the field of view video stream and / or the area of interest video stream. For example, the intermediate image processor may perform one or more of geometric correction or conversion, noise reduction, and chroma subsampling on the primary video stream and / or the secondary video stream.

[0086] In one or more exemplary conference devices, the conference device may include one or more output processors. For example, the output processor may be (or include) a processing engine, an encoder, an image encoder, etc.

[0087] The output processor may be integrated on a chip. The output processor may be integrated on a circuit. The output processor may be integrated on an integrated circuit. The output processor may be integrated on a field programmable gate array.

[0088] The output processor may be on the same chip and / or circuit and / or integrated circuit as the image processor. The output processor may be on a different chip and / or circuit and / or integrated circuit than the image processor.

[0089] The output processor may be on the same chip and / or circuit and / or integrated circuit as the intermediate image processor. The output processor may be on a different chip and / or circuit and / or integrated circuit than the intermediate image processor.

[0090] The output processor may store data. For example, the output processor may include a memory for data storage.

[0091] One or more output processors can communicate with one or more intermediate image processors (e.g., intermediate image processors). One or more output processors can interface with one or more intermediate image processors. For example, the Mobile Industry Processor Interface (MIPI) can be used for communication between one or more output processors and one or more intermediate image processors. However, the specific interface / standard is not limited. CSI-1, CSI-2, and CSI-3 standards can be used.

[0092] The communication between the one or more output processors and the one or more intermediate image processors may be wireless.The communication between the one or more output processors and the one or more intermediate image processors may be wired.

[0093] One or more output processors can communicate with one or more image processors, such as a first image processor and / or a second image processor and / or a third image processor. One or more output processors can interface with one or more image processors, such as a first image processor and / or a second image processor and / or a third image processor. For example, a Mobile Industry Processor Interface (MIPI) can be used for communication between one or more output processors and one or more image processors. However, the specific interface / standard is not limited. CSI-1, CSI-2, CSI-3 standards can be used. A display serial interface can be used. A serial bus can be used. eDP and HDMI can be used.

[0094] The communication between the one or more output processors and the one or more image processors may be wireless.The communication between the one or more output processors and the one or more image processors may be wired.

[0095] In one or more exemplary conferencing devices, the conferencing device may include a single output processor. The output processor may be configured to receive a field-of-view video stream. The output processor may be configured to receive a region-of-interest video stream. Thus, the output processor may be configured to receive a field-of-view video stream and / or a region-of-interest video stream.

[0096] Alternatively, in one or more exemplary conferencing devices, the conferencing device may include a 1:1 ratio of output processors to image processors. In one or more exemplary conferencing devices, the conferencing device may include a 1:1 ratio of output processors to intermediate image processors.

[0097] The number of output processors is not limited, and a conference device may include and / or be associated with 1, 2, 3, 4, 5, 6, 7, 8, 9, or 10 output processors. A conference device may include more than 1, 2, 3, 4, 5, 6, 7, 8, 9, or 10 output processors and / or be associated with more than 1, 2, 3, 4, 5, 6, 7, 8, 9, or 10 output processors. A conference device may include less than 2, 3, 4, 5, 6, 7, 8, 9, or 10 output processors and / or be associated with less than 2, 3, 4, 5, 6, 7, 8, 9, or 10 output processors.

[0098] In one or more exemplary conferencing devices, the output processor can be configured to provide a primary output video stream. The primary output video stream can be based on the field of view video stream. The primary output video stream can be based on the area of interest video stream. For example, the output processor can be configured to provide the primary output video stream based on the field of view video stream or the area of interest video stream.

[0099] In one or more exemplary conferencing devices, the output processor can be configured to provide a secondary output video stream. The secondary output video stream can be based on the field of view video stream. The secondary output video stream can be based on the area of interest video stream. For example, the output processor can be configured to provide the secondary output video stream based on the field of view video stream or the area of interest video stream.

[0100] In one or more exemplary conferencing devices, the output processor includes an interface, such as an output interface and a video stream interface. The output interface can be configured to provide a primary output video stream. The primary output video stream can be based on a field of view video stream. The output interface can be configured to provide a secondary output video stream. The secondary output video stream can be based on a region of interest video stream. The output interface can be configured to provide the primary output video stream and the secondary output video stream. The output interface can be configured for further output video streams, such as a tertiary output video stream, a quaternary output video stream, and the like.

[0101] In one or more exemplary conferencing devices, the primary output video stream can have a specific field of view. For example, the field of view of the primary output video stream can be at least 90, 100, 110, 120, 130, 140, 150, 160, 170, 180, 190, 200, 210, 220, 230, 240, 250, 260, 270, 280, 290, 300, 310, 320, 330, 340, 350, or 360 degrees. The primary output video stream may have a field of view of less than 90, 100, 110, 120, 130, 140, 150, 160, 170, 180, 190, 200, 210, 220, 230, 240, 250, 260, 270, 280, 290, 300, 310, 320, 330, 340, 350, or 360°. In one or more exemplary conferencing devices, the field of view of the primary output video stream may be in the range of 120° to 200° (horizontally), for example, in the range of 150° to 180°.

[0102] In one or more exemplary conferencing devices, the secondary output video stream can have a specific field of view. For example, the field of view of the secondary output video stream can be at least 30, 40, 50, 60, 70, 80, 90, 100, 110, 120, 130, 140, 150, 160, 170, 180, 190, 200, 210, 220, 230, 240, 250, 260, 270, 280, 290, 300, 310, 320, 330, 340, 350, or 360 degrees. The secondary output video stream may have an angle of less than 30, 40, 50, 60, 70, 80, 90, 100, 110, 120, 130, 140, 150, 160, 170, 180, 190, 200, 210, 220, 230, 240, 250, 260, 270, 280, 290, 300, 310, 320, 330, 340, 340, 350, or 360°.

[0103] In one or more exemplary conferencing devices, the field of view of the primary output video stream may be at least 120°, such as at least 150°, and the field of view of the secondary output video stream may be less than 90°.

[0104] The primary output video stream may have the same field of view as the secondary output video stream. The primary output video stream may have a different field of view than the secondary output video stream. The primary output video stream may have a larger field of view than the secondary output video stream. The primary output video stream may have a smaller field of view than the secondary output video stream.

[0105] The primary output video stream can have a field of view that is 1.1, 1.2, 1.3, 1.4, 1.5, 1.6, 1.7, 1.8, 1.9, 2.0, 2.1, 2.2, 2.4, 2.5, 2.6, 2.7, 2.8, 2.9, 3.0, 3.1, 3.2, 3.3, 3.4, 3.6, 3.7, 3.8, 3.0, 4.1, 4.2, 4.3, 4.4, 4.6, 4.7, 4.8, 4.9 or 5.0 times larger than the field of view of the secondary output video stream (horizontally). The secondary output video stream can have a field of view that is 1.1, 1.2, 1.3, 1.4, 1.5, 1.6, 1.7, 1.8, 1.9, 2.0, 2.1, 2.2, 2.3, 2.5, 2.6, 2.7, 2.8, 2.9, 3.0, 3.1, 3.2, 3.3, 3.4, 3.6, 3.7, 3.8, 3.0, 4.1, 4.2, 4.3, 4.4, 4.5, 4.6, 4.7, 4.8, 4.9 or 5.0 times larger than the field of view of the primary output video stream.

[0106] The output processor can be configured to process the field of view video stream. The output processor can be configured to process the region of interest video stream. The output processor can be configured to process the field of view video stream and the region of interest video stream.

[0107] The processing can be any number of different processes. For example, the output processor can be configured to perform one or more of image conversion, distortion correction, enhancement, and contrast correction. The output processor can be configured to perform one or more of image conversion, distortion correction, enhancement, and contrast correction on the area of interest video stream to provide a secondary output video stream. The output processor can be configured to perform one or more of image conversion, distortion correction, enhancement, perspective adjustment, and contrast correction on the field of view video stream to provide a primary output video stream. The output processor can provide additional signal processing, such as noise reduction and / or chroma subsampling on the field of view video stream and / or the area of interest video stream. In one or more exemplary conferencing devices, the output processor can be configured to "whiteboard" the area of interest video stream onto the secondary output video stream. Thus, the output processor can provide perspective adjustment to frame the video stream for the user so that the user appears to be viewing the area of interest, such as a whiteboard, head-on. This can be achieved even if the original image data is set at a certain angle.

[0108] The output processor may also include additional functionality. For example, the output processor may perform optical character recognition on any text within the field of view video stream and / or the region of interest video stream. The output processor may further perform object extraction on the field of view video stream and / or the region of interest video stream. The output processor may further perform object manipulation on the field of view video stream and / or the region of interest video stream. The output processor may also compensate for missing light on the field of view video stream and / or the region of interest video stream.

[0109] As described above, the output processor can be configured to perform image processing on the field of view video stream and / or the region of interest video stream. Alternatively, or in combination with processing, the output processor can be configured to provide control data to other components within the conferencing device. For example, as described above, the output device can communicate with one or more of the intermediate module and the image processors (e.g., the first image processor, the second image processor, and the third image processor).

[0110] In one or more exemplary conferencing devices, an output processor is configured to determine image control data, such as image control instructions or image control information. The image control data may indicate a region of interest. The output processor may then transmit and / or communicate the image control data. The control data, such as the first image control data, the second image control data, and / or the third image control data, may include instructions, parameters, standards, etc., to the corresponding image processor.

[0111] The first image control data may indicate a region of interest in the first image data, such as one or more of a position, size, and shape of the region of interest. The second image control data may indicate a region of interest in the second image data, such as one or more of a position, size, and shape of the region of interest. The third image control data may indicate a region of interest in the second image data, such as one or more of a position, size, and shape of the region of interest.

[0112] As an example, the output processor may be configured to determine first image control data. The first image control data may be based on the field of view video stream. The first image control data may not be based on the field of view video stream. The first image control data may be based on the area of interest video stream. The first image control data may not be based on the area of interest video stream. In one or more exemplary conferencing devices, the first image control data may be based on, for example, user input or external control input from a host device.

[0113] The output processor may then send the first image control data to the first image processor. As described above, the first image processor may be configured to provide a first secondary video stream based on the first image control data. Furthermore, the first image processor may be configured to provide a first primary video stream based on the first image control data. In one or more exemplary conferencing devices, the output processor may communicate with the intermediate image processor and be configured to determine the first image control data based on the field of view video stream and to send the first image control data to the first image processor.

[0114] Furthermore, the output processor may be configured to determine second image control data. The second image control data may be based on the field of view video stream. The second control data may not be based on the field of view video stream. In one or more exemplary conferencing devices, the second image control data may be based on, for example, user input or external control input from the host device. Furthermore, the second image processor may be configured to provide a second primary video stream based on the second image control data. The output processor may then send the second image control data to the second image processor. The second image processor may be configured to provide a secondary video stream based on the second image control data.

[0115] The output processor may also be configured to determine third image control data. The third image control data may be based on the field of view video stream. The third image control data may not be based on the field of view video stream. In one or more exemplary conferencing devices, the third image control data may be based on, for example, user input or external control input from the host device. Furthermore, the third image processor may be configured to provide a third primary video stream based on the third image control data. The output processor may then send the third image control data to the third image processor. The third image processor may be configured to provide a third primary video stream based on the third image control data.

[0116] In one or more exemplary conferencing devices, an output processor can determine and / or transmit / communicate image control data, such as first image control data, second image control data, and / or third image control data. As discussed above, the image control data can be used to adjust a secondary output to be provided based on a region of interest video stream. Thus, a specific region of interest within a field of view video stream can be determined using a field of view video stream.

[0117] Therefore, the output processor can be configured to determine control data, such as first image control data, second image control data, and / or third image control data. For example, the output processor can analyze the field of view video stream for a specific person who is speaking. Therefore, it may be advantageous to focus the region of interest on the person who is speaking. Therefore, the output processor can determine control data regarding the specific person who is speaking from the field of view video stream. Other regions of interest can also be used, such as the person who is speaking, a specific location, an area where a presentation is being presented, a whiteboard / blackboard, etc.

[0118] The output processor may then send the image control data to the specific image processor to provide a secondary video stream. In one or more exemplary conferencing devices, the output processor may send first image control data to the first image processor to provide a primary video stream. In one or more exemplary conferencing devices, the output processor may send second image control data to the second image processor to provide a secondary video stream. In one or more exemplary conferencing devices, the output processor may send third image control data to the third image processor to provide a tertiary video stream.

[0119] Thus, the control data can provide instructions to the image processor via the image control data to focus on a specific area of interest in the image data, such as the first image data, the second image data, and the third image data. The specific image processor can then provide a more focused secondary video stream, such as the first video stream, the second video stream, and / or the third video stream, to the intermediate image processor.

[0120] In one or more exemplary conferencing devices, the output processor may send first image control data to the first image processor to provide a first primary video stream. In one or more exemplary conferencing devices, the output processor may send second image control data to the second image processor to provide a second primary video stream. In one or more exemplary conferencing devices, the output processor may send third image control data to the third image processor to provide a third primary video stream.

[0121] Therefore, the control data can provide instructions to the image processor via the image control data to focus on a specific field of view in the image data, such as the first image data, the second image data, and the third image data. The specific image processor can then provide a more focused main video stream, such as the first main video stream, the second main video stream, and / or the third main video stream, to the intermediate image processor.

[0122] Advantageously, the output processor allows for focusing on a specific region of interest and can provide instructions to the image processor to do so. The output processor can only send control data to the relevant image processor. For example, if the region of interest is only found within the field of view of the second image sensor, the output processor can send second control data to the second image processor. If there are three image sensors, the output processor may not send the first control data to the first image processor, nor the third control data to the third image processor, as no changes will be made to the first and third image processors.

[0123] In one or more exemplary conferencing devices, the output processor may send control data to all available image processors, even if a particular image processor does not require further adjustment. Thus, if there are three image sensors, the output processor may provide first image control data to a first image processor, second image control data to a second image processor, and third image control data to a third image processor.

[0124] In one or more exemplary conferencing devices, the output processor may be configured to determine the region of interest selection control data.The output processor may send the region of interest selection control data to the intermediate image processor.

[0125] In one or more exemplary conferencing devices, the intermediate image processor can be configured to stitch together secondary video streams (such as the first video stream, the second video stream, and / or the third video stream) to provide an area of interest video stream. Alternatively, the intermediate image processor can select one of the first video stream, the second video stream, and / or the third video stream to provide the area of interest video stream. Thus, the area of interest video stream can be based on image data from only one of the image sensors.

[0126] Thus, in one or more exemplary conferencing devices, the output processor may send region of interest selection control data to the intermediate image processor.

[0127] The area of interest selection control data may be used to select which of the secondary video streams (such as the first video stream, the second video stream, and / or the third video stream) to select to provide the area of interest video stream.

[0128] In one or more exemplary conferencing devices, an output processor may analyze a field of view video stream for a specific person who is speaking. Therefore, it may be advantageous to select a secondary video stream containing the person who is speaking. Therefore, the output processor may determine region of interest selection control data regarding the specific person who is speaking from the field of view video stream.

[0129] However, the region of interest selection control data need not be based on the field of view video stream. The output processor can determine it on its own, for example by using a machine learning engine as discussed below. In addition, user input or external control input, such as from a host device, can be included to determine the region of interest selection control data.

[0130] Therefore, the region of interest selection control data can provide instructions to the intermediate image processor via the control data for selecting a specific secondary video stream to provide the region of interest video stream. The intermediate image processor can then select the best secondary video stream, such as the first video stream, the second video stream, and / or the third video stream.

[0131] Thus, the region of interest selection control data may include instructions, parameters, criteria, etc. for the intermediate image processor.

[0132] In one or more exemplary conference devices, the output processor may include a machine learning engine, such as an artificial intelligence (AI) engine. In one or more exemplary conference devices, the output processor may be associated with the machine learning engine. Thus, the output processor and / or the machine learning engine may provide a machine learning output. The first image control data may be based at least in part on the machine learning output. The region of interest selection control data may be based at least in part on the machine learning output. The first image control data and / or the region of interest selection control data may be based at least in part on the machine learning output.

[0133] In one or more exemplary conferencing devices, the second image control data and / or the third image control data may be based at least in part on machine learning output.

[0134] Thus, a machine learning engine can be used to determine certain control data in a conferencing device. For example, a machine learning engine can select a person speaking and provide a machine learning output to an output processor indicating the region of the person speaking. The output processor can use the machine learning output to determine region of interest selection control data in order to select and / or combine appropriate secondary video streams. Furthermore, the output processor can use the machine learning output to determine image control data so that the image processor can appropriately focus on the person speaking. The output processor can determine both region of interest selection control data and image control data from the machine learning output. The output processor can determine only region of interest selection control data or only image control data from the machine learning output.

[0135] In one or more exemplary conferencing devices, the machine learning output may include a region of interest selection machine learning output.The output processor may then utilize the region of interest selection machine learning output to determine the region of interest selection control data.

[0136] In one or more exemplary conferencing devices, the machine learning output may include an image control machine learning output. The output processor may then use the image control machine learning output to determine image control selection control data, such as first image control data, second image control data, and / or third image control data.

[0137] In one or more exemplary conferencing devices, the machine learning output may be used to determine region of interest selection control data and image control data, such as first image control data, second image control data, and / or third image control data.

[0138] In one or more exemplary conferencing devices, the conferencing device may include an interface. The interface may communicate with an output interface and / or an output processor. The interface may communicate with the output interface. The interface may communicate with the output processor.

[0139] An interface may include one or more connectors, such as ports, accessories, cables, outputs, inputs, male connectors, and female connectors. The type of connector is not limited. For example, one or more connectors may be configured to receive and / or supply electricity, receive and / or supply electrical signals, receive and / or supply video signals or video streams, receive and / or supply electrical data, receive and / or supply data, and / or receive and / or supply information.

[0140] The one or more connectors may include a first connector. The one or more connectors may include a second connector. The one or more connectors may include a third connector, a fourth connector, a fifth connector, a sixth connector, a seventh connector, an eighth connector, a ninth connector, a tenth connector, and so on. The first connector may be different from the second connector. The first connector may be the same as the second connector.

[0141] Connectors, such as ports, first connectors, and second connectors, can include, for example, a VGA connector, an Ethernet connector, a Thunderbolt connector, a DVI connector, an Ethernet connector, a Coax connector, a composite connector, a component connector, a USB connector, a 3.5mm audio connector, an HDMI connector, a PS / 2 connector, a USB-C connector, a parallel connector, and a DisplayPort connector. Any connector can be an input, an output, or both. Specific connectors, such as plugs, ports, and accessories, are not limited.

[0142] The conferencing device can be configured to provide a primary output video stream via the first connector. The conferencing device can be configured to provide a secondary output video stream via the first connector. The conferencing device can be configured to provide metadata, such as data not included in the video stream, via the first connector. Metadata can include many types of data. For example, metadata can include the number of people in a video stream. Metadata can include the faces of people in a video stream. Metadata can include text found in a video stream. For example, if there is text on any of the video streams, the conferencing device can output the text in the metadata. Thus, the first connector can provide both a field of view and an area of interest.

[0143] In one or more exemplary conference devices, the conference device can be configured to provide a primary output video stream, a secondary output video stream, and a metadata output via a first connector. The conference device can be configured to provide the primary output video stream and the secondary output video stream via a first connector. The first connector can be a single connector. Therefore, the primary output video stream, the secondary output video stream, and the metadata output can all be provided via a single connector. In addition, the primary output video stream and the secondary output video stream can all be provided via a single connector. In an alternative conference device, the primary output video stream can be provided via the first connector, and the secondary output video stream can be provided via the second connector.

[0144] In one or more exemplary conferencing devices, the interface can output the primary output video stream and the secondary output video stream simultaneously (or nearly simultaneously), e.g., simultaneously or concurrently. Alternatively, the output can be non-simultaneous, but the output can be fast enough so that a user would believe that the output of the video streams is simultaneous. The interface can output the primary output video stream and the secondary output video stream interleaved.

[0145] In one or more exemplary conference devices, the components discussed above may be in a single housing, such as a conference device housing. The conference device housing may be a single unit. The conference device housing may be multiple units attached together, such as components.

[0146] For example, the first image sensor and / or the second image sensor and / or the third image sensor may be within or on the conference device housing. The first image processor and / or the second image processor and / or the third image processor may be within the conference device housing. The intermediate image processor may be within the conference device housing. The output processor may be within the conference device housing. The first image sensor, the second image sensor, the third image sensor, the first image processor, the second image processor, the third image processor, the intermediate image processor, and the output processor may be within and / or on the conference device housing. The first image sensor, the second image sensor, the third image sensor, the first image processor, the second image processor, the third image processor, the intermediate image processor, and the output processor may be associated with the conference device housing. The interface and / or one or more connectors may be on and / or within the housing.

[0147] In one or more exemplary conferencing devices, the image sensors (eg, the first image sensor, the second image sensor, and the third image sensor) may be separate from the conferencing device housing.

[0148] The conference device and / or conference device housing may include additional ports, such as plugs, connectors, accessories. For example, the conference device and / or conference device housing may include one or more of a power connector, audio input, audio output, Bluetooth output, wireless output, a wired connector, an Ethernet connector, an auxiliary device connector, and an intermediate device connector. These connectors, such as ports, may include, for example, a VGA connector, an Ethernet connector, a Thunderbolt connector, a DVI connector, an Ethernet connector, a COAX connector, a composite connector, a component connector, a USB connector, a 3.5mm audio connector, an HDMI connector, a PS / 2 connector, a USB-C connector, a parallel connector, and a DisplayPort connector. Any one of the connectors may be input or output or both. Specific ports, such as plugs, connectors, accessories, are not limited. In one or more exemplary conference devices, the conference device may include a connector for attaching to a sound bar.

[0149] Advantageously, one or more exemplary conferencing devices can simplify user experience. For example, the conferencing device can be "plug-and-play." Thus, a user may simply plug the conferencing device into a host computer, such as a host computer, a user machine, a user computer, a host tablet, or a host laptop, to run multiple video streams on the host computer.

[0150] The conferencing device can be used with any video platform, such as a video streaming platform or a terminal platform. For example, the conferencing device can be used with Teams, Zoom, Skype, and others, regardless of the user's specific video platform. Therefore, users do not need expensive and / or high-powered equipment to run multiple streams from the conferencing device.

[0151] Thus, one or more exemplary conferencing devices are modular. Thus, the conferencing device can be incorporated into many different systems and used with many different types of hosts / devices.

[0152] As described above, the conferencing device can accept input from a user, such as commands and user input. The conferencing device can receive user input in any number of ways. For example, a user can operate a user machine, such as a host or main computer, such as a computer, phone, tablet, laptop, etc., to provide user input to the conferencing device. User input can be received wirelessly. User input can be received wirelessly. The user can provide user input using a controller associated with the conferencing device. The conferencing device can provide an interface for the user input and / or output and / or any results of the user input.

[0153] Users can provide user input to the conference device itself. Thus, the conference device may include one or more actuators, such as buttons, knobs, switches, or touch screens, to operate and / or input the conference device. The conference device may also include one or more displays for outputting information to the user. The conference device may also include a keyboard or any other mechanism for the user to input commands into the conference device.

[0154] The user can continuously provide user input throughout the entire use of the conference device. Thus, the user can adjust, for example, the area of interest that the user wishes to view during a video stream, such as a secondary output video stream. The user can make these adjustments in real time.

[0155] Alternatively, the user may be locked out from providing user input, for example, once the conferencing device is in use, such as during a meeting.

[0156] If the user does not provide any additional content, the conference device can include a default, such as standardized, user input. This can be stored in the conference device. For example, the conference device can have a memory. Furthermore, the machine learning engine can use the default user input.

[0157] In one or more exemplary conferencing devices, the conferencing device is configured to receive user input. The first image control data and / or the region of interest selection control data may be based at least in part on the user input. The first image control data may be based at least in part on the user input. The second image control data may be based at least in part on the user input. The third image control data may be based at least in part on the user input. The region of interest selection control data may be based at least in part on the user input.

[0158] Thus, user input can be used to determine certain control data within the conferencing device. For example, user input can indicate that the region of interest is a person speaking. The output processor can use the user input to determine region of interest selection control data to select the appropriate secondary video stream. Furthermore, the output processor can use the user input to determine image control data so that the image processor can appropriately focus on the person speaking. The output processor can determine both the region of interest selection control data and the image control data from the user input. The output processor can also determine only the region of interest selection control data or only the image control data from the user input.

[0159] The output processor may then use the user input to determine the region of interest selection control data. The output processor may then use the user input to determine image control data, such as first image control data, second image control data, and / or third image control data. In one or more exemplary conferencing devices, the user input may be used to determine the region of interest selection control data and the image control data, such as first image control data, second image control data, and / or third image control data.

[0160] Further disclosed herein are methods for achieving high-quality multi-video streaming capabilities.

[0161] In one or more exemplary methods, the method may include acquiring image data. For example, the method may include acquiring first image data, acquiring second image data, and / or acquiring third image data.

[0162] In one or more exemplary methods, the method may include outputting image data. For example, the method may include outputting first image data, outputting second image data, and / or outputting third image data. For example, the number of image data may be based on the number of image sensors used. Thus, the method may include outputting first image data from the first image sensor, outputting second image data from the second image sensor, and / or outputting third image data from the third image sensor.

[0163] Each output data may be output to an image processor. Each output data may be output to a specific image processor. Thus, first image data may be output to a first image processor, second image data may be output to a second image processor, and / or third image data may be output to a third image processor. Furthermore, the first image data may be received by the first image processor, the second image data may be received by the second image processor, and / or the third image data may be received by the third image processor.

[0164] The method may further include segmenting the image data. For example, each image processor may segment its respective image data into two, three, four, five, or six video streams, such as a primary video stream and a secondary video stream. In one example, the method may include segmenting the first image data into a first primary video stream and a secondary primary video stream. If more image data is used, the method may include segmenting the second image data into a second primary video stream and a secondary secondary video stream, and / or the method may include segmenting the third image data into a third primary video stream and a third tertiary video stream.

[0165] The primary video stream and the secondary video stream may be output to the intermediate image processor. Thus, the image processors may output their respective primary video streams and secondary video streams to the intermediate image processor. A first primary video stream and a secondary video stream may be output to the intermediate image processor. A second primary video stream and a secondary video stream may be output to the intermediate image processor. A third primary video stream and a secondary video stream may be output to the intermediate image processor. The first primary video stream and the secondary video stream may be received by the intermediate image processor. The second primary video stream and the secondary video stream may be received by the intermediate image processor. The third primary video stream and the secondary video stream may be received by the intermediate image processor.

[0166] Next, the method may include performing signal and / or image processing on the primary video stream and the secondary video stream. In one or more exemplary methods, the method may include splicing (e.g., combining) all of the primary video streams together. Thus, the first primary video stream and / or the second primary video stream and / or the third primary video stream may be combined or spliced together. The first primary video stream and / or the second primary video stream and / or the third primary video stream may be combined or spliced together to provide a field of view video stream. The method may further include classification processing, such as noise reduction or quality control of the primary video stream.

[0167] In one or more exemplary methods, the method may include splicing (e.g., combining) all of the secondary video streams together. The first video stream and / or the second video stream and / or the third video stream may be combined or spliced together to provide the region of interest video stream. Alternatively, the method may include selecting one of the secondary video streams. The method may include selecting one of the first video stream, the second video stream, and the third video stream to provide the region of interest video stream. The method may also include performing processing, such as noise reduction, on the secondary video streams.

[0168] The method may include outputting a field of view video stream based on the primary video stream. The field of view video stream may be based on all primary video streams.

[0169] The method may further include outputting an area of interest video stream based on the one or more secondary video streams.

[0170] The method may include outputting the field of view video stream and the region of interest video stream to an output processor. The method may include receiving the field of view video stream and the region of interest video stream by the output processor.

[0171] The method may then include image and / or signal processing of the field of view video stream and / or the area of interest video stream. In one or more exemplary methods, the method may include outputting a primary output video stream based on the field of view video stream and / or outputting a secondary output video stream based on the area of interest video stream. The method may further output metadata. The primary output video stream and the secondary output video stream may be output to a user device, such as a user machine. The primary output video stream and the secondary output video stream may be output via one or more connectors, such as a first connector.

[0172] In one or more exemplary methods, the method may further include determining image control data. The image control data may be based on the field of view video stream. The image control data may be determined in an output processor. The image control data may be output and / or sent to one of the image processors. The image control data may be output and / or sent via the output processor. The method may include determining first image control data, second image control data, and third image control data. The method may include outputting and / or sending the first image control data to the first image processor. The method may include outputting and / or sending the second image control data to the second image processor. The method may include outputting and / or sending the third image control data to the third image processor. The method may include receiving the first image control data by the first image processor. The method may include receiving the second image control data by the second image processor. The method may include receiving the third image control data by the third image processor.

[0173] In one or more exemplary methods, the method may further include determining region of interest selection control data. The method may include outputting the region of interest selection control data. The output may be performed via an output processor. For example, the region of interest selection control data may be output to an intermediate image processor. The region of interest selection control data may be received by the intermediate image processor. The region of interest selection control data may be based on a field of view video stream. Thus, the method may include determining and / or selecting a region of interest video stream based on the region of interest selection control data.

[0174] The method may further comprise receiving a user input.Any of the region of interest selection control data and / or the image control data, such as the first image control data, the second image control data and / or the third image control data may be based on the user input.

[0175] The method may further include outputting the machine learning output, such as from a machine learning engine.Any of the region of interest selection control data and / or the image control data, such as the first image control data, the second image control data, and / or the third image control data may be based on the machine learning output.

[0176] The various exemplary methods, devices, agents, and systems described herein are described in the general context of method steps and procedures. In one aspect, these method steps and procedures can be implemented by a computer program product contained in a computer-readable medium, which includes computer-executable instructions (e.g., program code) executed by a computer in a network environment. Computer-readable media can include removable and non-removable storage devices, including but not limited to read-only memory (ROM), random access memory (RAM), compact disks (CDs), digital versatile disks (DVDs), etc. In general, program modules can include routines, programs, objects, components, data structures, etc. that perform specified tasks or implement specific abstract data types. Computer-executable instructions, associated data structures, and program modules represent examples of program code for performing the steps of the methods disclosed herein. A specific sequence of such executable instructions or associated data structures represents an example of corresponding actions for implementing the functions described in such steps or procedures.

[0177] Figure 1 An example conferencing device of the present disclosure is shown. Figure 2 Provides a more detailed view of the example conferencing equipment.

[0178] As shown, the conferencing device 100 may include a first image sensor 102 for providing first image data 103. The conferencing device 100 may optionally have other sensors. The conferencing device 100 may include a second image sensor 104 for providing second image data 105. The conferencing device 100 may include a third image sensor 106 for providing third image data 107. The following will discuss a video conferencing device 100 having three image sensors, although any one of the three image sensors is optional.

[0179] The conference device 100 may further include a first image processor 110 for providing a first main video stream 202 and a first secondary video stream 204 based on the first image data 103. The conference device 100 may optionally include other image processors. For example, Figure 1 Three image processors are shown, each connected to an image sensor. A second image processor 112 can be used to provide a second primary video stream 206 and a secondary video stream 208 based on second image data 105. A third image processor 114 can be used to provide a third primary video stream 210 and a third video stream 212 based on third image data 107. Thus, each image processor communicates with one of the image sensors. There can be a 1:1 relationship between image sensors and image processors.

[0180] As shown, the conferencing device 100 may include an intermediate image processor 116. The intermediate image processor 116 may communicate with any image processor. Thus, the intermediate image processor 116 may communicate with the first image processor 110, the second image processor 112, and the third image processor 114. The intermediate image processor 116 may be configured to provide a field of view video stream 214 and an area of interest video stream 216. The area of interest video stream 216 may be based on one or more of the first video stream 204 and the second video stream 208. It may be based on any of the disclosed video streams.

[0181] When more image sensors are used, the field of view video stream 214 is based on the first primary video stream 202, the second primary video stream 206, and the third primary video stream 210. The area of interest video stream 216 is based on one or more of the first primary video stream 204, the second secondary video stream 208, and the third secondary video stream 212. The field of view video stream 214 can be spliced from the first primary video stream 202, the second primary video stream 206, and the third primary video stream 210, for example Figure 3 304 in , which provides an illustration of the components in the conferencing device 100 before the output processor 118.

[0182] Regarding the area of interest video stream 216, the intermediate image processor 116 may be configured to select one of the first video stream 204, the second video stream 208, and the third video stream 212 for the area of interest video stream 216. Alternatively, the area of interest 216 video streams may be combined and / or spliced from the first video stream 204, the second video stream 208, and the third video stream 212 for the area of interest video stream 216.

[0183] Advantageously, the first primary video stream 202 is independent of the first primary video stream 204. Therefore, the first primary video stream 204 is not a subset of the first primary video stream 202. Advantageously, the second primary video stream 206 is independent of the second secondary video stream 208. Therefore, the second secondary video stream 208 is not a subset of the second primary video stream 206. Advantageously, the third primary video stream 210 is independent of the third secondary video stream 212. Therefore, the third secondary video stream 212 is not a subset of the third primary video stream 210.

[0184] The conferencing device 100 may further include an output processor 118 in communication with the intermediate image processor 116. The output processor 118 may be configured to determine the first image control data 122 based on the field of view video stream 214 and send the first image control data 122 to the first image processor 110, for example.

[0185] Furthermore, the output processor 118 may be configured to determine region of interest selection control data 128 and send the region of interest selection control data 128 to the intermediate image processor 116 .

[0186] If more than one image sensor is used, the output processor 118 may provide additional image control data. For example, the output processor 118 may be configured to determine second image control data 124 based on the field of view video stream 214 and send the second image control data 124 to the second image processor 112. The second image processor 112 may be configured to provide a second video stream 208 based on the second image control data 124. Furthermore, the output processor 118 may be configured to determine third image control data 126 based on the field of view video stream 214 and send the third image control data 126 to the third image processor 114. The third image processor 114 may be configured to provide a third video stream 212 based on the third image control data 126.

[0187] Therefore, the first control data 122 , the second control data 124 , and / or the third control data 126 may be used to appropriately determine and / or prepare the first video stream 204 , the second video stream 208 , and the third video stream 212 .

[0188] The output processor 118 may include an output interface 224 configured to provide a primary output video stream 220 based on the field of view video stream 214 and a secondary output video stream 222 based on the area of interest video stream 216. The output processor 118 may also be configured to provide the primary output video stream 220 based on the field of view video stream 214 and / or the area of interest video stream 216. This may be provided to an external device 120, such as a user device.

[0189] In addition, if Figure 2 As shown, the output processor 118 is configured to perform one or more of image conversion 226 , distortion correction 228 , enhancement 230 , and contrast correction 132 on the area of interest video stream 216 to provide a secondary output video stream 222 .

[0190] Furthermore, the conferencing device 100 may include an interface 234 comprised of one or more connectors including a first connector 236. The conferencing device 100 may be configured to provide a primary output video stream 220, a secondary output video stream 222, and metadata 238 output via the first connector 236.

[0191] The field of view of the primary output video stream 220 may be at least 120°. The field of view of the secondary output video stream 222 may be less than 90°. Of course, other fields of view may also be used.

[0192] In addition, if Figure 2As shown, the output processor 118 may include a machine learning engine 218 for providing a machine learning output 242, wherein the first image control data 122 and / or the region of interest selection control data 128 are based at least in part on the machine learning output 242. The machine learning engine 218 may receive the field of view video stream 214 and / or the region of interest video stream 216 and be configured to determine the machine learning output 242 based on the field of view video stream 214 and / or the region of interest video stream 216. The second image control data 124 and / or the third image control data 126 may be based at least in part on the machine learning output 242. Furthermore, the conferencing device 100 is configured to receive a user input 240, wherein the first image control data 122 and / or the region of interest selection control data 128 are based at least in part on the user input 240.

[0193] Figure 3 3. Further details of the intermediate image processor 116 are shown. As shown, the intermediate image processor 116 can perform geometric correction 302, video stitching 304, noise reduction 306, and chroma subsampling 308 on the first primary video stream 202, the second primary video stream 206, and the third primary video stream 210. In addition, the intermediate image processor 116 can perform noise reduction 306 and chroma subsampling 308 on the first primary video stream 204, the second secondary video stream 208, and the third primary video stream 212.

[0194] It should be understood that not all connections are shown in the above figures, and the present disclosure should not be limited to the specific connections shown in the above figures.

[0195] An example of a conference device according to the present disclosure is stated as follows:

[0196] Item A1 - Conference equipment, including:

[0197] a first image sensor, configured to provide first image data;

[0198] a second image sensor, configured to provide second image data;

[0199] a first image processor configured to provide a first main video stream and a first secondary video stream based on the first image data;

[0200] a second image processor configured to provide a second primary video stream and a secondary video stream based on the second image data; and

[0201] An intermediate image processor communicates with the first image processor and the second image processor and is configured to provide a field of view video stream and an area of interest video stream, wherein the field of view video stream is based on the first primary video stream and the second primary video stream, and wherein the area of interest video stream is based on one or more of the first video stream and the second video stream.

[0202] Item A2 - A conferencing device according to Item A1, the conferencing device including a third image sensor for providing third image data, and a third image processor configured to provide a third main video stream and a third video stream based on the third image data, wherein the intermediate image processor communicates with the third image processor, wherein the field of view video stream is based on the first main video stream, the second main video stream and the third main video stream, and wherein the area of interest video stream is based on one or more of the first video stream, the second video stream and the third video stream.

[0203] Item A3 - A conference device according to Item A2, wherein the field of view video stream is spliced from the first main video stream, the second main video stream and the third main video stream.

[0204] Item A4 - The conferencing device of any one of Items A2 and A3, wherein the intermediate image processor is configured to select one of the first video stream, the second video stream, and the third video stream for the area of interest video stream.

[0205] Item A5 - A conferencing device according to any one of Items A2 and A3, wherein the area of interest video stream is spliced from the first video stream, the second video stream and the third video stream.

[0206] Item A6 - A conferencing device according to any one of items A1 to A5, wherein the first main video stream is independent of the first secondary video stream.

[0207] Item A7 - A conferencing device according to any one of items A1 to A6, the conferencing device including an output processor that communicates with the intermediate image processor and is configured to determine first image control data based on the field of view video stream and send the first image control data to the first image processor.

[0208] Item A8 - The conferencing device of Item A7, wherein the output processor is configured to determine region of interest selection control data and to send the region of interest selection control data to the intermediate image processor.

[0209] Item A9 - A conferencing device according to Item A7 or Item A8, wherein the output processor includes an output interface, and the output interface is configured to provide a primary output video stream based on the field of view video stream and a secondary output video stream based on the area of interest video stream.

[0210] Item A10 - The conferencing device of Item A9, wherein the output processor is configured to perform one or more of image conversion, distortion correction, enhancement, and contrast correction on the region of interest video stream to provide a secondary output video stream.

[0211] Item A11 - A conferencing device according to Item A9 or Item A10, wherein the conferencing device includes an interface, the interface includes one or more connectors, the connectors include a first connector, and the conferencing device is configured to provide a primary output video stream, a secondary output video stream, and metadata output via the first connector.

[0212] Item A12 - The conferencing device of any of Items A7 to A11, wherein the conferencing device is configured to receive user input, and wherein the first image control data and / or the region of interest selection control data are based at least in part on the user input.

[0213] Item A13 - A conferencing device according to any of items A7 to A12, wherein the output processor includes a machine learning engine for providing machine learning output, and wherein the first image control data and / or the region of interest selection control data is at least partially based on the machine learning output.

[0214] Item A14 - A conferencing device according to any of Items A1 to A12, wherein the first video stream is not a subset of the first main video stream.

[0215] Item B1 - Conference equipment, including:

[0216] a first image sensor, configured to provide first image data;

[0217] a first image processor configured to provide a first main video stream and a first secondary video stream based on the first image data;

[0218] an intermediate image processor in communication with the first image processor and configured to provide a field of view video stream and an area of interest video stream, wherein the field of view video stream is based on the first primary video stream and the area of interest video stream is based on the first secondary video stream; and

[0219] an output processor configured to determine first image control data based on the field of view video stream and send the first image control data to the first image processor;

[0220] The first image processor is configured to provide a first video stream based on the first image control data.

[0221] Item B2—The conferencing device of Item B1, wherein the output processor is configured to provide a primary output video stream based on the field of view video stream or the area of interest video stream.

[0222] Item B3 - A conferencing device according to any of items B1 to B2, wherein the output processor is configured to process the area of interest video stream to provide a secondary output video stream, wherein processing the area of interest video stream includes performing one or more of image conversion, distortion correction, enhancement, and contrast correction.

[0223] Item B4 - A conferencing device according to any one of items B1 to B3, wherein the conferencing device includes a second image sensor for providing second image data, and a second image processor configured to provide a second main video stream and a second secondary video stream based on the second image data, wherein the intermediate image processor communicates with the second image processor and is configured to provide a field of view video stream and an area of interest video stream, wherein the field of view video stream is based on the first main video stream and the second main video stream, and the area of interest video stream is based on one or more of the first video stream and the second video stream.

[0224] Item B5 - A conferencing device according to any one of items B1 to B4, wherein the conferencing device includes a third image sensor for providing third image data, and a third image processor configured to provide a third main video stream and a third video stream based on the third image data, wherein the intermediate image processor communicates with the third image processor and is configured to provide a field of view video stream and an area of interest video stream, wherein the field of view video stream is based on the first main video stream, the second main video stream and the third main video stream, and the area of interest video stream is based on one or more of the first video stream, the second video stream and the third video stream.

[0225] Item B6 - A conferencing device according to Item B5, wherein the intermediate image processor is configured to stitch together the first primary video stream, the second primary video stream and the third primary video stream to provide a field of view video stream.

[0226] Item B7 - A conferencing device according to any one of items B3 to B6, wherein the conferencing device includes an interface, the interface includes one or more connectors, the connectors include a first connector, and the conferencing device is configured to provide a primary output video stream, a secondary output video stream, and metadata output via the first connector.

[0227] Item B8 - The conferencing device of any of items B3 to B7, wherein the output processor is configured to determine region of interest selection control data and to send the region of interest selection control data to the intermediate image processor.

[0228] Item B9—The conferencing device of item B8, wherein the conferencing device is configured to receive user input, and wherein the first image control data and / or the region of interest selection control data are based at least in part on the user input.

[0229] Item B10 - A conferencing device according to any of items B1 to B9, wherein the output processor includes a machine learning engine for providing a machine learning output, and wherein the first image control data and / or the region of interest selection control data is at least partially based on the machine learning output.

[0230] Item B11 - A conferencing device according to any one of items B4 to B10, wherein the output processor is configured to determine second image control data based on the field of view video stream and send the second image control data to a second image processor, wherein the second image processor is configured to provide a second video stream based on the second image control data.

[0231] Item B12 - A conferencing device according to any one of items B4 to B11, wherein the output processor is configured to determine third image control data based on the field of view video stream and send the third image control data to a third image processor, wherein the third image processor is configured to provide a third video stream based on the third image control data.

[0232] Item B13 - A conferencing device according to any of items B3 to B12, wherein the field of view of the primary output video stream is at least 120° and the field of view of the secondary output video stream is less than 90°.

[0233] Item B14—A conferencing device according to any one of items B1 to B13, wherein the first secondary video stream is not a subset of the first main video stream.

[0234] Item B15 - A conferencing device according to any one of items B1 to B14, wherein the first main video stream is independent of the first secondary video stream.

[0235] The use of the terms "first," "second," "third," and "fourth," "primary," "secondary," and "tertiary," etc., does not imply any particular order, but rather is used to identify individual elements. Furthermore, the use of the terms "first," "second," "third," and "fourth," "primary," "secondary," and "tertiary," etc., does not imply any order or importance, but rather the terms "first," "second," "third," and "fourth," "primary," "secondary," and "tertiary," etc., are used to distinguish one element from another. Note that the terms "first," "second," "third," and "fourth," "primary," "secondary," and "tertiary," etc., here and elsewhere, are used for labeling purposes only and are not intended to imply any particular spatial or temporal order.

[0236] Furthermore, the presence of a first element does not imply the presence of a second element, and vice versa.

[0237] I understand. Figures 1 to 3Included are some modules or operations shown with solid lines and some modules or operations shown with dotted lines. The modules or operations included in the solid lines are the modules or operations included in the most extensive example embodiment. The modules or operations included in the dotted lines are example embodiments that can be included in the solid line example embodiment or a portion thereof, or are other modules or operations that can be taken in addition to the modules or operations of the solid line example embodiment. It should be understood that these operations do not need to be performed in the order presented. In addition, it should be understood that not all operations need to be performed. The exemplary operations can be performed in any order and in any combination.

[0238] It should be noted that the word "comprising" does not necessarily exclude the presence of other elements or steps than those listed.

[0239] It should be noted that the word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements.

[0240] It should also be noted that any reference signs do not limit the scope of the claims, that exemplary embodiments may be implemented at least partially by means of hardware and software, and that several "means," "units," or "devices" may be represented by the same item of hardware.

[0241] The various exemplary methods, devices, and systems described herein are described in the general context of method steps and procedures. In one aspect, these method steps and procedures can be implemented by a computer program product contained in a computer-readable medium, which includes computer-executable instructions (e.g., program code) executed by a computer in a network environment. Computer-readable media can include removable and non-removable storage devices, including but not limited to read-only memory (ROM), random access memory (RAM), compact disks (CDs), digital versatile disks (DVDs), etc. In general, program modules can include routines, programs, objects, components, data structures, etc. that perform specified tasks or implement specific abstract data types. Computer-executable instructions, associated data structures, and program modules represent examples of program code for performing the steps of the methods disclosed herein. A specific sequence of such executable instructions or associated data structures represents an example of corresponding actions for implementing the functions described in such steps or procedures.

[0242] While features have been shown and described, it should be understood that they are not intended to limit the claimed invention, and that it will be apparent to those skilled in the art that various changes and modifications may be made without departing from the spirit and scope of the claimed invention. Accordingly, the specification and drawings are to be regarded as illustrative rather than restrictive. The claimed invention is intended to encompass all alternatives, modifications, and equivalents.

[0243] Reference List

[0244] 100 conference equipment

[0245] 102 First Image Sensor

[0246] 103 First image data

[0247] 104 second image sensor

[0248] 105 Second image data

[0249] 106 third image sensor

[0250] 107 Third image data

[0251] 110 First Image Processor

[0252] 112 Second image processor

[0253] 114 Third Image Processor

[0254] 116 Intermediate Image Processor

[0255] 118 Output Processor

[0256] 120 external devices

[0257] 122 First image control data

[0258] 124 Second image control data

[0259] 126 Third image control data

[0260] 128 Focus Area Selection Control Data

[0261] 202 First main video stream

[0262] 204 First video stream

[0263] 206 Second main video stream

[0264] 208 Second video stream

[0265] 210 Third main video stream

[0266] 212 Third video stream

[0267] 214 field of view video stream

[0268] 216 Focus Area Video Stream

[0269] 218 Machine Learning Engine

[0270] 220 Main output video stream

[0271] 222 output video streams

[0272] 224 output interface

[0273] 226 Image Conversion

[0274] 228 Distortion Correction

[0275] 230 Enhancement

[0276] 232 Contrast Correction

[0277] 234 interface

[0278] 236 First Connector

[0279] 238 Metadata

[0280] 240 User Input

[0281] 242 Machine Learning Output

[0282] 302 Geometric Correction

[0283] 304 Video Splicing

[0284] 306 Noise Reduction

[0285] 308 chroma subsampling.

Claims

1. A conference device comprising: a first image sensor, configured to provide first image data; a first image processor configured to provide a first main video stream and a first secondary video stream based on the first image data; The conference device further includes a second image sensor for providing second image data, and a second image processor configured to provide a second main video stream and a secondary video stream based on the second image data, and a third image sensor for providing third image data, and a third image processor configured to provide a third main video stream and a third video stream based on the third image data. an intermediate image processor in communication with the first image processor and configured to provide a field of view video stream and an area of interest video stream, wherein the intermediate image processor is configured to stitch together the first main video stream, the second main video stream, and the third main video stream to provide the field of view video stream; and an output processor configured to determine first image control data based on the field of view video stream and send the first image control data to the first image processor; The first image processor is configured to provide the first video stream based on the first image control data. And wherein the output processor is configured to determine region of interest selection control data based on the field of view video stream, and send the region of interest selection control data to the intermediate image processor to select at least one of the first video stream, the second video stream, and the third video stream, wherein the region of interest video stream is based on at least one selected one of the first video stream, the second video stream, and the third video stream, and wherein the output processor uses machine learning output to determine the region of interest selection control data so as to select and / or combine appropriate secondary video streams.

2. The conference device according to claim 1, wherein: The output processor is configured to provide a main output video stream based on the field of view video stream or the area of interest video stream.

3. The conference device according to claim 2, wherein: The output processor is configured to process the area of interest video stream to provide a secondary output video stream, wherein processing the area of interest video stream includes performing one or more of image conversion, distortion correction, enhancement, and contrast correction.

4. The conference device according to claim 1, wherein: The intermediate image processor is in communication with the third image processor, and the intermediate image processor is configured to provide the field of view video stream and the region of interest video stream.

5. The conference device according to claim 1, wherein: The conferencing device includes an interface including one or more connectors, the one or more connectors including a first connector, the conferencing device being configured to provide a primary output video stream, a secondary output video stream, and metadata output via the first connector.

6. The conference device according to claim 1, wherein: The conferencing device is configured to receive user input, and wherein the first image control data and / or the region of interest selection control data are at least partially based on the user input.

7. The conference device according to claim 6, wherein: The output processor includes a machine learning engine for providing a machine learning output, and wherein the first image control data is based at least in part on the machine learning output.

8. The conference device according to claim 7, wherein: The output processor is configured to determine second image control data based on the field of view video stream and send the second image control data to the second image processor, wherein the second image processor is configured to provide the second secondary video stream based on the second image control data.

9. The conference device according to claim 8, wherein: The output processor is configured to determine third image control data based on the field of view video stream and send the third image control data to the third image processor, wherein the third image processor is configured to provide a third video stream based on the third image control data.

10. The conference device according to claim 9, wherein: The field of view of the primary output video stream is at least 120°, and the field of view of the secondary output video stream is less than 90°.

11. The conference device according to claim 10, wherein: The first secondary video stream is not a subset of the first primary video stream.

12. The conference device according to claim 11, wherein: The first main video stream is independent of the first secondary video stream.

Images