A system and method for sharing media resources for network-based communications

Abstract

The present disclosure relates to a system and method for facilitating sharing of media resources of a computing device with a data device integrated with the system to enable network-based communication with a remote device. In an example implementation, the proposed system receives device information related to available computing devices through a discovery manager. The system selects a computing device from the available computing devices. The system is capable of initiating communication between the data device and the remote device. Upon initiating the communication, the system receives communication data from the computing device. The system transmits the received communication data to the remote device to facilitate network-based communication with the remote device.

Description

Technical Field

[0001] This invention generally relates to data devices such as set-top boxes (STBs), and more particularly to sharing media resources of external computing devices with data devices used for network-based communication. Background Technology

[0002] The following description of related technologies is intended to provide background information relevant to the field of this disclosure. This section may include certain aspects of the prior art that may be related to various features of this disclosure. However, it should be understood that this section is intended only to enhance the reader's understanding of this disclosure and not to acknowledge prior art.

[0003] Data devices such as set-top boxes (STBs) have evolved in functionality, enabling users to access entertainment media, such as video and audio signals. STBs typically connect to external display devices to provide this access, but these external display devices may not have built-in cameras and microphones. While STBs may have built-in communication applications, without image capture devices like cameras and microphones, they cannot send audio or video signals to remote terminal devices when using these applications.

[0004] Therefore, users may need external cameras and microphones that can be directly connected to STB devices, increasing the overall cost. Furthermore, there are currently no cameras and microphones compatible with all types of STBs. Due to the limited compatibility of cameras and microphones, the use of communication applications in set-top box devices is restricted.

[0005] Therefore, there is a need in the art for an effective and economical system and method that can overcome the aforementioned problems in the prior art and can effectively enable network-based communication via data devices without the need for built-in resources by facilitating the sharing of media resources. Summary of the Invention

[0006] Purpose

[0007] Some of the objectives of this disclosure are satisfied by at least one embodiment of the present invention, as listed below.

[0008] One object of this disclosure is to implement a method and system that can share resources for communication, such as cameras and microphones typically available on devices like mobile phones or laptops.

[0009] One objective of this disclosure is to eliminate the need for external compatible devices such as cameras and microphones, which also avoids compatibility issues with external devices used for communication.

[0010] One object of this disclosure is to enable data devices such as STBs to transmit media data over the Internet during communications.

[0011] One purpose of this disclosure is to aggregate all local streams that can be used for a stream such as a video conference and to forward the aggregated stream to another party.

[0012] One object of this disclosure is to allow aggregation of individual Internet of Things (IoT) security camera streams available to a user when all security cameras are connected to the same network, from which the user can communicate with the STB, thus eliminating the need for an IoT server.

[0013] One purpose of this disclosure is to enable its use in applications such as those for sharing video and audio data, video editor applications, augmented reality applications, etc.

[0014] One purpose of this disclosure is to provide economical methods and systems.

[0015] Overview

[0016] This section is provided to introduce certain objects and aspects of the invention in a simplified form, wherein further details of these objects and aspects are set forth in the detailed description. This summary is not intended to identify key features or scope of the claimed subject matter.

[0017] On one hand, the proposed system is configured to facilitate the sharing of media resources of computing devices with data devices. The data devices are integrated with the system to enable network-based communication with remote devices. The system is configured with a data device and includes one or more processors coupled to memory, where the memory stores instructions. When executed by the one or more processors, the system receives device information related to available computing devices from a discovery manager coupled to the system; selects a computing device from the available computing devices; initiates communication between the data device and the remote device; and, upon initiating communication, receives communication data from the selected computing device through at least one port and transmits the received communication data to the remote device through the data device's communication module, facilitating network-based communication with the remote device.

[0018] This disclosure also provides a method for facilitating the sharing of media resources of a computing device with a data device to enable network-based communication with a remote device, wherein the method includes the following steps: receiving device information related to available computing devices from a discovery manager through a system configured with the data device; selecting a computing device from the available computing devices; initiating communication between the data device and the remote device; and, when initiating communication, receiving communication data from the selected computing device through at least one port, and sending the received communication data to the remote device through the communication module of the data device, thereby facilitating network-based communication with the remote device. Attached Figure Description

[0019] The accompanying drawings, which are incorporated herein and constitute a part of this invention, illustrate exemplary embodiments of the disclosed methods and systems, wherein similar reference numerals refer to the same components in different figures. Elements in the drawings are not necessarily drawn to scale, but are emphasized to clearly illustrate the principles of the invention. Some figures may use block diagrams to indicate elements and may not represent the internal circuitry of each component. Those skilled in the art will understand that the inventions depicted in these drawings include inventions of electrical elements, electronic elements, or circuits typically used to implement these elements.

[0020] Figure 1 An exemplary network architecture according to an embodiment of the present disclosure is shown, in which or through which the system of the present disclosure may be implemented.

[0021] Figure 2 Exemplary functional components of a system proposed according to an embodiment of the present disclosure are shown.

[0022] Figure 3A and 3B Exemplary representations of flowcharts relating to a system, a discovery manager, a computing device, and a communication module for outgoing and incoming communications, according to one embodiment of the present disclosure, are shown.

[0023] Figure 4 An exemplary representation of a flowchart according to an embodiment of the present disclosure is shown for facilitating the sharing of media resources of a computing device with a data device to enable network-based communication with a remote device.

[0024] Figure 5A An exemplary representation of a flowchart for registering a computing device over a network via a discovery manager, according to an embodiment of the present disclosure, is shown.

[0025] Figure 5B An exemplary representation of a flowchart for discovering available computing devices over a network via a discovery manager, according to an embodiment of the present disclosure, is shown.

[0026] Figure 6A and 6B An exemplary representation of a proposed system for incoming and outgoing telephone calls according to an embodiment of the present disclosure is shown.

[0027] Figure 7 An exemplary computer system according to an embodiment of the present disclosure is shown, in which embodiments of the present invention can be used or can be used in conjunction with the computer system.

[0028] The above will become more apparent from the following more detailed description of the invention. Detailed Implementation

[0029] In the following description, various specific details are set forth for purposes of explanation in order to provide a thorough understanding of embodiments of this disclosure. However, it will be apparent that embodiments of this disclosure can be practiced without these specific details. Several features described below may be used independently or in any combination with other features. A single feature may not solve all the problems discussed above, or may only solve some of the problems discussed above. Some of the problems discussed above may not be fully solved by any of the features described herein.

[0030] The following description provides exemplary embodiments only and is not intended to limit the scope, applicability, or configuration of this disclosure. Rather, the subsequent description of exemplary embodiments will provide those skilled in the art with enabling descriptions for implementing the exemplary embodiments. It should be understood that various changes can be made to the function and arrangement of the elements without departing from the spirit and scope of the invention as set forth.

[0031] Specific details are set forth in the following description to provide a thorough understanding of the embodiments. However, those skilled in the art will understand that the embodiments may be practiced without these specific details. For example, circuits, systems, networks, processes, and other components may be shown as components in the form of block diagrams to avoid obscuring the embodiments with unnecessary details. In other instances, well-known circuits, processes, algorithms, structures, and techniques may be shown without unnecessary details to avoid obscuring the embodiments.

[0032] This invention addresses the aforementioned problems in the art by providing a system and method for sharing media resources with a data device, such as a set-top box (STB), for applications involving communication, such as audio and video. Specifically, the system and method establish communication between the data device and an available computing device associated with a first user. During communication between the first and second users, data acquired through shared media resources, such as image capture and recording devices, is transmitted from the computing device to the data device (e.g., the STB), which further transmits the data to the second user, who has a remote device at a remote location. In this way, the system utilizes the resources of the first user's available computing device to enable network-based communication with the second user via the data device. In other words, the system and method provide a technical solution for utilizing publicly available resources such as cameras and microphones from computing devices, such as handheld mobile phones, laptops, and other such devices. Therefore, the system facilitates efficient utilization of resources in network communication without requiring the data device to have built-in resources. The system and method also facilitate dynamically discovering available computing devices from a plurality of registered computing devices, allowing for real-time selection of the most suitable computing device. Several other advantages are also possible.

[0033] Figure 1 An exemplary network architecture 100 according to an embodiment of the present disclosure is illustrated, in which or through which the system 110 of the present disclosure may be implemented. As shown, the system 110 may be configured with a data device 101. The system 110 enables communication between the data device 101 and one or more computing devices 102-1, 102-2…102-N (each individually referred to as computing device 102, and collectively as computing device 102) for sharing media resources of the computing device 102 with the data device 101 during communication with a remote device. The data device 101 may be an information device without any integrated / built-in media resources, such as a camera, microphone, or other accessory-based devices. In an exemplary embodiment, the data device 101 may be a set-top box (STB) device with communication features but without media resources integrated with the STB. The computing device 102 may be associated with a first user and may include electronic devices integrated with media resources. In an exemplary embodiment, the computing device 102 may include a mobile device, such as a tablet computer, laptop computer, Internet Protocol (IP) camera, mobile phone, or other such device. According to one embodiment and as follows Figure 1As shown, this architecture enables efficient communication between computing device 102 and data device 101 during communication between a first user associated with data device 101 and a second user located remotely. This is achieved by utilizing the resources of image capture devices such as cameras and recording devices such as microphones to transmit data to the second user. Media resources include accessory devices integrated with or coupled to the computing device, capable of capturing at least one of raw audio and raw video, or obtaining at least one of encoded audio data and encoded video data. For example, a media resource could be a built-in camera and / or microphone of a mobile phone or laptop. In another example, a media resource could be an external camera or IP camera that can be coupled to the computing device or data device.

[0034] In an example embodiment, communication may include at least one of audio calls / talks, video calls / talks, augmented virtual reality-based communication, and combinations thereof, such as a combination of audio and video calls. In one embodiment, data device 101 may have the capability to establish communication with a remote user via a remote device 107 associated with the second user. Communication module 105 may communicate with the remote device 107 via router 108 and base station 109. Because data device 101 may not have internal media resources, such as a built-in camera and microphone, it may not be able to directly provide data such as audio and / or video data to remote device 107. However, system 110 can share the media resources of computing device 102 with data device 101 to facilitate communication (including audio and video communication) between the first and second users. In other words, the proposed system 110 utilizes these readily available devices to perform audio, video communication, and any additional communication services. To share the media resources of computing device 102 with data device 101, data device 101 needs to be connected to computing device 102. Computing device 102, which is ready to share media resources, can register as a service provider. To achieve this, multiple registered computing devices can be registered with a network service. In an exemplary embodiment, each computing device 102 can register with the network service through a discovery manager 120. The system can be coupled to the discovery manager 120, which is capable of dynamically discovering available computing devices 102 from among the multiple registered computing devices.

[0035] In an exemplary embodiment, this dynamic discovery can be performed by assessing whether an available computing device is near system 110 or whether it is within reach of a network or, for example, a wireless network, based on a predefined threshold distance. The predefined threshold distance can be the physical distance of the computing device from the system, which allows for the detection / discovery of computing device 102. For example, one or more mobile phones of a first user may be nearby, for example, at a distance of 1 meter from the system and detectable by the discovery manager 120. Alternatively, one or more mobile phones of the first user may be nearby, for example, within the same Wi-Fi range or in the same network or reachability. It is understood that the aforementioned predefined threshold distance is merely exemplary and can vary in other scenarios. In an alternative embodiment, data device 101 / discovery manager 120 may discover computing devices via a network service module, such as a Network Service Discovery (NSD) module. In an example embodiment, the NSD module may allow the system to learn about available supporting devices within the nearby / desired network. For example, this could be within the same Wi-Fi range, or in other cases, this could be determined by explicitly configuring them to identify available external resource devices. During network service discovery, data devices 101 such as STBs can dynamically discover computing devices 102 through discovery manager 120, and select one of the available computing devices for effective network-based communication during outgoing and incoming communications. It is understood that network-based communication is not limited to call services and can include any communication service, such as screen sharing, video / audio sharing, whiteboard presentations, and other such features.

[0036] In one embodiment, system 110 may receive device information from discovery manager 120. The device information may relate to an available computing device. In an exemplary embodiment, the device information may be in the form of a collective list including identifier information related to an available computing device. For example, the identifier information may include at least one of port information and Internet Protocol (IP) address related to the available computing device 102. In an example embodiment, at least one port may include at least one of a predefined port or a dynamic port. The port information may relate to at least one port that enables communication between computing device 102 and data device 101.

[0037] In an example embodiment, the discovery manager 120 can receive identifier information through an NSD module associated with a network service. For example, system 110 configured in data device 101 can request the NSD module to provide IP address and port information of available devices. Based on the available devices in the set list, system 110 can allow selection of a computing device from the available computing devices. In an exemplary embodiment, computing device 102 can be selected from the available computing devices based on user input from a first user. In one embodiment, when computing device 102 is selected, system 110 can prepare to initiate communication between data device 101 and remote device 107. Since the communication can be outgoing or incoming, system 110 can trigger or receive communication, respectively.

[0038] In one embodiment, before initiating communication, system 110 may request computing device 102 to confirm the possibility of initiating communication. In an exemplary embodiment, this request may be sent and acknowledged via at least one port by exchanging signaling commands between data device 101 and the selected computing device 102. The exchange of signaling commands may occur via network 106. In one embodiment, upon receiving acknowledgment from computing device 102, the system may facilitate the transmission of parameters and encoding information between data device 101 and computing device 102. In one embodiment, the transmission of parameters may include negotiation that may occur when computing device 102 is discovered and ultimately determined, such that the communication parameters and media services of computing device 102 are synchronized with the requirements of communication. Therefore, the transmission of parameters enables the assessment of whether the media resources of the computing device meet the requirements of network-based communication. In an exemplary embodiment, the parameters may include at least one of audio parameters and video parameters corresponding to the media resources of computing device 102. For example, the parameters may correspond to an audio codec, a video codec, a corresponding resolution, and the orientation of the video. In one embodiment, system 110 may interact with communication module 105 to enable the reception of codec information to be sent to computing device 102. Codec information can be associated with the encoding or decoding attributes of the communication data. For example, codec information may include an audio-video codec associated with at least one of bitrate, sampling rate, video format, bitrate, profile level, and frame rate. In an exemplary embodiment, the exchange of signaling commands and the transmission of parameters and codec information can be performed via a socket service based on the IP address of computing device 102. The socket service can enable bidirectional communication endpoints for transmitting communication data from computing device 102 to data device 101. For example, a socket service can be initiated between a socket server thread 160 associated with computing device 102 and a socket client thread 162 belonging to data device 101. In an exemplary embodiment, the socket service can be enabled via at least one port associated with computing device 102. For example, at least one port may be a command port that allows all negotiation and signaling information. The primary purpose of the socket layer is to send information from one device to another and receive information from another device. This may not be limited to socket communication and can be extended to any other means of transmitting data from one device to another, such as Bluetooth, Zigbee, etc.

[0039] In one embodiment, when initiating communication, system 110 may receive communication data from a selected computing device. For example, the communication data may include at least one of text data, augmented virtual reality data, video data, and audio data. In an exemplary embodiment, the communication data may be received by data device 101 through at least one port. The audio and video data are in encoded form and originate from raw audio and raw video, which are captured using media resources of the selected computing device 102, respectively. In another exemplary embodiment, the audio and video data are simultaneously transmitted to data device 101 in encoded and / or raw form. In an exemplary embodiment, the raw audio and raw video may be encoded using an encoder coupled to computing device 102 and / or data device 101. Figure 1As shown, computing device 102 may include a camera 152, which can be used to acquire raw video data. Camera 152 can capture raw video data as needed by a first user. The raw video data can also be viewed by a local preview option 154, such as a display associated with computing device 102. The raw video data can be encoded using media codec video encoder 156, and the resulting encoded video can be sent to socket server thread 160 for further transmission to socket client thread 162 of data device 101. Similarly, computing device 102 may include a microphone or microphone 158, which can be used to acquire raw audio data and send it to socket server thread 160. In an exemplary embodiment, socket server thread 160 can receive raw audio data and send it to socket client thread 162, where the raw audio data can be encoded at media codec audio encoder 166 of data device 101 after preprocessing such as acoustic echo cancellation (AEC) using native AEC module 164 of data device 101. The socket client thread 162 of data device 101 can send communication data (encoded / raw audio data and / or encoded / raw video data) to the communication module 105 of data device 101. Communication module 105 can transmit the received communication data from the data device to remote device 107. In an exemplary embodiment, communication module 105 can use submodules 105-1 and 105-3 to send encoded / raw audio and encoded / raw video, respectively. Communication module 105 can communicate with remote device 107 via router 108 and base station 109. In an exemplary embodiment, communication module 105 can receive response communication data from remote device 107. Response communication data may include encoded audio response data and encoded video response data. Response communication data can be received from remote device 107 at submodules 105-2 and 105-4 of communication module 105, respectively. Data device 101 may also include a media codec video decoder 168 for decoding the received encoded video response data to obtain decoded video response data. Data device 101 may further include a media codec video decoder 173 for decoding the encoded audio response data to obtain decoded audio response data. After decoding, the response communication data can be viewed and listened to using a display 170 (for viewing video 172) and a speaker 174, respectively associated with data device 101. In an exemplary embodiment, the encoded audio response data may be sent to a local AEC module 164 for preprocessing (acoustic echo cancellation) prior to decoding. Thus, system 110 facilitates network-based communication between a first user using data device 101 and a second user using remote device 107.The communication may include at least one of outgoing communication and incoming communication, such that outgoing communication can be initiated by data device 101 to remote device 107 and incoming communication can be initiated by remote device 107 to data device 101.

[0040] In one embodiment, network 106 can be a wireless network, a wired network, or a combination thereof, and can be implemented as one of different types of networks, such as an intranet, a local area network (LAN), a wide area network (WAN), the Internet, etc. Furthermore, network 106 can be a private network or a shared network. A shared network can represent an association of different types of networks, where different types of networks can use various protocols, such as Hypertext Transfer Protocol (HTTP), Transmission Control Protocol / Internet Protocol (TCP / IP), Wireless Application Protocol (WAP), Automatic Repeat Request (ARQ), etc.

[0041] In one embodiment, when computing device 102 can receive a request for audio and / or video services, computing device 102 can open at least one port for audio and video transmission, wherein system 110 can receive communication data through these ports. In an exemplary embodiment, computing device 102 can begin recording raw data via a microphone and camera and send the recorded data to a corresponding encoder, wherein encoded data can be transmitted through the port. When system 110 receives data, the received data can be further processed if necessary and sent to a remote end as part of communication via one or more communication modules of data device 101.

[0042] In one embodiment, a system 110 configured to share media resources with a data device 101 (e.g., an STB) may include one or more processors coupled to memory, wherein the memory may store instructions that, when executed by the one or more processors, enable the system to perform the steps described above. Figure 2 refer to Figure 1This illustrates an exemplary representation of a system 110 for sharing media resources with a data device 101 according to one embodiment of the present disclosure. In one aspect, the system 110 may include one or more processors 202. The one or more processors 202 may be implemented as one or more microprocessors, microcomputers, microcontrollers, digital signal processors, central processing units, logic circuits, and / or any device that processes data according to operating instructions. Among other capabilities, the one or more processors 202 may be configured to retrieve and execute computer-readable instructions stored in a memory 204 of the system 110. The memory 204 may be configured to store one or more computer-readable instructions or routine operations in a non-transitory computer-readable storage medium, and may retrieve and execute these instructions or routine operations to create or share data packets via network services. The memory 204 may include any non-transitory storage device, including volatile memory such as RAM or non-volatile memory such as EPROM, flash memory, etc.

[0043] In one embodiment, system 110 may include interface 206. Interface 206 may include various interfaces, such as interfaces for data input and output devices, referred to as I / O devices, storage devices, etc. One or more interfaces 206 may facilitate communication within system 110. Interface 206 may also provide communication paths for one or more components of system 110. Examples of such components include, but are not limited to, one or more processing engines 208 and database 210.

[0044] Processing engine 208 can be implemented as a combination of hardware and programming (e.g., programmable instructions) to achieve one or more functions of processing engine 208. In the examples described herein, this combination of hardware and programming can be implemented in several different ways. For example, the program for processing engine 208 can be processor-executable instructions stored on a non-transitory machine-readable storage medium, and the hardware for processing engine 208 can include processing resources (e.g., one or more processors) to execute such instructions. In this example, the machine-readable storage medium can store instructions that implement processing engine 208 when executed by the processing resources. In such examples, system 110 can include machine-readable storage media storing instructions and processing resources to execute the instructions, or the machine-readable storage medium can be separate but provide access to system 110 and processing resources. In other examples, processing engine 208 can be implemented by electronic circuitry.

[0045] Processing engine 208 may include one or more engines selected from discovery engine 212, selection engine 214, synchronization engine 216, data transmission engine 218, and other engines 220. In one embodiment, discovery engine 212 may coordinate with a discovery manager to begin discovering computing devices for a desired service, such as an image capture service or a recording service on a network. To discover computing devices, discovery engine 212 may send a request to the discovery manager to send Internet Protocol (IP) and port information for available computing devices for the desired service. In response, discovery manager 120 may send a set of lists of available computing devices to system 110. The set of lists of available devices may be presented on an external device connected to data device 101.

[0046] In one embodiment, selection engine 214 may select one of the available computing devices for the desired service based on input received from the user. In an exemplary embodiment, selection engine 214 may select one of the available computing devices for an image capture service and / or a recording service.

[0047] In one embodiment, when communication is triggered to the network and communication is established, synchronization engine 216 can be configured to exchange signaling commands with computing devices associated with one or more parameters such as resolution, audio-video codec, direction, transmission, etc., to synchronize communication parameters and media services to perform communication.

[0048] In one embodiment, during synchronization, the data transmission engine 218 can receive data transmitted by the computing device via network 106, wherein the received data can be transmitted to the second computing device 107 via communication module 105. In another embodiment, data from the computing device can be directly transmitted to the remote second computing device 107.

[0049] In one embodiment, other engines 220 may assist at least one of the discovery engine 212, selection engine 214, synchronization engine 216, and data transmission engine 218 in communication. Database 210 may include data that can be stored or generated as a result of functionality implemented by any component of the processing engine 208 of system 110.

[0050] Figure 3A and 3B Exemplary representations of flowcharts illustrating communication between a system, a discovery manager, a computing device, and a communication module for outgoing and incoming communication, according to one embodiment of this disclosure, are shown respectively. Figure 3AAs shown, computing device 102 can register with discovery manager 120 (301). System 110 can then discover (302) available registered devices for the desired service from discovery manager 120. If no device is found, a notification to connect to a compatible device is sent to the user. Discovery manager 120 can send (303) a list of available registered devices to system 110. Based on input from the user, system 110 can select (304) computing device 102 from the list of available registered devices. Discovery manager 120 can communicate with computing device to initiate (305) a server socket on computing device, thereby forming an endpoint for bidirectional communication for sending or receiving data. In addition, discovery manager 120 can communicate with system 110 to initiate (306) a client socket on system 110 based on the device's IP address, thereby forming another endpoint for bidirectional communication for sending or receiving data. System 110 can then check with computing device whether outgoing communication can be performed (307). Based on confirmation (308), system 110 can request the communication module to perform outgoing communication (309). The communication module can send (310) codec information to system 110, and then send it (311) to computing device 102, wherein the codec information is associated with the encoding or decoding of data used for transmission. While transmitting codec information, computing device 102 can send (312) raw audio data to system 110, and then send it (313) to communication module 105. Computing device 102 can send (314) encoded video data to system 110, and then send it (315) to communication module 105. It is understood that the embodiments mentioned are merely exemplary, and that in 312 / 314, any raw or encoded data can be sent for any of the following audio and video data.

[0051] Figure 3B A flowchart for incoming communication is shown. (For example...) Figure 3BAs shown, computing device 102 can register with discovery manager 120 (321). Based on a request for incoming communication (322), system 110 can discover (323) registered devices for the required services from discovery manager 120. If no device is found, a notification to connect a compatible device is sent to the user. Discovery manager 120 can send (324) a list of available registered devices to system 110. Based on input from the user, system 110 can select (325) computing device 102 from the list of available registered devices. Discovery manager 120 can communicate with computing device to initiate (326) a server socket on computing device, thereby forming an endpoint for bidirectional communication for sending or receiving data. In addition, discovery manager 120 can communicate with system 110 to initiate (327) a client socket on system 110 based on the device's IP address, thereby forming another endpoint for bidirectional communication for sending or receiving data. System 110 can then check with computing device whether incoming communication can be performed (328). Based on confirmation (329), the system can request the communication module to accept (330) incoming communication. The communication module can send (331) codec information to system 110, and then send it (332) to computing device 102, wherein the codec information is associated with the encoding or decoding of the data used for transmission. While transmitting codec information, computing device 102 can send (333) raw audio data to system 110, and then send it (334) to communication module 105.

[0052] The computing device 102 can send (335) encoded video data to the system 110, and then send it (336) to the communication module 105.

[0053] Figure 4An exemplary representation of flowchart 400 according to an embodiment of this disclosure is shown for facilitating the sharing of media resources of a computing device with a data device to enable network-based communication with a remote device. Flowchart 400 can represent a general sequence of steps in the case of outgoing or incoming communication. At 402, the method may include the step of receiving device information related to available computing devices. The device information may be received from a discovery manager by a system integrated with the data device. At 404, the method may include the step of selecting a computing device from the available computing devices. At 406, the method may include the step of initiating communication between the data device and the remote device. Upon initiating communication, at 408, the method may include the step of receiving communication data from the selected computing device through at least one port. In an example embodiment, the communication data may include at least one of text data, augmented virtual reality data, video data, and audio data. The at least one port may include at least one of predefined ports and dynamic ports. At 410, the method may include the step of sending the received communication data to the remote device through the communication module of the data device to facilitate network-based communication with the remote device.

[0054] In one embodiment, the method may include the step of receiving dynamically discovered available computing devices from a plurality of registered computing devices by a discovery manager. The dynamic discovery step can be performed by assessing whether an available computing device is near the system or within reachable proximity to a network or wireless network based on a predefined threshold distance. The plurality of registered computing devices may register with a network service. Figure 5A An exemplary representation of a flowchart 500 for registering a computing device to a network via a discovery manager, according to an embodiment of the present disclosure, is shown. Figure 5A As shown, the computing device can start the (501) service and then register the (503) computing device with the local network through the discovery manager. Figure 5B An exemplary representation of flowchart 550 for discovering available computing devices over a network via a discovery manager, according to an embodiment of the present disclosure, is shown. The system may begin (551) discovering computing devices registered on a local network. In particular, the system may make a request for available registered computing devices for a desired service. In response, the system may receive (553) a list of available registered computing devices from the discovery manager.

[0055] Device information may be in the form of a list of identifiers associated with available computing devices. In an example embodiment, a computing device may be selected from available computing devices based on user input from a first user. Identifier information may include at least one of port information and Internet Protocol (IP) address associated with the available computing device. In an example embodiment, the IP address may be IPv4 or IPv6. Port information may be associated with at least one port. The discovery manager may receive the identifier information through a Network Service Discovery (NSD) module associated with a network service. Before initiating communication, the method may include a step of requesting the computing device through at least one port to confirm the possibility of initiating communication. In an exemplary embodiment, the request may be placed and confirmed by exchanging signaling commands between the data device and the selected computing device. Upon receiving confirmation from the computing device, the method may include the transmission of parameter and codec information facilitated by the system between the data device and the computing device. The transmission of parameters may enable the assessment of whether the media resources of the computing device meet the requirements of network-based communication. Parameters may include at least one of audio parameters and video parameters corresponding to the media resources of the computing device. Parameters may be associated with at least one of an audio codec, a video codec, a corresponding resolution of the video, and an orientation. The system can interact with a communication module to receive codec information to be sent to a computing device. The codec information can be associated with encoding or decoding attributes of the communication data. For example, the codec information may include an audio-video codec associated with at least one of bitrate, sampling rate, video format, bitrate profile level, and frame rate. In an exemplary embodiment, signaling command exchange and the transmission of parameters and codec information can be performed via a socket service based on the IP address of the computing device to enable bidirectional communication endpoints for transmitting communication data from the computing device to the data device. The socket service can be initiated between a socket server thread associated with the computing device and a socket client thread associated with the data device. The audio and video data can be in encoded form and can be derived, respectively, from raw audio and raw video captured using media resources of the selected computing device. The raw audio and raw video can be encoded using an encoder coupled to the computing device and / or the data device.

[0056] This method facilitates network-based communication between a first user using a data device and a second user using a remote device. The communication may include at least one of outgoing and incoming communication. Outgoing communication may be initiated by the data device to the remote device, and incoming communication may be initiated by the remote device to the data device. In one embodiment, the communication may include at least one of audio-based communication and video communication, such as audio calls, video calls, augmented virtual reality-based communication, and combinations thereof, for example, communication including both audio and video calls. In an alternative embodiment, the method may also be used in general methods that may include dependencies on streams from one or more external devices. Figure 6A and 6B An exemplary representation of a proposed system for incoming and outgoing telephone calls according to an embodiment of the present disclosure is shown. Figure 6A As shown, the proposed system can receive (601) a communication request in the form of an outgoing call request from a user or the communication module. Based on this request, the system can discover (602) available registered computing devices such as a camera or microphone. The system can then select (603) one of the available registered computing devices based on input received from the user. When an outgoing call is triggered to the call network and the call is successful, (604) signaling commands are exchanged between the STB and the computing device to initialize codec information and information associated with other parameters such as resolution, audio-video codecs, and orientation. During initialization, encoded / raw audio and / or video data can be sent (605) from the computing device, such as a mobile device, to the call module. Furthermore, if necessary, signaling commands can also be transmitted from the computing device to the call network during call communication.

[0057] In one embodiment, when the communication is in the form of an incoming call, such as Figure 6B As shown, the proposed system can receive (611) an incoming call request from a user or communication module. Based on the request, the system can discover (612) available registered computing devices such as a camera or microphone. The system can then select (613) one of the available registered computing devices based on input received from the user. The incoming call request is then accepted, and upon successful call establishment, signaling commands are exchanged (614) between the STB and the computing device to initialize codec information and information related to other parameters, such as resolution, audio-video codec, orientation, etc. During initialization, encoded / raw audio and / or video data can be sent (615) from the computing device, such as a mobile device, to the communication module. Furthermore, if necessary, signaling commands can be transmitted from the computing device to the calling network during call communication.

[0058] Figure 7An exemplary computer system according to an embodiment of the present disclosure is shown, in which embodiments of the invention can be used or used in conjunction with embodiments of the invention. Figure 7 As shown, computer system 700 may include external storage device 710, bus 720, main memory 730, read-only memory 740, mass storage device 750, communication port 760, and processor 770. Those skilled in the art will understand that the computer system may include more than one processor and communication port. Examples of processor 770 include, but are not limited to, those shown. Or an Itanium 2 processor, or Or Athlon processor, FortiSOC series processors TM An on-chip system processor or other future processor. Processor 770 may include various modules associated with embodiments of the invention. Communication port 760 may be any of the following: an RS-232 port for use with a modem-based dial-up connection, a 10 / 100 Ethernet port, a Gigabit or 10 Gigabit port using copper or fiber optic cable, a serial port, a parallel port, or other existing or future ports. Communication port 760 may be selected depending on the network, such as a local area network (LAN), a wide area network (WAN), or any network to which the computer system is connected. Memory 730 may be random access memory (RAM) or any other dynamic storage device known in the art. Read-only memory 740 may be any static storage device, such as, but not limited to, a programmable read-only memory (PROM) chip for storing static information, such as boot or BIOS instructions for processor 770. Mass storage 750 may be any current or future mass storage solution for storing information and / or instructions. Exemplary high-capacity storage solutions include, but are not limited to, Parallel Advanced Technology Attachment (PATA) or Serial Advanced Technology Attachment (SATA) hard disk drives or solid-state drives (internal or external, for example, with Universal Serial Bus (USB) and / or FireWire interfaces), such as available Seagate (e.g., Seagate Barracuda 7102 series) or Hitachi (e.g., Hitachi Deskstar 7K1000), one or more optical discs, and redundant array of independent disks (RAID) storage, such as disk arrays (e.g., SATA arrays), available from various vendors including Dot Hill Systems Corp., LaCie, Nexsan Technologies, Inc., and Enhance Technology, Inc.

[0059] Bus 720 couples the processor 770 to other memory, storage, and communication blocks. Bus 720 can be, for example, a Peripheral Component Interconnect (PCI) / PCI Expansion (PCI-X) bus, a Small Computer System Interface (SCSI) bus, USB, etc., for connecting expansion cards, drives, and other subsystems, as well as other buses, such as the Front Side Bus (FSB), which connects the processor 770 to the software system.

[0060] Optionally, operator and management interfaces, such as monitors, keyboards, and cursor control devices, can also be coupled to bus 720 to support direct operator interaction with the computer system. A network connection via communication port 760 can provide additional operator and management interfaces. External storage device 710 can be any type of external hard disk drive, floppy disk drive, etc. Zip drives, optical disc read-only memory (CD-ROM), rewritable optical disc (CD-RW), and digital video disk read-only memory (DVD-ROM). The above components are merely illustrative of various possibilities. The exemplary computer systems described above should in no way limit the scope of this disclosure.

[0061] Therefore, this invention provides a technical solution for sharing media resources with data devices such as STBs (Standard Telephone Bombs) that have suitable communication capabilities without requiring built-in cameras, microphones, or other resources. The technical solution includes service registration, service discovery, and data transmission, allowing the sharing of resources such as cameras and microphones with STB-Calling applications. The proposed system dynamically selects available computing devices and aggregates all local streams during outgoing and incoming communications, transmitting all aggregated local streams to other parties involved in the communication. In this way, the system enables STBs to utilize media resources associated with computing devices, such as cameras and microphones. In other words, the system facilitates STBs using the cameras and microphones of computing devices for various applications such as calling. In this way, users can easily communicate with another user through the STB during incoming or outgoing communications without any external resources.

[0062] This system and method are not limited to calling applications; they can be implemented for other applications or extended to any other communication service. For example, the system and method can be extended to publish media services over the Internet. In another example, the system and method can be extended to aggregate all available local streams into a single stream, such as video conferencing, and forward the aggregated frames to another party. In yet another example, the system and method can be extended to IoT security cameras, where all security cameras can connect to the same Optical Network Terminal (ONT) network, allowing users to communicate with the STB from a remote end via automatic response. Using this system and method, users can use aggregated or individual security camera streams, thus eliminating the need for an IoT server. In yet another example, the system and method can be extended to non-calling STB applications, such as brochures, photo albums, video editor applications, and augmented reality applications. In yet another example, the system and method can be extended to a technology bundled into a library that third-party STB application developers can leverage to utilize external cameras. Several other extended applications can be executed.

[0063] It should be understood that the embodiments described herein are for set-top box devices; however, the proposed systems and methods can be implemented in any computing device without departing from the scope of the invention, wherein any computing device does not have external devices, such as built-in cameras and microphones, or whenever it wishes to use external devices.

[0064] Although the preferred embodiments have been emphasized considerably herein, it should be understood that many embodiments can be made and many changes can be made to the preferred embodiments without departing from the principles of the invention. These and other variations in the preferred embodiments of the invention will be apparent to those skilled in the art from the disclosure herein, and it is thus clearly understood that the foregoing subject matter is merely illustrative and not intended to be limiting.

[0065] This patent document contains material protected by intellectual property rights, such as, but not limited to, copyright, product design, trademark, integrated circuit layout design and / or trade dress protection, which belong to (JPL) or its affiliates (hereinafter referred to as the Owner). While the Owner of the patent document or patent disclosure has no objection to any fax copying of the patent disclosure by any person because the patent document or patent disclosure appears in the Patent and Trademark Office documents or records, it retains all rights. All rights to this type of intellectual property belong entirely to the Owner.

[0066] Advantages of this disclosure

[0067] This disclosure provides a method and system capable of sharing resources such as cameras and microphones, which are typically found on devices such as mobile phones or laptops and are used for communication.

[0068] This disclosure provides an economical method and system that eliminates the need for external compatible devices (such as cameras and microphones), which also avoids compatibility issues with external devices used for communication.

[0069] This disclosure provides a method and system that enable STBs to transmit media data over the Internet during communication.

[0070] This disclosure provides a method and system that can aggregate all local streams available for use with a stream, such as a video conference, and forward the aggregated stream to another party.

[0071] This disclosure provides a method and system that can aggregate user-available Internet of Things (IoT) security camera streams when all security cameras are connected to the same ONT network, through which the user communicates with the STB, thereby avoiding the need for an IoT server.

[0072] This disclosure provides a method and a system that can be used in applications such as those for sharing video and audio data, video editing applications, augmented reality applications, and others.

Claims

1. A system for facilitating the sharing of media resources of multiple computing devices with a data device to enable network-based communication with remote devices, wherein each computing device registers with a network service via a discovery manager, the discovery manager dynamically discovers available computing devices from a plurality of registered computing devices, the system being configured with the data device and comprising: One or more processors coupled to memory, wherein the memory stores instructions that, when executed by the one or more processors, cause the system to: Receive device information related to available computing devices from the discovery manager coupled to the system; The computing device is selected from the available computing devices based on user input from a first user; Initiate communication between the data device and the remote device; When initiating the communication, communication data is received from the selected computing device through at least one port, and the received communication data is sent to the remote device through the communication module of the data device, facilitating network-based communication with the remote device. The communication data includes at least one of text data, augmented virtual reality data, video data, and audio data. The audio data and the video data are in encoded form and originate from raw audio and raw video, which are captured using media resources of the selected computing device. The raw audio and the raw video are encoded using at least one encoder coupled to the computing device and the data device. The media resources include accessory devices integrated with or coupled to the computing device, which are capable of capturing at least one of the raw audio and the raw video, or obtaining at least one of encoded audio data and encoded video data.

2. The system according to claim 1, characterized in that, The communication includes at least one of audio calls, video calls, augmented virtual reality-based communication, or combinations thereof, and wherein the discovery manager is capable of dynamically discovering the available computing devices from a plurality of registered computing devices.

3. The system according to claim 2, characterized in that, The dynamic discovery is performed by evaluating whether the available computing devices are near the system based on a predefined threshold distance, or by evaluating whether the available computing devices are within the reachable range of the network, wherein the plurality of registered computing devices register with the network service.

4. The system according to claim 1, characterized in that, The device information is in the form of a set list, which includes identifier information related to the available computing devices.

5. The system according to claim 4, characterized in that, The port includes at least one of a predefined port or a dynamic port, wherein the identifier information includes at least one of port information and an IP address associated with the available computing device, wherein the port information is associated with the at least one port, and wherein the discovery manager receives the identifier information through a network service discovery module associated with the network.

6. The system according to claim 1, characterized in that, Before initiating the communication, the system requests the computing device through the at least one port to confirm the possibility of initiating the communication, and the request is placed and confirmed through the exchange of signaling commands between the data device and the selected computing device.

7. The system according to claim 6, characterized in that, Upon receiving confirmation from the computing device, the system facilitates the transmission of parameter and codec information between the data device and the computing device.

8. The system according to claim 7, characterized in that, The transmission of the parameters enables the assessment of whether the media resources of the computing device meet the requirements of the network-based communication. The parameters include at least one of audio parameters and video parameters, the parameters correspond to the media resources of the computing device, and the parameters are related to at least one of the audio codec, video codec, corresponding resolution and orientation of the video.

9. The system according to claim 7, characterized in that, The system interacts with the communication module to receive codec information to be sent to the computing device, the codec information being associated with encoding or decoding attributes of the communication data, and wherein the codec information is related to at least one of bit rate, sampling rate, video format, profile level, and frame rate.

10. The system according to claim 7, characterized in that, The exchange of signaling commands and the transmission of parameters and encoding / decoding information are performed via socket services based on the IP address of the computing device, for enabling bidirectional communication endpoints to transmit the communication data from the computing device to the data device, and wherein the socket service is initiated between a socket server thread associated with the computing device and a socket client thread associated with the data device.

11. The system according to claim 4, characterized in that, The network-based communication enables communication between a first user using the data device and a second user using the remote device, wherein the communication includes at least one of outgoing communication and incoming communication, and wherein the outgoing communication is initiated by the data device to the remote device, and the incoming communication is initiated by the remote device to the data device.

12. The system according to claim 1, characterized in that, The discovery manager is directly integrated with the system or a router associated with the system, and the data device includes a set-top box.

13. A method for facilitating the sharing of media resources of multiple computing devices with a data device to enable network-based communication with remote devices, wherein each computing device registers with a network service via a discovery manager, the discovery manager dynamically discovering available computing devices from a plurality of registered computing devices, the method comprising: The system configured with the data device receives device information related to available computing devices from the discovery manager. The computing device is selected from the available computing devices based on user input from a first user; Initiate communication between the data device and the remote device; When initiating the communication, communication data is received from the selected computing device through at least one port, and the received communication data is sent to the remote device through the communication module of the data device, facilitating network-based communication with the remote device. The communication data includes at least one of text data, augmented virtual reality data, video data, and audio data. The audio data and video data are in encoded form and originate from raw audio and raw video, which are captured using media resources of the selected computing device, respectively. The raw audio and raw video are encoded using at least one encoder coupled to the computing device and the data device, and the media resources include accessory devices integrated with or coupled to the computing device for capturing at least one of the raw audio and raw video, or obtaining at least one of the encoded audio data and encoded video data.

14. The method according to claim 13, characterized in that, The method further includes: The discovery manager dynamically discovers the available computing devices from the plurality of registered computing devices. The dynamic discovery step is performed by assessing whether the available computing devices are near the system based on a predefined threshold distance, or by assessing whether the available computing devices are within the reachable range of the network, wherein the plurality of registered computing devices are registered to the network service.

15. The method according to claim 13, characterized in that, The communication includes at least one of audio calls, video calls, augmented virtual reality-based communication, and combinations thereof, wherein the device information is in the form of a set list, the set list including identifier information related to the available computing devices.

16. The method according to claim 15, characterized in that, The identifier information includes at least one of port information and IP address associated with the available computing device, wherein the port information is associated with the at least one port, and wherein the discovery manager receives the identifier information through a network service discovery module associated with the network.

17. The method according to claim 13, characterized in that, The method also includes: When selecting the computing device, a request is made to the computing device via the at least one port to confirm the possibility of initiating the communication, and the request is placed and confirmed by the exchange of signaling commands between the data device and the selected computing device.

18. The method according to claim 17, characterized in that, The method further includes: Upon receiving confirmation from the computing device, the system facilitates the transmission of parameter and codec information between the data device and the computing device.

19. The method according to claim 18, characterized in that, The transmission of the parameters enables the assessment of whether the media resources of the computing device meet the requirements of the network-based communication. The parameters include at least one of audio parameters and video parameters, the parameters correspond to the media resources of the computing device, and the parameters are related to at least one of the audio codec, video codec, corresponding resolution and orientation of the video.

20. The method according to claim 18, characterized in that, The system interacts with the communication module to receive codec information to be sent to the computing device, the codec information being associated with encoding or decoding attributes of the communication data, and wherein the codec information includes an audio-video codec related to at least one of bitrate, sampling rate, video format, profile level, and frame rate.

21. The method according to claim 18, characterized in that, The exchange of signaling commands and the transmission of parameters and encoding / decoding information are performed via socket services based on the IP address of the computing device, for enabling bidirectional communication endpoints to transmit the communication data from the computing device to the data device, and wherein the socket service is initiated between a socket server thread associated with the computing device and a socket client thread associated with the data device.

22. The method according to claim 13, characterized in that, The port includes at least one of a predefined port or a dynamic port.

23. The method according to claim 15, characterized in that, The network-based communication enables communication between a first user using the data device and a second user using the remote device, wherein the communication includes at least one of outgoing communication and incoming communication, and wherein the outgoing communication is initiated by the data device to the remote device, and the incoming communication is initiated by the remote device to the data device.

Images