Multiplexing of Application Channels on an Isochronous Stream

Abstract

A first device is provided. The first device includes an audio source, a sensor, and a processor. The audio source generates audio data, and the sensor captures sensor data. The processor generates a data packet that includes an audio data set generated by the audio source and a sensor data set captured by the sensor. In some examples, the data packet may also include audio payload length data and / or sensor payload length data, audio channel identification data and / or sensor channel identification data, and / or audio time offset data and / or sensor time offset data. The audio data set may have a first duration, and the sensor data set may have a second duration that is longer than the first duration. The processor transmits the data packet to a second device configured to reconstruct the audio data set and the sensor data set by demultiplexing the data packet.

Description

Technical Field

[0001] (Cross - Reference to Related Applications) This application claims priority to U.S. Provisional Patent Application No. 63 / 366,041, filed on June 8, 2022, entitled "Multiplexing Application Channels on Isochronous Streams", the entire disclosure of which is hereby incorporated by reference in its entirety.

Background Art

[0002] Immersive audio rendered in virtual reality or augmented reality applications often requires the capture of data in multiple modes by wearable audio devices. This captured data must be wirelessly transmitted to a central device (such as a smartphone) for further processing to provide the user with an immersive audio experience.

Summary of the Invention

[0003] The present disclosure generally relates to transmitting two or more types of data by an isochronous stream via a multiplexing scheme. These types of data can include various audio data (such as data captured by a microphone or data used by an acoustic transducer to generate audio) and / or non-audio data (such as data collected by different types of non-audio sensors). The multiplexing scheme can incorporate time offset values to enable accurate time demultiplexing and reconstruction of the data transmitted by the isochronous stream. The multiplexing scheme can also incorporate data regarding the payload length of data packets and channel identification information regarding the source of the captured data. The isochronous stream can be a Connected Isochronous Stream (CIS) or a Broadcast Isochronous Stream (BIS), depending on the application. The sensor data can include a wide variety of different data types, such as motion data captured by an inertial measurement unit (IMU), for example.

[0004] Generally, in one aspect, a first device is provided. The first device includes an audio source. The audio source is configured to generate audio data.

[0005] The first device further includes a sensor. The sensor is configured to capture sensor data. According to an example, the sensor can be an inertial measurement unit (IMU). In addition to this example, the sensor data can be motion data.

[0006] The first device further includes a processor. The processor is configured to generate data packets. The data packets include an audio data set generated by an audio source and a sensor data set captured by a sensor. According to one example, the data packets may further include audio payload length data and / or sensor payload length data. According to a further example, the data packets may further include audio channel identification data and / or sensor channel identification data. According to still a further example, the data packets further include audio time offset data and / or sensor time offset data. According to yet a further example, the audio data set may have a first duration, and the sensor data set may have a second duration longer than the first duration.

[0007] The processor is further configured to transmit the data packets to a second device. The second device is configured to reconstruct the audio data set and the sensor data set by demultiplexing the data packets. According to one example, the data packets may be transmitted via a Bluetooth connected isochronous stream or a Bluetooth broadcast isochronous stream.

[0008] According to one example, the first device may be a wearable audio device, and the second device may be a central device. In an alternative example, the first device may be a central device, and the second device may be a wearable audio device.

[0009] According to one example, the processor is further configured to: (1) receive an audio data positive acknowledgment before the end of the first duration, (2) generate a second data packet including the sensor data set, and (3) transmit the second data packet to the second device.

[0010] According to one example, the processor is further configured to: (1) generate a second audio data set via an audio source after the audio data set has ended; (2) generate a second data packet including the second audio data set and a sensor data set via the processor of the first device; and (3) transmit the second data packet to a second device. In addition to this example, the processor may be further configured to: (1) capture a second sensor data set via a sensor of the first device after the sensor data set has ended; (2) generate a third data packet including the second audio data set and the second sensor data set; and (3) transmit the third data packet to the second device.

[0011] Generally, in another aspect, a method for transmitting data is provided. The method includes capturing an audio data set via an audio source of a first device.

[0012] The method further includes capturing a sensor data set via a sensor of the first device.

[0013] The method further includes generating a data packet via the processor of the first device. The data packet includes the audio data set and the sensor data set. According to one example, the data packet may further include audio payload length data and / or sensor payload length data. According to another example, the data packet may further include audio channel identification data and / or sensor channel identification data. According to a further example, the data packet may further include audio time offset data and / or sensor time offset data. According to still a further example, the audio data set has a first duration, and the sensor data set has a second duration that is longer than the first duration.

[0014] The method further includes transmitting data packets to a second device via a transceiver of a first device. According to an example, the data packets may be transmitted via a Bluetooth connected isochronous stream or a Bluetooth broadcast isochronous stream.

[0015] The method further includes receiving data packets via a transceiver of the second device.

[0016] The method further includes reconstructing an audio data set and a sensor data set by demultiplexing the data packets via a processor of the second device.

[0017] According to an example, the method may further include (1) receiving an audio data positive acknowledgment via a transceiver of the first device before a first duration ends, (2) generating a second data packet including a sensor data set via a processor of the first device, and (3) transmitting the second data packet to a central device via a transceiver of the first device.

[0018] According to an example, the method may further include (1) generating a second audio data set via an audio source after the audio data set ends, (2) generating a second data packet including the second audio data set and the sensor data set via a processor of the first device, and (3) transmitting the second data packet to the second device via a transceiver of the first device. In addition to this example, the method may further include (1) capturing a second sensor data set via a sensor of the first device after the sensor data set ends, (2) generating a third data packet including the second audio data set and the second sensor data set via a processor of the first device, and (3) transmitting the third data packet to the second device via a transceiver of the first device.

[0019] In various embodiments, a processor or controller may be associated with one or more storage media (collectively referred to herein as "memory", such as volatile and non-volatile computer memories like ROM, RAM, PROM, EPROM, and EEPROM, floppy disks, compact disks, optical disks, magnetic tapes, flash, OTP-ROM, SSD, HDD, etc.). In some implementations, the storage media may be encrypted with one or more programs that, when executed on one or more processors and / or controllers, perform at least some of the functions described herein. The various storage media can be fixed within the processor or controller or can be portable, thereby loading one or more programs stored on the storage media into the processor or controller to implement the various aspects described herein. The term "program" or "computer program" is used herein in its general sense to refer to any kind of computer code (e.g., software or microcode) that can be used to program one or more processors or controllers.

[0020] It should be understood that all combinations of the foregoing concepts and additional concepts discussed in more detail below (subject to such concepts not being mutually inconsistent) are intended to be part of the subject matter of the invention disclosed herein. Specifically, all combinations of the claimed subject matter that appear at the end of this disclosure are intended to be part of the subject matter of the invention disclosed herein. It should also be understood that terms explicitly used herein that may also appear in any incorporated by reference disclosure should be given the meaning that most closely matches the particular concepts disclosed herein.

[0021] Other features and advantages will be apparent from the present specification and the claims.

[0022] In the drawings, like reference numerals generally refer to like parts throughout the different views. Also, the drawings are not necessarily to scale; instead, emphasis is generally placed on illustrating the principles of the various embodiments.

Brief Description of the Drawings

[0023]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

Figure 14

Embodiments for Carrying Out the Invention

[0024] The present disclosure generally aims to transmit two or more types of data on an isochronous stream via a multiplexing scheme. These types of data can include various audio data (such as data captured by a microphone or data used by an acoustic transducer to generate audio) and / or non-audio data (such as data collected by various types of non-audio sensors). The multiplexing scheme can incorporate a time offset value to enable accurate demultiplexing and reconstruction in time of the data transmitted by the isochronous stream. The multiplexing scheme can also incorporate data regarding the payload length of data packets and channel identification information regarding the source of the captured data.

[0025] In one non-limiting example, a wearable audio headset is used with a personal computer (PC) for gaming purposes. The headset can be connected to the PC via an isochronous stream. The headset includes a microphone for capturing the user's voice as audio data, and sensors such as an inertial measurement unit (IMU) for capturing the movement of the user's head as sensor data. The processor of the headset can multiplex the audio data with the sensor data such that the headset transmits data packets containing both the audio data and the sensor data to the PC via the isochronous stream. The data packets also include time offset data indicating when each payload of the audio data or the sensor data was captured. The PC receives the data packets, demultiplexes the packets, and uses the time offset to reconstruct the received audio and sensor data at the appropriate timing. In a further example, other types of audio data or non-audio data can be multiplexed and demultiplexed.

[0026] In another non-limiting example, an in-vehicle computing system of a vehicle is used with a pair of wireless earphones worn by a driver of the vehicle. The wireless earphones can be connected to the in-vehicle computing system via an isochronous stream. The in-vehicle computing system includes an audio source for generating audio data corresponding to a navigation subsystem, an entertainment subsystem, or any other in-vehicle subsystem capable of generating audio. The in-vehicle computing system also includes sensors configured to capture sensor data regarding the movement of the vehicle (such as speed or direction). A processor of the in-vehicle computing system can multiplex the audio data with the sensor data such that the in-vehicle computing system transmits data packets including both the audio data and the sensor data to the wireless earphones via the isochronous stream. The data packets can also include time offset data indicating when each payload of the audio data or the sensor data was captured. The wireless earphones receive the data packets, demultiplex the packets, and use the time offset to reconstruct the received audio and sensor data at appropriate timings. Thus, an immersive audio experience incorporating the movement of the vehicle can be created using the multiplexed audio and sensor data transmitted from the in-vehicle computing system to the wireless earphones.

[0027] As used in this application, the term "wearable audio device" is intended to mean, in addition to its ordinary meaning or the meaning known to those skilled in the art, a device that fits around, over, in, or near the ear (including open-ear audio devices worn on the user's head or shoulders) and radiates acoustic energy into or toward the ear. A wearable audio device may be referred to as headphones, earphones, earpieces, headsets, earbuds, or sports headphones, and may be wired or wireless. A wearable audio device includes an acoustic driver that converts an audio signal into acoustic energy. This acoustic driver can be housed within an earcup. Some of the following figures and descriptions show a single wearable audio device having a pair of earcups (each including an acoustic driver), but it should be understood that a wearable audio device may be a single stand-alone unit having only one earcup. Each earcup of a wearable audio device can be mechanically connected to another earcup or headset, for example, by a headband and / or by a lead wire that conveys an audio signal to an acoustic transducer within the earcup or headset. A wearable audio device can include components for receiving an audio signal wirelessly. A wearable audio device can include components of an active noise reduction (ANR) system. A wearable audio device can also include other functions, such as a microphone, so that the device can function as a headset. FIG. 1 shows examples of the form factor of in-ear headphones, the form factor of glasses, and the form factor of over-ear headsets, but in other examples, a wearable audio device can be an on-ear, around-ear, behind-ear, or near-ear headset.In some examples, a wearable audio device can be an open ear device that includes an acoustic driver that radiates acoustic energy toward an ear while leaving the ear open to the ear's environment and surroundings.

[0028] As used herein, the term "connected isochronous stream" is intended to refer to an isochronous data stream that utilizes a pre-established point-to-point communication link via LE Audio between a source device (also known as a central device or master device) and one or more audio devices (also known as peripheral devices or slave devices), in addition to including its ordinary meaning or the meaning known to those of ordinary skill in the art. In other words, a connected isochronous stream can provide an isochronous audio stream that utilizes at least one established and reliable communication channel and / or at least one confirmed communication channel between a source device and any respective audio device.

[0029] As used herein, the term "broadcast isochronous stream" is intended to refer to an isochronous data stream in which it is not necessary to establish a pre-established communication link between a source device that transmits data and an audio device that receives data, and in which it is not necessary to transmit or receive an acknowledgment or negative acknowledgment.

[0030] The following description is to be read with reference to FIGS. 1 - 14. FIG. 1 is a schematic diagram of the components of system 10 according to the present disclosure. In a non - limiting example of FIG. 1, system 10 includes at least one first device 100 and a second device 200. As shown in FIG. 1, at least one first device 100 may be embodied as a wearable audio device, while the second device 200 may be embodied as a central device. However, as demonstrated in subsequent examples, the first device 100 may alternatively be embodied as a central device, while the second device 200 may be embodied as a wearable audio device. Additionally, in some examples as illustrated in FIG. 1, system 10 includes a plurality of first devices 100A - 100C (collectively referred to as "first device 100" or "plurality of first devices 100"). The second device 200 is intended to be a device capable of establishing a wireless connection, e.g., wireless connection 138 (described later), with at least one first device 100. Although illustrated as a smartphone, the second device 200 can be selected from at least one of a tablet, a smart hub, a media hub, a stereo hub, a soundbar, a headphone case, or any device capable of transmitting or broadcasting wireless data (described later) to at least one wearable device 100. Further, although illustrated as a pair of fully wireless earphones 100A, a glasses - form factor device 100B, and an over - ear - type form - factor headset, it is to be understood that the first device 100 can be selected from any device capable of transmitting data to and / or receiving wireless data from the second device 200. In some examples, each first device 100 is intended to be a device capable of rendering audible acoustic energy, e.g., a device capable of rendering audio data, based on wireless data received from the second device 200.

[0031] Each device of the system 10, namely, the first device 100 and the second device 200, can establish one or more wireless connections 138A - 138C (collectively referred to as "wireless connection 138") between the second device 200 and each first device 100 using their respective communication modules and / or transceivers. Each wireless connection 138 can be used to transmit and / or receive wireless data via one or more wireless data streams 140A - 140C (collectively referred to as "wireless data stream 140"). In some examples, these wireless connections 138 include establishing one or more data streams between the second device 200 and each first device 100, and each data stream is an isochronous data stream, for example, a connected isochronous stream using the LE Audio standard. In other examples, the wireless connection 138 includes a broadcast isochronous stream, that is, the second device 200 broadcasts the data in one or more isochronous data streams received by one or more wireless audio devices 100. For example, the second device 200 may be configured to generate, broadcast, or in some cases wirelessly transmit the wireless data stream 140 received by each first device 100. Alternatively, the first device 100 can also transmit the wireless data stream 140 via a broadcast isochronous stream. The streams established by each wireless connection 138 can use various wireless data protocols, standards, or methods of transmission, such as the Bluetooth Low Energy protocol or the LE Audio standard.

[0032] FIG. 2 illustrates a modified form of the system 10 of FIG. 1. In the non-limiting example of FIG. 2, the first device 100 is embodied as a central device, while the second device 200 is embodied as a wearable audio device. In the example of FIG. 2, the first device 100 can be an in-vehicle computing system of a vehicle incorporating aspects such as a navigation subsystem, an entertainment subsystem, etc. The second device 200 is embodied as a pair of wireless earphones that can be worn by a driver of the vehicle. The wireless connection 138 enables the in-vehicle computing system to transmit a wireless data stream 140 to the wireless earphones.

[0033] FIG. 3 illustrates an example of multiplexing and demultiplexing of data transmitted by a wireless data stream 140 such as a Bluetooth isochronous stream. In this example, a plurality of applications of a first device 100 such as a wearable audio device capture data. These applications can be associated with different sensors such as a microphone, a motion sensor, etc. A multiplexer combines portions of data collected by each application into a single packet. The packet is transmitted via a transport 140 such as an isochronous stream and received by a second device 200 such as a PC or a smartphone. The PC or smartphone reconstructs the data captured by the first device 100 and provides the data to appropriate applications of the second device 200. In some examples, the multiplexing-demultiplexing scheme is bidirectional and can enable the second device 200 to transmit data packets containing multiple types of data to be demultiplexed by the first device 100.

[0034] The object of the present disclosure is to create a scheme for isochronous channels such as a connected isochronous stream (CIS) channel, such that the isochronous channel transmits packets containing data from multiple sources such as data collected by a microphone and a motion sensor, and each packet includes one or more channel identifiers and a length header. An isochronous adaptation layer (ISOAL) can be used to segment the packets.

[0035] FIG. 4 illustrates a data packet 106 created by multiplexing using a basic CIS multiplexing header. The illustrated packet includes a CIS header 142, isochronous adaptation layer (ISOAL) frame data 144, a first CIS multiplexing header 152, a first information payload 108, a second CIS multiplexing header 154, and a second information payload 110. In one example, the first CIS multiplexing header 152 and the first information payload 108 may correspond to audio data generated by an audio source 102. In some examples, the audio source may be a microphone configured to capture external audio. In other examples, the audio source 102 may be a hardware or software interface configured to receive audio information such as compressed media files from other aspects of the first device 100, from an entertainment or navigation subsystem. The second CIS multiplexing header 154 and the second information payload 110 may correspond to motion data captured by a non-audio sensor 104 such as a motion sensor. In some examples, the motion sensor may be an inertial measurement unit (IMU). The first CIS multiplexing header 152 includes a first service data unit (SDU) length 112 and a first channel identification data 116, while the second CIS multiplexing header 154 includes a second SDU length 114 and a first channel identification data 118. The service data unit (SDU) lengths 112, 114 indicate the lengths of the corresponding payloads 108, 110, while the channel identification data 116, 118 indicate the application or data type corresponding to the payload. In the non-limiting example of FIG. 4, the ISOAL frame data 144 is 24 bits, the SDU length data 112, 114 is 8 - 16 bits, and the channel identification data 116, 118 is 8 bits.

[0036] However, when transmitting a payload containing data from different sources (such as microphones, motion sensors, etc.) in the same packet, a timing offset is required to indicate when the data from each payload was captured. Due to different devices with various processing speeds or refresh rates, or different sensors of the same device, the data of each payload 108, 110 may be captured at different times. By incorporating a timing offset corresponding to each payload, the receiver can map all the data back to their original time domains based on the reference clock of the isochronous channel. Examples of the timing offsets 120, 122 are illustrated in FIG. 5 as components of the CIS multiplexing headers 152, 154 of the super SDU (SSDU). In FIG. 5, the timing offsets 120, 122 represent when the payloads 108, 110 were captured relative to the anchor point. The anchor point can correspond to the timing of the data packet received by the first device 100 from the second device 200 in an isochronous event, or any other relevant point in time.

[0037] Figure 6 illustrates an existing ISOAL architecture for creating CIS packets from multiple types of application data, while Figure 7 illustrates new layers in this architecture for segmentation and reassembly, retransmission and flow control, and encapsulation. Specifically, this new layer enables more efficient retransmission of the payload based on the duration of the data in the payload, as different types of data can have different durations. For example, audio data may end after 10 milliseconds (in a low-latency game application, etc.), while sensor data (such as motion sensor data) may end every 15 milliseconds. Thus, in this example, the data channel can be configured with a 5-millisecond duration (or flash timeout) while repeating the audio data twice and the sensor data three times before refresh. Further, if the audio data is acknowledged as received at the 8-millisecond mark but the sensor data is not acknowledged as received, the audio data is removed but not replaced with new data until the 10-millisecond mark is reached. During this period between 8 milliseconds and 10 milliseconds, the isochronous channel continues to transmit packets with only sensor data (no audio data) and saves energy by retransmitting only the data that has not yet been received.

[0038] Figures 8 and 9 are flow diagrams illustrating some of the scenarios discussed with reference to Figures 6 and 7. In the embodiments of Figures 8 and 9, the first device 100 can be a wearable audio device (such as an audio headset), and the second device can be a central device (such as a smartphone). In other embodiments of Figures 8 and 9, the first device 100 can be a central device (such as an in-vehicle computing system), and the second device can be a wearable audio device (such as a pair of wireless earphones).

[0039] In FIG. 8, the first device 100 wirelessly transmits a first data packet 106 (such as the data packet 106 shown in FIG. 5). The data packet 106 includes an audio data set 108 and a sensor data set 110. The sensor data set 110 may correspond to a motion sensor or other non-audio sensor. The audio data 108 has a first duration 124 of 10 milliseconds, and the sensor data 110 has a second duration 126 of 15 milliseconds. The first data packet 106 is received by the second device 200. In response, the second device 200 transmits an audio data positive acknowledgment 128 to the first device 100. Then, since the second device 200 has already positively acknowledged the reception of the audio data set 108, the first device 100 transmits a second data packet 130 having only the sensor data 110.

[0040] In FIG. 9, the first device 100 wirelessly transmits again a first data packet 106 that includes an audio data set 108 and a sensor data set 110. Then, the first device 100 generates a second data packet 130 after the first duration 124 has ended but before the second duration 126 has ended. Thus, the second data packet 130 includes a second audio data set 132 and the first sensor data set 110. Then, the first device 100 generates a third data packet 136 after both the first and second durations 124, 126 have ended. Thus, the third data packet 136 includes the second audio data set 132 and a second sensor data set 134. As time passes, new audio and sensor data are cycled into the transmitted data packets according to the data durations 124, 126.

[0041] FIG. 10 schematically illustrates one of the first devices 100 shown previously in FIGS. 1 and 2. The first device 100 can be a wearable audio device as shown in FIG. 1, or the first device can be a central device as shown in FIG. 2. As shown in the non-limiting example of FIG. 1, the first device 100 can be embodied in the form factor of in-ear headphones, the form factor of glasses, or an over-ear headset. As shown in the non-limiting example of FIG. 2, the first device 100 can be embodied as an in-vehicle computing system. The first device 100 includes an audio source 102, a sensor 104, a processor 125, a memory 175, and a transceiver 185. The audio source 102 can be embodied as a microphone for capturing audio, or a hardware or software interface for receiving audio information. The memory 175 is configured to store a first data packet 106, a second data packet 130, a first audio data set 108, a second audio data set 132, a first sensor data set 110, a second sensor data set 134, an audio payload length 112, a sensor payload length 114, a first channel identification data 116, a second channel identification data 118, audio time offset data 120, sensor time offset data 122, a first data duration 124, a second data duration 126, a data acknowledgment 128, and a third data packet 136. The processor 125 is configured to execute one or more applications such as app 1 111, app 2 113 to app N 1NN as shown in FIG. 3. In a non-limiting example, the processor 125 can be configured to multiplex the first audio data set 108 and the first sensor data set 110 to create the first data packet 106.

[0042] FIG. 11 schematically illustrates the second device 200 shown previously in FIGS. 1 and 2. As shown in the non-limiting example of FIG. 1, the second device 200 can be a smartphone. As shown in the non-limiting example of FIG. 2, the second device 200 can be a pair of wireless earphones. The second device 200 includes a processor 225, a memory 275, and a transceiver 285. The memory 275 is configured to store a first data packet 106, a second data packet 130, a first audio data set 108, a second audio data set 132, a first sensor data set 110, a second sensor data set 134, an audio payload length 112, a sensor payload length 114, first channel identification data 116, second channel identification data 118, audio time offset data 120, sensor time offset data 122, a first data duration 124, a second data duration 126, a data positive acknowledgment 128, and a third data packet 136. The processor 225 is configured to execute one or more applications such as app 1 211, app 2 213 to app Y 2YY as shown in FIG. 3. In a non-limiting example, the processor 225 can be configured to demultiplex the first data packet 106 and reconstruct the first audio data set 108 and the first sensor data set 110 for further processing.

[0043] FIGS. 12-14 are flowcharts of a method 900 for transmitting data according to various embodiments of the present invention. Referring to FIGS. 1-14, method 900 includes, at step 902, generating an audio data set 108 via an audio source 102 of a first device 100.

[0044] Method 900 further includes, at step 904, capturing a sensor data set 110 via a sensor 104 of the first device 100.

[0045] Method 900 further includes, at step 906, generating data packet 106 via processor 125 of the first device 100. Data packet 106 includes audio data set 108 and sensor data set 110. According to one example, data packet 106 may further include audio payload length data 112 and / or sensor payload length data 114. According to another example, data packet 106 may further include audio channel identification data 116 and / or sensor channel identification data 118. According to a further example, data packet 106 may further include audio time offset data 120 and / or sensor time offset data 122. According to yet a further example, audio data set 108 has a first duration 124, and sensor data set 110 has a second duration 126 that is longer than the first duration 124.

[0046] Method 900 further includes, at step 908, transmitting data packet 106 to the second device 200 via transceiver 185 of the first device 100. According to one example, data packet 106 may be transmitted via a Bluetooth connected isochronous stream or a Bluetooth broadcast isochronous stream.

[0047] Method 900 further includes, at step 910, receiving data packet 106 via transceiver 285 of the second device 200.

[0048] Method 900 further includes, at step 912, reconstructing audio data set 108 and sensor data set 110 by demultiplexing data packet 106 via processor 225 of the second device 200.

[0049] According to one example, method 900 may further include, in optional steps 914, 916, and 918, respectively: (1) receiving an audio data positive response 128 via transceiver 185 of the first device 100 before the end of the first duration 124; (2) generating a second data packet 130 including the sensor data set 110 via processor 125 of the first device 100; and (3) transmitting the second data packet 130 to the second device 200 via transceiver 185 of the first device 100.

[0050] According to one example, method 900 may further include, in optional steps 920, 922, and 924, respectively: (1) generating a second audio data set 132 via audio source 102 after the audio data set 108 has ended; (2) generating a second data packet 130 including the second audio data set 132 and the sensor data set 110 via processor 125 of the first device 100; and (3) transmitting the second data packet 130 to the second device 200 via transceiver 185 of the first device 100. In addition to this example, method 900 may further include, in optional steps 926, 928, and 930, respectively: (1) capturing a second sensor data set 134 via sensor 926 of the first device 100 after the sensor data set 110 has ended; (2) generating a third data packet 136 including the second audio data set 132 and the second sensor data set 134 via processor 125 of the first device 100; and (3) transmitting the third data packet 136 to the second device 200 via transceiver 185 of the first device 100.

[0051] All definitions defined and used herein are to be understood as controlling the dictionary definitions, definitions in incorporated documents by reference, and / or the ordinary meaning of the defined terms.

[0052] As used in this specification and the claims, the indefinite articles "a" and "an" should be understood to mean "at least one" unless specifically stated otherwise.

[0053] As used in this specification and the claims, the phrase "and / or" means "either or both" of the elements so combined, i.e., elements that are present conjunctively in some cases and disjunctively in other cases. The plurality of elements listed with "and / or" should be construed in the same manner, i.e., as "one or more" of the elements so combined. Optional elements may exist in addition to those specifically identified by the "and / or" clause, whether or not other elements are related or unrelated to the specifically identified elements.

[0054] As used in this specification and the claims, "or" should be understood to have the same meaning as "and / or" as defined above. For example, when separating items in a list, "or" or "and / or" should be construed as inclusive, i.e., including not only at least one of the number of elements or the list, but also more than one, and optionally including items not in the additional list. Terms such as "only one of" or "exactly one of" or, when used in the claims, "consisting of" indicate inclusion of exactly one element of the number of elements or the list only when specifically stated otherwise. Generally, the term "or" as used in this specification is construed as indicating exclusive alternatives (i.e., "one or the other but not both") only when preceded by exclusive terms such as "either", "only one of", "only one of the", or "exactly one of".

[0055] As used in this specification and the claims, the phrase "at least one" with respect to a list of one or more elements means at least one element selected from any one or more of the elements in the list of elements, but does not necessarily include at least one of each and every element specifically listed in the list of elements, and is not to be construed as excluding any combination of elements in the list of elements. This definition also allows elements to optionally exist outside of the specifically identified elements in the list of elements to which the phrase "at least one" refers, whether or not they are related to the specifically identified elements.

[0056] It should also be understood that, unless otherwise explicitly indicated, in any method claimed in this specification that includes more than one step or act, the order of the steps or acts of the method is not necessarily limited to the order in which the steps or acts of the method are recited.

[0057] In the claims, as well as in the above specification, all transitional phrases such as "comprising", "including", "carrying", "having", "containing", "involving", "holding", "composed of", etc. are to be understood to mean without limitation, i.e., including but not limited to. Only the transitional phrases "consisting of" and "consisting essentially of" are respectively limiting or semi - limiting transitional phrases.

[0058] The above examples of the described subject matter can be realized in any of a number of ways. For example, some aspects can be realized using hardware, software, or a combination thereof. If at least a portion of any aspect is realized as software, the software code can be executed by any suitable processor, or set of processors, whether provided on a single device or computer or distributed among a plurality of devices / computers.

[0059] The present disclosure can be realized as a system, method, and / or computer program product at any of the assumed levels of integration of any technically detailed level. The computer program product can include a computer-readable storage medium (or media) having thereon computer-readable program instructions for causing a processor to execute aspects of the present disclosure.

[0060] A computer-readable storage medium can be a tangible device that can hold and store instructions for use by an instruction execution device. The computer-readable storage medium can be, for example, an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing, but is not limited thereto. A non-exhaustive list of more specific examples of computer-readable storage media includes the following: portable computer diskettes, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM, or flash memory), static random access memory (SRAM), portable compact disc read-only memory (CD-ROM), digital versatile disk (DVD), memory stick, floppy disk, punch card or mechanically encoded device with instructions recorded thereon in grooves or raised structures, as well as suitable combinations of the foregoing. As used herein, a computer-readable storage medium is not construed to be a transient signal per se, such as a freely propagating electromagnetic wave like radio waves, an electromagnetic wave propagating through a transmission medium such as a waveguide (e.g., an optical pulse passing through an optical fiber cable), or an electrical signal transmitted through an electrical wire.

[0061] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to respective computing / processing devices via a network, such as, for example, the Internet, a local area network, a wide area network, and / or a wireless network, or to an external computer or an external storage device. The network can include copper transmission cables, optical transmission fibers, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge servers. A network adapter card, or network interface, of each computing / processing device receives the computer-readable program instructions from the network and transfers the computer-readable program instructions for storage in a computer-readable storage medium within each respective computing / processing device.

[0062] Computer-readable program instructions for performing the operations of this disclosure may be source code or object code written in any combination of one or more programming languages, including assembler instructions, instruction-set-architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state-setting data, integrated circuit configuration data, or object-oriented programming languages such as Smalltalk, C++, procedural programming languages such as the "C" programming language, or similar programming languages. The computer-readable program instructions may execute entirely on the user's computer, partially on the user's computer, as a stand-alone software package, partially on the user's computer, partially on a remote computer, or entirely on a remote computer or server. In the latter scenario, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider). In some examples, an electronic circuit, including, for example, a programmable logic circuit, a field-programmable gate array (FPGA), or a programmable logic array (PLA), may execute the computer-readable program instructions by using the state information of the computer-readable program instructions to customize the electronic circuit to perform aspects of this disclosure.

[0063] Aspects of the present disclosure are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to examples of the present disclosure. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer-readable program instructions.

[0064] The computer-readable program instructions may be provided to a processor of a special purpose computer, or other programmable data processing apparatus to produce a machine, such that the instructions executed via the processor of the computer or other programmable data processing apparatus create means for implementing the functions / operations specified in one or more blocks of the flowchart and / or block diagram. Further, these computer-readable program instructions may be stored in a computer-readable storage medium that can direct a computer, programmable data processing apparatus, and / or other devices to function in a particular manner, such that the computer-readable storage medium storing the instructions comprises an article of manufacture including instructions for implementing the functions / operations specified in the flowchart and / or block diagram, or modes of the blocks.

[0065] The computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable apparatus, or other device to produce a computer-implemented process, such that the instructions executed on the computer, other programmable apparatus, or other device implement the functions / acts specified in one or more blocks of the flowchart and / or block diagram.

[0066] The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of the implementation forms assumed for the systems, methods, and computer program products according to various examples of the present disclosure. In this regard, each block of the flowchart or block diagram may correspond to a module, segment, or portion of instructions that includes one or more executable instructions for performing the specified logical function(s). In some alternative implementation forms, the functions described in the blocks may occur out of the order described in the figures. For example, two blocks shown in succession may actually be executed substantially simultaneously, or, in some cases, the blocks may be executed in the reverse order depending on the functions involved. Further, it should also be noted that each block of the block diagram and / or flowchart illustration, and combinations of blocks in the block diagram and / or flowchart illustration, can be implemented in a system based on dedicated hardware that performs a specific function or operates or executes a combination of dedicated hardware and computer instructions.

[0067] Other implementation forms are within the scope of the following claims and other claims that the applicant may have the right to.

[0068] In this specification, various examples have been described and illustrated. However, those skilled in the art will readily conceive of various other means and / or functions for implementing and / or achieving the results and / or obtaining one or more of the advantages described herein, and each of such variations and / or modifications is considered to be within the scope of the examples described herein. More generally, those skilled in the art will readily understand that all of the parameters, dimensions, materials, and configurations described herein are exemplary, and that the actual parameters, dimensions, materials, and / or configurations depend on the particular application or the application for which the teachings of the invention are used. Those skilled in the art will be able to recognize or confirm many equivalents corresponding to the specific examples described herein with only routine experimentation. Accordingly, the foregoing examples are presented only by way of illustration, and it should be understood that the examples can be implemented in other ways than as explicitly described within the scope of the appended claims and their equivalents. The examples of the present disclosure are directed to each individual feature, system, article, material, kit, and / or method described herein. Further, any combination of two or more such features, systems, articles, materials, kits, and / or methods is included within the scope of the invention of the present disclosure if such features, systems, articles, materials, kits, and / or methods do not mutually conflict.

Explanation of Reference Numerals

[0069] 10 System 100 First Device 100A - C First Device 102 Audio Source 104 Sensor 106 Data Packet 108 Payload 110 Payload 112 Audio Payload Length Data 114 Sensor Payload Length Data 116 First Channel Identification Data 118 Second Channel Identification Data 120 Timing Offset 122 Timing offset 124 First duration 125 Processor 126 Second duration 128 Audio data positive response 130 Second data packet 132 Second audio data set 134 Second sensor data set 136 Third data packet 138 Wireless connection 138A~138C Wireless connection 140 Wireless data stream 140A~140C Wireless data stream 142 CIS header 144 Frame data 152 First CIS multiplexing header 154 Second CIS multiplexing header 175 Memory 185 Transceiver 200 Second device 225 Processor 275 Memory 285 Transceiver 900 Method 902 Step 904 Step 906 Step 908 Step 910 Step 912 Step 914 Step 916 Step 920 Step 922 Step 924 Step 926 Step 928 Step

Claims

1. A first device, comprising: an audio source configured to generate audio data; a sensor configured to capture sensor data; a processor configured to: generate a data packet comprising an audio data set generated by the audio source and a sensor data set captured by the sensor; transmit the data packet to a second device configured to reconstruct the audio data set and the sensor data set by demultiplexing the data packet.

2. The first device according to claim 1, wherein the first device is a wearable audio device and the second device is a central device.

3. The first device according to claim 1, wherein the first device is a central device and the second device is a wearable audio device.

4. The first device according to claim 1, wherein the data packet further comprises audio payload length data and / or sensor payload length data.

5. The first device according to claim 1, wherein the data packet further comprises audio channel identification data and / or sensor channel identification data.

6. The first device according to claim 1, wherein the data packet further comprises audio time offset data and / or sensor time offset data.

7. The first device according to claim 1, wherein the sensor is an inertial measurement unit (IMU) and the sensor data is motion data.

8. The first device according to claim 1, wherein the data packet is transmitted via a Bluetooth connected isochronous stream or a Bluetooth broadcast isochronous stream.

9. The audio data set has a first duration, the sensor data set has a second duration longer than the first duration, and the processor is configured to: receive an audio data positive acknowledgment before the end of the first duration; generate a second data packet comprising the sensor data set. The first device according to claim 1, further configured to transmit the second data packet to the second device.

10. The audio data set has a first duration, the sensor data set has a second duration longer than the first duration, and the processor generates a second audio data set via the audio source after the audio data set has ended, generates a second data packet including the second audio data set and the sensor data set via the processor of the first device, The first device according to claim 1, further configured to transmit the second data packet to the second device.

11. The processor captures a second sensor data set via the sensor of the first device after the sensor data set has ended, generates a third data packet including the second audio data set and the second sensor data set, The first device according to claim 10, further configured to transmit the third data packet to the second device.

12. A method for transmitting data, comprising: generating an audio data set via an audio source of a first device; capturing a sensor data set via a sensor of the first device; generating, via a processor of the first device, a data packet including the audio data set and the sensor data set; transmitting the data packet to a second device via a transceiver of the first device; receiving the data packet via a transceiver of the second device; reconstructing the audio data set and the sensor data set by demultiplexing the data packet via a processor of the second device.

13. The method according to claim 12, wherein the data packet further includes audio payload length data and / or sensor payload length data.

14. The method according to claim 12, wherein the data packet includes audio channel identification data and / or sensor channel identification data.

15. The method according to claim 12, wherein the data packet further includes audio time offset data and / or sensor time offset data.

16. The method according to claim 12, wherein the data packet is transmitted via a Bluetooth connected isochronous stream or a Bluetooth broadcast isochronous stream.

17. The method according to claim 12, wherein the audio data set has a first duration, and the sensor data set has a second duration longer than the first duration.

18. Receiving an audio data positive response via the transceiver of the first device before the first duration ends; Generating a second data packet including the sensor data set via the processor of the first device; The method according to claim 17, further comprising transmitting the second data packet to the second device via the transceiver of the first device.

19. Generating a second audio data set via the audio source after the audio data set ends; Generating a second data packet including the second audio data set and the sensor data set via the processor of the first device; The method according to claim 17, further comprising transmitting the second data packet to the second device via the transceiver of the first device.

20. Capturing a second sensor data set via the sensor of the first device after the sensor data set ends; Generating a third data packet including the second audio data set and the second sensor data set via the processor of the first device; The method according to claim 19, further comprising transmitting the third data packet to the second device via the transceiver of the first device.

Images