A method and apparatus for transmitting and receiving audio while transmitting analog video over a single coaxial cable.
By quantizing and buffering audio during video blanking periods and multiplexing it into video intervals, the method addresses the complexity and cost issues of existing audio-video transmission, achieving efficient and cost-effective transmission over a single coaxial cable.
Patent Information
- Application Number
- JP2023578984
- Authority / Receiving Office
- JP · JP
- Patent Type
- Patents
- Current Assignee / Owner
- Priority Date
- 2021-07-12
- Filing Date
- 2022-07-12
- Publication Date
- 2026-11-11
- Estimated Expiration
- 2042-07-12
AI Technical Summary
Existing methods for transmitting analog audio and video over a single coaxial cable require complex computation and large buffers due to the need to calculate and store audio duration for each video frame, leading to increased cost and complexity.
A method that involves quantizing analog audio signals, buffering them during video blanking periods, and multiplexing audio data into video blanking intervals without intensive calculation, using a first-in, first-out buffer and digital-to-analog conversion.
Reduces computational burden and manufacturing costs while maintaining efficient audio and video transmission, allowing for bidirectional audio transmission over a single coaxial cable.
Smart Images

Figure 0007948974000002 
Figure 0007948974000003 
Figure 0007948974000004
Abstract
Description
[Technical Field]
[0001] This application relates in general to digital video transmission, and more particularly to a method and apparatus for transmitting and receiving audio while transmitting analog video over a single coaxial cable. [Background technology]
[0002] When both analog audio and video data need to be transmitted over long distances, it is clear that one cable is superior to two cables in terms of cost and complexity. Various methods and devices have been proposed to reduce the cost and complexity of transmitting both analog audio and video signals over a single coaxial cable. To avoid interference between analog audio and video data over a single cable, there are two ways to achieve the necessary separation. It is known that audio and video data can be frequency multiplexed and transmitted in separate, non-overlapping frequency bands. It is known that audio and video data can be interleaved in the time domain so that they exist in the signal at different times without interfering with each other. It is also known that audio data can be interleaved when there is no active video during video blanking. However, current signaling protocols are complex and require intensive computation to interleave audio data into video data.
[0003] For example, U.S. Patent Application No. 14 / 442,803 of Zhejiang Dahua Technology Co., Ltd. discloses a method and apparatus for transmitting high-definition video signals to address the problems of the prior art, namely the cost and complexity of transmitting analog and audio over long distances using a single coaxial cable. In particular, application 803 proposes buffering audio data by calculating the duration of the audio for each row in the video frame over which the audio data is superimposed. However, the method proposed in application 803 requires calculating and storing the audio duration size for each active row of video data. This requires storing and buffering audio durations that are by no means small, resulting in burdensome buffering and storage requirements and thus complicating the encoding scheme. The exemplary implementation described herein does not require calculating the number of audio samples stored in the buffer and transmitted per frame, employs a much smaller buffer, and reduces manufacturing costs and operating power requirements.
[0004] Therefore, there remains a need in this field for solutions to address the aforementioned problems in particular. [Overview of the project]
[0005] One or more embodiments herein generally relate to methods and systems for transmitting digital audio and video data over an analog transmission medium using a single coaxial cable. More specifically, one or more embodiments herein disclose a method or apparatus for efficiently transmitting audio data during the video blanking period of an analog video signal in a manner that does not require intensive calculation of the audio duration for each line.
[0006] According to one embodiment, a method of one or more embodiments of this specification may include the steps of receiving an audio signal in analog or digital format, and quantizing the analog signal to a digital format to obtain a plurality of quantized audio data. This method includes the steps of buffering one or more of the plurality of quantized audio data, and receiving a video signal including a blank section and at least one portion of video data. In some embodiments, this method includes the steps of generating one or more quantized audio pulses corresponding to one or more of the plurality of quantized audio data stored in the buffer, and an audio header corresponding to the one or more quantized audio pulses. This method includes the step of identifying, without calculation, at least one allowable portion of a blank section corresponding to a first duration exceeding a predetermined duration of the audio header and one or more quantized audio pulses. In some embodiments, this method may end with the step of obtaining a combined audio / video signal representing at least a portion of the audio data and video data by multiplexing the audio header and one or more audio pulses with at least one portion of video data in at least one allowable portion of the blank section.
[0007] According to another aspect of the present invention, a method of one or more embodiments of this specification may include the step of receiving an analog video signal, the analog video signal including an audio header and one or more quantized audio pulses corresponding to quantized audio data. The method may also include the step of detecting the audio header in a blank section of the analog video. In some embodiments, the method includes the steps of determining a reference level for the audio header and extracting one or more quantized audio pulses in response to detecting the audio header. In some embodiments, the method includes the steps of converting one or more quantized audio pulses to the original values of one or more quantized audio data based on the reference level for the audio header and storing one or more quantized audio data of the original values in a first-in, first-out (FIFO) buffer. In some embodiments, the method includes using the FIFO buffer to reconstruct continuous audio data from one or more quantized audio data of the original values. For example, audio sample data stored in the FIFO buffer and a local audio clock having substantially the same frequency as the original sampling clock on the transmitter side. Substantially the same means that the difference is very small.
[0008] Another aspect of one or more embodiments may include a method and apparatus for transmitting digital audio over analog video data using a single cable. This method includes, in the case of analog audio, a transmitter receiving a digital video signal and an audio signal in analog or digital format, and, in the case of analog audio, sampling the analog audio signal with an analog-to-digital converter (ADC). The method includes, storing audio bits corresponding to the digital audio signal or the sampled analog audio signal in a first-in, first-out (FIFO) buffer, and, in response to an arbiter detecting the availability of data in the FIFO buffer, reading and outputting the serialized audio bits. Upon detecting the video blanking period and the availability of audio sample data in the FIFO, the arbiter may release a single data sample from the FIFO by serializing the sample data and simultaneously adding a digital header or start code to the beginning of the sequence. Each FIFO entry contains one audio sample data converted by the ADC. The number of bits per sample depends on the resolution of the ADC. Bit serialization for digital transmission is performed on the output side. These bits are represented as digital values with a specific duration, which are then converted by a DAC into pulses with a specific height and duration. Analog audio transmission operates in a similar manner, except that the arbiter output is a digital code representing the height of the start pulse, followed by an audio sample code, all of which have some duration, and thus become analog pulses after being converted to analog format by a DAC as shown in the figure. In some embodiments, this method includes the steps of formatting serialized audio bits with a digital start code and generating a combined digital audio / video signal by inserting the serialized audio bits and the digital start code into blanking periods of digital video data.Some embodiments include the steps of generating a combined analog audio / video signal containing a native audio data stream by converting the combined digital audio / video signal to analog using a digital-to-analog converter (DAC), and transmitting the combined analog audio / video signal to a receiver in one direction. The method may include the steps of receiving the combined analog audio / video signal at the receiver and converting the combined analog audio / video signal to a combined digital audio / video signal using an ADC. Such a method may include the step of extracting the audio data stream from the combined digital audio / video signal using an audio decoder. In some embodiments, the method includes the video decoder providing timing information and video blanking period positions to assist the audio decoder in retrieving the audio data.
[0009] Another aspect of one or more embodiments may include a method for bidirectional audio transmission. In addition to the transmitter-to-receiver transmission described above, the sampled audio signal is sampled at a first frequency, the transmitter includes a reverse audio signal receiver, and the receiver includes a reverse audio signal transmitter. Such a bidirectional configuration includes the steps of: receiving a second audio signal in the reverse audio signal transmitter included in the receiver; determining one or more vertical blanking lines of the second digital video signal and one or more time slots without audio data in the one or more vertical blanking lines using an arbiter in the reverse audio transmitter and a video decoder in the receiver; if the second audio signal is an analog audio signal, sampling the second audio signal at a second frequency using another analog-to-digital converter (ADC) located in the reverse signal transmitter; and storing a second digitized audio sample corresponding to the second audio signal in another first-in, first-out (FIFO) buffer located in the reverse signal transmitter, wherein the second digitized audio The process includes: the sample being either the second audio signal in digital form or a second digitized sample of the second (analog) audio signal; inserting a start pattern, one or more second serialized audio data, and / or an end pattern into a vertical blanking section of the combined analog audio / video stream; and transmitting an audio stream corresponding to the start pattern and / or the end pattern and one or more second serialized audio bits to the transmitter, including the audio receiver, via the opposite-direction signal transmitter located in the receiver, to the opposite-direction signal receiver located in the transmitter, wherein the transmission of the audio stream is performed in a different direction opposite to the one direction, over the single cable. The inserted audio data may be one or more per line.
[0010] The reverse audio signal receiver in the transmitter uses a pair of clock frequencies that must be substantially the same as the frequency used by the corresponding reverse audio signal transmitter in the receiver, although this may differ from the frequency used in the reverse direction. For the video decoder on the receiver side to correctly decode the video, it is desirable that the video line information, including synchronization, blanking, and active video position, be the same on both sides. This method includes extracting the serialized audio data based on one or more vertical blanking lines and one or more time slots, and storing the extracted audio samples in a first-in, first-out buffer. In some embodiments, this method includes utilizing a predetermined clock frequency that is substantially the same as the clock frequency of the reverse audio transmitter in the (video) receiver. This method includes, in the reverse audio signal receiver in the (video) transmitter, selecting an output sample frequency based on the transmitting clock frequency, acquiring an audio sample, and outputting an audio stream reconstructed based on the output sample frequency. In some embodiments, the audio clock is set to a frequency substantially the same as the audio clock frequency used on the transmitter side (e.g., 8KHz, 16KHz, etc.). The output audio samples are extrapolated from data stored in a first-in, first-out buffer based on the difference between clocks, in order to obtain equally spaced audio data that matches the receiver's audio clock. [Brief explanation of the drawing]
[0011] These and other aspects and features of the embodiments described herein will become apparent to those skilled in the art by considering the following description of specific embodiments of the invention in conjunction with the accompanying drawings.
[0012] [Figure 1] This is a block diagram of an exemplary voice transmitter according to one or more embodiments of the present invention.
[0013] [Figure 2]is a schematic diagram of an output analog video signal according to one or more embodiments of the present invention.
[0014] [Figure 3] is a block diagram of an exemplary audio receiver according to one or more embodiments.
[0015] [Figure 4] is a flowchart of a method for transmitting audio signal data according to one or more embodiments of the present invention.
[0016] [Figure 5] is a flowchart of a method for receiving an analog video signal according to one or more embodiments of the present invention.
[0017] [Figure 6] is a block diagram of an exemplary video transmitter according to one or more embodiments of the present invention.
[0018] [Figure 7] is a block diagram of an exemplary video receiver according to one or more embodiments of the present invention.
[0019] [Figure 8] is a schematic diagram of analog video / audio signals according to one or more embodiments of the present invention.
[0020] [Figure 9] is a block diagram of an exemplary video receiver including an audio signal transmitter in the opposite direction according to one or more embodiments of the present invention.
[0021] [Figure 10] is a block diagram of an exemplary video transmitter including an audio signal receiver in the opposite direction according to one or more embodiments of the present invention.
[0022] [Figure 11] is a schematic diagram of analog video / audio signals according to one or more embodiments of the present invention. [Modes for carrying out the invention]
[0023] The present invention will be described in detail with reference to drawings and images provided as examples to illustrate the invention, so that those skilled in the art may carry it out. In particular, the following figures and examples are not intended to limit the scope of the invention to a single embodiment, but other embodiments are possible by substituting some or all of the elements described or illustrated.
[0024] Furthermore, if any particular element of the present invention can be partially or completely implemented using known components, only the portion of such known components necessary for understanding the present invention will be described, and detailed descriptions of other parts of such known components will be omitted so as not to obscure the present invention.
[0025] As will be apparent to those skilled in the art, unless otherwise explicitly stated herein, embodiments described as being implemented in software are not limited thereto and may include embodiments implemented in hardware or a combination of software and hardware, and vice versa. Embodiments showing a single component should not be considered limiting; rather, unless otherwise expressly stated herein, the present invention is intended to encompass other embodiments containing multiple identical components, and vice versa. Furthermore, unless expressly stated herein, the applicant does not intend any term in the specification or claims to be non-generic or to have a special meaning. The present invention also encompasses current and future known equivalents of components mentioned herein as examples.
[0026] Figure 1 shows a transmitter-side audio encoder system 100 to which the embodiment can be applied. As shown in Figure 1, the system 100 includes an analog-to-digital converter (ADC) 104, a buffer 106, an arbiter 108, a video encoder 110, a signal combiner 112, and a digital-to-analog converter (DAC) 114. In an exemplary embodiment, the system 100 is configured to receive a continuous analog audio signal 102 and process the analog audio signal 102 for transmission along with a video signal 111 over a single transmission line.
[0027] In one embodiment, the ADC 104 may be configured to receive the audio signal 102 in analog or digital format, and if in analog format, to sample the analog audio signal 102 at a predetermined frequency. In one embodiment, the audio sampling rate may be set lower than the video line rate of the video signal 111, thereby preventing buffer 106 from overflowing. The ADC 104 may be configured to output quantized audio data 103 corresponding to the sampled analog audio signal 102. This is necessary when the audio signal is in analog format, but not when the audio signal is in digital format, as is well understood. In one embodiment, the ADC 104 may be configured to sample the audio signal 102 at 8 kHz. In another embodiment, the ADC 104 may be configured to sample the audio signal 102 at 16 kHz. In yet another embodiment, the ADC 104 may be configured to sample the audio signal 102 at 32 kHz.
[0028] In some embodiments, the ADC 104 may be configured to receive an analog audio signal 102, sample the analog audio signal 102 in a digital format, quantize it, and thereby generate quantized audio data 103. In some embodiments, a buffer 106 may be configured to buffer the quantized audio data 103. The buffer 106 may receive the quantized audio data 103 from the ADC 104 and store entries until a video blanking period, which will be described in more detail below, becomes available, or it may be configured to directly receive various parts of the digital audio signal. In one embodiment, the buffer 106 may include a first-in, first-out (FIFO) buffer configured so as not to change the order in which quantized / digital audio data is received and transmitted by the buffer 106. In another embodiment, the buffer 106 may be configured to timestamp incoming quantized / digital audio data. In one embodiment, in response to buffering the quantized audio data 103, the buffer 106 may be configured to notify the arbiter 108 that quantized / digital audio data is available in the buffer 106. In one embodiment, the buffer 106 may be configured to include an interrupt I / O circuit that communicates with the arbiter 108. In another embodiment, the arbiter 108 may include a polling I / O circuit configured to poll the buffer 106 to determine the presence of quantized / digital audio data 103 in the buffer 106.
[0029] In some embodiments, the arbiter 108 analyzes the video signal 111 to determine the timing and presence of video blanking intervals in the video signal 111, which will be described in more detail below. In one embodiment, in response to polling and / or interrupts, the arbiter 108 may be configured to determine the availability of video blanking intervals (not shown in Figure 1) in the video signal 111. In one embodiment, the video signal 111 includes blanking intervals and at least a portion of the video data placed therein. In one embodiment, the video blanking interval may include horizontal sync pulses, vertical sync pulses, color sync pulses, and / or other data corresponding to the video data of the video signal 111. In another embodiment, the video blanking interval does not include sync pulses and / or other data corresponding to the video data of the video signal 111, and the arbiter 106 or video encoder 110 may be configured to insert sync pulses at predetermined positions in the video blanking interval based on a predetermined video format, which will be described in more detail below.
[0030] In some embodiments, in response to determining the timing and presence of a video blanking interval, the arbiter 108 may be configured to instruct buffer 106 to transmit quantized audio data 103 corresponding to the availability of the video blanking period of the video signal 111. If, at the time of determining the presence of the video blanking interval, buffer 106 does not contain a data entry (i.e., quantized audio data 103), the row is skipped until an entry is stored in buffer 106. In one embodiment, the arbiter 108 may include a synchronization circuit that includes one or more clocking circuits that process the video signal 111 and execute a task management protocol for interleaving the quantized audio data 103 with the video blanking period of the video signal 111. In one embodiment, the arbiter 108 is configured to fetch or retrieve the quantized audio data from buffer 106.
[0031] In one embodiment, the arbiter 108 is configured to receive a digital video signal 111 from the video encoder 110. The arbiter 108 may be configured to check whether quantized / digital audio data 103 stored in the buffer 106 is available in response to approaching a blank section of the video signal 111. In response to determining that quantized / digital audio data 103 is available in the buffer 106, the arbiter 108 can generate one or more synchronization pulses, one or more quantized audio pulses corresponding to the quantized / digital audio data 103 stored in the buffer 106, and an audio header describing the one or more quantized audio pulses. For example, the synchronization pulses are generated by the video encoder 110, while the arbiter 108 can determine the timing for inserting the audio data and format the data accordingly. In one embodiment, the header may indicate the presence of audio data and be used to decode the audio on the receiver side, which will be described later. In other embodiments, the synchronization pulses are generally generated by the video encoder (encoder). The arbiter determines when to insert the audio data and formats the data accordingly. It is understood that, in this specification, the terms “insert” or “inserting” should be interpreted broadly to include superposition, combination, injection, or other forms of the signal and / or its components, such as combining an audio signal with an existing combined audio-video stream.
[0032] In one embodiment, the video encoder 110 is configured to generate a digital video signal 111 and output the video signal 111 to an arbiter 108 and a signal combiner 112. In another embodiment, the video encoder 110 may relay an existing video signal received from a video source (not shown). In some embodiments, the video encoder 110 may be configured to output a video signal 111 which can be composed of 8-bit, 10-bit, or 12-bit 4:4:4 or 4:2:2 YUV data. For example, in some embodiments, since there is only one cable, the UV data may be modulated and synchronized before being added to the Y to form a single digital data before being converted to analog. In other embodiments, the YUV data may be interleaved with synchronization in a predefined manner before DAC conversion. In some embodiments, the video signal 111 can be composed using other subsample formats. In some embodiments, the video encoder 110 may include one or more analog processors, a chrominance processor, a luminance processor, a clocking circuit, and a host communication interface (not shown in Figure 1).
[0033] As described above, the arbiter 108 may be configured to determine the availability of the video blanking period for the video signal 111. In some embodiments, the arbiter 108 may include processing circuitry and memory for storing firmware and / or other types of non-transient computer-readable instructions. The processing circuitry of the arbiter 108 may be configured to execute instructions contained in the firmware and / or memory for carrying out the exemplary embodiments described herein. In some embodiments, the processing circuitry within the arbiter 108 may include one or more hardware processors, one or more field-programmable gate arrays (FPGAs), individually and / or in combination.
[0034] In one embodiment, the arbiter 108 may be configured to manage the timing for interleaving and / or inserting quantized / digital audio data 103 stored in buffer 106 with video signal 111 via a clocking function (not shown in Figure 1). In some embodiments, the arbiter 108 may include a clocking function for synchronizing the quantized / digital audio data 103 with video signal 111. The arbiter 108 may include one or more digital signal processors (DSPs), field-programmable gate arrays (FPGAs), and / or application-specific integrated circuits (ASICs). In one embodiment, the clock circuit of the arbiter 108 may include one or more of the following, individually and / or in combination: a phase-locked loop (PLL) clock, a non-PLL clock, a crystal oscillator, and an LC resonant tank circuit. The clocking circuit may include additional timing and signal conditioning functions, for example, fan-out buffers, multipliers and dividers, and combiners, individually and / or in combination.
[0035] As shown in Figure 1, the signal combiner 112 can be configured to receive quantized audio data 103 from the buffer 106 and to receive a digital video signal 111 from the video encoder 110. The signal combiner 112 may be configured to interleave the quantized / digital audio data 103 during the video blanking period of the video data 111, which will be described in more detail later. In one embodiment, the signal combiner 112 may be configured to interleave the quantized / digital audio data 103 with the video signal 111 by inserting the quantized / digital audio data 103 into the video blanking portion of the video signal 111, and to output the combined audio / video (A / V) digital signal 113 to a digital-to-analog converter (DAC) 114.
[0036] In some embodiments, the DAC 114 includes an 8-bit DAC, a 10-bit DAC, or a DAC of any higher bit width, and can operate at a specific frequency required by the video signal 111. For example, the DAC 114 can operate at 148.5 MHz, which is twice the input data rate of standard HD digital video (e.g., 1080p30 / 25 or 720p60 / 50). In some embodiments, the amplitude of the DAC output signal can be set by an external current setting resistor (not shown in Figure 1). The DAC 114 may be configured to receive the A / V signal 113 from the signal combiner 112 in digital format, convert the combined audio / video signal 113 into analog format, and output an analog audio / video (A / V) signal 116 for transmission over a single transmission line (not shown in Figure 1). In one embodiment, the signal transmission line may be a coaxial cable configured to transmit the analog signal over one or more optical fiber cables and / or twisted-pair wire transmission lines.
[0037] In one embodiment, the video signal 111 may include a resolution of 1280 x 720 pixels. The video blanking section 210 may correspond to the video blanking period of the 1280 x 720 pixel video signal. The video signal 111 may include streaming video frames, each frame containing 1650 pixels per line, of which 1280 pixels are active video pixels. In one embodiment, the blanking section of the video signal 111 includes 370 pixels configured to include horizontal sync pulses, color sync pulses, and / or other data corresponding to video data.
[0038] In one embodiment, for example, the video signal 111 may correspond to the SMPTE 296M standard. In this standard, a 1280 x 720 pixel video frame has 750 horizontal lines or rows, of which 30 lines are vertical blanking sections that do not contain video data. According to the SMPTE 296M numbering scheme, lines 26 to 745 are active video frame data. Both the active frames and the vertical blanking sections may contain their respective video blanking sections 210. Due to the periodic nature of the video blanking sections, the video blanking sections may be the same size for each row of the video signal 111. In other embodiments, the video signal 111 may correspond to other SMPTE standards, including, but not limited to, 259M, 344M, 292M, 372M, 424M, ST-2081, ST-2082, and / or ST-2083. It should be noted that the exemplary embodiments described herein are not limited to the line numbering schemes described above. Those skilled in the art can apply the principles of the present invention to any other numbering scheme by reading the disclosures described herein.
[0039] In some embodiments, the arbiter 108 may be configured in conjunction with the signal combiner 112 to generate one or more quantized audio pulses (not shown in Figure 1) corresponding to one or more of the quantized audio data 103 stored in the buffer 106. In the digital domain, the generation of these quantized audio pulses corresponds to digital data for the duration of the pulse. The arbiter 108 may also be configured in conjunction with the signal combiner 112 to generate audio headers (not shown in Figure 1) corresponding to one or more quantized audio pulses. In some embodiments, the arbiter 108 may be configured to multiplex the buffer 106 into a combined A / V signal 113 by having the signal combiner 112 transmit the audio headers along with the quantized audio data 103. In one embodiment, the signal combiner 112 and the arbiter 108 may utilize a time-division multiplexing protocol (TDMA) to combine the quantized audio data 113 and the video signal 111, which will be discussed later.
[0040] Referring here to Figure 2 in relation to Figure 1, Figure 2 shows an exemplary A / V signal 216 according to one or more embodiments. The A / V signal 216 may correspond to the A / V signal 116 in Figure 1. In some embodiments, the A / V signal 216 shown in Figure 2 may correspond to a single active row of a single video frame of the video signal 111. As shown in Figure 2, the A / V signal 216 may include video data 211 and a video blanking section 210. The video data 211 may correspond to the video data of the video signal 111 generated or received by the video encoder 110, as described above. The video blanking section 210 of the combined A / V signals 116, 216 may include one or more audio pulses 213, an audio header 215, and a synchronization pulse 216. As shown in Figure 2, the video blanking section 210 may be before or after the synchronization pulse 216. In some embodiments, the synchronization pulse 216 may include a horizontal synchronization pulse, a vertical synchronization pulse, a color synchronization pulse, and / or other data corresponding to the CODEC of the video signals 116,216.
[0041] In some embodiments, the arbiter 108 may be configured to determine the synchronization pulse position of the synchronization pulse 216 in the video blanking section 210. The arbiter 108 may be configured to identify, without calculation, at least one allowable portion of the blanking section corresponding to a first duration exceeding a predetermined duration of the audio header and one or more quantized audio pulses.
[0042] In another embodiment, the arbiter 108 may be configured to identify, without calculation, a synchronization portion configured to include one or more synchronization pulses based on a predefined video standard, wherein the synchronization portion is different from at least one allowable portion of the blanking section. In some embodiments, the A / V signal 216 may include other data corresponding to the synchronization pulse and / or video data 211 within the video blanking section 210. In one embodiment, identifying the synchronization pulse location may include digitizing an existing analog video signal (e.g., via the ADC 104) and identifying the synchronization pulse location and blanking section from the digitized video stream.
[0043] In another embodiment, the video signal 111 may include a predefined digital format having video blanking sections without synchronization pulses, and determining the location of the synchronization pulses may involve communication between the video encoder 110 and the audio arbiter 108 regarding where (i.e., the temporal location) the synchronization pulses 216 are inserted according to a predefined video standard. For example, the video encoder 110 may combine or modulate active YUV video data and include synchronization data to form a single video data stream for transmission. In some embodiments, the arbiter 108 may include an audio encoder and / or video encoder (not shown) having a pixel counter. Utilizing the pixel counter, the arbiter 108 may determine the insertion locations of predefined synchronization and audio data based on the pixel counter's count value.
[0044] In one embodiment, since the transmission of a digital video signal in a digital format does not require synchronization or color synchronization data, identifying the location of the synchronization pulse and the blank sections without a synchronization pulse may involve communication between the video encoder (e.g., combiner 112) and the audio arbiter 108 regarding where the synchronization pulse 216 is inserted according to a predefined video standard. Therefore, determining the location of the synchronization pulse may involve determining one or more portions of the video blanking section 210 that do not contain video-related signals corresponding to the video data 211 (e.g., horizontal / vertical / color synchronization pulses and / or other data corresponding to the video data 211). However, analog video transmissions do contain synchronization pulses, and the temporal location of the synchronization pulse present in the video data 211 can be determined by digitizing the analog video stream (e.g., via an ADC 104). In another embodiment, the arbiter 108 may be configured to determine the location of other data within the blanking section (e.g., horizontal / vertical / color synchronization pulses and / or other data corresponding to the video data 211) based on a predefined video format.
[0045] In some embodiments, the video blanking section 210 may include an audio header 215 and one or more quantized audio pulses 213. A single quantized audio pulse 213 may correspond to a single digitized audio data (e.g., 8 / 10 / 12 / 16-bit format) stored as a single buffer entry. The level (i.e., amplitude or voltage level) of the audio pulse 213 can linearly represent the digital data corresponding to the quantized / digital audio data 103 stored in buffer 106. In one embodiment, for example, for 8-bit sample data, 00hex includes the lowest level audio pulse 213, and FFhex includes the highest level audio pulse 213. In some embodiments, other hexadecimal leveling protocols can be utilized for various video formats without departing from the scope of the embodiments described herein. In some embodiments, the duration of the quantized / digital audio pulse 213 may be predefined based on the specific video format of signals 116, 216.
[0046] As shown in Figure 2, the audio header 215 and the quantized / digital audio pulse 213 each contain a predetermined pulse duration (i.e., pulse width). Because audio pulses can be distorted at the far end of the cable (i.e., the receiver side) due to their imperfect frequency characteristics, the pulse duration needs to be appropriately selected. In some embodiments, the duration of each audio pulse 213 can be programmed and predetermined for each video format based on the available blank space and cable length. Each video blanking section 210 of the signal 216 may have the same size / duration due to the periodic nature of the video blanking section in each row of each frame of the video signal 216. Thus, the duration or pulse width of the quantized / digital audio pulse 213 and the audio header 215 can be predetermined for a particular video format such that the duration of at least one audio header 215 and one quantized / digital audio pulse 213 fits within the blanking section 210. In some embodiments, based on a specific predefined video format, the duration of the audio header 215 and the quantized / digital audio pulses 213 are selected such that the video blanking interval includes at least two quantized / digital audio data pulses 213.
[0047] For example, some video formats (e.g., the SMPTE standard mentioned above) can include wider video blanking intervals that provide sufficient space for two or more quantized / digital audio pulses 213. In some embodiments, to accommodate inserting two or more quantized / digital audio pulses 213 into each blanking interval 210, the pulse duration of the quantized / digital audio pulses 213 and / or audio header 215 can be selected to be smaller, based on the duration of the video blanking interval 210 for a particular video format. Doing so allows for an increase in the transfer data rate when two or more audio pulses 213 can be inserted into the video blanking interval 210.
[0048] As described above, the quantized / digital voice pulse 213 may include an analog representation of the quantized voice data 102 stored in the buffer 106, referencing the header level of the voice header 215. In some embodiments, the voice header 215 may include the maximum signal value or an intermediate level signal value of the quantized voice pulse 213. In one embodiment, the voice header 215 may be configured to function as a flag indicating valid voice data and as a reference level for decoding the quantized voice pulse 213 and for a downstream receiver to reconstruct the voice data into its original format, which will be discussed in more detail in the considerations in Figure 3.
[0049] In one embodiment, the audio header 215 corresponds to a voltage level different from that of the video blanking section 210 and can be easily identified by a downstream audio receiver. The voltage level of the audio header 215 may be configured to be easily distinguishable in the video blanking section 210, facilitating a downstream receiver to determine that a particular video blanking section 210 does not contain audio data. In one embodiment, the audio header 215 includes a zero audio level or an integer multiple of the zero audio level. The audio header may function as a zero reference for decoding the quantized audio pulses 213 into bidirectional AC waves. In some embodiments, one or more quantized audio pulses 213 are superimposed (i.e., interleaved or multiplexed) with a DC offset.
[0050] In one embodiment, the audio header 215 may correspond to the DC offset during transmission, and the audio header 215 may be used in a downstream video receiver to restore the DC offset. During the transmission of signal 216, the voltage level of the audio pulse 213 may change for various reasons. The quantization level on the receiver side may also differ from the original DAC conversion level. Therefore, two different levels are required to accurately restore the audio digital value of the audio pulse 213. Thus, in some embodiments, the audio header 215 may correspond to a predefined header level and a video blank level as a reference for appropriately scaling the received audio level. Thus, in some embodiments, the audio header 215 may include a value that is the half-amplitude or full-amplitude of the audio signal in addition to the DC offset to assist decoding by a downstream transmitter, for example, as described in Figure 3A below.
[0051] In some embodiments, an arbiter 108 communicating with a signal combiner 112 may be configured to interleave an audio header 215 and one or more audio samples (or a portion thereof, if digital) 213 at a predetermined time in the video blanking section 210. In one embodiment, the interleave includes multiplexing the audio header 215 and one or more audio pulses 213 with at least one portion of the video data 211 in an allowable portion of the blanking section 210, resulting in a combined audio / video signal 216 representing at least a portion of the audio and video data. Figure 2 shows the audio header 215 and audio samples 213 positioned between the video data 211 and the synchronization pulse 216, but in some embodiments, the header 215 and audio samples 213 may be inserted before or after the synchronization pulse 216. In some embodiments, the audio header 215 and audio samples 213 may be included both before and after the synchronization pulse 216.
[0052] Referring next to Figure 3, in relation to Figures 1 and 2, Figure 3 shows an exemplary audio decoder system 300 corresponding to the audio encoder system 100 of Figure 1. The audio decoder system 300 working in conjunction with the audio encoder system 100 can constitute an audio / video CODEC system. The decoder system 300 may include an ADC 302, an audio detection module 304, an audio extraction module 306, and a FIFO buffer 303. As shown in Figure 3, the ADC 302 may be configured to receive A / V signals 116, 216, and 316 in analog form and to convert the analog A / V signals 116, 216, and 316 into the digital domain. The ADC 302 may be configured to output the A / V signals 116, 216, and 316 in digital form as a digital A / V signal 313. The ADC 302 may be configured to output the A / V signal 313 to the audio detection module 304, the audio extraction arbiter 306, and a video transmission destination (not shown in Figure 3). In some embodiments, the decoder system 300 may be configured to receive analog video signals 116, 216, 316, including an audio header 215 and quantized audio pulses 213 placed in the video blanking section 210 of the analog video signals 116, 216, 316, based on a predetermined sampling frequency.
[0053] In some embodiments, the audio extraction arbiter 306 may include a video decoder synchronous slicer (not shown in Figure 3). Detecting the presence of an audio header may involve using information from the video decoder's synchronous slicer function to assist in detecting the audio header 215 at one or more predefined position / time intervals according to the video transmission protocol and the resolution of the A / V signal 313. In one embodiment, synchronous position detection enables a predetermined search range for detecting the presence of an audio header. The search range may include time intervals of the video signal 111 that include video blanking sections. In other embodiments, other functions of the video decoder may assist in detecting the audio header 215.
[0054] In some embodiments, the audio detection module 304 may be configured to detect the presence of an audio header 215 included in the video blanking section 210. In response to the detection of the audio header 215, the audio detection module 304 may be configured to communicate the presence of audio data to the audio extraction arbiter 306. In response, the audio extraction arbiter 306 may be configured to extract audio samples 213 from the video blanking section 210 by extracting digitized AV samples 313 at predetermined timing intervals corresponding to the positions of the header 215 detected by the audio detection module 304. In some embodiments, the audio extraction arbiter 305 may be configured to convert the extracted audio data back to its original value by referencing the header level of the header 215. In one embodiment, the audio extraction section 306 utilizes the header level to restore the DC offset value added to the quantized audio data 103 during transmission. This prevents the quantized audio data from being identified as a horizontal synchronization signal when the signal swings low. Next, the audio extraction unit 306 can be configured to send the restored audio data samples to the buffer 303 for later retrieval. In one embodiment, the audio extraction unit 306 may include a local audio clock encoder (not shown in Figure 3) having substantially the same corresponding audio sampling frequency as employed on the transmitter side. Substantially the same means that the difference is negligible. In some embodiments, reconstructing a continuous audio signal involves using a predetermined audio sampling rate corresponding to the sampling rate of an upstream audio encoder (e.g., audio encoder system 200). Note that the terms “upstream” and “downstream” are used for illustrative purposes and specifically indicate a particular direction.
[0055] In one embodiment, the audio extraction unit 306 may include a clocking function for synchronizing the reconstruction of audio data. The audio extraction arbiter 306 may include one or more digital signal processors (DSPs), field-programmable gate arrays (FPGAs), and / or application-specific integrated circuits (ASICs). In one embodiment, the clock circuit of the audio extraction arbiter 306 may include one or more of the following, individually and / or in combination: a phase-locked loop (PLL) clock, a non-PLL clock, a crystal oscillator, and an LC resonant tank circuit. The local audio clock circuit of the audio extraction arbiter 306 may include additional timing and signal conditioning functions, for example, a fan-out buffer, multipliers and dividers, and a combiner, individually and / or in combination. The local audio clock is used to periodically extract audio data from the FIFO buffer 306, reconstruct the original audio signal in digital format, and output a continuous audio signal 303. However, because the clock references of the transmit and receive clocks are different, there may be a clock frequency difference that causes variations in sampling frequency between the transceiver and the receiver, which may require correction.
[0056] In some embodiments, the receiver's reconstruction clock frequency should be substantially the same as the transmitter's sampling clock frequency. Substantially the same means that any difference, if any, is negligible. For example, these differences stem from the fact that the receiver and transmitter each have independent clock references. Even if the specified frequencies are the same on both sides, they cannot be made identical or synchronized due to the free-operating nature of these clocks. In some embodiments, if the transmit sampling frequency and the receive sampling frequency are not substantially the same, interpolation may be performed on the received data to reconstruct the original audio signal at the receiver's sampling rate, preventing overrun or underrun of the FIFO buffer over time due to the difference in sampling frequencies. This reduces the size of the FIFO buffer. Interpolation, also known as rate matching, eliminates the need for complex and resource-intensive synchronization of audio and video signals, and also eliminates the need to consider video frame boundaries. Furthermore, according to one or more embodiments, interpolation eliminates the need to calculate and buffer audio data based on frame periodicity. Rather, the difference in sampling frequency between the transmitter and receiver can be compensated for by interpolating the received audio sample according to the difference (for example, the frequency difference between the receiver and transmitter).
[0057] Referring now to Figure 4, which shows a flowchart of an exemplary method 400 according to one or more exemplary embodiments described herein. Method 400 may begin with step 402, which involves receiving an audio signal in analog or digital format. In step 404, the analog audio signal is quantized in digital format, or the digital portion is received. In step 406, one or more quantized audio data are buffered. In step 408, a video signal is received, which includes a blank section and a video section. In step 410, an audio header and at least a portion of the buffered audio data are combined into the blank section of the video signal. In step 412, the combined digital audio / video signal is converted to analog, and the combined audio / video signal, which includes the audio header and at least a portion of the buffered audio data, is transmitted to a downstream video receiver, where the header data is detected, extracted, and used to reconstruct the audio signal. The method may then end.
[0058] Referring now to Figure 5, which shows a flowchart of an exemplary method 500 according to one or more exemplary embodiments described herein. Method 500 may begin with step 502 of receiving an analog video signal, which includes one or more audio headers and one or more quantized audio pulses corresponding to quantized audio data or digital data portions. For example, each of the one or more audio headers is followed by one or more quantized audio pulses for a video blanking time. Step 504 is to detect the audio headers within the blanking section of the analog video. Step 506 is to determine the reference level of the audio headers. Step 508 is to extract one or more quantized audio pulses in response to the detection of the audio headers. Step 510 is to convert one or more quantized audio pulses to their original values based on the reference level of the audio headers. Step 512 is to store one or more of the quantized audio data in a first-in, first-out (FIFO) buffer at their original values. In step 514, a continuous audio signal is reconstructed from one or more quantized audio data of the original values using the FIFO buffer, local audio clock, and extrapolation unit. This may then terminate the method.
[0059] As described above, Figures 1 to 5 illustrate several embodiments in which digitally sampled audio data or digital audio data is converted to an analog voltage level. The analog voltage level is set proportionally to the digital sample value and superimposed on the analog video signal in a predetermined time slot of the horizontal blanking period following the start pulse. However, there are limitations to the dynamic range and SNR (signal-to-noise ratio) that can be achieved, due to the resolution limitations of the DAC (digital-to-analog converter) on the transmitting side and the ADC (analog-to-digital converter) on the receiving side. As a result, the SNR decreases, and background noise is introduced into the performance of the audio channel. Improved audio performance can be achieved by using high-resolution (14 bits or more) DACs and ADCs. However, this approach is resource-intensive. However, by transmitting the audio sample data in a native digital format according to one or more embodiments described herein, the SNR can be mitigated, and high-quality audio channel performance can be achieved while also saving resources.
[0060] Referring next to Figures 6 and 7, in relation to Figures 1 and 3, Figures 6 and 7 show transceiver systems having a video transmitter 600 and a video receiver 700, respectively. For example, Figure 6 may be similar to Figure 1, but differs in that the data output from the arbiter represents analog or digital pulse data. The block diagrams of the video transmitter 600 and video receiver 700 may be similar to those of the transmitters / receivers 100 and 300 shown in Figures 1 and 3, where the same reference numerals and units and tens digit numbers correspond to similar structures with similar functions. The transmitters / receivers 600 and 700 are configured to transmit audio data over the video signal by transmitting the audio data in a native digital format (e.g., a 12-bit or 24-bit serial digital data stream) instead of the analog level representation described in Figures 1 to 5 above. For example, in some embodiments corresponding to voice-grade audio, 13 to 16 bits are usually sufficient. In other embodiments with high-definition audio, 24 bits may be used.
[0061] In some embodiments, the video transmitter 600 (also simply called the “transmitter”) may include an ADC 604, a FIFO 606, an arbiter 608, a video formatter 610, a combiner 612, and a DAC 614. As shown in Figure 6, the ADC 604 receives an analog audio signal 602 and a clock signal 607 as inputs. The clock signal 607 may correspond to a phase-locked loop (PLL) (e.g., PLL 605) configured to provide clock functionality. In some embodiments, the PLL 605 may be part of the video transmitter 600 or it may be separate from the video transmitter 600 and wirelessly communicate with the video transmitter 600. For example, in some embodiments, the video transmitter 600 may be configured to receive the clock signal 607 from an external source (not shown). The clock signal 607 has a frequency F S It operates with F S It can include 8, 16, 32, 44.1, 48, and 96 kHz.
[0062] In some embodiments, the operation of the video transmitter 600 may include receiving an analog audio signal 602 via the ADC 604 or a digital audio signal as input. The ADC 604 uses a sampling clock signal 607 to generate F S The analog audio signal 602 is sampled periodically at a rate of F. The sampled audio data or digital audio data is then sent to the FIFO 606 and stored in chronological order. Since the extracted audio data samples are not necessarily sent periodically, the FIFO buffer 606 also facilitates aperiodic insertion of audio samples. For example, the audio signal is F S It can sample periodically at a rate of F. The FIFO buffer actually facilitates the aperiodic insertion of audio samples into the video blanking time available for the audio slot. (The period of the video line is F. SIt is not synchronized. Some of the vertical blanking lines are reserved for other data communications and are not available for voice transmission. The FIFO buffer 606 outputs the stored voice samples in response to command signals from the arbiter 608.
[0063] In some embodiments, the arbiter 608 uses timing signals 611 to check whether audio data samples stored in the FIFO buffer 606 are available and whether a video blanking period from the video encoder / formatter is available. Checking whether audio data stored in the FIFO buffer 606 is available can be performed, for example, by command signals including polling / calling functionality of the arbiter 608. The arbiter 608 can read / call the audio data stored in the FIFO buffer 606, format the digital audio data for transmission during the blanking time, and thereby output a digital audio stream 609. Formatting the audio data to generate the digital audio stream 609 may include generating a start code header in serial bit format.
[0064] In some embodiments, as shown and described in Figure 1, the video formatter 610 may receive a video input signal 603 as input and output a digital video signal 613 to the combiner 612. As described above, the video formatter 610 transmits a timing signal 611 to the arbiter 608. For example, in some embodiments, the arbiter 608 may format the audio packet by adding a digital audio header and a serialized audio data bitstream according to a predefined format, or it may digitally encode or add redundant bits for error checking and correction purposes to improve the reliability of transmission in the presence of noise. Each bit in the stream is a fixed number of bytes / codes of the same digital data that, when converted to the analog domain, represent the height and duration of a digital bit pulse. This is similar to audio transmission, but the height of the pulse is usually fixed because the audio data information is transmitted digitally as a high-speed 1 / 0 bitstream rather than a low-speed, variable-height pulse. Each variable-height pulse in analog transmission corresponds to 13 to 16 consecutive shorter pulses of 1s and 0s in digital transmission, with optional redundant bits added.
[0065] Along with the start code header, the digital audio stream 609 may be combined with the digital video signal 613 by superposition via the combiner 612. The superposition of the digital audio stream 609 may be performed by utilizing the timing signal 611 and implementing time division multiplexing (TDMA). For example, the timing signal 611 facilitates the transmission of the digital audio stream 609 in mutually exclusive time slots. The combined digital audio / video data 615, which includes both audio and video data in the digital domain, is then converted to the analog domain by the DAC 614. Thus, the DAC 614 outputs an analog audio / video (A / V) stream 616 for downstream transmission to the video receiver 700.
[0066] Referring now to Figure 7, which shows a video receiver 700 (also simply referred to as the “receiver”) according to several embodiments described herein. As shown in Figure 7, the video receiver 700 includes an ADC 704, an audio extraction unit 708, a FIFO buffer 706, a data synchronizer 712, and a video decoder 710. The video receiver 700 receives an analog A / V stream 616 as input via the ADC 704. The analog A / V stream 616, which contains both audio and video data, is converted to the digital domain by the ADC 704. After converting the analog A / V stream 616 to the digital domain, the ADC 704 outputs the A / V stream 616 to the audio extraction unit 708 and the video decoder 710.
[0067] The video decoder 710 receives the A / V stream 616 via the ADC 704 and decodes and outputs a baseband video signal (YUV) 713 for display to the user. The video signal 713 corresponds to the original video input signal 603. The video decoder 710 transmits a timing signal 711 to the audio extraction unit 708. In some embodiments, the video decoder 710 can utilize time-division multiplex access to process and output the digital video signal 713. The video decoder 710 decodes the video signal and provides the necessary timing information to the audio extraction unit 708. In some embodiments, the extraction unit 708 may be separated into an audio (header) detection unit and an extraction unit, similar to the corresponding functions in analog systems. Based on the timing signal provided by the video decoder 710 and the detected digital audio header, the audio extraction unit 708 extracts audio data from the A / V stream 616 and outputs the audio data to the FIFO buffer 706.
[0068] Since the video blanking period (i.e., the video line period) is not the same as the audio sample period, the extracted audio data samples do not arrive periodically, so the extracted audio samples may be stored in the FIFO buffer 706. The FIFO buffer 706 outputs the stored audio data to the synchronizer 712. In some embodiments, the synchronizer 712 operates at the frequency of the clock 605 (F S ) and a signal frequency (F at a rate substantially the same as S' ) receives a clock signal from the PLL 705. As described above, the transmitter and the receiver are usually installed at locations distant from each other and connected only by a single video cable. This is desirable to reduce cost and improve efficiency, so that wiring between the transmitter (such as a camera) and the receiver (such as a display / recorder) can be achieved with only one low-cost cable. Each side has its own free-running reference clock, and it cannot be guaranteed that the frequencies and phases are completely identical or synchronized. Therefore, only substantially the same frequency can be expected. Even if both sides are operating at 8 KHz, they will not be the same. In some embodiments, the synchronizer 712 retrieves the audio data samples stored in the FIFO buffer 706, and processes the signal F S' to calculate the correct data at its output based on. Based on the corrected output data, the data synchronizer 712 outputs a digital audio stream 716. In some embodiments, calculating the correct data at the output is performed by checking for the presence or absence of input data samples for F S and F S' based on constantly tracking the difference between, performed by extrapolation from adjacent samples. For example, if F S' is faster, there will be no samples left to use.
[0069] Referring here to Figures 6-7 and Figure 8, instead of transmitting audio using the digital-to-analog converted analog level corresponding to the digital code (as described in Figures 1-5 above, for example), a downstream audio transmission waveform 800 can be used. As shown in Figure 8, the waveform 800 may include a video blanking interval 802 containing a synchronization pulse 816, a digital start code pattern 804, audio data 806, and a digital end code pattern 806. For example, in some embodiments, the arbiter 608, together with the video formatter 610, manages the detection of the video blanking interval 802 in the A / V stream 616. This can be done in the same or similar manner as the video encoder 110 and arbiter 108 described in the discussion of Figures 1-5 above. In some embodiments, the A / V stream 616 is the final output generated by the arbiter 608 and the video formatter 610. The arbiter 608 acquires timing information, including, for example, H / V synchronization, line count, and pixel count, to determine an appropriate video blanking interval in which audio data packets can be inserted.
[0070] In some embodiments, the format of the transmitted audio data corresponding to the A / V stream 616 may correspond to a native serial digital data stream. For example, a start code pattern 804 may be transmitted first, followed by sampled digital audio data 806 in a serial bitstream format with optional redundant bits for error correction. In one embodiment, the digital audio data 806 may be in a native format where the most significant bits are transmitted first. In another embodiment, the digital audio data 806 may be in a native format where the least significant bits are transmitted first. In some embodiments, the arbiter 608 and the audio extraction unit 708 may or may not have error correction functionality when processing the digital audio data 806.
[0071] In some embodiments, the A / V stream 616 may be encoded for various purposes, including facilitating user communication channels between upstream and downstream users. In some embodiments, other types of data payloads can be implemented (e.g., video or metadata related to control data). In these embodiments, maintaining periodicity may not be critical, and a receiver-side synchronizer / extrapolator may not be required.
[0072] As will be described in more detail below, in some embodiments, two or more digital audio data samples may be transmitted based on a predefined protocol, depending on the blanking space (i.e., vertical blanking space or horizontal blanking space). For example, in the analog domain, one audio sample is represented by one pulse of variable height. By adding additional sample pulses after the first sample, more audio samples can be transmitted under the same header. This is also true for the digital domain. The number of samples transmitted together must be predefined so that the receiver knows how many samples it should receive. If the number of samples is variable, a transmission end flag is required after a valid sample to indicate that there are no more valid samples. In some embodiments, transmitting native digital audio data superimposed on (analog) video can be extended to bidirectional simultaneous audio communication, which will be discussed later.
[0073] Next, referring to Figures 9 to 11, Figures 9 to 10 show embodiments corresponding to a video transmission protocol that includes the function of bidirectional simultaneous voice transmission by utilizing an upstream channel. The block diagrams of the video receiver 900 and video transmitter 1000 may be similar to those of the transmitter / receiver 600, 700 shown in Figures 6 and 7, and parts and numbers that are similarly labeled with respect to the ones and tens digits correspond to similar structures having similar functions. Accordingly, Figure 9 shows a block diagram of the video receiver 900 including a reverse-direction voice signal transmitter configured for upstream (reverse) voice transmission, and Figure 10 shows a block diagram of the video transmitter 1000 including a reverse-direction signal receiver configured for upstream (reverse) voice reception.
[0074] The audio transmission and reception processing is similar to the downstream audio processing described above in Figures 6-8 (for example, similar to the transmitter / receiver 600 and 700), but because video transmission is unidirectional (downstream), the methods for inserting and extracting audio data in the upstream / reverse audio transmitter 901 and the upstream audio receiver 1001 differ. For example, in this TDMA operation, the space available for other uses not occupied by video data and video synchronization timing is the video blanking time, either line (horizontal) blanking or field / frame (vertical) blanking. In the case of bidirectional transmission, as an exemplary implementation, one blanking time can be used downstream and the other blanking time can be used upstream. Other implementations may require other types of identification, such as using different headers to avoid confusion.
[0075] For example, typically, downstream data is transmitted during the horizontal blanking time, excluding the vertical blanking line. Upstream data is transmitted over a predefined vertical blanking line that is not used for any other purpose. Thus, the upstream audio transmission protocol described herein facilitates reverse transmission (e.g., upstream transmission from video receiver 700 to video transmitter 600) by utilizing the vertical blanking line of the analog video input signal 603 that is not used for video transmission or downstream audio transmission. Generally, these occupy mutually exclusive lines. That is, the horizontal blanking time on the vertical blanking line used for upstream transmission is not used for downstream transmission. Due to this fact, and the fact that the frequency of the video line period differs from the audio sample frequency, upstream audio transmission is also not periodic. Upstream audio data is collected over the frame period before transmitting everything during the vertical blanking line, and the transmission is also not periodic. Therefore, at the receiving end, the local F S' It is necessary to use this to convert it back into a periodic signal.
[0076] As shown by dashed lines in Figures 9 and 10, in some embodiments, the video receiver 900 includes an upstream / reverse audio transmitter 901. The video receiver 900 is located on the video receiver side (e.g., corresponding to video receiver 700). The video transmitter 1000 includes an upstream / reverse audio receiver 1001. The video transmitter 1000 is located on the video transmitter side (e.g., video transmitter 600). The upstream audio transmitter 901 and the upstream audio receiver 1001 are configured to implement bidirectional simultaneous audio transmission, details of which will be described later.
[0077] Figure 11 shows an exemplary upstream audio transmission waveform 1100 according to several embodiments described herein. To achieve the minimum audio sample rate (e.g., 8K, 16K, or 32 samples per second), there are limitations on the number of vertical / horizontal blanking lines per frame of the video signal; therefore, multiple digital audio samples may be transmitted per horizontal line. As shown in Figure 11, transmission on each video blanking line 1102 is initiated by a digital start pattern 1104 following horizontal synchronization (HS). Following the digital start pattern 1104 are multiple digital audio data samples 1106. N This may continue. In some embodiments, audio data sample 1106 N An termination pattern 1108 may be implemented to indicate the end of a signal. In some embodiments, the termination pattern 1108 may be used when the number of audio samples differs from line to line. In other embodiments, a fixed number of samples per line is predetermined. If a fixed number of samples per line is predetermined, an termination pattern is not necessary.
[0078] In some embodiments, the arbiter 908 of the upstream audio transmitter 901 uses the video decoder 910 to determine the appropriate vertical blanking line for the received video (corresponding to 603 in Figure 6) and the time slot for transmission within that vertical blanking line. For example, the arbiter 908 retrieves audio data from the FIFO buffer 906 and obtains a start code 1104 and one or more audio data samples 1106. N The signal can then be formatted and transmitted appropriately. The start and / or end code patterns 1106, 1108 of each line are inserted by the combiner 912 before being electrically superimposed on the input video signal (corresponding to 603 in Figure 6). Such insertion allows the start and / or end codes 1104, 1108 to be detected at the video transmitter by the upstream audio receiver 1001, as described in Figure 10.
[0079] Returning to Figure 10, in some embodiments, the upstream audio receiver 1001 of the video transmitter 1000 obtains information from the video formatter 1010 to determine which video blanking lines to monitor, according to a predefined protocol. In some embodiments, the predefined protocol includes the start code format, bit pulse duration, number of bits per sample, number of samples per line (for a fixed number), end code, signed / unsigned, and bit order (MSB or LSB first). The upstream audio receiver 1001 then uses the audio extraction unit 1008 to extract digital audio samples. After detecting the start code 1104, the audio extraction unit 1008 extracts audio data samples 1106 N It may also be extracted. When start code 1104 is detected, the audio extraction unit 1008 extracts audio data sample 1106 N The data is sent to the FIFO buffer 1006. The FIFO buffer 1006 stores the sampled audio data in time series so that it can be reconstructed by the synchronizer 1012. In some embodiments, the synchronizer 1012 retrieves the data from the FIFO buffer 1006 and transmits it at a transmission rate F S The received audio clock rate F is essentially the same as S' The audio output samples are calculated based on this, and in this calculation, interpolation, also known as rate matching as mentioned earlier, can be used. Being substantially the same means the frequencies are the same, but depending on the accuracy of the reference clock, there may be a difference of several hundred ppm (i.e., the difference is negligible). Also, the clocks are not synchronized, and the phase and frequency relationships are not fixed. For example, the calculation here is similar to that described above for the downstream receiver. The data is not periodic in the time domain and is considered to be packetized. Regarding interpolation, the audio data rate is matched between two substantially identical but independent sampling clocks (e.g., PLL605 in Figure 6 and PLL705 in Figure 7) of the transmitter and receiver. On the audio receiver side (having an upstream audio receiver 1001 of the video transmitter 1000; see also Figure 6), resampling of the input audio data sampled by the transmitter clock (e.g., PLL605 in Figure 6) is required to reconstruct the audio data based on a local free-running audio clock (not shown). An arbiter (e.g., arbiter 608 in Figure 6) monitors the rate difference between the input data stored in FIFO 1006 and the target output sampling rate from which the data is extracted from FIFO 606. The resampling position of the data at the output of FIFO 1006 is dynamically adjusted by synchronizer 1012 based on this rate difference, so that the original audio signal is correctly reconstructed using the local free-running audio clock. The figure below illustrates this interpolation.
number
[0080] Although the present invention has been described in detail with reference to its preferred embodiments, it will be readily apparent to those skilled in the art that modifications and alterations of form and detail can be made without departing from the spirit and scope of the exemplary embodiments described herein. The appended claims are intended to encompass such modifications and alterations.
Claims
1. A method for transmitting audio over analog video data to a receiver using a single cable, A process of receiving a digital video signal and an audio signal with a transmitter, wherein the audio signal is either an analog audio signal or a digital audio signal, If the audio signal is an analog audio signal, the process involves sampling the audio signal using an analog-to-digital audio converter (ADC), The steps include storing the sampled analog audio signal or the digital audio data corresponding to the digital audio signal in a first-in, first-out (FIFO) buffer, The arbiter reads the digital audio data in the first-in, first-out buffer, The process of formatting serialized audio bits having a digital start code from the aforementioned digital audio data, A step of generating a combined digital audio / video signal by inserting the serialized audio bits having the digital start code into the blanking period of the digital video signal, A process of generating a combined analog audio / video stream containing native format audio data by converting the combined digital audio / video signals to analog using a digital-to-analog converter (DAC), The process of transmitting the combined analog audio / video stream to the receiver in one direction over the single cable, A method for providing this.
2. The receiver receives the combined analog audio / video stream, The process of converting the combined analog audio / video stream into a combined digital audio / video signal using another analog-to-digital converter included in the receiver, The method according to claim 1, further comprising the step of extracting an audio data stream from the combined digital audio / video signals.
3. The process involves storing the aforementioned audio data stream in a separate first-in, first-out buffer, A step of acquiring audio data samples from another first-in, first-out buffer at substantially the same rate as the audio sampling frequency of the transmitter, and periodically reconstructing the audio data samples into a continuous audio stream, The process of outputting the continuous audio stream The method according to claim 2, further comprising:
4. The method according to claim 3, wherein the acquisition and reconstruction step involves interpolating the audio data sample based on the frequency difference between the receiver and the transmitter.
5. The method further includes a bidirectional audio transmission method wherein the sampled audio signal is sampled at a first frequency, the transmitter includes an audio signal receiver in the opposite direction to the one direction, and the receiver includes an audio signal transmitter in the opposite direction to the one direction. The process includes receiving a second audio signal, which is either a second analog audio signal or a second digital audio signal, with the opposite audio signal transmitter included in the receiver, The process involves determining one or more vertical blanking lines of the coupled analog audio / video stream and one or more time slots without audio data within the one or more vertical blanking lines, using the arbiter of the audio signal transmitter in the opposite direction and the video decoder of the receiver. If the second audio signal is the second analog audio signal, the process involves sampling the second analog audio signal at a second frequency using another analog-to-digital converter (ADC) located in the audio signal transmitter in the opposite direction, A step of storing a second digitized audio sample corresponding to either the second digital audio signal or the sampled second analog audio signal in another first-in, first-out (FIFO) buffer located in the audio signal transmitter in the opposite direction, The process involves inserting a start pattern, one or more second serialized audio data, and an end pattern into the vertical blanking section of the combined analog audio / video stream. A step of transmitting an audio stream corresponding to the start pattern and / or end pattern and the second serialized audio bit to the transmitter, including the reverse audio signal receiver, via the reverse audio signal transmitter located in the receiver, to the reverse audio signal receiver located in the transmitter, wherein the transmission of the audio stream is performed in a different direction opposite to the one direction via the single cable. The method according to claim 1, comprising:
6. The process of receiving the audio stream using the audio signal receiver in the opposite direction, which is located inside the transmitter, A step of extracting the second serialized audio data based on the one or more vertical blanking lines and the one or more time slots to obtain a second extracted audio sample, The process includes storing the second extracted audio sample in yet another first-in, first-out (FIFO) buffer located within the audio signal receiver in the opposite direction, A step of selecting an output sample frequency that is substantially the same as the sampling frequency of the audio signal transmitter in the opposite direction, The steps include acquiring the aforementioned audio sample and outputting an audio stream reconstructed based on the aforementioned output sample frequency. The method according to claim 5, further comprising:
7. The method according to claim 6, wherein the acquisition and outputting steps are performed by interpolating the audio sample based on the frequency difference between the receiver and the transmitter.
8. The method according to claim 1, wherein the step of reading the digital audio data by the arbiter is performed in response to the detection that data in the first-in, first-out buffer is available.
9. The receiver receives the combined analog audio / video stream, The process of converting the combined analog audio / video stream into a combined digital audio / video signal using another analog-to-digital converter included in the receiver, The method according to claim 8, further comprising the step of extracting an audio data stream from the combined digital audio / video signals.
10. The process involves storing the aforementioned audio data stream in a separate first-in, first-out buffer, A step of acquiring audio data samples from another first-in, first-out buffer at substantially the same rate as the audio sampling frequency of the transmitter, and periodically reconstructing the audio data samples into a continuous audio stream, The process of outputting the continuous audio stream The method according to claim 9, further comprising:
11. The method according to claim 10, wherein the acquisition and reconstruction step involves interpolating the audio data sample based on the frequency difference between the receiver and the transmitter.
12. The method further includes a bidirectional audio transmission method wherein the sampled audio signal is sampled at a first frequency, the transmitter includes an audio signal receiver in the opposite direction to the one direction, and the receiver includes an audio signal transmitter in the opposite direction to the one direction. The process includes receiving a second audio signal, which is either a second analog audio signal or a second digital audio signal, with the opposite audio signal transmitter included in the receiver, The process involves determining one or more vertical blanking lines of the coupled analog audio / video stream and one or more time slots without audio data within the one or more vertical blanking lines, using the arbiter of the audio signal transmitter in the opposite direction and the video decoder of the receiver. If the second audio signal is the second analog audio signal, the process involves sampling the second analog audio signal at a second frequency using another analog-to-digital converter (ADC) located in the audio signal transmitter in the opposite direction, A step of storing a second digitized audio sample corresponding to either the second digital audio signal or the sampled second analog audio signal in another first-in, first-out (FIFO) buffer located in the audio signal transmitter in the opposite direction, The process involves inserting a start pattern, one or more second serialized audio data, and an end pattern into the vertical blanking section of the combined analog audio / video stream. A step of transmitting an audio stream corresponding to the start pattern and / or end pattern and the second serialized audio bit to the transmitter, including the reverse audio signal receiver, via the reverse audio signal transmitter located in the receiver, to the reverse audio signal receiver located in the transmitter, wherein the transmission of the audio stream is performed in a different direction opposite to the one direction via the single cable. The method according to claim 8, comprising:
13. The process of receiving the audio stream using the audio signal receiver in the opposite direction, which is located inside the transmitter, A step of extracting the second serialized audio data based on the one or more vertical blanking lines and the one or more time slots to obtain a second extracted audio sample, The process includes storing the second extracted audio sample in yet another first-in, first-out (FIFO) buffer located within the audio signal receiver in the opposite direction, A step of selecting an output sample frequency that is substantially the same as the sampling frequency of the audio signal transmitter in the opposite direction, The steps include acquiring the aforementioned audio sample and outputting an audio stream reconstructed based on the aforementioned output sample frequency. The method according to claim 12, further comprising:
14. The method according to claim 13, wherein the acquisition and output steps are performed by interpolating the audio sample based on the frequency difference between the receiver and the transmitter.
Citation Information
Patent Citations
Digital signal converter
JP1999317916A
Signal transmitter and signal receiver
JP2002125207A
Data transmitting method, data receiving method, data transmitter, data receiver, digital signal transmission system, and data transmission / reception method
JP2004208337A
Method and apparatus for transmitting and receiving audio over analog video transmission over a single coaxial cable
US10750117B1
Method of transmitting data having insertion of audio information
US20080095518A1
