Transmission device, reception device, and transmission / reception system

By converting the vibration signal into a 16-bit half-precision floating point format and sending it using the frame structure of IEC 60958 standard, the problem of low arithmetic processing accuracy on the reception side is solved, and efficient audio and tactile signal transmission is achieved, ensuring the stability of the signal and high precision processing.

CN118923043APending Publication Date: 2024-11-08SONY GROUP CORP
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202380029510.9
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Priority Date
2022-03-30
Filing Date
2023-03-02
Publication Date
2024-11-08

AI Technical Summary

Technical Problem

The prior art is difficult to efficiently perform vibration signals, especially audio data and tactile vibration signals, on the receiving side, which leads to reduced accuracy or failure of processing.

Method used

Convert the vibration signal into 16-bit half-precision floating point format data and send it through the HDMI transmission path. It is arranged in the audio sample word or auxiliary sample bit area using the IEC 60958 standard frame structure to provide channel status information to ensure the correct reception side Identify and process.

Benefits of technology

Complex arithmetic operation processing on the receiving side is realized, which ensures high-precision transmission and processing of the signal, avoids sharp waveform fluctuations at zero crossing and unbalanced DC components, and improves the stability of the system and the audio reproduction quality.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118923043A_ABST
    Figure CN118923043A_ABST
Patent Text Reader

Abstract

The vibration signal is transmitted in such a manner that arithmetic operation processing can be advantageously performed on the receiving side. The conversion unit converts the vibration signal into data in a floating point format. For example, the vibration signal is an audio signal or a tactile vibration signal. For example, the data in the floating point format is 16-bit half-precision floating point data. The transmission unit transmits the floating point format data to the external device via the transmission path. For example, the transmission path is an HDMI transmission path. Further, for example, the transmission unit transmits the data in the floating point format using a transmission signal structure including each block for a plurality of frames for the audio signal.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present technology relates to a transmitting device, a receiving device, and a transceiver system, and particularly to a transmitting device that transmits a vibration signal to an external device via a transmission path so that complex arithmetic operation processing can be advantageously performed on the receiving side. Background Art

[0002] It has been considered that, for example, object audio data including sound data and position data related to a sound source is transmitted from the transmitting side, and realistic surround effect enhanced sound reproduction is performed on the receiving side. In addition, it has been considered that a tactile vibration signal is transmitted from the transmitting side as a tactile signal, and a tactile stimulation is provided to the user on the receiving side.

[0003] In this case, with respect to the object audio data, an arithmetic operation process of allocating the sound of the sound source to each of the plurality of speakers arranged in the space is applied on the receiving side based on the position of the sound source. In addition, with respect to the tactile vibration signal, an arithmetic operation process of adjusting the gain according to the sensitivity of each user, the sensitivity of the part to be stimulated, the sensitivity of the vibration device, and the nonlinearity of the vibration device is applied on the receiving side.

[0004] For example, in PTL 1, it is disclosed that when data in a floating point format is compressed, efficiency is improved by dividing processing into processing for an exponential part and processing for a decimal part.

[0005] Citation List

[0006] Patent Literature

[0007] [PTL 1] Japanese Patent Publication No. 2018-037891 Summary of the invention

[0008] [Technical issues]

[0009] In order to maintain high precision even if arithmetic processing is performed on the receiving side as described above, it is necessary to send a vibration signal (sound data related to the sound source in the object audio data, a tactile vibration signal as a tactile signal, etc.) from the sending side in a signal format suitable for arithmetic processing.

[0010] An object of the present technology is to transmit a vibration signal in such a manner that arithmetic operation processing can be favorably performed on the receiving side.

[0011] [Solution to the problem]

[0012] The present technology is conceived to provide a transmitting device including a conversion section that converts a vibration signal into data in a floating point format, and a transmitting section that transmits the data in the floating point format to an external device via a transmission path.

[0013] In the present technology, the conversion unit converts the vibration signal into data in a floating point format. For example, the vibration signal may be an audio signal or a tactile vibration signal. In addition, for example, the data in the floating point format may be 16-bit half-precision floating point (binary16) data. In this way, the data in the floating point format may express a range of values ​​from -65504 to 65504 and have 11-bit precision.

[0014] In addition, for example, the conversion section may convert the vibration signal into data in a floating point format by setting the maximum displacement of the vibration signal to a predetermined value, such as 1, which is smaller than the maximum value of the range of values ​​determined by the number of bits of the exponent part of the data in the floating point format. In this way, even if arithmetic operations on the data in the floating point format are continued on the receiving side, the range of values ​​has a margin, and therefore, arithmetic operations that process signals exceeding the maximum displacement (100%) of the vibration signal can be performed.

[0015] The transmitting unit transmits the data in floating point format to the external device via a transmission path. For example, the transmission path may be an HDMI transmission path. In addition, for example, the transmitting unit may transmit the data in floating point format using a transmission signal structure including each block of a plurality of frames for an audio signal. In this way, the data in floating point format obtained by converting the vibration signal may be advantageously transmitted to the external device.

[0016] In this case, for example, the transmission signal structure may be a frame structure of the IEC 60958 standard, and the transmitting section may arrange the data in the floating point format in the area of ​​the audio sample word or in the area of ​​the audio sample word and the auxiliary sample bit, and transmit the data in the floating point format. In addition, in this case, the data in the floating point format may be arranged in the area of ​​the audio sample word or in the area of ​​the audio sample word and the auxiliary sample bit in the order of the sign part, the exponent part, and the decimal part without intervals starting from the most significant bit side of the audio sample word. In this configuration, the exponent part in the data in the floating point format may include 5 bits, and the decimal part may include 10 bits, 14 bits, or 18 bits.

[0017] As a result of the arrangement without intervals starting from the most significant bit side, for example, as data in a floating-point format, when data in which the number of bits of the exponent part is not changed and only the number of bits of the decimal part is increased to improve the precision is sent, even if the receiving side erroneously processes the data when it believes that a number in a floating-point format with a smaller number of bits in the decimal part is arranged, only the precision is reduced and failure can be avoided.

[0018] For example, the channel status provided in the transmission signal structure of each block may include information indicating that the data in the floating point format is arranged in the area of ​​the audio sample word, or in the area of ​​the audio sample word and the auxiliary sample bit. In this way, the receiving side can easily recognize that the data in the floating point format is arranged in the area of ​​the audio sample word, or in the area of ​​the audio sample word and the auxiliary sample bit.

[0019] In addition, in this case, for example, the channel status provided in the transmission signal structure of each block may include information indicating the number of bits of data in a floating point format arranged in the region of the audio sample word, or the region of the audio sample word and the auxiliary sample bit. In this way, the receiving side can easily recognize the number of bits of data in a floating point format arranged in the region of the audio sample word, or the region of the audio sample word and the auxiliary sample bit.

[0020] In addition, for example, when the sign portion indicates a negative sign, the transmission section can apply bit inversion processing to the exponent portion, and then can transmit the floating point format data. In this way, even if the floating point format data is mistakenly recognized as signed integer format data on the receiving side, the sharp waveform fluctuation and unbalanced DC component at the time of zero crossing can be suppressed.

[0021] In addition, for example, when the sign portion of the data in the floating point format indicates a negative sign, the transmission section can apply a bit reversal process to the exponent portion, and perform a process of converting the decimal portion into a 2's complement, and then can transmit the data in the floating point format. In this way, even if the data in the floating point format is mistakenly recognized as data in the signed integer format on the receiving side, the sharp waveform fluctuation and unbalanced DC component at the time of zero crossing can be suppressed. In this case, by applying not only the bit reversal process to the exponent portion but also the process of converting the decimal portion into a 2's complement, the waveform at the time of a negative value can be made clean, compared with the case where the bit reversal process is applied only to the exponent portion.

[0022] As described above, in the present technology, the vibration signal is converted into data in a floating point format, which is sent to an external device via a transmission path, and therefore, complex arithmetic operation processing applied to the vibration signal can be advantageously performed in the external device.

[0023] Furthermore, another concept of the present technology resides in a receiving device including a receiving section that receives data in a floating point format obtained by converting a vibration signal from an external device via a transmission path, and a processing section that processes the data in the floating point format.

[0024] In addition, another concept of the present technology is a transceiver system, wherein a sending device and a receiving device are connected to each other via a transmission path, the sending device includes a conversion unit that converts a vibration signal into data in a floating point format, and a sending unit that sends the data in the floating point format to the receiving device via the transmission path, and the receiving device includes a receiving unit that receives the data in the floating point format from the sending device via the transmission path, and a processing unit that processes the data in the floating point format. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Figure 1 is a block diagram illustrating a configuration example of a transmission and reception system as a first embodiment.

[0026] Figure 2 is a diagram illustrating a frame structure of the IEC 60958 standard.

[0027] Figure 3 is a diagram illustrating a subframe structure of the IEC 60958 standard.

[0028] Figure 4 It is a diagram illustrating the structure of binary 16-bit data.

[0029] Figure 5 is a diagram illustrating the composition of frames in an audio stream having a frame structure of the IEC 60958 standard.

[0030] Figure 6 A diagram for describing binary 16-bit data (binary16) in comparison with signed integer 16-bit data (signed int16) is depicted.

[0031] Figure 7 is a diagram illustrating a case where signed integer 16-bit data (signed int16) and binary 16-bit data (binary16) are applied to an audio signal.

[0032] Figure 8 is a table schematically illustrating the format of channel status in the IEC 60958 standard.

[0033] Fig. 9 1 is a table illustrating an example of definition of "b" of the first bit of the 0th byte and "d" of the third to fifth bits of the 0th byte in the channel status.

[0034] Fig.10 is a table illustrating another example of the definition of "b" in the first bit of the 0th byte and "d" in the third to fifth bits of the 0th byte in the channel status.

[0035] Fig.11 is a table illustrating an example of the definition of "word length" and "sample word length" in the channel status.

[0036] Fig.12 It is a diagram illustrating a waveform example in which a data string expressing a sine wave with an amplitude of 12.5% ​​(vibrating within a range of -0.125 to +0.125) using binary 16-bit data (binary16) is mistaken for signed integer 16-bit data (signedint16).

[0037] Fig.13 It is a diagram for describing the processing applied to binary 16-bit data (binary16) in Improvement Plan 1.

[0038] Fig.14 It is a diagram illustrating a waveform example in which, when a data string representing a sine wave with an amplitude of 12.5% ​​(vibrating within the range of -0.125 to +0.125) using binary 16-bit data (binary16) is processed by Improvement Plan 1, the data string is mistakenly regarded as signed integer 16-bit data (signed int16).

[0039] Fig.15 It is a diagram for describing the processing applied to binary 16-bit data (binary16) in Improvement Plan 2.

[0040] Fig.16 It is a diagram illustrating a waveform example in which, when a data string representing a sine wave with an amplitude of 12.5% ​​(vibrating within the range of -0.125 to +0.125) using binary 16-bit data (binary16) is processed by Improvement Plan 2, the data string is mistakenly regarded as signed integer 16-bit data (signed int16).

[0041] Fig.17 A diagram for describing a case where a sine wave having an amplitude of 50% (vibrating within a range of -0.5 to +0.5) is depicted.

[0042] Fig.18 A diagram depicting improvements in precision for describing data in floating point format.

[0043] Fig.19 is a diagram illustrating an arrangement example of a frame structure of the IEC60958 standard in an audio stream in the case of using binary 20-bit data or binary 24-bit data.

[0044] Fig. 20 is a block diagram illustrating a configuration example of a transmission and reception system as a second embodiment.

[0045] Fig.21 This is a table illustrating an example of a packet structure of a new InfoFrame including position data related to a sound source.

[0046] Fig. 22 is a block diagram illustrating a configuration example of a transmission and reception system as a third embodiment. DETAILED DESCRIPTION

[0047] A description will now be given of a form for specifically implementing the present invention (hereinafter referred to as an "embodiment"). Note that the description is made in the following order.

[0048] 1. First Embodiment

[0049] 2. Second Embodiment

[0050] 3. Third embodiment

[0051] 4. Modifications

[0052] <1. First Embodiment>

[0053] (Configuration example of a transmission and reception system)

[0054] Figure 1 The diagram illustrates a configuration example of a transceiver system 10 as a first embodiment. The transceiver system 10 has a configuration in which an AV (audio / visual) amplifier 100 as an HDMI (High Definition Multimedia Interface) source and a smart TV (television) 200 as an HDMI sink are connected to each other via an HDMI cable 300. Note that "HDMI" is a registered trademark.

[0055] The AV amplifier 100 transmits differential signals corresponding to uncompressed pixel data associated with an image of one screen to the smart TV 200 on multiple channels during an effective image period (hereinafter also appropriately referred to as an active video period), which is a period obtained by excluding a horizontal blanking period and a vertical blanking period from a period from one vertical synchronization signal to the next vertical synchronization signal, and transmits differential signals corresponding to at least audio data and control data accompanying an image, other types of auxiliary data, and the like to the smart TV 200 on multiple channels during the horizontal blanking period or the vertical blanking period.

[0056] That is, the AV amplifier 100 includes an HDMI transmitter 101. For example, the HDMI transmitter 101 converts pixel data related to an uncompressed image into corresponding differential signals and transmits the differential signals serially to the smart TV 200 connected via the HDMI cable 300 on three TMDS channels #0, #1, and #2 as a plurality of channels.

[0057] In addition, the HDMI transmitter 101 converts audio data accompanying uncompressed images as well as required control data, other types of auxiliary data, etc. into corresponding differential signals, and transmits the differential signals serially on three TMDS channels #0, #1 and #2 to the smart TV 200 connected via the HDMI cable 300.

[0058] In addition, the HDMI transmitter 101 transmits a pixel clock synchronized with the pixel data transmitted on the three TMDS channels #0, #1, and #2 on the TMDS clock channel to the smart TV 200 connected via the HDMI cable 300. On one TMDS channel #i (i=0, 1, 2), 10 bits of pixel data are transmitted during one clock of the pixel clock.

[0059] In this configuration, TMDS coding is an 8-bit / 10-bit conversion coding that converts 8-bit data into 10-bit data, and is a coding that reduces transition points compared to previous data to suppress adverse effects such as unnecessary radiation and maintains DC balance. Therefore, the coding stroke cannot be guaranteed in theory, so DC coupling and independent transmission of the clock are essential.

[0060] The smart TV 200 receives differential signals corresponding to pixel data sent from the AV amplifier 100 on multiple channels during an active video period, and receives differential signals corresponding to audio data and control data sent from the AV amplifier 100 on multiple channels during a horizontal blanking period or a vertical blanking period.

[0061] That is, the smart TV 200 includes an HDMI receiver 201. The HDMI receiver 201 receives differential signals corresponding to pixel data and differential signals corresponding to audio data and control data transmitted from the AV amplifier 100 connected via the HDMI cable 300 on TMDS channels #0, #1, and #2 in synchronization with a pixel clock similarly transmitted from the AV amplifier 100 on the TMDS clock channel.

[0062] Note that an example in which image data, audio data, and control data are transmitted on TMDS channels #0, #1, and #2, and a pixel clock is transmitted on a TMDS clock channel is described above, and this example corresponds to HDMI 1.4 or earlier versions as well as HDMI 2.0. In the case of HDMI 2.1, transmission by using FRL channels #0, #1, #2, and #3 is performed. In this case, the TMDS clock channel corresponds to FRL channel #3.

[0063] In this case, data transmission is performed on a fixed rate link (FRL) packet using three lanes #0 to #2 or four lanes #0 to #3. In this configuration, the FRL character encoding is a 16-bit / 18-bit conversion encoding that converts 16-bit data into 18-bit data, is an encoding that maintains DC balance, and is an encoding that allows clock extraction.

[0064] Furthermore, the smart TV 200 transmits object audio data acquired from, for example, an AV streaming service to the AV amplifier 100 .

[0065] That is, the smart TV 200 includes a transmission processing unit 202 and an ARC (audio return channel) / eARC (enhanced audio return channel) transmission unit (ARC / eARC Tx) 203. The transmission processing unit 202 inputs object audio data and generates an audio stream including the object audio data. In this configuration, the object audio data includes sound data (audio signal) and position data related to the sound source. In this configuration, the sound data (audio signal) constitutes a vibration signal. The details of the audio stream are further described later.

[0066] The ARC / eARC transmission section 203 transmits the audio stream generated in the transmission processing section 202 to the AV amplifier 100 over an audio return channel or an enhanced audio return channel using the common line and the HPD line of the HDMI cable 300 .

[0067] The AV amplifier 100 receives an audio stream transmitted from the smart TV 200 on an audio return channel or an enhanced audio return channel, and generates an audio signal for each speaker included in the speaker system 104 by performing a rendering process using object audio data included in the audio stream.

[0068] That is, the AV amplifier 100 includes an ARC / eARC receiving part (ARC / eARC Rx) 102 and an audio processing part 103. The ARC / eARC transmitting part 102 receives an audio stream transmitted from the smart TV 200 on an audio return channel or an enhanced audio return channel.

[0069] The audio processing unit 103 extracts object audio data from the audio stream received by the ARC / eARC receiving unit 102 , performs rendering processing using the object audio data, thereby generating an audio signal for each speaker included in the speaker system 104 , and provides the audio signal to the corresponding speaker of the speaker system 104 .

[0070] Note that the transmission channels of the HDMI system include a transmission channel called DDC (Display Data Channel), as well as a CES line and an HPD line, in addition to the above-mentioned transmission channel for TMDS or FRL.

[0071] The DDC includes two lines (signal lines) included in the HDMI cable 300 and not shown. The DDC is used by the AV amplifier 100 to read the EDID stored in the EDID ROM (Extended Display Identification ROM) included in the smart TV 200 via the HDMI cable 300. In addition, the DDC is used by the AV amplifier 100 to read and write data related to the SCDCS stored in the SCDC (Status and Control Data Channel) register included in the smart TV 200 via the HDMI cable 300.

[0072] In addition, the CEC line is used for bidirectional communication of control data between the AV amplifier 100 and the smart TV 200. The HPD line is used by the AV amplifier 100 to detect connection of the smart TV 200 and the like.

[0073] "Audio Stream Details"

[0074] Now, the details of the audio stream generated by the object audio data processing section 202 of the smart TV 200 are described. The transmission processing section 202 generates the audio stream in a transmission signal structure including each block of a plurality of frames for an audio signal. In this embodiment, the frame structure of the IEC 60958 standard is used as the transmission signal structure.

[0075] Figure 2 The diagram illustrates the frame structure of the IEC 60958 standard. Each frame includes 2 subframes. In the case of two-channel stereo audio, the left channel signal is included in the first subframe, and the right channel signal is included in the second subframe.

[0076] A preamble is set to the head of the subframe, "M" is added as a preamble to the left channel signal, and "W" is added as a preamble number to the right channel signal. Note that "B" indicating the start of a block is added to the head preamble in every 192 frames. That is, one block includes 192 frames. A block is a unit included in the channel state described later.

[0077] Figure 3 The diagram illustrates the subframe structure of the IEC 60958 standard. The subframe includes a total of 32 time slots, including the 0th to 31st time slots. The 0th to 3rd time slots indicate a preamble (synchronization preamble). As described above, the preamble indicates any one of "M", "W" and "B" to indicate the distinction between the left channel and the right channel and the start position of the block.

[0078] The 4th to 27th time slots correspond to the main data field. When a code range equal to or less than 20 bits, for example, a code range of 20 bits or a code range of 16 bits is adopted, an area of ​​the audio sample word from the 8th to 27th time slots is used. In this case, the audio signal is arranged on the MSB (most significant bit) side without intervals, and each remaining bit on the LSB (least significant bit) side is filled with 0. In addition, when a 24-bit code range is adopted, in addition to the area of ​​the audio sample word from the 8th to 27th time slots, an area of ​​the auxiliary sample bit from the 4th to 7th time slots is used. In this case, the audio signal is arranged in the 4th to 27th time slots.

[0079] The 28th time slot is a valid indicator of the main data field. The 29th time slot represents 1 bit of user data. By accumulating these 29th time slots across each frame, a series of user data can be formed. The message of the user data includes an 8-bit IU (information unit) as a unit, and one message includes 3 to 129 information units.

[0080] There may be 0 to 8 bits of "0" between information units. The header of an information unit is identified by the start bit "1". The first 7 information units in a message are reserved, and the user can set any information in the 8th and subsequent information units. Messages are separated from each other by 8 or more bits of "0".

[0081] The 30th time slot represents a 1-bit channel state. By accumulating the 30th time slot across frames for each block, the channel state can be formed. Note that, as described above, the head position of the block is indicated by the preamble "B" (0th to 3rd time slots).

[0082] The 31st time slot is a parity bit. This parity bit is added so that the number of "0" and "1" included in the 4th to 31st time slots is an even number.

[0083] As described above, the object audio data includes the sound data (audio signal) and the position data related to the sound source. The transmission processing unit 202 converts the sound data (audio signal) related to the sound source into data in a floating point format. In this sense, the transmission processing unit 202 includes a conversion unit. After that, the transmission processing unit 202 arranges the data in the floating point format in the area of ​​the audio sample word in the audio stream having the frame structure of the IEC60958 standard, or in the area of ​​the audio sample word and the auxiliary sample bit.

[0084] In the present embodiment, data in a floating point format is defined as data in a 16-bit half-precision floating point format (hereinafter appropriately referred to as "binary 16-bit data"). Figure 4The diagram illustrates the structure of binary 16-bit data. The binary 16-bit data has a structure in which a 1-bit sign part, a 5-bit exponent part, and a 10-bit fraction part are arranged from the MSB side.

[0085] Figure 5 It is a diagram illustrating the frame configuration in an audio stream having a frame structure of the IEC 60958 standard. Binary 16-bit data obtained by converting sound data (audio signal) related to a sound source is arranged without intervals in the area of ​​the audio sample word on the MSB side, and 0 is filled in each remaining bit on the LSB side.

[0086] here, Figure 6 A diagram for describing binary 16-bit data (binary16) in comparison with signed integer 16-bit data (signed int16) is depicted. Figure 6 (a) illustrates a signed integer 16-bit data, while Figure 6 (b) illustrates binary 16-bit data. Signed integer 16 data is represented as 2's complement, can represent a range of -32768 to 32767, and has a maximum precision of 15 bits.

[0087] In binary 16-bit data, the 5-bit exponent part indicates a range of -14 to 15, but actually stores a value of 1 to 30 obtained by adding 15. Note that 0 indicates a subnormal number, that is, a state where the decimal part cannot be normalized and the precision is insufficient, and 31 indicates positive infinity or NaN (not a number). Binary 16-bit data has an expression range determined by the 5-bit exponent part, and can express a range of -65504 to 65504.

[0088] In addition, in binary 16-bit data, the decimal part has 10 bits, but the configuration in which the most significant bit is always 1 is used to omit the most significant bit when encoding to express the binary 16-bit data. Therefore, in the case of a normalized number, 1 is added to the MSB of the 10 bits of the decimal part to express the decimal value "1.xxxxxxxxxx", and the significant bit is extended to 11 bits. Therefore, the binary 16-bit data has 11 bits of precision in the case of a normalized number. On the other hand, in the case of a denormalized number, the 10 bits of the decimal part express the decimal value "0.xxxxxxxxxx", and the significant bit is still 10 bits. Therefore, in the case of a denormalized number, the binary 16-bit data is expressed in a range of approximately -6.1*10^-5 to 6.10^5, and has a precision of less than 10 bits.

[0089] In the present embodiment, when the sound data (audio signal) associated with the sound source is converted into binary 16-bit data, the maximum displacement of the sound data is set to a predetermined value smaller than the maximum value (=65504) in the range determined by the exponential part of the binary 16-bit data, for example, 1. As a result of this setting, even if arithmetic operations in the binary 16-bit data format are continued on the receiving side, the range of values ​​has a margin, and therefore, arithmetic operations that process signals exceeding the maximum displacement (100%) of the sound data (audio signal) associated with the sound source can be performed.

[0090] Figure 7 The diagram illustrates a case where signed integer 16-bit data (signed int16) and binary 16-bit data (binary16) are applied to an audio signal. The diagram illustrates a case where the maximum displacement (100%) of the audio signal is set to "32768" in the signed integer 16-bit data, and the maximum displacement (100%) of the audio signal is set to "1" in the binary 16-bit data.

[0091] In this case, -20% of the audio signal is represented as "1110 0110 0110 0110b (E666h)" in signed integer 16-bit data, and as "1011 0010 0110 0110b (B266h)" in binary 16-bit data. Note that when this "1011 0010 0110 0110b (B266h)" is mistakenly recognized as signed integer 16-bit data, it represents -19866, which is approximately -61%.

[0092] Furthermore, -10% of the audio signal is expressed as "11110011 00110011b (F333h)" in signed integer 16-bit data, and as "10101110 0110 0110b (AE66h)" in binary 16-bit data. Note that when this "1010 1110 0110 0110b (AE66h)" is mistakenly recognized as signed integer 16-bit data, it represents -20890, which is approximately -64%.

[0093] Furthermore, +10% of the audio signal is expressed as "00001100 11001101b(0CCDh)" in signed integer 16-bit data, and as "00101110 0110 0110b(2E66h)" in binary 16-bit data. Note that when this "0010 1110 0110 0110b(2E66h)" is mistakenly regarded as signed integer 16-bit data, it represents 11878, which is approximately +36%.

[0094] Furthermore, +20% of the audio signal is expressed as "00011001 10011010b(199Ah)" in signed integer 16-bit data, and as "00110010 0110 0110b(3266h)" in binary 16-bit data. Note that when this "0011 0010 0110 0110b(3266h)" is mistakenly regarded as signed integer 16-bit data, it indicates 12902, which is approximately +39%.

[0095] Furthermore, the transmission processing unit 202 includes the position data related to the sound source in the channel status provided in the frame structure of the IEC 60958 standard, that is, the channel status in IEC 60958.

[0096] For example, it is conceivable to allocate two bytes to each of X, Y, and Z of the three-dimensional coordinates included in the position data related to the sound source. In this case, it is conceivable that (1) signed integer 16-bit data is used and decimeters (for example, "1234" represents 123.4 meters) or centimeters (for example, "1234" represents 12 meters 34 centimeters) are used as units. In addition, in this case, it is conceivable that (2) binary 16-bit data is used and meters (for example, "1.0" represents 1 meter and "12.3" represents 12.3 meters) are used as units.

[0097] Figure 8 : is a table schematically illustrating the format of the channel status in the IEC 60958 standard. The channel status is obtained by accumulating the 30th time slot in the subframe for each block (see Figure 3 ). In this table, the contents of the channel status are arranged byte by byte in the vertical direction, and the bit configuration of each byte is indicated in the horizontal direction. The format used by the consumer is assumed here, and only the main part is described.

[0098] "a" in the 0th bit of the 0th byte indicates the channel status, and is set to "0" here, indicating that the channel status is for consumer use. "b" in the 1st bit of the 0th byte indicates the data type, "0" indicates a linear PCM audio signal, and "1" indicates an audio signal other than linear PCM.

[0099] "d" in the 3rd to 5th bits of the 0th byte indicates additional format information. Fig. 9 is a table illustrating an example of the definition of "b" in the first bit of the 0th byte and "d" in the third to fifth bits of the 0th byte. In this case, assuming that the data in the floating point format is a type of linear PCM audio signal, b="0" and d="001" are newly defined to indicate the transmission of the data in the floating point format. In addition, Fig.10001 is a table illustrating another example of the definition of "b" in the first bit of the 0th byte and "d" in the third to fifth bits of the 0th byte. In this case, assuming that the data in the floating point format is not a linear PCM audio signal, b="1" and d="001" are newly defined to indicate the transmission of the data in the floating point format.

[0100] In addition, return reference Figure 8 , the "word length" in the 0th bit of the 4th byte and the "sample word length" in the 1st to 3rd bits of the 4th byte indicate the bit length of the audio signal to be transmitted. Fig.11 Illustrated is an example of the definition of "word length" and "sample word length".

[0101] For example, "word length = 0" and "sample word length = 100" indicate that the bit length of the audio signal is 16 bits. In the present embodiment, as the aforementioned audio signal, binary 16 data is transmitted, and "word length = 0" and "sample word length = 100" are defined to indicate that the bit length is 16 bits. In addition, for example, "word length = 0" and "sample word length = 101" are defined to indicate that the bit length of the audio signal is 20 bits. In addition, for example, "word length = 1" and "sample word length = 101" are defined to indicate that the bit length of the audio signal is 24 bits.

[0102] In addition, return reference Figure 8 , the 6 bytes from the 18th byte to the 23rd byte are a new setting area for arranging the position data related to the sound source. In the 18th byte to the 19th byte, 2 bytes of X data in the three-dimensional coordinates are arranged. In the 20th byte to the 21st byte, 2 bytes of Y data in the three-dimensional coordinates are arranged. In the 22nd byte to the 23rd byte, 2 bytes of Z data in the three-dimensional coordinates are arranged. Note that the area for arranging the position data related to the sound source is not limited to the 18th byte to the 23rd byte.

[0103] As mentioned above, in Figure 1 In the transceiver system 10 illustrated in the figure, the smart TV 200 generates an audio stream having a frame structure of the IEC 60958 standard including object audio data, and transmits the audio stream to the AV amplifier 100 on the audio return channel or the enhanced audio return channel. At this time, the smart TV 200 converts the sound data (audio signal) related to the sound source included in the object audio data into binary 16-bit data as data in a floating point format, and transmits the converted sound data. Therefore, in the audio processing section 103 of the AV amplifier 100, a rendering process (complex arithmetic operation process) of generating an audio signal for each speaker included in the speaker system 104 can be advantageously performed.

[0104] In addition, as mentioned above, Figure 1 In the transceiver system 10 illustrated in FIG. 1 , when generating an audio stream having a frame structure of the IEC 60958 standard including target audio data, the smart TV 200 includes information indicating that data in a floating point format is arranged in an area of ​​an audio sample word or an area of ​​an audio sample word and an auxiliary sample bit, and information indicating the number of bits of the data in the floating point format, in the channel status of the IEC 60958 standard. Therefore, in the AV amplifier 100, these information can be acquired from the channel status, and the rendering process in the audio processing section 103 can be appropriately performed.

[0105] In addition, Figure 1 In the transceiver system 10 illustrated in FIG. 1 , when generating an audio stream having a frame structure of the IEC 60958 standard, the smart TV 200 arranges data in a floating point format obtained by converting sound data (audio signal) related to a sound source in an area of ​​an audio sample word or an area of ​​an audio sample word and an auxiliary sample bit, and includes position data related to the sound source in a channel state in the IEC 60958 standard. Therefore, the sound data and position data related to the sound source and included in the object audio data can be transmitted from the smart TV 200 to the AV amplifier 100 in synchronization with each other, and therefore, a good 3D audio space can be reproduced in the AV amplifier 100.

[0106] "Improved handling of binary 16-bit data"

[0107] Note that the above describes the conversion of sound data (audio signal) related to the sound source included in the object audio data into binary 16-bit data as data in a floating point format, and the binary 16-bit data is directly arranged in the area of ​​the audio sample word of the audio stream having the frame structure of the IEC 60958 standard.

[0108] In this case, for example, even if the AV amplifier 100 does not support binary 16-bit data, or even if the AV amplifier 100 supports binary 16-bit data, the channel status is not correctly configured due to the absence of channel bits, etc., and the state in which the binary 16-bit data is transmitted cannot be recognized, there is a possibility that the binary 16-bit data is mistaken for signed integer 16-bit data and is processed as such, thereby generating sharp waveform fluctuations at zero crossings and unbalanced DC components.

[0109] For example, Fig.12The diagram illustrates an example of a waveform in which a data string expressing a sine wave with an amplitude of 12.5% ​​(vibrating within the range of -0.125 to +0.125) with binary 16-bit data (binary16) is mistaken for signed integer 16-bit data (signed int16). The dotted line indicates a waveform in which a sine wave with an amplitude of 12.5% ​​(y=0.125*sin x) is expressed with binary 16-bit data. The solid line indicates a waveform in which the binary 16-bit data is mistaken for signed integer 16-bit data. Note that the full scale of signed integer 16-bit data is -32768 to +32767.

[0110] As described above, when binary 16-bit data is mistaken for signed integer 16-bit data, a sharp waveform fluctuation occurs when crossing zero as shown by a dotted oval frame P1, and an unbalanced DC component appears as shown by a dotted oval frame P2.

[0111] It is believed that these phenomena are caused by the numbers in the exponent part, which are offset values ​​obtained by adding 15 to the original value in the binary 16-bit data. The exponent is also a value equal to or less than 0 near zero, but becomes a value equal to or less than 15 due to the addition of 15. In this case, the exponent part has a length of 5 bits, so the MSB is 0. For example, 15 is 01111b. Signed integer 16-bit data is represented as 2's complement, and when 0 exists in a position close to the MSB and the signed integer 16-bit data represents a negative value, the signed integer 16-bit data has a large negative value. For example, as binary 16-bit data, 0.0625, which is half of 0.125, is represented as 0x2C00, and -0.0625 is represented as 0xAC00. When they are mistaken for integer 16-bit data, they are 11264 and -21504, respectively.

[0112] Here, in order to suppress the sharp waveform fluctuation and unbalanced DC component at the zero crossing when the binary 16-bit data is mistaken for the signed integer 16-bit data, Improvement Plan 1 and Improvement Plan 2 are proposed to apply predetermined processing to the binary 16-bit data obtained by converting the sound data (audio signal) related to the sound source and included in the object audio data, and arrange the processed binary 16-bit data in the area of ​​the audio sample word of the audio stream having the frame structure of the IEC 60958 standard.

[0113] First, the improvement plan 1 is described. Fig.13 As shown in the diagram, in the case where the sign portion indicates a negative sign for the unprocessed data, the improvement plan 1 applies a bit reversal process to the 5-bit exponent portion of the data, thereby obtaining processed data.

[0114] Fig.14 1 is a diagram illustrating a waveform example in which a data string expressing a sine wave (vibrating in the range of -0.125 to +0.125) with an amplitude of 12.5% ​​with binary 16-bit data (binary16) is mistaken for signed integer 16-bit data (signed int16) when the data string is processed by improvement plan 1. The dotted line indicates a waveform expressing a sine wave (y=0.125*sin x) with an amplitude of 12.5% ​​with binary 16-bit data. The solid line indicates a waveform in which a data string after processing binary 16-bit data by improvement plan 1 is mistaken for signed integer 16-bit data.

[0115] In the case where a data string of binary 16-bit data is processed by the improvement plan 1 as described above, even in the case where the data string is mistaken for signed integer 16-bit data, Fig.12 Compared with the case shown in the diagram, the sharp waveform fluctuation at the time of zero crossing is also suppressed as shown in the dotted oval box P1, and the unbalanced DC component is suppressed as shown in the dotted oval box P2. In the case of this improvement plan 1, when the sign part is checked and it is determined to be a negative sign, the entire exponential part of 5 bits is simply inverted, so its implementation is simple and even hardware implementation can be easily completed.

[0116] Improvement Plan 2 is now described. Fig.15 As shown in the diagram, in the case where the sign portion indicates a negative sign, the improvement plan 2 applies bit reversal processing to the 5-bit exponent portion of the unprocessed data and applies 2's complement conversion processing to the 10-bit fractional portion, thereby obtaining processed data.

[0117] Fig.16 1 is a diagram illustrating a waveform example in which, in the case where a data string expressing a sine wave (vibrating in the range of -0.125 to +0.125) with an amplitude of 12.5% ​​with binary 16-bit data (binary16) is processed by Improvement Plan 2, the data string is mistaken for signed integer 16-bit data (signed int16). The dotted line indicates a waveform expressing a sine wave (y=0.125*sin x) with an amplitude of 12.5% ​​with binary 16-bit data. The solid line indicates a waveform in the case where a data string obtained after processing binary 16-bit data by Improvement Plan 2 is mistaken for signed integer 16-bit data.

[0118] In the case where a data string of binary 16-bit data is processed by the improvement plan 2 as described above, even in the case where the data string is mistaken for signed integer 16-bit data, Fig.12Compared with the case shown in the diagram, the sharp waveform fluctuation at zero crossing is also suppressed as shown in the dotted oval box P1, and the unbalanced DC component is suppressed as shown in the dotted oval box P2. In the case of this improvement plan 2, compared with the above-mentioned improvement plan 1, complicated processing is required, and compared with the improvement plan 1, the waveform at the negative value is clean.

[0119] In the description given above, a sine wave having an amplitude of 12.5% ​​(vibrating within a range of -0.125 to +0.125) is exemplified, and the same applies to a case where the amplitude is larger than that in this example. Fig.17 (a) is a diagram illustrating an example of a waveform in the case where a data string expressing a sine wave with an amplitude of 50% (vibrating in the range of -0.5 to +0.5) using binary 16-bit data (binary16) is mistaken for signed integer 16-bit data. The dotted line indicates a waveform obtained by expressing a sine wave with an amplitude of 50% (y=0.5*sin x) using binary 16-bit data. The solid line indicates a waveform in the case where the binary 16-bit data is mistaken for signed integer 16-bit data. Note that the full scale of signed integer 16-bit data is -32768 to +32767.

[0120] Fig.17 (b) is a diagram illustrating a waveform example in which, in the case where a data string expressing a sine wave (vibrating in the range of -0.5 to +0.5) with an amplitude of 50% with binary 16-bit data (binary16) is processed by the improvement plan 1, the data string is mistaken for signed integer 16-bit data (signed int16). The dotted line indicates a waveform obtained by expressing a sine wave (y=0.5*sin x) with an amplitude of 50% with binary 16-bit data. The solid line indicates a waveform in the case where a data string obtained after processing binary 16-bit data by the improvement plan 1 is mistaken for signed integer 16-bit data.

[0121] Furthermore, in such a case where the amplitude increases, in which the binary 16-bit data is simply used and the data string is processed while being misunderstood as the signed integer 16-bit data, the binary 16-bit data may generate sharp waveform fluctuations at zero crossing and unbalanced DC components. Furthermore, in this case, by processing the binary 16-bit data with the improvement plan 1, the sharp waveform fluctuations at zero crossing are suppressed, and the unbalanced DC components are suppressed, as shown in the dotted oval frame P2. Illustration of the waveform regarding the improvement plan 2 is omitted, but of course, a processing effect similar to that of the improvement plan 1 is provided.

[0122] (The precision of floating point format data is improved)

[0123] In the description given above, an example of converting the sound data (audio signal) related to the sound source into binary 16-bit data (16-bit data in a half-precision floating point format) as the object audio data is described. As described above, the binary 16-bit data has 11-bit precision within the range of the normalized number, but it is conceivable to use data in a floating point format with an increased number of bits in the decimal part to further improve the precision.

[0124] Fig.18 (b) and Fig.18 (c) illustrate examples of data in floating point format that can improve precision compared to binary 16-bit data. Note that Fig.18 (a) Graphical illustration and Figure 4 Binary 16-bit data (binary16) similar to that shown in the diagram.

[0125] Fig.18 The data of the floating point format of (b) (hereinafter appropriately referred to as "binary 20-bit data (binary20)") has a length of 20 bits, and has a configuration in which a 1-bit sign part, a 5-bit exponent part, and a 14-bit fraction part are arranged from the MSB side. Also in the case of this binary 20-bit data, similarly to the binary 16-bit data, the exponent part has 5 bits. In the case of this binary 20-bit data, the number of bits of the fraction part is 14, and therefore has 15 bits of precision within the range of the normalized number (corresponding to a 16-bit signed integer).

[0126] also, Fig.18 The data of the floating point format of (c) (hereinafter appropriately referred to as "binary 24-bit data (binary24)") has a length of 24 bits, and has a configuration in which a 1-bit sign part, a 5-bit exponent part, and an 18-bit fraction part are arranged from the MSB side. Also in the case of this binary 20-bit data, similarly to the binary 16-bit data, the exponent part has 5 bits. In the case of this binary 24-bit data, the number of bits of the fraction part is 18, and therefore has 19 bits of precision within the range of the normalized number (corresponding to a 20-bit signed integer).

[0127] Fig.19 1 is a diagram illustrating an example of arrangement of a frame structure of the IEC60958 standard in an audio stream when binary 20-bit data or binary 24-bit data is used. The binary 20-bit data is arranged in the area of ​​the audio sample word. In addition, the binary 24-bit data is arranged in the area of ​​the audio sample word and the area of ​​the auxiliary sample bit.

[0128] In this case, a part of the area of ​​the audio sample word up to the exponent part as viewed from the MSB side has the same structure as that in the case where binary 16-bit data is arranged. In addition, the decimal part is normalized, so even if 4 bits or 8 bits on the low-order side of the decimal part of binary 20-bit data or binary 24-bit data are cut off to become 16-bit data, a value rounded off as binary 16-bit data (a value close to the original value) is obtained. That is, even if the receiving side mistakenly processes the arranged binary 20-bit data or binary 24-bit data as binary 16-bit data, only a decrease in accuracy occurs, and failure can be avoided.

[0129] In addition, in the description given above, an example of transmitting 1 (1 channel) of object audio data from the smart TV 200 to the AV amplifier 100 is described, but 2 (2 channels) or more object audio data may be transmitted, and the object audio data may also be transmitted together with a normal audio signal. In this case, in the audio stream having the frame structure of the IEC 60958 standard, each data only needs to be divided and arranged in each frame.

[0130] Furthermore, in the description given above, an example is described in which the HDMI sink is a smart TV, but the HDMI sink is not limited thereto, and may be, for example, a TV, a set-top box, or a PC that does not have an Internet connection function.

[0131] "Second Embodiment"

[0132] (Configuration example of a transmission and reception system)

[0133] Fig. 20 The diagram illustrates a configuration example of a transceiver system 20 as a second embodiment. The transceiver system 20 is configured such that a media player 400 as an HDMI source and an AV amplifier 500 as an HDMI sink are connected via an HDMI cable 600 .

[0134] The media player 400 transmits differential signals corresponding to uncompressed pixel data associated with an image of one screen to the AV amplifier 500 on multiple channels during an effective image period (hereinafter also appropriately referred to as an active video period), which is a period obtained by excluding the horizontal blanking period and the vertical blanking period from the period from one vertical synchronization signal to the next vertical synchronization signal, and transmits differential signals corresponding to at least audio data and control data accompanying the image, other types of auxiliary data, etc. to the AV amplifier 500 on multiple channels during the horizontal blanking period or the vertical blanking period.

[0135] That is, the media player 400 includes an HDMI transmitter 401. For example, the HDMI transmitter 401 converts pixel data related to an uncompressed image into corresponding differential signals and transmits the differential signals serially to the AV amplifier 500 connected via the HDMI cable 600 on three TMDS channels #0, #1, and #2 as a plurality of channels.

[0136] In addition, the HDMI transmitter 401 converts the audio data accompanying the uncompressed image as well as the required control data, other types of auxiliary data, etc. into corresponding differential signals, and transmits the differential signals serially on three TMDS channels #0, #1 and #2 as multiple channels to the AV amplifier 500 connected via the HDMI cable 600.

[0137] Furthermore, the HDMI transmitter 401 transmits a pixel clock synchronized with the pixel data transmitted on the three TMDS channels #0, #1, and #2 on the TMDS clock channel to the AV amplifier 500 connected via the HDMI cable 600. On one TMDS channel #i (i=0, 1, 2), 10 bits of pixel data are transmitted during one clock of the pixel clock.

[0138] In this configuration, TMDS coding is 8-bit / 10-bit conversion coding that converts 8-bit data into 10-bit data, and is coding that reduces transition points from previous data to suppress adverse effects such as unnecessary radiation and maintains DC balance. Therefore, the coding stroke cannot be guaranteed in theory, so DC coupling and independent transmission of the clock are essential.

[0139] The AV amplifier 500 receives differential signals corresponding to pixel data sent from the media player 400 on multiple channels during an active video period, and receives differential signals corresponding to audio data and control data sent from the media player 400 on multiple channels during a horizontal blanking period or a vertical blanking period.

[0140] That is, the AV amplifier 500 includes an HDMI receiver 501. The HDMI receiver 501 receives differential signals corresponding to pixel data and differential signals corresponding to audio data and control data transmitted from the media player 300 connected via the HDMI cable 600 on TMDS channels #0, #1, and #2 in synchronization with a pixel clock similarly transmitted from the media player 400 on the TMDS clock channel.

[0141] Note that an example in which image data, audio data, and control data are transmitted on TMDS channels #0, #1, and #2, and a pixel clock is transmitted on a TMDS clock channel is described above, and this example corresponds to HDMI 1.4 or earlier versions as well as HDMI 2.0. In the case of HDMI 2.1, transmission by using FRL channels #0, #1, #2, and #3 is performed. In this case, the TMDS clock channel corresponds to FRL channel #3.

[0142] In this case, data transmission on a fixed rate link (FRL) packet using three lanes #0 to #2 or four lanes #0 to #3 is performed. In this configuration, the FRL character encoding is a 16-bit / 18-bit conversion encoding that converts 16-bit data into 18-bit data, is an encoding that maintains DC balance, and is an encoding that allows clock extraction.

[0143] In the transceiver system 20 , the media player 400 extracts object audio data associated with the image together with the image data from, for example, the HDD, and transmits the image data and the object audio data to the AV amplifier 400 .

[0144] That is, the AV amplifier 400 includes a transmission processing unit 402. The transmission processing unit 402 inputs the target audio data and generates an audio stream including the target audio data. The detailed description is omitted, but the transmission processing unit 402 is similar to Figure 1 The transmission processing unit 202 of the smart TV 200 of the transmission and reception system 10 shown in the diagram inputs the target audio data and generates an audio stream including the target audio data, that is, an audio stream having a frame structure of the IEC 60958 standard.

[0145] In this case, the transmission processing section 402 converts the sound data (audio signal) related to the sound source included in the target audio data into data in a floating point format, and arranges the data in the floating point format in an area of ​​an audio sample word, or an area of ​​an audio sample word and auxiliary sample bits in an audio stream having a frame structure in the IEC 60958 standard (see Figure 5 and Fig.19 ). In addition, the transmission processing unit 402 includes the position data related to the sound source included in the target audio data in the channel state provided in the frame structure of the IEC 60958 standard, that is, the channel state in the IEC 60958 standard (see Figure 8 ).

[0146] The audio stream generated in the transmission processing section 402 is supplied as audio data to the HDMI transmitter 401. The HDMI transmitter 401 packets the audio stream, inserts the packetized audio stream into the data island period of the TMDS transmission data, and transmits the packetized audio stream to the AV amplifier 500.

[0147] In the transceiver system 20, the AV amplifier 500 outputs the audio stream having the frame structure of the IEC 60958 standard transmitted from the media player 400 as audio data from the HDMI receiver 501. The AV amplifier 500 performs rendering processing by using the object audio data included in the audio stream, thereby generating an audio signal for each speaker included in the speaker system.

[0148] That is, the AV amplifier 500 includes an audio processing section 502. The audio processing section 503 extracts object audio data from the audio stream output from the HDMI receiver 501, performs rendering processing by using the object audio data, thereby generating an audio signal for each speaker included in the speaker system 503, and provides the audio signal to the corresponding speaker in the speaker system 503.

[0149] As mentioned earlier, in Fig. 20 In the transceiver system 20 illustrated in the figure, the media player 400 generates an audio stream including object audio data having a frame structure of the IEC 60958 standard, groups the audio stream, inserts the grouped audio stream into the data island period of the TMDS transmission data, and transmits the audio stream to the AV amplifier 500, and at this time, converts the sound data (audio signal) related to the sound source included in the object audio data into data in a floating point format, and transmits the converted sound data. Therefore, as Figure 1 As in the audio processing section 103 of the AV amplifier 100 of the transceiver system 10 illustrated in FIG. 2 , the rendering processing (complex arithmetic operation processing) of generating an audio signal for each speaker included in the speaker system 503 in the audio processing section 502 of the AV amplifier 500 can be advantageously performed.

[0150] Note that the above describes a configuration in which the position data related to the sound source included in the object audio data is included in the channel status provided in the frame structure of the IEC 60958 standard, that is, the channel status in the IEC 60958 standard and then transmitted. However, for example, it is conceivable to define a new InfoFrame packet including the position data related to the sound source, insert the InfoFrame packet into the data island period of the TMDS transmission data, and transmit the inserted InfoFrame packet to the AV amplifier 500.

[0151] Fig.21: is a table illustrating an example of a packet configuration of a new information frame including position data related to a sound source. The "information frame type" definition indicating the type of information frame grouping is defined in the 0th byte. The version information "information frame version number" related to the packet data definition is described in the 1st byte. The information "information frame length" indicating the packet length is described in the 2nd byte. In this configuration example, it is assumed that N channels, that is, N object audio data, are transmitted simultaneously, so "information frame length = N*6+2".

[0152] Information "starting channel ID" indicating the starting channel is described in the 3rd byte. Also, information "channel number" indicating the number of channels is described in the 4th byte. Also, in the following bytes, 2-byte data of each of X, Y, and Z three-dimensional coordinates of position data related to the sound sources of the N channels are sequentially arranged.

[0153] Note that in the case of HDMI, the information frame is limited to 30 bytes or less, so N is limited to 4 or less. Therefore, when the position data related to the sound source included in the object audio data of 5 channels (5) or more is to be sent, multiple information frames are used to send it. For example, when the position data related to the sound source included in the object audio data of 7 channels (7) is to be sent, 2 information frames are used. In this case, in the first information frame in which the position data related to the sound source included in the object audio data of the 1st to 4th channels is described, settings such as "starting channel ID = 1" and "number of channels = 4" are set, and in the second information frame in which the position data related to the sound source included in the object audio data of the 5th to 7th channels is described, settings such as "starting channel ID = 5" and "number of channels = 3" are set.

[0154] "Third Embodiment"

[0155] (Configuration example of a transceiver system)

[0156] Fig. 22 The configuration example of the transceiver system 30 as the third embodiment is illustrated in FIG. The transceiver system 30 is configured such that a television receiver 700 as an HDMI source and an audio amplifier 700 as an HDMI sink are connected via an HDMI cable 900 .

[0157] For the television receiver 700, an HDMI terminal 701 to which an HDMI receiving unit (HDMI RX) 702 and an ARC / eARC transmitting unit (ARC / eARCTx) 703 are connected is provided. For the audio amplifier 800, an HDMI transmitting unit (HDMI TX) 802 and an ARC / eARC receiving unit (ARC / eARC Rx) 803 are provided. One end of an HDMI cable 900 is connected to the HDMI terminal 701 of the television receiver 700, and the other end thereof is connected to the HDMI terminal 801 of the audio amplifier 800.

[0158] The television receiver 700 includes an HDMI receiving section 702, an ARC / eARC transmitting section 703, and an audio transmitting circuit 704. In addition, the television receiver 700 includes a system controller 705, a digital broadcast receiving circuit 707, a content reproduction circuit 708, a display section 709, and a network interface 710. In addition, in the example shown in the diagram, in order to simplify the description, the components of the image system are appropriately omitted.

[0159] The system controller 705 controls the operation of each part of the television receiver 700. The digital broadcast receiving circuit 707 processes the television broadcast signal input from the receiving antenna 721, and outputs a signal of a first mode related to the broadcast content (a video signal, a multi-channel audio signal (linear PCM signal) and a tactile vibration signal on a predetermined number of channels), or a signal of a second mode (a tactile vibration signal on a predetermined number of channels).

[0160] In this configuration, the multi-channel audio signal includes audio signals on multiple channels. In addition, the tactile vibration signals on a predetermined number of channels associated with the signal of the first mode are used to obtain vibrations synchronized with the video and audio. In addition, the tactile vibration signals on a predetermined number of channels associated with the signal of the second mode are used to obtain vibrations that are not directly related to the video or audio but are used for massage, rehabilitation, etc.

[0161] The network interface 710 communicates with an external server via the Internet 723 and outputs a first mode of signals related to network content (video signals, multi-channel audio signals (linear PCM signals) and tactile vibration signals on a predetermined number of channels), or a second mode of signals (tactile vibration signals on a predetermined number of channels).

[0162] In addition, the BD player 722 outputs a signal of a first mode (a video signal, a multi-channel audio signal (linear PCM signal) and a tactile vibration signal on a predetermined number of channels) related to the reproduced content, or a signal of a second mode (a tactile vibration signal on a predetermined number of channels) through a reproduction operation.

[0163] The content reproduction circuit 708 selectively extracts a signal of the first mode or a signal of the second mode acquired in the digital broadcast reception circuit 707 , the network interface 710 , or the BD player 722 .

[0164] Thereafter, when the signal of the first mode is extracted, the content reproduction circuit 708 transmits the video signal to the display section 709. The display section 709 displays an image based on the video signal.

[0165] In addition, when extracting the signal of the first mode, the content reproduction circuit 708 sends the multi-channel audio signal and the tactile vibration signal on the predetermined number of channels to the audio transmission circuit 704. The audio transmission circuit 704 sends the multi-channel audio signal (linear PCM signal) and the tactile vibration signal on the predetermined number of channels to the audio amplifier 800 at the same time.

[0166] In addition, when extracting the signal of the second mode, the content reproduction circuit 708 transmits the haptic vibration signal on the predetermined number of channels to the audio transmission circuit 704. The audio transmission circuit 704 transmits the haptic vibration signal on the predetermined number of channels to the audio amplifier 800.

[0167] When the signal of the first mode is extracted in the content reproduction circuit 708 and the multi-channel audio signal and the tactile vibration signal on the predetermined number of channels are transmitted accordingly, the audio transmission circuit 704 generates an audio stream including the multi-channel audio signal and the tactile vibration signal on the predetermined number of channels. In addition, when the signal of the second mode is extracted in the content reproduction circuit 708 and the tactile vibration signal on the predetermined number of channels is transmitted accordingly, the audio transmission circuit 704 generates an audio stream including the tactile vibration signal on the predetermined number of channels.

[0168] Detailed description is omitted, but the audio transmission circuit 704 is similar to Figure 1 The sending processing unit 202 of the smart TV 200 of the transceiver system 10 shown in the diagram generates an audio stream including a multi-channel audio signal and a tactile vibration signal on a predetermined number of channels, or including a tactile vibration signal on a predetermined number of channels, that is, an audio stream having a frame structure of the IEC 60958 standard.

[0169] In this case, the audio transmission circuit 704 converts the tactile vibration signal into data in a floating point format, and arranges the data in the floating point format in an area of ​​an audio sample word in an audio stream having a frame structure of the IEC 60958 standard, or in an area of ​​an audio sample word and auxiliary sample bits (see Figure 5 and Fig.19). In addition, in this case, when audio signals and tactile vibration signals on multiple channels are included in an audio stream having a frame structure of the IEC 60958 standard, the audio transmission circuit divides data related to each channel into frames and arranges the frames.

[0170] The ARC / eARC transmission section 703 transmits the audio stream generated in the audio transmission circuit 704 to the audio amplifier 800 over an audio return channel or an enhanced audio return channel using a common line and an HPD line of the HDMI cable 900 .

[0171] In this case, in the audio stream, a multi-channel audio signal related to the signal of the first mode and tactile vibration signals on a predetermined number of channels are included, or tactile vibration signals on a predetermined number of channels related to the signal of the second mode are included. Configuration information related to the signal included in the audio stream is included in, for example, a channel state configured for each block, i.e., a channel state in the IEC 60958 standard, and its detailed description is omitted.

[0172] The audio amplifier 800 includes an HDMI transmission section 802, an ARC / eARC reception section 803, and an audio reception circuit 804. In addition, the audio amplifier 800 includes a system controller 805, an audio reproduction circuit 808, and a haptic vibration reproduction circuit 809. The system controller 805 controls the operation of each section of the audio amplifier 800.

[0173] The ARC / eARC receiving unit 803 receives an audio stream having a frame structure of the IEC 60958 standard from the television receiver 700 on an audio return channel or an enhanced audio return channel. As described above, the audio stream includes a multi-channel audio signal related to the signal of the first mode and a haptic vibration signal on a predetermined number of channels, or includes a haptic vibration signal on a predetermined number of channels related to the signal of the second mode.

[0174] The audio receiving circuit 804 obtains a multi-channel audio signal related to the signal of the first mode and a tactile vibration signal on a predetermined number of channels, or a tactile vibration signal on a predetermined number of channels related to the signal of the second mode, included in the audio stream received by the ARC / eARC receiving unit 803. In this case, based on the configuration information included in the transmission signal, the multi-channel audio signal related to the signal of the first mode and the tactile vibration signal on a predetermined number of channels, or the tactile vibration signal on a predetermined number of channels related to the signal of the second mode is extracted.

[0175] The audio reproduction circuit 808 amplifies the multi-channel audio signal related to the signal of the first mode acquired by the audio receiving circuit 804 for each channel, and transmits the multi-channel audio signal to the speaker system 850 including speakers corresponding to each channel. As a result, audio reproduction is performed by using the multi-channel audio signal in the speaker system 850.

[0176] In addition, the tactile vibration reproduction circuit 809 amplifies the tactile vibration signals on a predetermined number of channels associated with the signal of the first mode acquired in the audio receiving circuit 804, or the tactile vibration signals on a predetermined number of channels associated with the signal of the second mode for each channel, and sends the tactile vibration signals to a tactile vibration system 860 having a vibration device corresponding to each channel, respectively. As a result, in the tactile vibration system 860, vibration reproduction is performed based on the tactile vibration signals on the predetermined number of channels. Note that in this case, in the tactile vibration reproduction circuit 809, an arithmetic operation process for adjusting the gain according to the sensitivity of each user, the sensitivity of the part to be stimulated, the sensitivity of the vibration device, and the nonlinearity of the vibration device is applied.

[0177] In this case, as described above, in the case where the signal of the first mode is processed, the tactile vibration signals on the predetermined number of channels are transmitted simultaneously with the multi-channel audio signal, and therefore, the vibration reproduction is correctly synchronized with the audio reproduction and also synchronized with the video display in the display section 709 of the television receiver 700. In addition, in the case where the signal of the second mode is processed, only the tactile vibration signals on the predetermined number of channels are transmitted, and therefore, the audio reproduction is not performed, and only the vibration reproduction for example for massage and rehabilitation is performed.

[0178] As mentioned above, in Fig. 22 In the transceiver system 30 illustrated in the figure, the television receiver 700 generates an audio stream having a frame structure of the IEC 60958 standard including a tactile vibration signal, transmits the audio stream to the audio amplifier 800 on the audio return channel or the enhanced audio return channel, and at this time converts the tactile vibration signal into data in a floating point format, thereby transmitting the data in the floating point format. Therefore, in the tactile vibration reproduction circuit 809 of the audio amplifier 800, an arithmetic operation process (complex arithmetic operation process) for adjusting the gain according to the sensitivity of each user, the sensitivity of the part to be stimulated, the sensitivity of the vibration device, and the nonlinearity of the vibration device can be advantageously performed.

[0179] <4. Modifications>

[0180] Note that in the above embodiments, an example is described in which an HDMI ARC / eARC or HDMI transmission path is used as a transmission path for an audio stream having an IEC 60958 structure, but it is conceivable that an IEC 61883-7 transmission path, an MHL transmission path, a DisplayPort transmission path (DP transmission path), and a coaxial cable or an optical cable are used as an IEC 60958 transmission path.

[0181] In addition, the preferred embodiments of the present disclosure are described in detail with reference to the accompanying drawings, but the technical scope of the present disclosure is not limited to these examples. It is obvious that a person skilled in the art of the present disclosure can conceive various variations and modifications within the scope of the technical ideas recorded in the scope of the claims. Needless to say, it should be understood that these examples also belong to the technical scope of the present disclosure.

[0182] In addition, the effects described in this specification are merely illustrative and exemplary, and not restrictive. That is, in addition to or in place of the above effects, the technology disclosed herein can achieve other effects that are obvious to those skilled in the art based on the description in this specification.

[0183] In addition, the present technology can also adopt the following configurations.

[0184] (1) A sending device, comprising:

[0185] A conversion section that converts the vibration signal into data in a floating point format; and

[0186] The data in the floating point format is transmitted to a transmission unit of an external device via a transmission path.

[0187] (2) The transmitting device according to (1) above, wherein the data in the floating point format includes 16-bit half-precision floating point data.

[0188] (3) A transmitting device according to (1) or (2) above, wherein the conversion unit sets the maximum displacement of the vibration signal to a predetermined value that is smaller than the maximum value of a range of values ​​determined by the number of bits of the exponential part of the data in the floating point format, thereby converting the vibration signal into the data in the floating point format.

[0189] (4) The transmitting device according to (3) above, wherein the predetermined value includes 1.

[0190] (5) The transmission device according to any one of (1) to (4) above, wherein the transmission path includes an HDMI transmission path.

[0191] (6) The transmitting device according to any one of (1) to (5) above, wherein the transmitting section transmits the data in the floating point format using a transmission signal structure including each block for a plurality of frames of an audio signal.

[0192] (7) The transmitting device according to (6) above, wherein

[0193] The transmission signal structure includes a frame structure of the IEC 60958 standard, and

[0194] The transmitting section arranges the data in the floating point format in a region of audio sample words or a region of audio sample words and auxiliary sample bits, and transmits the data in the floating point format.

[0195] (8) A transmitting device according to (7) above, wherein the data in the floating point format is arranged without interval in the area of ​​the audio sample word, or in the area of ​​the audio sample word and auxiliary sample bits, in the order of a sign part, an exponent part and a decimal part, starting from the most significant bit side of the audio sample word.

[0196] (9) The transmission device according to (8) above, wherein an exponent part in the data in the floating point format includes 5 bits, and a fraction part includes 10 bits, 14 bits, or 18 bits.

[0197] (10) A transmitting device according to any one of (6) to (9) above, wherein the channel status provided in the transmission signal structure of each block includes: information indicating that the data in the floating point format is arranged in the area of ​​the audio sample word or in the area of ​​the audio sample word and auxiliary sample bits.

[0198] (11) A transmitting device according to (10) above, wherein the channel status provided in the transmission signal structure of each block includes: information indicating the number of bits of the data in the floating point format arranged in the area of ​​the audio sample words or in the area of ​​the audio sample words and auxiliary sample bits.

[0199] (12) The transmitting device according to any one of (1) to (11) above, wherein when the sign portion of the floating-point format data indicates a negative sign, the transmitting unit applies a bit inversion process to the exponent portion and then transmits the floating-point format data.

[0200] (13) A transmitting device according to any one of (1) to (11) above, wherein when the sign portion of the data in the floating-point format indicates a negative sign, the transmitting unit applies a bit-reversal process to the exponent portion, converts the decimal portion into a 2's complement, and then transmits the data in the floating-point format.

[0201] (14) The transmitting device according to any one of (1) to (13) above, wherein the vibration signal includes an audio signal.

[0202] (15) The transmitting device according to any one of (1) to (13) above, wherein the vibration signal includes a tactile vibration signal.

[0203] (16) A receiving device comprising:

[0204] a receiving section that receives data in a floating point format obtained by converting the vibration signal from an external device via a transmission path; and

[0205] A processing unit processes the data in the floating point format.

[0206] (17) The receiving device according to (16) above, wherein the transmission path includes an HDMI transmission path.

[0207] (18) The receiving device according to (16) or (17) above, wherein the data in the floating point format includes 16-bit half-precision floating point data.

[0208] (19) The receiving device according to any one of (16) to (18) above, wherein the receiving section receives the data in the floating point format using a transmission signal structure including each block of a plurality of frames for an audio signal.

[0209] (20) A transceiver system,

[0210] The sending device and the receiving device are connected to each other via a transmission path.

[0211] The sending device comprises

[0212] a conversion section for converting the vibration signal into floating point format data, and

[0213] transmitting the data in the floating point format to a transmitting section of the receiving device via the transmission path, and

[0214] The receiving device comprises

[0215] a receiving section for receiving the data in the floating point format from the transmitting device via the transmission path, and

[0216] A processing unit processes the data in the floating point format.

[0217] Reference numerals list

[0218] 10,20,30: Transceiver system

[0219] 100:AV Amplifier

[0220] 101:HDMI Transmitter

[0221] 102:ARC / eARC receiving unit

[0222] 103: Audio Processing Unit

[0223] 104: Speaker System

[0224] 200: Smart TV

[0225] 201:HDMI Receiver

[0226] 202: Sending processing unit

[0227] 203:ARC / eARC sending unit

[0228] 300:HDMI cable

[0229] 400:Media Player

[0230] 401:HDMI transmitter

[0231] 402: Send processing unit

[0232] 500:AV Amplifier

[0233] 501:HDMI receiver

[0234] 502: Audio Processing Unit

[0235] 503: Speaker System

[0236] 600:HDMI cable

[0237] 700:TV receiver

[0238] 701:HDMI terminal

[0239] 702: HDMI receiving unit

[0240] 703:ARC / eARC sending unit

[0241] 704: audio transmission circuit

[0242] 705: System Controller

[0243] 707: Digital broadcast receiving circuit

[0244] 708: Content reproduction circuit

[0245] 709: Display unit

[0246] 710: Network interface

[0247] 721: Receiving antenna

[0248] 722:BD player

[0249] 723: Internet

[0250] 800:Audio Amplifier

[0251] 801:HDMI terminal

[0252] 802: HDMI transmitter

[0253] 803:ARC / eARC receiving unit

[0254] 804: audio receiving circuit

[0255] 805: System Controller

[0256] 808: Audio reproduction circuit

[0257] 809: tactile vibration generating circuit

[0258] 850: Speaker System

[0259] 860:Tactile Vibration System

Claims

1. A sending device, comprising: A conversion unit for converting the vibration signal into floating point format data; and The data in the floating point format is transmitted to a transmission unit of an external device via a transmission path.

2. The transmitting device according to claim 1, wherein the data in the floating point format comprises 16-bit half-precision floating point data.

3. A transmitting device according to claim 1, wherein the conversion section sets the maximum displacement of the vibration signal to a predetermined value smaller than a maximum value of a range of values ​​determined by the number of bits of an exponential portion of the data in the floating point format, thereby converting the vibration signal into the data in the floating point format. The transmitting device according to claim 3 , wherein the predetermined value comprises 1.

5. The transmission device according to claim 1, wherein the transmission path includes an HDMI transmission path.

6. The transmitting device according to claim 1, wherein the transmitting section transmits the data in the floating point format using a transmission signal structure including each block for a plurality of frames of an audio signal.

7. The transmitting device according to claim 6, wherein The transmission signal structure includes a frame structure of the IEC 60958 standard, and The transmitting section arranges the data in the floating point format in a region of audio sample words or a region of audio sample words and auxiliary sample bits, and transmits the data in the floating point format.

8. The transmitting device according to claim 7, wherein the data in the floating point format is arranged in an area of ​​the audio sample word, or an area of ​​the audio sample word and auxiliary sample bits, in the order of a sign part, an exponent part and a decimal part without intervals, starting from the most significant bit side of the audio sample word.

9. The transmitting device according to claim 8, wherein an exponent part in the data in the floating point format includes 5 bits, and a fraction part includes 10 bits, 14 bits or 18 bits.

10. The transmitting device according to claim 6, wherein the channel status provided in the transmission signal structure of each block includes: Information indicating that the data in the floating point format is arranged in the area of ​​the audio sample word or in the area of ​​the audio sample word and auxiliary sample bits.

11. The transmitting device according to claim 10, wherein the channel status provided in the transmission signal structure of each block includes: Information indicating the number of bits of the data in the floating point format arranged in the region of the audio sample word or in the region of the audio sample word and auxiliary sample bits.

12. The transmission device according to claim 1, wherein when a sign portion of the data in the floating point format indicates a negative sign, the transmission section applies a bit inversion process to the exponent portion and then transmits the data in the floating point format.

13. The transmitting device according to claim 1, wherein when the sign portion of the data in the floating point format indicates a negative sign, the transmitting section applies a bit reversal process to an exponent portion and performs a process of converting a decimal portion into a 2's complement, and then transmits the data in the floating point format.

14. The transmitting device according to claim 1, wherein the vibration signal comprises an audio signal.

15. The transmitting device of claim 1, wherein the vibration signal comprises a tactile vibration signal.

16. A receiving device, comprising: a receiving section that receives data in a floating point format obtained by converting the vibration signal from an external device via a transmission path; and A processing unit processes the data in the floating point format.

17. The receiving device according to claim 16, wherein the transmission path comprises an HDMI transmission path.

18. The receiving device according to claim 16, wherein the data in the floating point format comprises 16-bit half-precision floating point data.

19. The receiving device according to claim 16, wherein the receiving section receives the data in the floating point format using a transmission signal structure including each block for a plurality of frames of an audio signal.

20. A transceiver system, The sending device and the receiving device are connected to each other via a transmission path. The sending device comprises a conversion section for converting the vibration signal into floating point format data, and transmitting the data in the floating point format to a transmitting section of the receiving device via the transmission path, and The receiving device comprises a receiving section for receiving the data in the floating point format from the transmitting device via the transmission path, and A processing unit processes the data in the floating point format.