Audio transmission method and device

Abstract

The invention discloses an audio transmission method and device, and relates to the technical field of multimedia. And the audio sending end generates at least one audio frame and sends the at least one audio frame. A single audio frame only comprises two descriptive information messages used for describing the audio format of the ASP message, namely, one audio control message and one audio information frame message, compared with three SDP messages in the prior art, the number of the SDP messages is smaller, the bandwidth occupied by the two descriptive information messages is smaller, and the service life of the ASP message is prolonged. The data volume of effective audio data which can be transmitted by a channel in unit time can be improved, and the audio transmission efficiency between devices is improved.

Description

[0001] This application claims priority to the Chinese patent application filed with the State Intellectual Property Office on October 17, 2023, with application number 202311349951.5 and application name “A method and device for audio transmission”, the entire contents of which are incorporated by reference in this application. Technical Field

[0002] The present application relates to the field of multimedia technology, and in particular to an audio transmission method and device. Background Art

[0003] With the continuous development of computer technology, various types of multimedia data continue to emerge. Taking the wired transmission of audio data between different devices as an example, different devices can use the display port (DP) bus to transmit pure audio data. In the DP protocol, pure audio data is carried in a data frame for wired transmission between devices. However, in the DP protocol, each data frame carrying pure audio data carries audio data and various types of symbols, such as symbols for audio clock recovery or symbols for frame delimitation. When a large amount of audio data needs to be transmitted between different devices, a single data frame includes 3 secondary-data packets (SDP) messages for frame delimitation. These SDP messages will occupy the bandwidth of the DP bus, resulting in a reduction in the effective audio data that can be transmitted by the DP bus per unit time, and the audio transmission efficiency between devices is affected. Summary of the invention

[0004] The present application provides an audio transmission method and device, which solves the problem that the effective audio data transmitted per unit time in wired communication between devices is reduced and the audio transmission efficiency is affected.

[0005] This application adopts the following technical solution.

[0006] In a first aspect, the present application provides an audio transmission method. The audio transmission method is executed by a first device or a chip or processor in the first device, such as the first device may refer to a set-top box, etc. Exemplarily, the audio transmission method includes: the first device generates at least one audio frame, the audio frame includes: an audio control message, an audio information frame message and multiple audio sample packets (audio sample packets, ASP messages) carrying audio data, the audio control message and the audio information frame message are used to indicate: the audio format of the audio data. And, the first device sends the aforementioned at least one audio frame.

[0007] In the present application, a single audio frame only includes two descriptive information messages for describing the audio format of the audio data in the ASP message, namely, an audio control message and an audio information frame message. Compared with the three SDP messages in conventional technology, the number is smaller, and the bandwidth occupied by these two descriptive information messages is also smaller, which is conducive to increasing the amount of effective audio data that can be transmitted by the channel per unit time and improving the audio transmission efficiency between devices.

[0008] In combination with the audio transmission method provided in the first aspect, in an optional implementation, the audio transmission method provided in the present application also includes: the first device notifies the audio change; and the first device updates the audio control message and the audio information frame message, and sends the updated audio control message and the audio information frame message. In the present application, when there is a change in the audio format, the first device can immediately respond to the change in the audio format (such as audio update information) to update the descriptive information message, and resend the updated audio control message, audio information frame message and ASP message, avoiding the problem of audio transmission errors caused by untimely audio updates between devices, which is conducive to improving the efficiency of the second device in updating the played audio content based on the received updated audio frame, thereby improving the user's experience of playing audio using the second device.

[0009] In combination with the audio transmission method provided in the first aspect, in an optional implementation, after the first device notifies the audio change and before updating the audio control message and the audio information frame message, the first device sends a first audio control message, and the first audio control message carries an audio mute flag with a first value. And, after the first device updates the audio control message and the audio information frame message, the first device sends a second audio control message, and the second audio control message carries an audio mute flag with a second value. Exemplarily, the first value is 1 and the second value is 0. For example, the audio mute flag with a first value is used to indicate that the second device is muted, and the audio mute flag with a second value is used to indicate that the second device turns off the mute and plays audio content in a new audio format, so as to avoid the second device outputting audio with user-perceivable errors such as noise or plosive sounds during the audio format update process, which is conducive to improving the user experience of playing audio through the second device.

[0010] In the second aspect, the present application provides another audio transmission method. The audio transmission method is executed by a second device or a chip or processor in the second device, such as a TV or an amplifier. Exemplarily, the audio transmission method includes: the second device receives at least one audio frame, the audio frame includes: an audio control message, an audio information frame message and multiple audio sampling ASP messages carrying audio data, an audio control message and an audio information frame message are used to indicate: the audio format of the audio data. And, the second device plays audio according to at least one audio frame. A single audio frame includes only two descriptive information messages for describing the audio format of the audio data in the ASP message, namely, an audio control message and an audio information frame message, which is less than the number of three SDP messages in the conventional technology, and the bandwidth occupied by these two descriptive information messages is also smaller, which is conducive to increasing the amount of effective audio data that can be transmitted by the channel per unit time and improving the audio transmission efficiency between devices.

[0011] In combination with the audio transmission method provided in the second aspect, in an optional implementation, the second device includes an audio buffer for caching audio data, and the audio transmission method provided in the present application also includes: the second device receives the first audio control message of the first device, and responds to the first audio control message to delete the data stored in the audio buffer; the first audio control message carries an audio mute flag with a first value. And, the second device receives the second audio control message of the first device, and responds to the second audio control message to receive the ASP message carrying audio data; the second audio control message carries an audio mute flag with a second value. In the present application, the audio mute flag with a first value is used to indicate that the second device is muted, and the audio mute flag with a second value is used to indicate that the second device turns off the mute and plays audio content in a new audio format, so as to avoid the second device outputting audio with user-perceivable errors such as noise or plosive sounds during the audio format update process, which is conducive to improving the user's experience of playing audio through the second device.

[0012] In combination with the audio transmission method provided in the first aspect and the second aspect, in an optional implementation, the first device and the second device provided in the present application are connected via a unified multimedia interconnection channel, such as a UMI (unified multimedia interconnection) channel.

[0013] In combination with the audio transmission method provided by the first aspect and the second aspect, in an optional implementation, the audio format described in the present application may include a combination of one or more of the following parameters: audio sampling rate, number of audio channels, parity flag of the audio channel, audio encoding type and audio sampling bit width.

[0014] In combination with the audio transmission method provided in the first aspect and the second aspect, in an optional implementation, each of the aforementioned audio frames further includes: at least one encryption description packet (EDP or EDP message), the EDP being used to transmit encryption parameters of audio data. Exemplarily, the EDP is used to transmit parameters related to audio and video encryption, the parameters being used to determine the encryption key of the ASP message, so that the second device can parse the ASP message based on the encryption key and then play the audio data carried by the ASP, which is beneficial to improving the security and reliability of audio transmission.

[0015] For example, the EDP is transmitted between an audio information frame message and a plurality of ASP messages.

[0016] In combination with the audio transmission method provided in the first aspect and the second aspect, in an optional implementation, each of the aforementioned audio frames also includes: at least one key distribution packet (KDP or KDP message), and the KDP is used to indicate the encryption key of the aforementioned audio data. Exemplarily, the number of KDPs in each audio frame is associated with the number of audio playback devices multicasted by the first device, such as the number of KDPs is consistent with the number of audio playbacks, and different KDPs correspond to different audio playback devices, so that the audio playback device corresponding to the KDP parses the EDP based on the KDP to determine the encryption key of the audio data in the ASP message. Thus, the audio playback device parses the ASP message in the audio frame according to the determined encryption key to obtain the audio data carried by these ASP messages, and plays the audio content corresponding to the audio data.

[0017] For example, the KDP is transmitted between at least one EDP message and a plurality of ASP messages.

[0018] In combination with the audio transmission method provided by the first aspect and the second aspect, in an optional implementation, the number of multiple ASP messages in each audio frame is 1280.

[0019] In combination with the audio transmission method provided in the first aspect and the second aspect, in an optional implementation, in the absence of video transmission between the first device and the second device, if the second device supports processing pure audio data packets or the second device has pure audio processing capabilities, the above-mentioned audio frames can be transmitted between the first device and the second device.

[0020] In a third aspect, the present application provides an audio device, wherein the audio device includes a module for executing the method of any one of the implementation modes of the first aspect or the second aspect.

[0021] Exemplarily, the audio device includes a processing module and a transceiver module.

[0022] For example, when the audio device is used to execute the method of any implementation mode in the first aspect, the processing module is used to generate at least one audio frame, the audio frame includes: an audio control message, an audio information frame message and multiple ASP messages carrying audio data, an audio control message and an audio information frame message are used to indicate: the audio format of the audio data. The transceiver module is used to send at least one audio frame.

[0023] For another example, when the audio device is used to execute the method of any implementation mode in the second aspect, the transceiver module is used to receive at least one audio frame, the audio frame includes: an audio control message, an audio information frame message and multiple ASP messages carrying audio data, an audio control message and an audio information frame message are used to indicate: an audio format of the audio data. The processing module is used to play audio according to the at least one audio frame.

[0024] In a fourth aspect, the present application provides an audio processing device. The audio processing includes: a memory, a transceiver, and a processor. The memory is used to cache audio data. The processor is used to generate at least one audio frame, the transceiver is used to send the at least one audio frame, and the memory, the transceiver, and the processor are used to collaboratively perform the operation steps of the method of any implementation method in the first aspect.

[0025] In a fifth aspect, the present application provides an audio playback device. The audio playback device includes: a transceiver, a processor, and an audio unit. The transceiver is used to receive at least one audio frame, the processor is used to parse the at least one audio frame and instruct the audio unit to play the audio data carried by the at least one audio frame, and the transceiver, the processor, and the audio unit are used to collaboratively perform the operation steps of the method of any implementation method in the second aspect.

[0026] In a sixth aspect, the present application provides an audio transmission system. The audio transmission system includes an audio processing device provided in the fourth aspect and an audio playback device provided in the fifth aspect, the audio processing device and the audio playback device are connected via a unified multimedia interconnection channel, the audio processing device can be used to implement the function of the first device in the first aspect or the second aspect, and the audio playback device can be used to implement the function of the second device in the first aspect or the second aspect. Therefore, the audio transmission system can also achieve the beneficial effects of the audio transmission method in the first aspect and the second aspect, which will not be described in detail here.

[0027] In a seventh aspect, the present application provides a computer-readable storage medium. The computer-readable storage medium includes: computer software instructions.

[0028] When the computer software instructions are executed in a computing device, the computing device executes the operation steps of the method in the first aspect or any possible implementation of the first aspect, and executes the operation steps of the method in the second aspect or any possible implementation of the second aspect. For example, the computing device is the aforementioned audio processing device, audio playback device, first device, or second device.

[0029] In an eighth aspect, the present application provides a computer program product. When the computer program product is run on a computer, the computing device executes the operation steps of the method in the first aspect or any possible implementation of the first aspect, and executes the operation steps of the method described in the second aspect or any possible implementation of the second aspect. For example, the computer is the aforementioned audio processing device, audio playback device, first device, or second device.

[0030] Regarding the beneficial effects of the third to eighth aspects, reference may be made to the description of any implementation in the first or second aspect, and no further description is given here. Based on the implementations provided in the above aspects, this application may be further combined to provide more implementations. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] Figure 1 A schematic diagram of an audio transmission system provided by the present application;

[0032] Figure 2 A schematic diagram of an audio and video encoding and decoding system provided by the present application;

[0033] Figure 3 A schematic diagram of the process of an audio transmission method provided in this application Figure 1 ;

[0034] Figure 4A A schematic diagram of an audio stream provided by this application Figure 1 ;

[0035] Figure 4B A schematic diagram of the process of an audio transmission method provided in this application Figure 2 ;

[0036] Figure 5 A schematic diagram of an audio stream provided by this application Figure 2 ;

[0037] Figure 6 A schematic diagram of an audio stream provided by this application Figure 3 ;

[0038] Fig. 7A A schematic diagram of the process of an audio transmission method provided in this application Figure 3 ;

[0039] Figure 7B Schematic diagram 4 of a flow chart of an audio transmission method provided in this application;

[0040] Figure 8 A schematic diagram of the process of an audio transmission method provided in this application Figure 5 ;

[0041] Fig. 9 A schematic diagram of the structure of an audio device provided by the present application;

[0042] Fig.10 A schematic diagram of the structure of the audio processing device provided by this application;

[0043] Fig.11 A schematic diagram of the structure of the audio playback device provided in this application. DETAILED DESCRIPTION

[0044] The present application provides an audio transmission method, in which an audio transmitter generates at least one audio frame and sends the at least one audio frame. A single audio frame includes only two descriptive information messages for describing the audio format of an ASP message, namely, an audio control message and an audio information frame message. Compared with the three SDP messages in the conventional technology, the number is smaller, and the bandwidth occupied by the two descriptive information messages is also smaller, which is conducive to increasing the amount of effective audio data that can be transmitted by the channel per unit time and improving the audio transmission efficiency between devices.

[0045] For example, when the audio transmitter actively reads the capability description information of the audio playback device and confirms that the audio playback device supports processing pure audio data packets, the audio transmitter sends the audio stream to the audio playback device through the unified multimedia interconnection channel, thereby avoiding the invalid communication process between devices caused by the audio playback device not supporting processing pure audio data packets, and the problem of occupied channel bandwidth resources between devices. Among them, the data packets in the audio stream (such as the above-mentioned audio frames) only include two descriptive information packets (DIP or DIP messages), which are less than the number of three SDP messages in the conventional technology, and the bandwidth occupied by these two DIP messages is also smaller, which is conducive to increasing the amount of effective audio data that can be transmitted by the unified multimedia interconnection channel per unit time, and improving the audio transmission efficiency between devices.

[0046] The technical solutions involved in this application may not only be applied to current audio and video transmission technologies or audio and video standards, but may also be applied to future audio and video transmission technologies or audio and video standards. The terms used in the implementation methods of this application are only used to explain the specific embodiments of this application and are not intended to limit this application. The following is a brief introduction to some concepts that may be involved in this application.

[0047] Audio stream: A data stream that supports streaming technology and may include one or more data packets, each of which carries audio data to be played.

[0048] Descriptive Information Message (DIP Message): used to describe different data in a data packet.

[0049] Audio sampling message (ASP message): used to carry audio data in the audio stream.

[0050] Encryption Description Message (EDP Message): used to transmit parameters related to audio and video encryption. These parameters are used to indicate the encryption key of the ASP message.

[0051] Key Distribution Message (KDP Message): Encryption key used to transmit video and audio.

[0052] In order to make the description of the following embodiments clear and concise, an introduction to the related art is first given.

[0053] Figure 1 A schematic diagram of an audio transmission system provided by the present application. The audio processing process may include audio acquisition, audio encoding, audio transmission, audio decoding and playback processes.

[0054] Figure 1 The audio transmission system in the embodiment includes a set-top box 110, a smart TV 120, multiple audio playback devices and a server 130. The set-top box 110 is connected to the operator network through a network cable and can receive video and audio streams from the server 130. The network can realize the function of video and audio transmission, and the network can include one or more network devices, such as a network device 131, which can be a router or a switch. In some optional implementations, the set-top box 110 and the server 130 can also communicate through wireless communication, which is not limited in this application.

[0055] The set-top box 110 is a video and audio processing device, which is used to realize functions such as receiving, processing and pushing video streams or audio streams. In some possible situations, the set-top box 110 may also be called an Internet TV set-top box, a network high-definition player or other. Exemplarily, the set-top box 110 may refer to a TV box provided by a network operator or a TV box purchased by a user. For the hardware implementation of the set-top box 110, please refer to the following Fig.10 The description is not repeated here.

[0056] Smart TV 120 is a display device with video and audio processing functions, which can receive, process, push and play video or audio streams. In some possible cases, the smart TV 120 can refer to video and audio devices such as conference tablets, smart TVs or projectors, which are not limited in this application. For the hardware implementation of smart TV 120, please refer to the following Fig.10 or Fig.11 The description is not repeated here.

[0057] The multiple audio playback devices include: audio playback devices 121 to audio playback devices 124. For example, these audio playback devices may include but are not limited to: speakers, smart speakers, amplifiers, multimedia consoles, or other devices that support audio playback functions, such as virtual reality (VR) terminal devices or augmented reality (AR) terminal devices, etc. For the hardware implementation of the audio playback device, please refer to the following Fig.11 The description is not repeated here.

[0058] In this embodiment, the set-top box 110 and the smart TV 120 can be connected via a channel 125, the set-top box 110 and each audio playback device can also be connected via a channel 125, and the smart TV 120 and each audio playback device can also be connected via a channel 125. Exemplarily, the channel 125 can be a data bus supporting video and audio transmission, for example, the channel 125 refers to a unified multimedia interconnection channel.

[0059] The unified multimedia interconnection channel can support uncompressed video and compressed video transmission, and can also support various advanced features such as Quick Video Transport (QVT), Auto Low Latency Mode (ALLM), and Dynamic Frame Rate Refresh (DFR). In addition, the unified multimedia interconnection channel can also support LPCM format audio defined by IEC 60958 and various HDR protocols, such as the HDR protocol specified in T / UWA 005.1-2022, such as HDR Vivid, etc. The unified multimedia interconnection also supports encryption control protection for data transmission such as audio and video. In some optional implementations, the channel 125 may also refer to other types of data connections, which can realize the functions supported by the above-mentioned unified multimedia interconnection channel.

[0060] The unified multimedia interconnection channel refers to a channel connected based on the unified multimedia interconnection interface provided by the source device and the sink device. The unified multimedia interconnection channel can be used to connect a charger to charge an electronic device (such as the above-mentioned smart TV, set-top box or audio and video playback device), and can also be used to transmit data between electronic devices and peripheral devices, and can also be used to connect headphones to play audio through headphones. The above-mentioned unified multimedia interconnection interface can also be used to connect other electronic devices, such as AR devices. When the unified multimedia interconnection channel is used to implement the data communication function between devices, the unified multimedia interconnection channel can support uncompressed video and compressed video transmission, and can also support various advanced features such as Quick Video Transport (QVT), Auto Low Latency Mode (ALLM), and Dynamic Frame Rate Refresh (DFR). In addition, the unified multimedia interconnection channel can also support LPCM format audio and video defined by IEC60958 and various HDR protocols, such as the HDR protocol specified in T / UWA 005.1-2022, such as HDR Vivid, etc. The unified multimedia interconnection channel also supports encryption control protection for audio and video data transmission. In some optional implementations, the channel 125 may also refer to other types of channels that can implement the functions supported by the above-mentioned unified multimedia interconnection channel.

[0061] In this embodiment, the unified multimedia interconnection interface is suitable for transmitting high-speed signals for audio and video devices, third-party protocol devices (such as USB / PCIe / Ethernet), etc. A device using the unified multimedia interconnection interface is called a unified multimedia interconnection device, and a system consisting of multiple unified multimedia interconnection devices connected together is called a unified multimedia interconnection system. The capabilities provided by the unified multimedia interconnection interface include: ①, bidirectional transmission of audio and video, to meet the audio and video transmission requirements of devices such as televisions, PCs, and mobile phones, and to support audio and video content transmission protection, visual lossless compression, etc. ②, third-party protocol data transmission, to achieve interaction between the unified multimedia interconnection device and the third-party protocol device, such as the supported third-party protocols include USB3, and Ethernet, PCIe, etc. will be supported in the future. ③, bidirectional power supply, to meet the power supply requirements of electronic devices whose power does not exceed a certain power value, such as the certain power value of 480W, 500W or other values.

[0062] In some examples, the communication network formed by the channels used to realize audio and video transmission between devices is also called: audio and video interface network or audio and video interface network, etc. It is worth noting that in this embodiment, channel is a general term, which refers to the communication channel or the medium for signal transmission. Channel can also be called bus, data line, communication channel, communication connection, link or channel, etc., which is not limited in this application. In some optional implementations, the channel can be implemented through wired or wireless connection technology.

[0063] The server 130 may be an application server or an authentication and authorization server. The server 130 may provide video services, game services, message services, music services, authentication and authorization services, etc. In one example, the functions of multiple services may be integrated on the server 130, for example, game services and music services may be deployed on the server 130. In another example, the functions of some services may be integrated on the server 130, for example, some services of the game service and some services of the video service may be deployed on the server 130. The server 130 may also provide multiple virtual machines using virtualization technology, and various services are provided by the virtual machines. The embodiments of the present application do not limit the deployment form of the server. The network device 131 is connected to the server 130 wirelessly or wiredly. Figure 1 This is just a schematic diagram. The network can also include other devices. Figure 1 Not drawn in.

[0064] It can be understood that the above-mentioned audio is a general term. The audio includes multiple audio frames. One audio frame corresponds to a group of ASP messages, and the group of ASP messages carries audio data to be parsed and played. Figure 1 This is just a schematic diagram. The audio transmission system may also include other devices. Figure 1 The embodiments of the present application do not limit the number and type of each device included in the system.

[0065] exist Figure 1 The audio and video transmission system shown is based on Figure 2 This is a schematic diagram of an audio and video codec system provided by the present application. The audio and video codec system includes a source device 210 and a sink device 220. The source device 210 establishes a communication connection with the sink device 220 through a unified multimedia interconnection channel.

[0066] The source device 210 can implement the video and audio encoding function, such as Figure 1 As shown, the source device 210 may be a set-top box 110 or a smart TV 120 . The source device 210 may also be an audio and video control center with audio and video encoding capabilities. For example, the audio and video control center includes one or more servers.

[0067] The source device 210 may include a data source 211 , a pre-processing module 212 , an audio and video transmission adapter 213 , and a communication interface 214 .

[0068] The data source 211 may include or may be any type of electronic device for collecting audio and video, and / or any type of source audio and video generating device, such as a computer graphics processor for generating computer animation scenes or any type of device for acquiring and / or providing source audio and video, or computer generating source audio and video. The data source 211 may be any type of memory or storage for storing the above-mentioned source audio and video. The above-mentioned source audio and video may include multiple audio and video streams or images collected by multiple audio and video collecting devices (such as cameras).

[0069] The preprocessing module 212 is used to receive the source video and audio, and preprocess the source video and audio to obtain pure audio or multiple frames of images. For example, the preprocessing performed by the preprocessing module 212 may include color format conversion (e.g., from RGB to YCbCr), octree structuring, video and audio splicing, audio track merging and deletion, or channel number adjustment, etc.

[0070] The video and audio transmission adapter 213 is used to receive video and audio or images, and encode the audio, image or image to obtain coded data. In some optional situations, the coded stream (coded data) obtained by encoding can also be called a bit stream. If the coded data is obtained by encoding audio data, the bit stream refers to the audio stream.

[0071] The communication interface 214 in the source device 210 can be used to receive encoded data (such as a video stream or an audio stream), and send the encoded data (or a version of the encoded data after any other processing) to another device such as the host device 220 or any other device through a unified multimedia interconnection channel for storage, display, playback or image reconstruction.

[0072] Optionally, the source device 210 includes a bitstream buffer, which is used to store bitstreams corresponding to one or more coding units.

[0073] The above-mentioned sink device 220 can realize the function of video and audio decoding, such as Figure 1 As shown, the sink device 220 may be Figure 1 Any of the smart TV 120 or audio playback devices shown.

[0074] The sink device 220 may include a video and audio playback unit 221 , a post-processing module 222 , a video and audio receiving adapter 223 , and a communication interface 224 .

[0075] The communication interface 224 in the sink device 220 is used to receive the encoded data (or a version of the encoded data after any other processing is performed on the encoded data) from the source device 210 or any other source device such as a storage device.

[0076] The communication interface 214 and the communication interface 224 may be used to communicate via a direct communication link between the source device 210 and the sink device 220, such as a direct wired connection. Figure 2 For more information about the unified multimedia interconnection channel, please refer to Figure 1 The description is not repeated here.

[0077] The communication interface 224 corresponds to the communication interface 214, for example, and can be used to receive transmission data and process the transmission data using any type of corresponding transmission decoding or processing and / or decapsulation to obtain encoded data (such as a video stream or an audio stream).

[0078] The communication interface 224 and the communication interface 214 can be configured as follows: Figure 2 A unidirectional communication interface or a bidirectional communication interface indicated by an arrow pointing from the source device 210 to the corresponding unified multimedia interconnect channel of the sink device 220, and can be used to send and receive messages, etc. to establish a connection, confirm and exchange any other information related to the communication link, or data transmission such as encoded compressed data transmission, etc.

[0079] The video and audio receiving adapter 223 is used to receive encoded data and decode the encoded data to obtain decoded data (video or pure audio, etc.).

[0080] The post-processing module 222 is used to post-process the decoded data to obtain post-processed data (such as an image to be displayed or an audio to be played). The post-processing performed by the post-processing module 222 may include, for example, color format conversion (such as conversion from YCbCr to RGB), octree reconstruction, video and audio splitting and fusion, or any other processing for generating data for output by the video and audio playback unit 221.

[0081] The video and audio playback unit 221 is used to receive post-processed data for display or playback to a user or viewer, etc. The video and audio playback unit 221 may be or include any type of display for representing the reconstructed image, such as an integrated or external display screen or display. For example, the display screen may include a liquid crystal display (LCD), an organic light emitting diode (OLED) display, a plasma display, a projector, a micro LED display, a liquid crystal on silicon (LCoS), a digital light processor (DLP), or any other type of display screen. The video and audio playback unit 221 may include one or more audio playback modules, each of which may refer to a speaker, an intelligent speaker, an amplifier, etc.

[0082] As an optional implementation, the source device 210 and the sink device 220 may transmit the encoded data through a data forwarding device. For example, the data forwarding device may be a router or a switch. It is worth noting that the data forwarding device needs to support a unified multimedia interconnection interface.

[0083] The implementation of the embodiments of the present application will be described in detail below with reference to the accompanying drawings.

[0084] Here, the audio transmission method of the embodiment of the present application is as follows: Figure 2 The source device 210 and the sink device 220 are shown as an example for explanation. Figure 3 A schematic diagram of the process of an audio transmission method provided in this application Figure 1 . The first device 31 is used to implement the function of the source device 210, and the second device 32 is used to implement the function of the sink device 220. In this embodiment, the first device 31 may also be called an audio-visual sending device, an audio-visual sending terminal, an audio sending device, or an audio processing device, and the second device 32 may also be called an audio-visual receiving device, an audio-visual playing device, or an audio receiving device. In this embodiment, the first device 31 and the second device 32 are connected via a channel 33, which may refer to a unified multimedia interconnection channel.

[0085] In a first possible application scenario, the first device 31 may be Figure 1 The set-top box 110 and the second device 32 may be Figure 1 The smart TV 120 in the embodiment of the present invention pushes video and audio data to the smart TV, such as a set-top box.

[0086] In a second possible application scenario, the first device 31 may be Figure 1The set-top box 110 and the second device 32 may be Figure 1 Any audio playback device in, such as any one of the audio playback devices 121 to the audio playback device 124. For example, a set-top box pushes audio data to the audio playback device.

[0087] In a third possible application scenario, the first device 31 may be Figure 1 The smart TV 120 and the second device 32 may be Figure 1 Any audio playback device in the audio playback device, such as any one of the audio playback devices 121 to the audio playback device 124. For example, a smart TV pushes audio data to the audio playback device.

[0088] The above three possible application scenarios are only examples provided in this embodiment and should not be understood as limiting the present application. In other possible examples, the first device 31 may be Figure 1 Any one of the audio playback devices (such as audio playback device 121), the second device 32 is another audio playback device (such as audio playback device 122) different from the aforementioned audio playback device.

[0089] See also Figure 3 The audio transmission method provided in this embodiment includes the following steps S310 to S330.

[0090] S310 , the first device 31 reads capability description information of the second device 32 through the channel 33 .

[0091] The capability description information is used to indicate the audio processing capability of the second device 32. In some possible situations, the capability description information may also be referred to as a capability descriptor, which may include one or more bits of information.

[0092] In an optional example, the second device 32 includes a memory, in which the above-mentioned capability description information is stored. The memory may include, but is not limited to, random access memory (RAM), flash memory, read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), register, hard disk, mobile hard disk, CD-ROM or any other form of storage medium known in the art.

[0093] In another optional example, the second device 32 includes a dedicated register, and the state of the dedicated register is used to indicate the above-mentioned capability description information. If the value indicated by the register is 1, the capability description information indicates that the second device 32 has the ability to process pure audio data packets; if the value indicated by the register is 0, the capability description information indicates that the second device 32 does not have the ability to process pure audio data packets.

[0094] It is worth noting that since the capability description information indicates the audio processing capability of the device, in some cases, the capability description information is also called device comprehensive capability description information. The area in the second device that stores the DCCD information is called the DCCD area or simply DCCD, which is not limited in this application.

[0095] S320: When the capability description information indicates that the second device 32 supports processing of pure audio data packets, the first device 31 generates an audio stream to be sent.

[0096] The audio stream includes one or more data packets. In some examples, if a data packet only includes audio data and does not include video data, the data packet may also be referred to as a pure audio data packet. In addition, when multiple pure audio data packets are transmitted in the form of an audio stream, each data packet in the audio stream may also be referred to as an audio frame, and the present application does not limit the name of the data packet.

[0097] Combination Figure 3 and Figure 4A , an exemplary description of the structure of the audio stream, Figure 4A A schematic diagram of an audio stream provided by this application Figure 1 The audio stream includes multiple data packets (or audio frames), such as data packet 1, data packet 2, data packet 3, data packet 4, data packet 5, etc. Each data packet includes: an audio control message, an audio information frame message and multiple ASP messages carrying audio data. The ASP message is used to transmit the audio sample data or compressed audio sample data of the audio stream. Figure 4A Taking data packet 1 as an example, data packet 1 includes: an audio control message, an audio information frame message and 1280 ASP messages (such as ASP0 to ASP1279).

[0098] The above-mentioned one audio control message and one audio information frame message are used to indicate: the audio format of the audio data in the multiple ASP messages. In this embodiment, the audio control message and the audio information frame message are messages used to describe the ASP message, that is, DIP messages.

[0099] The audio control message is also called an audio control DIP message, and the audio control message includes a head byte (HB) and a data byte. HB is also called a message header of the audio control message, and DB is also called a payload of the audio control message. The structure of the audio control message is exemplarily described below in conjunction with Table 1.

[0100] Table 1

[0101]

[0102]

[0103] The header bytes include HB0 to HB3, the data bytes include DB1 to DB31, and the audio control message also includes a CRC32 check byte. The CRC32 check value carried by the CRC32 check byte is used to verify the accuracy of the aforementioned header bytes and data bytes.

[0104] The audio information frame message is also called the audio information frame DIP message, and the audio control message also includes a header byte (HB) and a data byte (DB). Among them, HB is also called the message header of the audio information frame message, and DB is also called the payload of the audio information frame message. The message format and structure of the audio information frame message can refer to the description of the general technology. In some optional implementation methods, the content of the information frame message transmitted by the unified multimedia interconnection interface shall comply with the definition of the information frame in CTA-861-H and be carried by the information frame DIP message. The information frame DIP message includes the following types: source product description information frame DIP message, audio information frame DIP message and MPEG source information frame DIP message.

[0105] Regarding the mapping relationship between the audio information frame message and CTA-861-H in the embodiment of the present application, the following Table 2 provides the mapping relationship between the message header and payload of the audio information frame DIP message and the information frame in CTA-861-H, as shown in the following Table 2.

[0106] Table 2

[0107]

[0108]

[0109] The above Table 2 is only a schematic illustration of the structure of an audio information frame message in this embodiment and should not be construed as a limitation to the present application.

[0110] In this embodiment, the packet sending mechanism of the audio information frame message is consistent with the packet sending mechanism of the audio control message, that is, in the video mode, when the audio format does not change, it is sent in the vertical blanking area, and it is sent once per frame. When the audio format changes, it is sent immediately in the blanking area. In the pure audio mode, 1280 ASPs are sent once, and when the audio format changes, it is sent immediately.

[0111] When the information in the audio information frame DIP conflicts with the information in the audio control DIP, the information in the audio control DIP takes precedence. The following is an exemplary description in conjunction with the accompanying drawings, which will not be repeated here.

[0112] Regarding the audio format of the audio data in the above ASP message, the audio format includes one or more combinations of the following parameters: audio sampling rate, number of audio channels, parity flags of audio channels, audio encoding type and audio sampling bit width.

[0113] The audio sampling rate refers to how much the recording device samples the analog signal per unit time. The higher the sampling frequency, the more real and natural the waveform of the mechanical wave is. For example, the audio sampling rate may include but is not limited to: 32k Hertz (Hz), 44.1kHz, 48kHz, 88.2khz, 96kHz, 176.4kHz, 192kHz or other values. Combined with the contents of Table 1, the audio sampling frequency is carried in the "AudioSampleFreq" field included in DB1-DB3 in the above audio control message.

[0114] The number of audio channels refers to the number of channels sampled when playing the audio data carried by the ASP message, such as 2 channels, 4 channels, 6 channels, 8 channels, 10 channels, 12 channels, 14 channels, 16 channels, 18 channels, 20 channels, 22 channels, 24 channels, 26 channels, 28 channels, 30 channels, 32 channels, etc. As explained in conjunction with the contents of Table 1, the number of audio channels is carried in the "Number of Audio Channels" field included in DB5 in the above audio control message.

[0115] The parity flag of the audio channel is used to indicate whether the actual number of channels during audio playback is an odd channel or an even channel. This is explained in conjunction with the contents of Table 1. For example, the parity flag of the audio channel is carried in the "audio channel parity flag" field included in DB5. When the actual number of audio channels is an even number, the audio channel number field value is equal to the actual number of audio channels, and the audio channel parity flag is set to 0; when the actual number of audio channels is an odd number, the audio channel number field value is equal to the actual number of audio channels plus 1, and the audio channel parity flag is set to 1.

[0116] The audio coding type refers to the coding method used for the data packet (audio frame). When the audio coding type selects the AudioVivid audio format, the DB7 configuration field takes effect. The host side (second device 32) can ignore this field in the LPCM format and IEC-61937 format.

[0117] The audio sampling bit width is also called the sampling bit number, sampling bit depth or sampling bit depth, which refers to the number of points used to describe the strength of the sound signal. The audio sampling bit width reflects the audio sampling device's sound resolution. The higher the bit number, the finer the sound recording, so it is also called sampling accuracy. The audio sampling bit width directly affects the signal-to-noise ratio / dynamic range of the collected audio signal. A larger audio sampling bit width can provide more possible amplitude values, resulting in a larger dynamic range, lower noise floor and higher fidelity.

[0118] The following takes the transmission of compressed audio data in Audio Vivid format as an example to illustrate the configuration of the audio control message: the audio coding type field value is set to 0x2; the audio channel number field value is set to 0x0, and the real channel number is obtained by parsing the compressed audio code stream on the host side. When the audio coding type field value is 0x2 (Audio Vivid), the host side should ignore the audio information frame message.

[0119] In an optional situation, if the information in the audio information frame message conflicts with the information in the audio control message, the information in the audio control message takes precedence. That is, the priority of the audio control message is higher (or superior) than the audio information frame message. Thus, the problem of errors in parsing data packets in the audio stream by the audio playback device caused by the conflict of information in the two DIP messages can be avoided.

[0120] The above audio control message and the structure and implementation of the audio format are only examples of the data packets (audio frames) provided in this embodiment and should not be construed as limiting the present application. Figures 5 and 6 Other messages that may exist in the data packet are also exemplified and will not be described in detail here.

[0121] Please continue to see Figure 3 , the audio transmission method provided in this embodiment also includes the following step S330.

[0122] S330 : The first device 31 sends the message carried by each data packet in the audio stream through the channel 33 .

[0123] Corresponding to the process of S330 , the second device 32 receives the audio stream sent by the first device 31 through the channel 33 .

[0124] In this embodiment, the first device can directly read the capability description information of the second device through the channel, thereby obtaining the audio processing capability of the second device, and only sends the audio stream to the second device when the second device supports processing pure audio data packets. In this way, the invalid communication process between devices caused by the second device not supporting processing pure audio data packets and the problem of channel bandwidth resources being occupied between devices are avoided.

[0125] In addition, a single data packet (audio frame) in the audio stream only includes two descriptive information messages for describing the audio format of the audio data in the ASP message, namely, an audio control message and an audio information frame message. Compared with the three SDP messages in conventional technology, the number is smaller, and the bandwidth occupied by these two descriptive information messages is also smaller, which is conducive to increasing the amount of effective audio data that can be transmitted by the channel per unit time and improving the audio transmission efficiency between devices.

[0126] After the second device 32 receives the message carried by each data packet in the audio stream, the second device 32 parses the received audio stream and plays the audio content indicated by the audio data in the audio stream.

[0127] Regarding the above Figure 3 The audio transmission process is as follows: Figure 4B A possible implementation is also provided, such as Figure 4B As shown, Figure 4B A schematic diagram of the process of an audio transmission method provided in this application Figure 2 For the introduction of the first device 31 and the second device 32, please refer to the aforementioned Figure 3 The description is not repeated here.

[0128] See also Figure 4B , the audio transmission method provided in this embodiment includes the following steps S410 to S430.

[0129] S410: The first device 31 generates at least one audio frame.

[0130] For the description of the audio frame, please refer to the above Figure 3 , Figure 4A And the following Figure 5 and Figure 6 The description is not repeated here.

[0131] S420: The first device 31 sends at least one audio frame to the second device 32.

[0132] S430: The second device 32 plays audio according to at least one audio frame.

[0133] In this embodiment, since a single audio frame only includes two descriptive information messages for describing the audio format of the ASP message, the computing power required for the second device 32 to parse the audio frame is reduced, and the audio data in the audio frame is not prone to lag problems during playback in the second device 32, which is beneficial to improving the audio playback effect in the second device 32.

[0134] The structure of the data packet (audio frame) is further exemplarily described below in conjunction with the accompanying drawings.

[0135] exist Figure 4A To improve the security of audio streaming, Figure 5 A schematic diagram of an audio stream provided by this application Figure 2 The audio stream includes multiple data packets (or audio frames), such as data packet 1, data packet 2, data packet 3, data packet 4, data packet 5, and so on.

[0136] Figure 5 and Figure 4A The similarities are that each data packet (audio frame) includes, in sequence, an audio control message, an audio information frame message, and multiple ASP messages carrying audio data.

[0137] Figure 5 and Figure 4A The difference is that the data packet (audio frame) also includes one or more encryption description messages (EDP messages) located between an audio information frame message and multiple ASP messages, and the encryption description message is used to indicate the encryption keys of multiple ASP messages. Exemplarily, the EDP message carries parameters related to audio encryption, and the audio playback device (such as the second device 32 mentioned above) can obtain the encryption key of the ASP message according to the parameters carried in the EDP message.

[0138] In a possible example, a data packet (audio frame) has an EDP message, such as Figure 5 Data packet 1 in the ASP message is encrypted using the EDP message, and then, during the audio transmission process, the transmission security of the audio data carried in the ASP message is increased.

[0139] In another possible example, a data packet (audio frame) has multiple EDP messages, such as 2 EDP messages, 3 EDP messages (such as Figure 5In the case where a data packet includes multiple EDP messages, even if there is a packet loss problem during audio transmission, it can be guaranteed to the greatest extent that the audio playback device can receive at least one EDP message, so that the audio playback device can encrypt the ASP message based on the received EDP message. Then, during the audio transmission process, the transmission security and robustness of the audio data carried in the ASP message are increased.

[0140] The above two possible examples are only examples provided in this embodiment and should not be construed as limiting the present application. In this application, regardless of the unicast scenario or the multicast scenario, the audio transmitter can use EDP messages to describe the encryption and decryption method of the audio data, thereby improving the transmission security of the audio.

[0141] exist Figure 5 To further improve the security of audio streaming, Figure 6 A schematic diagram of an audio stream provided by this application Figure 3 The audio stream includes multiple data packets (or audio frames), such as data packet 1, data packet 2, data packet 3, data packet 4, data packet 5, and so on.

[0142] Figure 6 and Figure 4A , Figure 5 The similarities are that each data packet includes, in sequence, an audio control message, an audio information frame message, and a plurality of ASP messages carrying audio data.

[0143] Figure 6 and Figure 5 The difference is that the data packet also includes one or more key distribution messages (KDP messages) located between the encryption description message (EDP message) and multiple ASP messages, and each key distribution message corresponds to an audio playback device.

[0144] Exemplarily, each key distribution message carries the encryption and decryption key of the audio playback device corresponding to the key distribution message, so that when the first device sends an audio stream to multiple audio playback devices (multicast scenario), different audio playback devices can parse the KDP message corresponding to the audio playback device in the data packet, and after obtaining the encryption key in the KDP message, decrypt the ASP message according to the encryption key. In this way, even if the first device sends an audio stream to multiple audio playback devices, if some users use illegal methods to intercept each data packet in the audio stream, since the ASP messages carried by these data packets have been encrypted, the audio data carried by these ASP messages can also be guaranteed to be safe.

[0145] In a possible example, a data packet (audio frame) has a KDP message, such as Figure 6 Data packet 1 in the ASP message is encrypted using the KDP message, and then, during the audio transmission process, the transmission security of the audio data carried in the ASP message is increased.

[0146] In another possible example, a data packet (audio frame) has multiple KDP messages, such as 2 KDP messages, 3 KDP messages (such as Figure 6 In the embodiment of the present application, the number of KDP messages in a single data packet is associated with the number of audio playback devices multicasted by the first device, such as the number of KDP messages in a single data packet is consistent with the number of audio playback devices multicasted by the first device. When the number of audio playback devices multicasted by the first device is 3, each data packet has 3 KDP messages.

[0147] The above two possible examples are only examples provided in this embodiment and should not be construed as limiting the present application. Regardless of the unicast scenario or the multicast scenario, the audio transmitter (first device) can use KDP messages to encrypt and decrypt audio data, thereby improving the transmission security of the audio.

[0148] It is worth noting that in the above FIG. 4A to FIG. 6 In the example of , the number of multiple ASP messages in each data packet is 1280. That is, during the audio transmission process, the first device will continue to send the next audio control message every time it sends 1280 ASP messages.

[0149] However, it should be understood that the inclusion of 1280 ASP messages in each data packet is merely an example provided in this embodiment and should not be construed as a limitation to the present application.

[0150] For example, in some possible situations, a single data packet includes k ASP messages, where k is a positive integer, such as k=128, 256, 512, 640, 2560 or other values, which are not limited in the present application.

[0151] Combined with the above Figures 3 to 6 The content of the present application is used to exemplify the audio transmission method provided by the present application: when the audio and video transmission adapter in the first device only sends audio, the audio control DIP message and the audio information frame DIP message should be sent first. When the advanced digital content protection (ADCP) encryption function is enabled, the first device should send three EDP messages in succession; if the multicast function is also enabled, the first device also needs to send K KDP messages (K is the number of multicast devices).

[0152] Exemplarily, when the audio specification (audio format) of the audio does not change, the first device sends an audio control DIP message, an audio information frame DIP message, an EDP, and a KDP message once every 1280 ASP messages.

[0153] As another example, when the audio specification (audio format) of the audio changes, the first device should immediately update the audio control DIP message, the audio information frame DIP message, the EDP, and the KDP message. Fig. 7A and Figure 7B The embodiment shown.

[0154] When designing an audio / video receiving adapter, the designer needs to consider that it takes some time for the ADCP module to receive the EDP and output the XOR material for decryption. Therefore, the ASP needs to be cached and the decryption can start after the ADCP can output the XOR material.

[0155] The above embodiments, combined with the accompanying drawings, provide an exemplary description of the audio transmission method and data packet structure provided by the present application. However, in the process of audio transmission, it is inevitable that the audio specification (audio format) changes, such as user switching or the audio transmitter adjusting the audio specification (audio format) according to the local operating conditions. In conventional technology, changes in audio specifications (audio format) usually cause plosive sounds in the audio playback device. To solve the above problems, the present application provides a method for transmitting audio data and a method for transmitting audio data. Figures 3 to 6 Based on the example shown, a possible implementation is provided, such as Fig. 7A As shown, Fig. 7A A schematic diagram of the process of an audio transmission method provided in this application Figure 2 The hardware implementation of the first device 31 and the second device 32 may refer to the description of the aforementioned embodiment, which will not be described in detail here.

[0156] See also Fig. 7A The audio transmission method provided in the embodiment of the present application also includes the following steps S710 to S730.

[0157] S710: The first device 31 obtains audio update information.

[0158] The audio update information indicates the change of the audio specifications (audio formats) of the multiple ASP messages. The contents of the audio specifications (audio formats) can refer to the relevant contents of the aforementioned S310, which will not be described in detail here.

[0159] In a possible example, the audio update information is generated according to a user request received by the first device 31. Figure 1When the user requests the smart TV 120 in the example, the user request may refer to a command input by the user through a remote control, a button on the TV, or a touch screen, or a command input by the user through voice, or a command sent by the user through a mobile phone, a tablet computer, or other smart wearable device, or a gesture control command implemented by the user through air operation, etc. For another example, the first device 31 is Figure 1 When the set-top box 110 is in use, the user request may refer to an instruction input by the user through a remote control, a button on the set-top box, etc., or an instruction sent by the user through a mobile phone, a tablet computer or other control device.

[0160] In another possible example, the audio update information is generated by the first device 31 according to the operation of the first device 31. For example, the first device 31 can identify the audio specification (audio format) to be used for the audio data during the process of pushing the audio stream, and generate the audio update information when the audio specification (audio format) of the audio data changes.

[0161] The above two examples are merely examples of audio update information provided in this embodiment. Audio update may also occur in some other possible scenarios, which are not limited in this application.

[0162] S720: The first device 31 updates the descriptive information message in the unsent data packet in the audio stream according to the audio update information to obtain an updated data packet.

[0163] The descriptive information message includes: an updated audio control message and an updated audio information frame message.

[0164] In an optional situation, if the audio stream is encrypted during transmission, the above-mentioned descriptive information message also includes an encrypted description message (EDP message).

[0165] In another optional scenario, if the first device 31 sends an audio stream to multiple devices, the above-mentioned descriptive information message also includes a mask description message (EDP message) and a key distribution message (KDP message).

[0166] For more information about the above descriptive information message, please refer to the above Figures 3 to 6 The illustrated embodiments are not described in detail here.

[0167] S730 . The first device 31 sends the message carried by the updated data packet through the channel 33 .

[0168] Corresponding to the process of S730, the second device 32 receives the message carried by the updated data packet, and parses and plays the audio based on the message carried by the updated data packet. For example, during the audio transmission process, if the audio specification (audio format) has not changed, the first device 31 sends a group of DIP messages for every 1280 ASP messages; if the audio specification (audio format) has changed, the first device 31 immediately sends the updated DIP message.

[0169] It is worth noting that, in addition to the second device 32, the first device 31 can also send messages carried in the updated data packets to other audio playback devices. The specific implementation can refer to the aforementioned multicast scenario, which will not be described in detail here.

[0170] In this way, when there is a change in the audio specification (audio format), the first device can immediately respond to the change in the audio specification (audio format) (audio update information) to update the descriptive information message, and resend the updated descriptive information message and ASP message (i.e., the updated data packet), thereby avoiding the problem of audio transmission errors caused by untimely audio updates between devices, and is conducive to improving the efficiency of the second device in updating the played audio content based on the received updated data packet, thereby enhancing the user experience of playing audio using the second device.

[0171] In order to further reduce the probability of audio playback devices outputting plosive sounds during audio switching, Fig. 7A Based on this, the embodiment of the present application also provides a possible implementation method, such as Figure 7B As shown, Figure 7B A schematic diagram of the process of an audio transmission method provided in this application Figure 3 .exist Fig. 7A On the basis of, the audio transmission method provided in this embodiment also includes the following steps S711, S712, S721 and S722.

[0172] Among them, after the first device 31 obtains the audio update information and before the first device 31 obtains the updated data packet, that is, after S710 and before S720, the audio transmission method provided in this embodiment also includes the following S711 and S712.

[0173] S711. The first device 31 sends a first audio control message to the second device 32.

[0174] Corresponding to the process of S712 , the second device 32 receives the first audio control message sent by the first device 31 .

[0175] The first audio control message carries an audio mute flag whose value is a first value.

[0176] For example, the audio mute flag refers to the "AudioMuteFlag" field in Table 1. Therefore, in some cases, the audio mute flag is also called an audio mute flag or a mute flag; the first value is 1, that is, the first audio control message indicates that the second device 32 is muted.

[0177] S712: The second device 32 responds to the first audio control message and deletes the data stored in the audio buffer.

[0178] Among them, the audio buffer is a storage area in the second device 32 for caching audio data. For example, the storage area can be implemented by but not limited to the following types of memory support: RAM, flash memory, hard disk, mobile hard disk, CD-ROM or any other form of storage medium known in the art, etc.

[0179] After the second device 32 deletes the data stored in the audio buffer, the second device 32 has no audio data to play, that is, the second device 32 is in a mute state.

[0180] Optionally, after receiving the first audio control message, the second device 32 may also refuse to receive the ASP message carrying audio data. Alternatively, after receiving the first audio control message, the second device 32 not only deletes the data stored in the audio buffer, but also discards the non-descriptive information message (non-DIP message). Thus, the second device 32 implements muting (such as Figure 7B T mute ), and avoids the problem of popping sounds when the audio is updated or the audio is switched.

[0181] After the first device 31 obtains the updated data packet, that is, after S720, the audio transmission method provided in this embodiment further includes the following S721 and S722.

[0182] S721. The first device 31 sends a second audio control message to the second device 32.

[0183] Corresponding to the process of S721 , the second device 32 receives the second audio control message sent by the first device 31 .

[0184] The second audio control message carries an audio mute flag with a second value. The difference between the second audio control message and the first audio control message is that the values ​​of the audio mute flags of the two are different, and the value of the audio mute flag in the second audio control message can be a second value, such as the second value is 0, that is, the second audio control message instructs the second device 32 to turn off the mute and start playing the audio.

[0185] S722. The second device 32 responds to the second audio control message and receives the ASP message carrying audio data.

[0186] In an embodiment of the present application, an audio mute flag having a first value is used to indicate that the second device is muted, and an audio mute flag having a second value is used to indicate that the second device turns off the mute and plays audio content of a new audio specification (audio format), thereby preventing the second device from outputting audio with user-perceptible errors such as noise or plosive sounds during the process of updating the audio specification (audio format), which is beneficial to improving the user experience of playing audio through the second device.

[0187] above Figure 7B In the illustrated embodiment, the first value is 1 and the second value is 0, but in some optional cases, the first value may be 0 and the second value may be 1, which is not described in detail in this application. It is worth noting that when different devices negotiate to use more bits to record the aforementioned audio mute flag, the first value and the second value may also be other values.

[0188] For the above Fig. 7A and Figure 7B The present application embodiment also provides a possible specific example, such as Figure 8 As shown, Figure 8 A schematic diagram of the process of an audio transmission method provided in this application Figure 5 .exist Figure 8 In the example, the function of the source side can be implemented by the aforementioned first device 31, and the function of the sink side can be implemented by the aforementioned second device 32. The audio transmission method provided in this embodiment includes the following steps ① to ⑥.

[0189] Step ①: The software solute audio in the source side is about to change, such as obtaining the audio change flag.

[0190] Step ②: For source measurement, after obtaining (such as receiving) the audio change flag, an audio control DIP message with AudioMuteFlag=1 should be sent.

[0191] Step 3: For the sink side, after receiving the audio control DIP message with AudioMuteFlag=1, it should perform mute processing and clear the audio buffer. Before receiving the audio control DIP with AudioMuteFlag=0, it will not receive the ASP message.

[0192] Step ④: For source measurement, perform audio changes, and after the audio changes are completed, ensure that all data in the ASP are data of the switched format (audio format / audio specification / standard).

[0193] Step ⑤: For the source side, wait for the audio change to be completed and T mute After more than 1us, an audio control DIP message with AudioMuteFlag=0 is sent.

[0194] Step ⑥: For the sink side, after receiving the audio control DIP message with AudioMuteFlag=0, it starts to receive the ASP message.

[0195] DB1-DB3 of the audio control DIP message payload jointly identify the audio sampling frequency, namely AudioSampleFreq. The sampling rate identified by AudioSampleFreq should deviate from the actual audio sampling rate by no more than 0.5%.

[0196] The audio control DIP message payload DB4 identifies the audio encoding type, supporting the uncompressed LPCM encoding format, the compressed IEC-61937 format, and the Audio Vivid audio format. The payload DB5 identifies the number of audio channels. When the actual number of audio channels is an even number, the audio channel number field value is equal to the actual number of audio channels, and the audio channel parity flag is set to 0; when the actual number of audio channels is an odd number, the audio channel number field value is equal to the actual number of audio channels plus 1, and the audio channel parity flag is set to 1. The payload DB6 identifies the audio sampling bit width.

[0197] The DB7 configuration field takes effect only when the Audio Vivid audio format is selected for the audio encoding type. The sink side can ignore this field in the LPCM format and IEC-61937 format. In order to provide more flexibility for different transmission media versions with different bandwidths, Audio Vivid selects four different repetition periods for each frame length. For Audio Vivid with a frame length of 1024, four options are provided: 1024, 4096, 8192, and 16384. For Audio Vivid with a frame length of 512, four options are provided: 512, 2048, 4096, and 8192.

[0198] For the reserved domain, the source side should fill 0x00; the sink side should have the ability to identify and ignore the reserved domain. For future versions of the standard protocol applicable to this application, the reserved domain may become a valid domain. For example, it is recommended that the sink side can extend the ability to parse the reserved domain to facilitate version upgrades.

[0199] It is understandable that, in order to implement the functions in the above-mentioned embodiments, the first device and the second device include hardware structures and / or software modules corresponding to the execution of each function. It should be easily appreciated by those skilled in the art that, in combination with the units and method steps of each example described in the embodiments disclosed in this application, the present application can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is executed in the form of hardware or computer software driving hardware depends on the specific application scenario and design constraints of the technical solution.

[0200] Combined with the above Figures 1 to 8 , describes in detail the audio transmission method provided according to this embodiment, and will now be combined with Fig. 9 , describing the audio device provided according to this embodiment.

[0201] Fig. 9 Schematic diagram of the structure of an audio device provided by the present application. The audio device 900 can be used to implement the functions of any one of the devices in the above method embodiments, and thus can also achieve the beneficial effects of the above method embodiments. In this embodiment, the audio device 900 can be as follows: Figure 1 The set-top box 110, smart TV 120 or any audio playback device shown may also be Figure 2 The source device 210 or sink device 220 shown may also be the first device or the second device provided in the subsequent embodiments. It should be understood that the audio device 900 may also be a module (such as a chip) applied to any of the aforementioned devices.

[0202] like Fig. 9 As shown, the audio device 900 includes: a transceiver module 910 and a processing module 920 .

[0203] For example, when the audio device 900 is used to perform the function of the first device in the above embodiment, the processing module 920 is used to generate at least one audio frame, the audio frame includes: an audio control message, an audio information frame message and multiple ASP messages carrying audio data, an audio control message and an audio information frame message are used to indicate: the audio format of the audio data. The transceiver module 910 is used to send at least one audio frame.

[0204] Optionally, the processing module 920 is further used to notify the audio change and update the audio control message and the audio information frame message. The transceiver module 910 is further used to send the updated audio control message and the audio information frame message.

[0205] Optionally, after notifying the audio change and before updating the audio control message and the audio information frame message, the transceiver module 910 is further used to send a first audio control message, the first audio control message carrying an audio mute flag with a first value. After updating the audio control message and the audio information frame message, the transceiver module 910 is further used to send a second audio control message, the second audio control message carrying an audio mute flag with a second value.

[0206] For another example, when the audio device 900 is used to perform the function of the second device in the above embodiment, the transceiver module 910 is used to receive at least one audio frame, the audio frame includes: an audio control message, an audio information frame message and multiple ASP messages carrying audio data, an audio control message and an audio information frame message are used to indicate: the audio format of the audio data. The processing module 920 is used to play audio according to the at least one audio frame.

[0207] Optionally, the transceiver module 910 is further used to receive a first audio control message from the first device, the first audio control message carrying an audio mute flag with a first value. The processing module 920 is further used to respond to the first audio control message and delete the data stored in the audio buffer. The transceiver module 910 is further used to receive a second audio control message from the first device, the second audio control message carrying an audio mute flag with a second value; the processing module 920 responds to the second audio control message and receives an ASP message carrying audio data.

[0208] Optionally, the audio frame further includes: at least one encryption description message, and the encryption description message is used to transmit encryption parameters of the audio data.

[0209] Optionally, the audio frame further includes: at least one key distribution message, each key distribution message corresponds to an audio playback device.

[0210] Optionally, the number of the multiple ASP messages is 1280.

[0211] The transceiver module 910 and the processing module 920 can cooperate to implement each step in the above method embodiment. A more detailed description of the transceiver module 910 and the processing module 920 can be directly obtained by referring to the relevant description of the device in the method embodiment shown in the above figures, and will not be repeated here.

[0212] When the audio device implements the audio transmission method shown in any of the above-mentioned figures through software, the audio device and its various units may also be software modules. The above-mentioned audio transmission method is implemented by calling the software module through a processor. The processor may be a central processing unit (CPU), an application-specific integrated circuit (ASIC), or a programmable logic device (PLD), and the above-mentioned PLD may be a complex programmable logical device (CPLD), a field programmable gate array (FPGA), a generic array logic (GAL) or any combination thereof.

[0213] Understandably, Fig. 9 The audio device shown is only an example provided in this embodiment. The audio device may include more or fewer units according to different audio transmission processes, and this application is not limited to this.

[0214] When the audio device is implemented by hardware, the hardware may be implemented by a processor or a chip system. The chip system includes one or more chips, each chip including an interface circuit and a control circuit. The interface circuit is used to receive data from other devices outside the chip and transmit it to the control circuit, or to send data from the control circuit to other devices outside the chip.

[0215] The control circuit and the interface circuit are used to implement any possible implementation method in the above embodiments through logic circuits or execution code instructions. The beneficial effects can be found in the description of any aspect of the above embodiments, which will not be repeated here.

[0216] It is understood that the processor in the embodiments of the present application may be a CPU, or other general-purpose processors, digital signal processors (DSP), ASICs, FPGAs or other programmable logic devices, transistor logic devices, hardware components or any combination thereof. The general-purpose processor may be a microprocessor or any conventional processor.

[0217] in addition, Fig. 9 The audio device 900 shown may also be processed by an audio processing device such as Fig.10 As shown, Fig.10 The structural diagram of the audio processing device provided in the present application, the audio processing device 1000 includes: a memory 1010 and at least one processor 1020, the processor 1020 can implement the audio transmission method provided in the above embodiment, and the memory 1010 is used to store the software instructions corresponding to the above audio transmission method. As an optional implementation, in hardware implementation, the audio processing device 1000 may refer to a chip or chip system encapsulated with one or more processors 1020. For example, when the audio processing device 1000 is used to implement the method steps in the above embodiment, the processor 1020 included in the audio processing device 1000 executes the steps of the first device 31 in the above method and its possible sub-steps. In an optional situation, the audio processing device 1000 may also include a communication interface 1030, which may be used to send and receive data. For example, the communication interface 1030 is used to receive audio data, or send an audio stream, etc.; the communication interface 1030 may be implemented by an interface circuit included in the audio processing device 1000. Therefore, in some examples, the communication interface 1030 may also be referred to as a transceiver of the audio processing device. In this embodiment, the communication interface 1030 supports wired connection using a unified multimedia interconnection interface.

[0218] In the embodiment of the present application, the communication interface 1030, the processor 1020 and the memory 1010 can be connected via a bus 1040, and the bus 1040 can be divided into an address bus, a data bus, a control bus, etc. The bus 1040 can be a Peripheral Component Interconnect Express (PCIe) bus, or an extended industry standard architecture (EISA) bus, a unified bus (Ubus or UB), a computer express link (CXL), a cache coherent interconnect for accelerators (CCIX), a unified multimedia interconnect channel or other types of buses, etc.

[0219] It is worth noting that the audio processing device 1000 can also perform Fig. 9 The functions of the audio device 900 shown are not described in detail here.

[0220] The audio processing device 1000 provided in this embodiment may be the above set-top box 110, smart TV 120, source device 210 or first device 31, etc., or other audio devices with audio processing functions, which are not limited in this application. For example, when the aforementioned audio playback device also has an audio processing function, the audio processing device 900 may refer to any one of the aforementioned audio playback devices.

[0221] in addition, Fig. 9 The audio device 900 shown can also be implemented by an audio playback device. When the audio device 900 is implemented by an audio playback device, this embodiment provides a possible example, such as Fig.11 As shown, Fig.11 The audio playback device 200 includes: a processor 210, an external memory interface 220, an internal memory 221, a universal serial bus (USB) interface 230, a unified multimedia interconnection interface 231, an antenna 1, an antenna 2, a mobile communication module 250, a wireless communication module 260, an audio unit 270, a speaker 270A, a receiver 270B, a microphone 270C, an earphone interface 270D, a sensor module 280, a button 290, a motor 291, an indicator 292, a camera 293, a display screen 294, and a subscriber identification module (SIM) card interface 295.

[0222] Among them, the above-mentioned sensor module 280 may include sensors such as pressure sensor, gyroscope sensor, air pressure sensor, magnetic sensor, acceleration sensor, distance sensor, proximity light sensor, fingerprint sensor, temperature sensor, touch sensor, ambient light sensor and bone conduction sensor.

[0223] It is to be understood that the structure illustrated in this embodiment does not constitute a specific limitation on the audio playback device 200. In other embodiments, the audio playback device 200 may include more or fewer components than shown in the figure, or combine some components, or split some components, or arrange the components differently. The components shown in the figure may be implemented in hardware, software, or a combination of software and hardware.

[0224] The processor 210 may include one or more processing units, for example, the processor 210 may include an application processor (AP), a modem processor, a graphics processor (GPU), an image signal processor (ISP), a controller, a memory, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), etc. Different processing units may be independent devices or integrated into one or more processors.

[0225] The controller may be the nerve center and command center of the audio playback device 200. The controller may generate an operation control signal according to the instruction operation code and the timing signal to complete the control of fetching and executing instructions.

[0226] The processor 210 may also be provided with a memory for storing instructions and data. In some embodiments, the memory in the processor 210 is a cache memory. The memory may store instructions or data that the processor 210 has just used or cyclically used. If the processor 210 needs to use the instruction or data again, it may be directly called from the memory. This avoids repeated access, reduces the waiting time of the processor 210, and thus improves the efficiency of the system.

[0227] In some embodiments, the processor 210 may include one or more interfaces. The interface may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a USB interface, etc.

[0228] It is understandable that the interface connection relationship between the modules shown in this embodiment is only a schematic illustration and does not constitute a structural limitation on the audio playback device 200. In other embodiments, the audio playback device 200 may also adopt different interface connection methods in the above embodiments, or a combination of multiple interface connection methods.

[0229] The wireless communication function of the audio playback device 200 can be implemented through antenna 1, antenna 2, mobile communication module 250, wireless communication module 260, modem processor and baseband processor, etc. In some embodiments, the antenna 1 of the audio playback device 200 is coupled with the mobile communication module 250, and the antenna 2 is coupled with the wireless communication module 260, so that the audio playback device 200 can communicate with the network and other devices through wireless communication technology.

[0230] The wired communication function of the audio playback device 200 can be implemented through the USB interface 230 or the unified multimedia interconnection interface 231. For example, the audio playback device 200 receives or sends an audio stream through a channel connected to the unified multimedia interconnection interface 231.

[0231] The audio playback device 200 implements the display function through a GPU, a display screen 294, and an application processor. The GPU is a microprocessor for image processing, which connects the display screen 294 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. The processor 210 may include one or more GPUs, which execute program instructions to generate or change display information.

[0232] The display screen 294 is used to display images, videos, etc. The display screen 294 includes a display panel.

[0233] The audio playback device 200 can realize the shooting function through the ISP, the camera 293, the video codec, the GPU, the display screen 294 and the application processor. The ISP is used to process the data fed back by the camera 293. The camera 293 is used to capture a static image or a video. In some embodiments, the audio playback device 200 may include 1 or N cameras 293, where N is a positive integer greater than 1.

[0234] The external memory interface 220 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the audio playback device 200. The external memory card communicates with the processor 210 through the external memory interface 220 to implement a data storage function. For example, files such as music and videos are stored in the external memory card.

[0235] The internal memory 221 may be used to store computer executable program codes, which may include instructions. The processor 210 executes various functional applications and data processing of the audio playback device 200 by running the instructions stored in the internal memory 221. For example, in an embodiment of the present application, the processor 210 may execute the instructions stored in the internal memory 221, and the internal memory 221 may include a program storage area and a data storage area.

[0236] The program storage area may store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc. The data storage area may store data created during the use of the audio playback device 200 (such as audio data, a phone book, etc.), etc. In addition, the internal memory 221 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, a universal flash storage (UFS), etc.

[0237] The audio playback device 200 can implement audio functions such as music playback and recording through the audio unit 270, the speaker 270A, the receiver 270B, the microphone 270C, the headphone interface 270D, and the application processor.

[0238] The button 290 includes a power button, a volume button, etc. The button 290 may be a mechanical button. It may also be a touch button. The motor 291 may generate a vibration prompt. The motor 291 may be used for an incoming call vibration prompt or for touch vibration feedback. The indicator 292 may be an indicator light, which may be used to indicate the charging status, the change in power, or may be used to indicate messages, missed calls, notifications, etc.

[0239] The method steps in the embodiments of the present application can also be implemented by a processor executing software instructions. The software instructions can be composed of corresponding software modules, and the software modules can be stored in random access memory (RAM), flash memory, read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), registers, hard disks, mobile hard disks, CD-ROMs or any other form of storage medium known in the art. An exemplary storage medium is coupled to the processor so that the processor can read information from the storage medium and write information to the storage medium. Of course, the storage medium can also be a component of the processor. The processor and the storage medium can be located in an ASIC. In addition, the ASIC can be located in a network device or a terminal device. Of course, the processor and the storage medium can also be present as discrete components in an audio processing device and an audio playback device.

[0240] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented by software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer programs or instructions. When the computer program or instruction is loaded and executed on a computer, the process or function described in the embodiment of the present application is executed in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, a network device, a user device or other programmable device. The computer program or instruction may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer program or instruction may be transmitted from one website site, computer, server or data center to another website site, computer, server or data center by wired or wireless means. The computer-readable storage medium may be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium may be a magnetic medium, for example, a floppy disk, a hard disk, a tape; it may also be an optical medium, for example, a digital video disc (DVD); it may also be a semiconductor medium, for example, a solid state drive (SSD).

[0241] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed in the present application, and these modifications or replacements should be included in the protection scope of the present application. Therefore, the protection scope of the present application shall be based on the protection scope of the claims.

Claims

1. An audio transmission method, characterized in that: The method comprises: Generate at least one audio frame, the audio frame comprising: an audio control message, an audio information frame message and a plurality of audio sampling ASP messages carrying audio data, the audio control message and the audio information frame message are used to indicate: an audio format of the audio data; The at least one audio frame is transmitted.

2. The method according to claim 1, characterized in that The audio frame further includes: at least one encryption description message, and the encryption description message is used to transmit encryption parameters of the audio data.

3. The method according to claim 2, characterized in that The audio frame further includes: at least one key distribution message, each key distribution message corresponding to an audio playback device.

4. The method according to any one of claims 1 to 3, characterized in that The number of the multiple ASP messages is 1280.

5. The method according to any one of claims 1 to 4, characterized in that The method further comprises: Notification of audio changes; Update audio control message and audio information frame message; Send the updated audio control message and audio information frame message.

6. The method according to claim 5, characterized in that The audio format includes one or more combinations of the following parameters: audio sampling rate, number of audio channels, parity flags of audio channels, audio encoding type, and audio sampling bit width.

7. The method according to claim 5 or 6, characterized in that: After the notification of audio change and before the update audio control message and the audio information frame message, sending a first audio control message, wherein the first audio control message carries an audio mute flag having a first value; After the updating audio control message and the audio information frame message, a second audio control message is sent, wherein the second audio control message carries an audio mute flag having a second value.

8. The method according to any one of claims 1 to 7, characterized in that The method is executed by a first device, and the first device is connected to a second device via a unified multimedia interconnection channel.

9. An audio transmission method, characterized in that: The method comprises: Receive at least one audio frame, the audio frame comprising: an audio control message, an audio information frame message and a plurality of audio sampling ASP messages carrying audio data, the audio control message and the audio information frame message being used to indicate: an audio format of the audio data; Audio is played according to the at least one audio frame.

10. The method according to claim 9, characterized in that The audio frame further includes: at least one encryption description message, and the encryption description message is used to transmit encryption parameters of the audio data.

11. The method according to claim 10, characterized in that The audio frame further includes: at least one key distribution message, each key distribution message corresponding to an audio playback device.

12. The method according to any one of claims 9 to 11, characterized in that: The number of the multiple ASP messages is 1280.

13. The method according to any one of claims 9 to 12, characterized in that: The method is performed by a second device, the second device comprising an audio buffer for caching audio data, and the method further comprises: receiving a first audio control message from a first device, wherein the first audio control message carries an audio mute flag having a first value; In response to the first audio control message, delete the data stored in the audio buffer; receiving a second audio control message from the first device, wherein the second audio control message carries an audio mute flag having a second value; In response to the second audio control message, an ASP message carrying audio data is received.

14. The method according to claim 13, characterized in that The first device and the second device are connected via a unified multimedia interconnection channel.

15. An audio device, characterized in that: The audio device comprises: A processing module, configured to generate at least one audio frame, wherein the audio frame includes: an audio control message, an audio information frame message, and a plurality of audio sampling ASP messages carrying audio data, wherein the audio control message and the audio information frame message are used to indicate: an audio format of the audio data; The transceiver module is used to send the at least one audio frame.

16. An audio device, characterized in that: The audio device comprises: A transceiver module, configured to receive at least one audio frame, wherein the audio frame includes: an audio control message, an audio information frame message, and a plurality of audio sampling ASP messages carrying audio data, wherein the audio control message and the audio information frame message are used to indicate: an audio format of the audio data; A processing module is used to play audio according to the at least one audio frame.

17. An audio processing device, characterized in that: include: memories, transceivers, and processors; The memory is used for caching audio data; The processor is used to generate at least one audio frame, the transceiver is used to send the at least one audio frame, and the memory, the transceiver and the processor are used to collaboratively execute the method of any one of claims 1-8.

18. An audio playback device, characterized in that: include: A transceiver, a processor and an audio unit; the transceiver is used to receive at least one audio frame, the processor is used to parse the at least one audio frame and instruct the audio unit to play the audio data carried by the at least one audio frame, and the transceiver, the processor and the audio unit are used to collaboratively execute the method described in any one of claims 9 to 14.

19. An audio transmission system, characterized in that: include: The audio processing device of claim 17 and the audio playback device of claim 18, wherein the audio processing device and the audio playback device are connected via a unified multimedia interconnection channel.

20. A computer-readable storage medium, characterized in that: The computer-readable storage medium comprises computer software instructions, and when the computer software instructions are executed in a computing device, the computing device performs the method of any one of claims 1-8 or the method of any one of claims 9-14.

21. A computer program product, characterized in that When the computer program product is executed on a computer, the computing device executes the method of any one of claims 1 to 8 or the method of any one of claims 9 to 14.

Images