Multimedia data transmission method and multimedia equipment

Abstract

The invention discloses a multimedia data transmission method and multimedia equipment, and relates to the technical field of multimedia. The source device and the sink device are connected through the UMI bus, and the multimedia data are encrypted in the transmission process in the UMI bus through the EDP message, so that the load data in the multimedia data message cannot be analyzed and obtained under the condition that the sink device does not support the analysis of the EDP message, the load data in the multimedia data message are effectively protected, and the user experience is improved. And the data security between the devices connected through the bus is improved.

Description

Technical Field

[0001] The present application relates to the field of multimedia technology, and in particular to a multimedia data transmission method and a multimedia device. Background Art

[0002] With the development of big data, artificial intelligence (AI), and cloud computing technology, various types of multimedia data continue to emerge. Taking the wired transmission of multimedia data (such as audio or video) between different devices as an example, different devices can use the display port (DP) bus or the high definition multimedia interface (HDMI) bus to transmit video data. In the DP protocol and the HDMI protocol, multimedia data is transmitted between the source device and the host device through the bus. However, when unauthenticated or unauthorized multimedia devices are connected to the source device through the DP bus or the HDMI bus, these unauthenticated or unauthorized multimedia devices can also receive the multimedia data sent by the source device, that is, these multimedia data cannot be effectively protected and have low security. Summary of the invention

[0003] The present application provides a multimedia data transmission method and a multimedia device, which solves the problem of low data security between a source device and a sink device connected via a bus, and can effectively protect the multimedia data transmitted between devices.

[0004] This application adopts the following technical solution.

[0005] In a first aspect, the present application provides a multimedia data transmission method. The multimedia data transmission method is performed by a source device, the source device is connected to a sink device through a unified multimedia interconnection interface (UMI) bus, and the multimedia data transmission method includes: the source device enables a content protection function and sends a multimedia data stream to the sink device, the multimedia data stream includes one or more data packets, the data packets include: one or more encryption description packets (EDP or EDP message) and multimedia data packets, the encryption description message (EDP) is used to transmit encryption parameters of the multimedia data message.

[0006] In the present application, the source device and the sink device are connected via a UMI bus, and the multimedia data is encrypted using an EDP message during transmission in the UMI bus, so that if the sink device does not support parsing of the EDP message, the payload data in the multimedia data message cannot be parsed and obtained, thereby effectively protecting the payload data in these multimedia data messages and improving the data security between devices connected via the bus.

[0007] In a second aspect, the present application provides a multimedia data transmission method. The multimedia data transmission method is executed by a sink device, the sink device is connected to a source device via a first bus, and the multimedia data transmission method includes: the sink device receives a multimedia data stream sent by a source device, the multimedia data stream includes one or more data packets, the data packets include: one or more EDPs and multimedia data messages, the EDP is used to transmit encryption parameters of the multimedia data messages. And, the sink device parses the multimedia data message according to the encryption description message in the multimedia data stream.

[0008] Since multimedia data is encrypted with EDP messages during transmission in the UMI bus, if the host device does not support parsing of EDP messages, it is impossible to parse and obtain the payload data in the multimedia data messages, which effectively protects the payload data in these multimedia data messages and improves the data security between devices connected through the bus.

[0009] In combination with the multimedia data transmission method provided in the first aspect and the second aspect, in an optional implementation, the row blanking area of ​​the above-mentioned data packet includes: one or more vertical blanking packets (vertical blanking packets, VBP or VBP packets), and the EDP is located after the VBP. The VBP packet is used to mark the video line where the rising edge (in terms of positive polarity) of the vertical synchronization (Vsync) signal of a video frame is located. The EDP packet is sent after the VBP packet, so that the multimedia data packets after the EDP packet can be encrypted, that is, the payload data in these multimedia data packets can be effectively encrypted, thereby improving the data security between devices.

[0010] In combination with the multimedia data transmission method provided in the first aspect and the second aspect, in an optional implementation, the EDP includes a first tunnel message header, the first tunnel message header includes a content protection flag, the value of the content protection flag is a first value or a second value, the first value indicates that the EDP is not encrypted, and the second value indicates that the EDP is encrypted. For example, the first value is 0 and the second value is 1. By setting the content protection flag in the tunnel message header of the EDP, and indicating whether the EDP is encrypted or not by setting different values ​​for the content protection flag, it is beneficial to improve the parsing efficiency of the EDP.

[0011] In combination with the multimedia data transmission method provided in the first aspect and the second aspect, in an optional implementation, the first tunnel message header further includes: a first message length and a first type, the value of the first message length is 28 bytes, and the value of the first type is a third value. For example, the third value is 4b'1110(14).

[0012] In combination with the multimedia data transmission method provided in the first aspect and the second aspect, in an optional implementation, the tail of the EDP provided by the present application carries a cyclic redundancy check (CRC), which is used to verify the accuracy of encryption parameters in the EDP.

[0013] In combination with the multimedia data transmission method provided in the first aspect and the second aspect, in an optional implementation, the row blanking area of ​​the data packet also includes: one or more key distribution packets (KDP or KDP message), the KDP is located after the EDP, and the KDP is used to transmit the encryption key of the multimedia data message. Exemplarily, each key distribution message corresponds to a multimedia playback device (such as a TV or stereo). For example, the number of KDPs in each data packet is associated with the number of host devices sent by the source device multicast, such as the number of KDPs and the number of audio playbacks are consistent, and different KDPs correspond to different host devices, so that the host device corresponding to the KDP parses the EDP based on the KDP to determine the encryption key of the ASP message. Thus, the host device parses the ASP message in the data packet according to the determined encryption key to obtain the audio data carried by these ASP messages, and plays the audio content corresponding to the audio data.

[0014] In combination with the multimedia data transmission method provided in the first aspect and the second aspect, in an optional implementation, the aforementioned KDP includes a second tunnel message header, the second tunnel message header includes: a second message length, a second type and a tag, the value of the second message length is 48 bytes, the value of the second type is a fourth value, and the value of the content protection flag in the tag is 0. For example, the fourth value is 4b'1111(15).

[0015] In combination with the multimedia data transmission method provided by the first aspect and the second aspect, in an optional implementation manner, the multimedia data message provided by the present application includes: one or both of an audio sample message (audio sample packet, ASP or ASP message) or an active video message (active video packet, AVP or AVP message).

[0016] In combination with the multimedia data transmission method provided in the first aspect and the second aspect, in an optional implementation, the multimedia data message provided in the present application includes a tunnel message header and payload data, and the value of the content protection flag in the tunnel message is a fifth value, and the fifth value indicates that the payload data has been encrypted. For example, the fifth value is 1.

[0017] In combination with the multimedia data transmission method provided in the first aspect and the second aspect, in an optional implementation, the above-mentioned load data includes multiple encrypted data blocks, wherein the encrypted i-th data block is determined based on the XOR material and the i-th data block to be encrypted, the XOR material is generated based on the counting flag of the EDP and the counting flag of the i-th data block, the counting flag of the EDP is the high 64 bits of the counter, and the counting flag of the i-th data block is the low 64 bits of the counter.

[0018] In combination with the multimedia data transmission method provided by the first aspect and the second aspect, in an optional implementation, the count mark of the encrypted i-th data block is the sixth value, and the count mark of the i+1-th data block in the encrypted multiple data blocks is the seventh value. If the sixth value is less than the first maximum value, the seventh value is the sixth value plus one; if the sixth value is equal to the first maximum value, the seventh value is 0.

[0019] In combination with the multimedia data transmission method provided by the first aspect and the second aspect, in an optional implementation, the count mark of the j-th EDP in the multimedia data stream is the eighth value, and the count mark of the j+1-th EDP in the multimedia data stream is the ninth value. If the eighth value is less than the second maximum value, the ninth value is the eighth value plus one; if the eighth value is equal to the second maximum value, the ninth value is 0.

[0020] In a third aspect, the present application provides a multimedia transmission device, which includes a module for executing the method of any one of the implementation modes of the first aspect or the second aspect.

[0021] In a fourth aspect, the present application provides a multimedia device. The multimedia device includes a processor and a transceiver. Exemplarily, the processor is used to enable a content protection function, and cooperates with the transceiver to perform the method of the first aspect and any optional implementation of the first aspect. Also exemplarily, the transceiver is used to receive a video stream, and cooperates with the processor to perform the method of the second aspect and any optional implementation of the second aspect.

[0022] In a fifth aspect, the present application provides a multimedia data transmission system. The multimedia data transmission system includes two multimedia devices provided in the fourth aspect, wherein the multimedia device for sending multimedia data is a source device, and the multimedia device for receiving multimedia data is a sink device. The source device can be used to implement the function of the source device in the first aspect or the second aspect, and the sink device can be used to implement the function of the sink device in the first aspect or the second aspect. Therefore, the multimedia data transmission system can also achieve the beneficial effects of the methods in the first aspect and the second aspect, which will not be repeated here.

[0023] In a sixth aspect, the present application provides a computer-readable storage medium. The computer-readable storage medium includes: computer software instructions. When the computer software instructions are executed in a computing device, the computing device executes the operation steps of the method in the first aspect or any possible implementation of the first aspect, and executes the operation steps of the method described in the second aspect or any possible implementation of the second aspect. For example, the computing device is the aforementioned source device or sink device, etc.

[0024] In a seventh aspect, the present application provides a computer program product. When the computer program product is run on a computer, the computing device executes the operation steps of the method in the first aspect or any possible implementation of the first aspect, and executes the operation steps of the method described in the second aspect or any possible implementation of the second aspect. For example, the computer is the aforementioned source device or sink device.

[0025] Regarding the beneficial effects of the third to seventh aspects, reference may be made to the description of any implementation in the first or second aspect, and no further description is given here. Based on the implementations provided in the above aspects, this application may also be further combined to provide more implementations. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Figure 1 A schematic diagram of a video transmission system provided for this application;

[0027] Figure 2 A schematic diagram of a multimedia encoding and decoding system provided by this application;

[0028] Figure 3 A flowchart of a multimedia data transmission method provided by this application;

[0029] Figure 4 A schematic diagram of the structure of a video frame provided in this application;

[0030] Figure 5 A schematic diagram of the structure of an EDP message provided for this application;

[0031] Figure 6A schematic diagram of the structure of a KDP message provided for this application;

[0032] Figure 7 A schematic diagram of the multimedia data encryption and decryption method provided by this application;

[0033] Figure 8 A schematic diagram of the structure of a multimedia transmission device provided by this application;

[0034] Fig. 9 A schematic diagram of the structure of the multimedia device provided in this application. DETAILED DESCRIPTION

[0035] The present application provides a multimedia data transmission method, in which a source device and a sink device are connected via a UMI bus, and multimedia data is encrypted using an EDP message during transmission in the UMI bus, so that when the sink device does not support parsing of the EDP message, it is impossible to parse and obtain the payload data in the multimedia data message, thereby effectively protecting the payload data in the multimedia data message and improving the data security between devices connected via the bus.

[0036] The technical solutions involved in this application may not only be applied to current audio and video transmission technologies or audio and video standards, but may also be applied to future audio and video transmission technologies or audio and video standards. The terms used in the implementation methods of this application are only used to explain the specific embodiments of this application and are not intended to limit this application. The following is a brief introduction to some concepts that may be involved in this application.

[0037] Multimedia data: refers to the general term for language, text, data, audio, image or video data.

[0038] Multimedia data message: a message used to carry multimedia data.

[0039] Multimedia data stream: A data stream transmitted from a source device to a sink device in the form of a continuous media stream, including multiple continuous multimedia data packets.

[0040] Video streaming refers to the transmission of video data, for example, a video stream can be processed as a stable and continuous stream through a network. A video stream includes multiple video frames, each of which corresponds to a frame of image.

[0041] Descriptive information packet (DIP or DIP message): used to describe different data in a data packet.

[0042] In this embodiment, video is a general term, and video includes a sequence of multiple continuous frames, where one frame corresponds to one image. Video and audio are information application technology terms, and refer to video, audio, or multimedia content including video and audio.

[0043] In order to make the description of the following embodiments clear and concise, an introduction to the related art is first given.

[0044] Figure 1 A schematic diagram of a video transmission system provided by the present application. The video processing process may include but is not limited to: video acquisition, video encoding, video transmission, video decoding and playback process.

[0045] Figure 1 The video and audio transmission system in the embodiment includes a set-top box 110, a smart TV 120, a plurality of video and audio playback devices and a server 130. The set-top box 110 is connected to the operator network through a network cable and can receive video and audio streams from the server 130. The network can realize the function of video and audio transmission, and the network can include one or more network devices, such as a network device 131 which can be a router or a switch. In some optional implementations, the set-top box 110 and the server 130 can also communicate through wireless communication, which is not limited in this application.

[0046] The set-top box 110 is a video and audio processing device, which is used to realize functions such as receiving, processing and pushing video streams or video and audio streams. In some possible situations, the set-top box 110 may also be called an Internet TV set-top box, a network high-definition player or other. Exemplarily, the set-top box 110 may refer to a TV box provided by a network operator or a TV box purchased by a user. For the hardware implementation of the set-top box 110, please refer to the following Fig. 9 The description is not repeated here.

[0047] Smart TV 120 is a display device with video and audio processing functions, which realizes functions such as receiving, processing, pushing and playing video streams or video and audio streams. In some possible situations, the smart TV 120 may refer to video and audio devices such as conference tablets, smart TVs or projectors, which are not limited in this application. For the hardware implementation of smart TV 120, please refer to the following Fig. 9 The description is not repeated here.

[0048] The plurality of video and audio playback devices include: video and audio playback devices 121 to video and audio playback devices 124. For example, these video and audio playback devices may include but are not limited to: a multimedia control platform or other devices supporting video and audio playback functions, such as a virtual reality (VR) terminal device or an augmented reality (AR) terminal device, etc. For the hardware implementation of the video and audio playback device, please refer to the following Fig. 9 The description is not repeated here.

[0049] In this embodiment, the set-top box 110 and the smart TV 120 may be connected via a bus 125, the set-top box 110 and each video and audio playback device may also be connected via the bus 125, and the smart TV 120 and each video and audio playback device may also be connected via the bus 125. Exemplarily, the bus 125 may be a data bus supporting video and video and audio transmission, for example, the bus 125 is a UMI bus.

[0050] The UMI bus refers to a bus connected based on the UMI interface provided by the source device and the sink device. The UMI bus can be used to connect a charger to charge electronic devices (such as the above-mentioned smart TV, set-top box or audio and video playback device), and can also be used to transmit data between electronic devices and peripheral devices. It can also be used to connect headphones and play audio through headphones. The above-mentioned UMI interface can also be used to connect other electronic devices, such as AR devices, etc.

[0051] When the UMI bus is used to implement data communication functions between devices, the UMI bus can support uncompressed video and compressed video transmission, and can also support various advanced features such as Quick Video Transport (QVT), Auto Low Latency Mode (ALLM), and Dynamic Frame Rate Refresh (DFR). In addition, the UMI bus can also support LPCM format audio and video defined by IEC 60958 and various HDR protocols, such as the HDR protocol specified in T / UWA 005.1-2022, such as HDR Vivid, etc. UMI also supports encryption control protection for data transmission such as audio and video. In some optional implementations, the bus 125 may also refer to other types of buses that can implement the functions supported by the above-mentioned UMI bus.

[0052] The server 130 may be an application server or an authentication and authorization server. The server 130 may provide video services, game services, message services, music services, authentication and authorization services, etc. In one example, the functions of multiple services may be integrated on the server 130, for example, game services and music services may be deployed on the server 130. In another example, the functions of some services may be integrated on the server 130, for example, some services of the game service and some services of the video service may be deployed on the server 130. The server 130 may also provide multiple virtual machines using virtualization technology, and various services are provided by the virtual machines. The embodiments of the present application do not limit the deployment form of the server. The network device 131 is connected to the server 130 wirelessly or wiredly. Figure 1 This is just a schematic diagram. The network can also include other devices. Figure 1 Not drawn in.

[0053] It can be understood that the above-mentioned video and audio are a general term. The video and audio include multiple video frames. One video frame corresponds to a group of messages, and the group of messages carries multimedia data to be parsed and played. Figure 1 This is just a schematic diagram. The video and audio transmission system may also include other equipment. Figure 1 The embodiments of the present application do not limit the number and type of each device included in the system.

[0054] exist Figure 1 Based on the video and audio transmission system shown, the following exemplary description is given by taking the multimedia data as one or both of the audio data and the video data as an example. Figure 2 A schematic diagram of a multimedia codec system provided by the present application. The multimedia codec system includes a source device 210 and a sink device 220, and the source device 210 establishes a communication connection with the sink device 220 via a UMI bus.

[0055] The source device 210 can implement the video and audio encoding function, such as Figure 1 As shown, the source device 210 may be a set-top box 110 or a smart TV 120 . The source device 210 may also be an audio and video control center with audio and video encoding capabilities. For example, the audio and video control center includes one or more servers.

[0056] The source device 210 may include a data source 211 , a pre-processing module 212 , an audio and video transmission adapter 213 , and a communication interface 214 .

[0057] The data source 211 may include or may be any type of electronic device for collecting audio and video, and / or any type of source audio and video generating device, such as a computer graphics processor for generating computer animation scenes or any type of device for acquiring and / or providing source audio and video, or computer generating source audio and video. The data source 211 may be any type of memory or storage for storing the above-mentioned source audio and video. The above-mentioned source audio and video may include multiple audio and video streams or images obtained by multiple audio and video acquisition devices (such as cameras), such as ultra high definition (UHD) video, high definition (HD) video, 4K video, etc.

[0058] The preprocessing module 212 is used to receive source video and audio, and preprocess the source video and audio to obtain video and audio or multiple frames of images. For example, the preprocessing performed by the preprocessing module 212 may include color format conversion (e.g., conversion from RGB to YCbCr), octree structuring, video and audio splicing, audio track merging and deletion, or channel number adjustment, etc.

[0059] The video and audio transmission adapter 213 is used to receive video and audio or images, and encode the video and audio, images or images to obtain coded data. In some optional situations, the coded stream (coded data) obtained by encoding can also be called a bit stream. If the coded data is obtained by encoding multimedia data, the bit stream refers to the video and audio stream.

[0060] The communication interface 214 in the source device 210 may be used to receive encoded data (such as a video stream or an audio / video stream), and send the encoded data (or a version of the encoded data after any other processing) to another device such as the sink device 220 or any other device through a UMI bus for storage, display, playback, or image reconstruction.

[0061] exist Figure 2 In the example of , the video and audio transmission adapter 213 and the communication interface 214 in the source device 210 may be referred to as the UMI controller of the source device 210 , and the video and audio reception adapter 223 and the communication interface 223 in the sink device 220 may be referred to as the UMI controller of the sink device 220 .

[0062] Optionally, the source device 210 includes a bitstream buffer, which is used to store bitstreams corresponding to one or more coding units.

[0063] The above-mentioned sink device 220 can realize the function of video and audio decoding, such as Figure 1 As shown, the sink device 220 may be Figure 1 Any of the smart TV 120 or audio and video playback devices shown.

[0064] The sink device 220 may include a video and audio playback unit 221 , a post-processing module 222 , a video and audio receiving adapter 223 , and a communication interface 224 .

[0065] The communication interface 224 in the sink device 220 is used to receive the encoded data (or a version of the encoded data after any other processing is performed on the encoded data) from the source device 210 or any other source device such as a storage device.

[0066] The communication interface 214 and the communication interface 224 may be used to communicate via a direct communication link between the source device 210 and the sink device 220, such as a direct wired connection. Figure 2 For more information about the UMI bus, see Figure 1 The description is not repeated here.

[0067] The communication interface 224 corresponds to the communication interface 214, for example, and can be used to receive transmission data and process the transmission data using any type of corresponding transmission decoding or processing and / or decapsulation to obtain encoded data (such as a video stream or an audio / video stream).

[0068] The communication interface 224 and the communication interface 214 can be configured as follows: Figure 2 The unidirectional communication interface indicated by the arrow pointing from the source device 210 to the corresponding UMI bus of the sink device 220, or the bidirectional communication interface, can be used to send and receive messages, etc. to establish a connection, confirm and exchange any other information related to the communication link, or data transmission such as encoded compressed data transmission, etc.

[0069] The video and audio receiving adapter 223 is used to receive encoded data and decode the encoded data to obtain decoded data (video or video and audio, etc.).

[0070] The post-processing module 222 is used to post-process the decoded data to obtain post-processed data (such as an image to be displayed or an audio and video to be played). The post-processing performed by the post-processing module 222 may include, for example, color format conversion (such as conversion from YCbCr to RGB), octree reconstruction, audio and video splitting and fusion, etc., or any other processing for generating data for output by the audio and video playback unit 221.

[0071] The video and audio playback unit 221 is used to receive post-processed data for display or playback to a user or viewer, etc. The video and audio playback unit 221 may be or include any type of display for representing the reconstructed image, such as an integrated or external display screen or display. For example, the display screen may include a liquid crystal display (LCD), an organic light emitting diode (OLED) display, a plasma display, a projector, a micro LED display, a liquid crystal on silicon (LCoS), a digital light processor (DLP), or any other type of display screen. The video and audio playback unit 221 may also include one or more video and audio playback modules, each of which may refer to a speaker, an intelligent speaker, an amplifier, etc.

[0072] As an optional implementation, the source device 210 and the sink device 220 may transmit the encoded data through a data forwarding device. For example, the data forwarding device may be a router or a switch. It is worth noting that the data forwarding device needs to support the UMI interface. It is understandable that the interface connection relationship between the modules illustrated in the embodiment of the present application is only a schematic illustration and does not constitute a structural limitation on the source device and the sink device. In other embodiments of the present application, the structural limitation of the source device and the sink device may also adopt different interface connection methods in the above embodiments, or a combination of multiple interface connection methods.

[0073] The implementation of the multimedia data transmission method provided in the embodiment of the present application will be described in detail below with reference to the accompanying drawings.

[0074] Here, the multimedia data transmission method of the embodiment of the present application is as follows: Figure 2 The source device 210 and the sink device 220 are shown as an example for explanation. Figure 3 A flowchart of a multimedia data transmission method provided in the present application. Among them, the source device 31 is used to implement the function of the source device 210, and the sink device 32 is used to implement the function of the sink device 220. In this embodiment, the source device 31 may also be referred to as an audio-visual sending device, an audio-visual sending terminal or a video sending device, and the sink device 32 may also be referred to as an audio-visual receiving device, an audio-visual playing device or a video receiving device. In this embodiment, the source device 31 and the sink device 32 are connected via a bus 33, and the bus 33 may refer to a UMI bus. For the specific implementation of the UMI bus, please refer to the aforementioned Figure 2 In some examples, the bus 33 may also be referred to as a connection bus, a first bus or other names.

[0075] In a first possible application scenario, the source device 31 may be Figure 1 The set-top box 110 and the sink device 32 may be Figure 1 The smart TV 120 in the embodiment of the present invention pushes multimedia data to the smart TV, such as a set-top box.

[0076] In a second possible application scenario, the source device 31 may be Figure 1 The set-top box 110 and the sink device 32 may be Figure 1 Any video and audio playback device in the video and audio playback device, such as any one of the video and audio playback devices 121 to the video and audio playback device 124. For example, a set-top box pushes multimedia data to the video and audio playback device.

[0077] In a third possible application scenario, the source device 31 may be Figure 1 The smart TV 120 and the sink device 32 may be Figure 1 Any video and audio playback device in the video and audio playback device, such as any one of the video and audio playback devices 121 to 124. For example, a smart TV pushes multimedia data to the video and audio playback device.

[0078] The above three possible application scenarios are only examples provided in this embodiment and should not be construed as limiting the present application. In other possible examples, the source device 31 may be Figure 1 Any one of the audio and video playback devices (such as the audio and video playback device 121), the host device 32 is another audio and video playback device (such as the audio and video playback device 122) different from the aforementioned audio and video playback device.

[0079] See also Figure 3 The multimedia data transmission method provided in this embodiment includes the following steps S310 to S330.

[0080] S310 : The source device 31 enables a content protection function.

[0081] The content protection function means that the source device 31 needs to encrypt data transmitted via the UMI bus to improve the security of the data transmitted via the UMI bus.

[0082] As an optional implementation manner, after the source device 31 and the sink device 32 perform capability negotiation, if the source device 31 determines that the sink device 32 supports the content protection function, the source device 31 enables the content protection function.

[0083] Exemplarily, the sink device 32 provides capability description information, which includes a comprehensive capability statement of the sink device 32, such as whether it supports content protection functions, color display capabilities, and other contents in audio and video processing capabilities, such as audio processing capabilities (whether the sink device supports processing pure audio data packets), HDR display capabilities (whether the sink device supports displaying a specific type of HDR video, such as Brilliant HDR), or others. In some cases, the capability description information may also be referred to as a capability descriptor, which may include one or more bits of information.

[0084] In an optional example, the host device 32 includes a memory in which the capability description information is stored. The memory may include, but is not limited to, random access memory (RAM), flash memory, read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), registers, hard disk, mobile hard disk, CD-ROM or any other form of storage medium known in the art.

[0085] In another optional example, the sink device 32 includes a dedicated register, and the state of the dedicated register is used to indicate the above-mentioned capability description information. If the value indicated by the register is 1, the capability description information indicates that the sink device 32 supports the content protection function; if the value indicated by the register is 0, the capability description information indicates that the sink device 32 does not support the content protection function.

[0086] In some cases, the capability description information is also called device comprehensive capability description (DCCD) information, and the area in the host device storing the DCCD information is called a DCCD area or simply DCCD, which is not limited in this application.

[0087] S320: The source device 31 sends a multimedia data stream to the sink device 32 via the UMI bus.

[0088] The multimedia data stream includes one or more data packets, and the data packets include: one or more encryption description packets (EDPs) and multimedia data packets, wherein the EDP is used to transmit encryption parameters of the multimedia data packets.

[0089] The following is combined with Figure 4, taking the data packet as a video frame as an example, the format of the video frame in the multimedia data stream is described exemplarily, Figure 4 A schematic diagram of the structure of a video frame provided in this application. Figure 4 In the embodiment of the present invention, a video frame includes: a multimedia data message and a timing signal. In some cases, the multimedia data message is also referred to as a multimedia message for short, which will not be repeated later.

[0090] Multimedia data packets include one or both of valid video packets (AVP packets) and audio sample packets (audio samplepacket, ASP or ASP packets). AVP packets are used to transmit valid video data or valid pixels or valid pixel data (or valid video pixel data), and ASP packets are used to transmit audio data. Figure 4 In the video frame, multiple AVP messages are arranged in rows in the video frame, and multiple AVP messages belonging to the same row are a video row.

[0091] It is worth noting that Figure 4 The structure of the video frame is only an example provided by the present application and should not be understood as a limitation of the present application. In some possible implementations, a single video frame may also include only one line of AVP messages, which is not limited by the present application.

[0092] Optionally, when the link bandwidth is greater than the video bandwidth, an ASP can be inserted between two AVPs, but the priority of the ASP is lower than that of the AVP.

[0093] The timing signals include: horizontal synchronization signal (Hsync), vertical synchronization signal (Vsync) and display enable (DE) signal, which are used to describe the video timing of the video, that is, when should what message or signal be transmitted. Hsync represents the start of scanning a row of pixels (such as the signal received by the row register in the host device 32 is 1), Vsync represents the start of scanning a video frame, and display enable DE represents whether valid data can be received (such as DE=1 indicates that valid data can be received, and DE=0 indicates that valid data cannot be received).

[0094] Combination Figure 4, briefly explain the video timing of the video frame: during the blanking period (Hblank) of the total row pixels (Htotal) or the row blanking area, the video frame cannot transmit AVP messages, but can transmit other messages, such as ASP, DIP messages, etc.; during the active period (Hactive) of the total row pixels (Htotal) or the row active area, the video frame can transmit AVP messages. The starting point of Hactive is marked by the horizontal blanking packet (HBP), which indicates the end of the row blanking area. Exemplarily, the audio and video sending adapter in the source device sends HBP for each video line and immediately sends it at the end of Hblank.

[0095] exist Figure 4 In the video frame shown, the line blanking area of ​​the video frame includes a VBP message, which is used to indicate the video line where the Vsync rising edge (in terms of positive polarity) of a video frame is located. Exemplarily, the video and audio transmission adapter in the source device sends one or more VBPs (such as Figure 4 For example, if Vsync is positive polarity, the video and audio transmission adapter replaces HBP with VBP in the video line where the rising edge of Vsync is located, and sends two more VBPs immediately; if Vsync is negative polarity, the video and audio transmission adapter replaces HBP with VBP in the video line where the falling edge of Vsync is located, and sends two more VBPs immediately.

[0096] For the above VBP message, the VBP message includes a tunnel message header and a payload part. Exemplarily, the tunnel message header occupies 4 bytes, the payload part occupies 28 bytes, and a single VBP message occupies 32 bytes in total. The VBP message carries: an identifier indicating the format of the multimedia data message, which is carried in the payload part of the VBP message. In conjunction with Table 1, the tunnel message header of the VBP message is exemplarily described below.

[0097] Table 1: Tunnel header of VBP message

[0098]

[0099] Among them, ShuttleID is used to identify a data transmission channel on the link (Link), and the value is 0 to 127; for the audio and video adapter (audio and video sending adapter or audio and video receiving adapter), this field is the adapter ID set at the factory.

[0100] Combination Figure 4The sending position of the EDP message is exemplarily described. The EDP message is located in the line blanking area of ​​the video frame and is located after the VBP message. Exemplarily, after sending the VBP message, the video and audio sending adapter sends one or more EDP messages ( Figure 4 Three EDP messages are shown in FIG), and are sent in one row.

[0101] In a possible example, a video frame has an EDP message. The ASP message is encrypted using the EDP message, and then, during the multimedia data transmission process, the transmission security of the multimedia data carried in the ASP message is increased.

[0102] In another possible example, a video frame has multiple EDP messages, such as 2 EDP messages, 3 EDP messages (such as Figure 4 In the case where a video frame includes multiple EDP messages, even if there is a packet loss problem during the multimedia data transmission process, it can be guaranteed to the greatest extent that the sink device can receive at least one EDP message, so that the sink device can parse the ASP message based on the received EDP message. Then, during the multimedia data transmission process, the transmission security and robustness of the multimedia data carried in the ASP message are increased.

[0103] The above two possible examples are only examples provided in this embodiment and should not be construed as limiting the present application. In this application, regardless of the unicast scenario or the multicast scenario, the source device can use EDP messages to describe the encryption and decryption methods of multimedia data, thereby improving the transmission security of multimedia data.

[0104] The EDP message includes a tunnel message header and a payload, which is used to transmit the encryption parameters of the aforementioned multimedia data message. For the payload part of the EDP message, please refer to the following Figure 5 The following is an exemplary description of the format of the tunnel message header of the EDP message in conjunction with Table 2.

[0105] Table 2: Tunnel header of EDP message

[0106]

[0107]

[0108] For the audio and video adapter, the ShuttleID field is the adapter ID set at the factory.

[0109] The tunnel message header of the EDP message may also be referred to as a first tunnel message header, and the first tunnel message header includes a content protection flag (CP in Table 2), a first message length (Length in Table 2), and a first type (Type in Table 2).

[0110] The value of the content protection flag is a first value or a second value, the first value indicates that the EDP is not encrypted, and the second value indicates that the EDP is encrypted.

[0111] In Table 2, the first value is 0, and the second value may be 1. However, in some other possible examples, the first value may be 1 and the second value may be 0, which is not limited in this example.

[0112] The difference between the tunnel header of the EDP message and the tunnel header of the VBP message is that the message length value in the EDP message is 28 bytes and the value of Type is 14 (4b'1110). Optionally, the value of the type in the EDP message is also called the third value.

[0113] In this embodiment, by setting a content protection flag in the tunnel message header of the EDP and setting different values ​​for the content protection flag to indicate whether the EDP is encrypted or not, the parsing efficiency of the EDP is improved.

[0114] Please continue to see Figure 4 , the line blanking area of ​​the video frame also includes: one or more KDPs, and the KDP is located after the EDP, and the KDP is used to transmit the encryption key of the multimedia data message. Exemplarily, each key distribution message carries the encryption and decryption key of the host device corresponding to the key distribution message. In the multicast scenario, the source device sends K KDP messages immediately after sending the EDP message (K is the number of multicast devices), and the sending is completed within one line.

[0115] In a possible example, a video frame has a KDP message. The KDP message is used to encrypt the ASP message, and then, during the multimedia data transmission process, the transmission security of the multimedia data carried in the multimedia data message is increased.

[0116] In another possible example, a video frame has multiple KDP packets, such as 2 KDP packets, 3 KDP packets (such as Figure 4 In an embodiment of the present application, the number of KDP messages in a single video frame is associated with the number of sink devices sent by the source device multicast, such as the number of KDP messages in a single video frame is consistent with the number of sink devices sent by the source device multicast. When the number of sink devices sent by the source device multicast is 3, each video frame has 3 KDP messages.

[0117] The above two possible examples are only examples provided in this embodiment and should not be construed as limiting the present application. Regardless of the unicast scenario or the multicast scenario, the multimedia sender (source device) can use KDP messages to encrypt and decrypt multimedia data, thereby improving the transmission security of multimedia data.

[0118] When the source device sends an audio stream to multiple sink devices (multicast scenario), different sink devices can parse the KDP message corresponding to the sink device in the data packet, and after obtaining the encryption key in the KDP message, decrypt the multimedia data message according to the encryption key. In this way, even if the source device sends a multimedia data stream to multiple sink devices, if some users intercept each data packet in the multimedia data stream in an illegal way, since the multimedia data messages carried by these data packets have been encrypted, the multimedia data carried by these multimedia data messages can also be guaranteed to be safe.

[0119] The KDP message includes a tunnel message header and a payload, which is used to transmit the encryption key of the aforementioned multimedia data message. For the payload part of the KDP message, please refer to the following Figure 6 The following is an exemplary description of the format of the tunnel message header of the KDP message in conjunction with Table 3.

[0120] Table 3: Tunnel header of KDP message

[0121]

[0122]

[0123] Among them, the tunnel message header of the KDP message can also be called a second tunnel message header, and the second tunnel message header includes a label (Flags in Table 3), a second message length (Length in Table 3) and a second type (Type in Table 3).

[0124] The value of the content protection flag (CP in Table 3) in the tag is 0, the value of the second message length is 48 bytes, and the value of the second type is a fourth value. For example, the fourth value is fixed to 4b'1111 (15).

[0125] Corresponding to the process of S320 , the sink device 32 receives the multimedia data stream sent by the source device 31 .

[0126] S330 , the sink device 32 parses the multimedia data message according to the EDP in the multimedia data stream.

[0127] Since multimedia data is encrypted with EDP messages during transmission in the UMI bus, if the host device does not support parsing of EDP messages, it is impossible to parse and obtain the payload data in the multimedia data messages, which effectively protects the payload data in these multimedia data messages and improves the data security between devices connected through the bus.

[0128] The following is an exemplary description of the encryption process of multimedia data in combination with the payload part of the EDP message, the payload part of the KDP message and the content included in the multimedia data message.

[0129] First, the payload part of the EDP message and the payload part of the KDP message are introduced.

[0130] like Figure 5 As shown, Figure 5 A schematic diagram of the structure of an EDP message provided for this application. The EDP message includes a tunnel message header and a payload part. For the tunnel message header of the EDP message, please refer to the contents of Table 2, which will not be repeated here. The payload part (payload) of the EDP message includes the following fields: type (Type), version (Version), total message length (Length, Len), CurCKID, CurCKType, NextCKID, NextCKType, ID_A, EncAlgorithm, high count flag (CtrHigh) and reserved field (Reserved, RSVD).

[0131] The meaning of each field is explained below.

[0132] Type: data message type, the value is 0x02.

[0133] Version: Encrypted description message version number, value is 0x01.

[0134] Len: The length of the entire message in bytes.

[0135] CurCKID: The content key identifier currently in use.

[0136] CurCKType: The type of content key currently in use. Definitions include: ①, 00b: unicast, the content key is generated using a derivative; ②, 01b: multicast, the content key is distributed using a key distribution message.

[0137] NextCKID: The next content key identifier to be used.

[0138] NextCKType: The type of the next content key to be used. The definitions include: ①, 00b: unicast, the next content key to be used is generated using a derivative; ②, 01b: multicast, the next content key to be used is distributed using a key distribution message.

[0139] ID_A: Unique device identifier of sender A of multimedia data stream (audio and video stream), binary code.

[0140] EncAlgorithm: content encryption algorithm and mode, including definitions: 0001b→SM4-CTR.

[0141] High count flag (CtrHigh): CtrHigh is the high 64 bits of the counter used for encrypted content, and its uniqueness should be guaranteed. The initial CtrHigh can be generated based on a random number. When a new encryption description message is generated, 1 is added to the previous CtrHigh to generate a new CtrHigh. The low 64 bits CtrLow of the counter used for encrypted content is 0, and the counter is increased by 1 after each 16-byte audio and video stream data block is encrypted.

[0142] Others are reserved fields (RSVD): filled with 0.

[0143] In the case where the multimedia data stream includes multiple EDP messages, the count flag (CtrHigh) of each EDP message is exemplarily described: the count flag of the EDP is the high 64 bits of the counter. For example, the count flag of the jth EDP is the eighth value, and the count flag of the j+1th EDP in the multimedia data stream is the ninth value.

[0144] In the first possible case, if the eighth value is less than the second maximum value, the ninth value is the eighth value plus 1. The second maximum value here refers to the maximum value set for the value of the EDP message, such as 255, 511, 1023 or other values, which is not limited in this application.

[0145] In a second possible case, if the eighth value is equal to the second maximum value, the ninth value is 0.

[0146] Combining the above two possible situations, a feasible specific example is provided for the value of the EDP message: the initial CtrHigh can be generated based on a random number, and when a new encrypted description message (EDP) is generated, 1 is added to the previous CtrHigh to generate a new CtrHigh (i.e., CtrHigh new = CtrHigh + 1), when it is full, add 1 and wrap around to 0, and then continue to accumulate. It is worth noting that the optional way of taking the value of the EDP message provided in the above embodiment should not be understood as a limitation to the present application.

[0147] It should be understood that different counting flags are set for different EDP messages in the same multimedia data stream, so that the host device can effectively distinguish the encryption parameters of different data packets, avoid using the EDP message in data packet 1 to parse the multimedia data message in data packet 2, and realize the respective encryption of different data packets, which is beneficial to improve the security of multimedia data messages transmitted between devices.

[0148] like Figure 6 As shown, Figure 6 A schematic diagram of the structure of a KDP message provided for this application. The KDP message includes a tunnel message header and a payload part. For the tunnel message header of the KDP message, please refer to the contents of Table 3, which will not be repeated here. The payload part (payload) of the KDP message includes the following fields: type (Type), version (Version), total message length (Length, Len), CKID, CurCKType, ID_B, ECKCtr, ECK high count flag (CtrHigh) and reserved field (RSVD).

[0149] The meaning of each field is explained below.

[0150] Type: data message type, the value is 0x01.

[0151] Version: Key distribution message version number, value is 0x01.

[0152] Len: The length of the entire message in bytes.

[0153] CKID: The identifier corresponding to the content key in the current message.

[0154] ID_B: Unique device identifier of the video and audio stream receiver B, in binary code.

[0155] ECKCtr: Counter of encrypted content key usage.

[0156] RSVD: Filled with 0.

[0157] ECK: Encrypted content key, encrypted using the content key encryption key (Content Key Encryption Key, CKEK) and the counter ECKCtr, CKEK is derived from the shared master key Km. The generation method of CKEK is as shown in the following formula (1).

[0158] CKEK=KDF(Km, Random_A||Random_B||ID_A||ID_B, "ContentKeyEncryptionKey", 128) Formula (1)

[0159] With respect to the encryption method of the payload data in the multimedia data message, an optional implementation method is provided below: the payload data includes a plurality of encrypted data blocks, wherein the encrypted i-th data block is determined according to the XOR material and the i-th data block to be encrypted. The XOR material is generated according to the count flag of the EDP and the count flag of the i-th data block, the count flag of the EDP is the upper 64 bits of the counter, and the count flag of the i-th data block is the lower 64 bits of the counter.

[0160] Regarding the value taking method of the counting mark of the data block, the counting mark of the i-th data block after encryption is the sixth value, and the counting mark of the i+1-th data block among the multiple encrypted data blocks is the seventh value.

[0161] In the first possible case, if the sixth value is less than the first maximum value, the seventh value is the sixth value plus 1. The first maximum value here refers to the maximum value set for the value of the count flag of the encrypted content, such as 255, 511, 1023 or other values, which are not limited in this application.

[0162] In a second possible case, if the sixth value is equal to the first maximum value, the seventh value is 0.

[0163] The following is a feasible specific example: the low 64 bits CtrLow of the counter used for encrypted content are set to 0 when CtrHigh in each frame of EDP is updated, and the counter is incremented by 1 after each 16-byte (4 symbols) video and audio stream data block is encrypted (i.e., CtrLow is set to 0). new =CtrLow+1), when it is full, add 1 and wrap back to 0, and then continue to accumulate.

[0164] The following example takes a single data block with a data size of 128 bits as an example. Figure 7 , the encryption and decryption process of the data block is illustrated by way of example, Figure 7 Schematic diagram of the multimedia data encryption and decryption method provided by this application. For the software implementation of the source device 31 and the sink device 32, please refer to Figure 2 The relevant description of is not repeated here. The UMI TX (transmitting side) can be Figure 2 The UMI controller of the source device 210 supports the implementation, and the UMI RX (receiver side) can be Figure 2 The UMI controller of the host device 220 supports the implementation. The advanced digital content protection (ADCP) module is used to provide the XOR material stream required for encryption and decryption. The XOR material stream is a data stream including multiple XOR materials. In this embodiment, the XOR material refers to the input parameter used in the data encryption and decryption process. In terms of data content, the XOR material can be 1 bit of data, such as 0 or 1.

[0165] The encryption and decryption method provided in this embodiment includes the following steps ① to ③.

[0166] Step ①: The source device 31 encrypts the 128-bit audio and video data (data block to be encrypted or data block before encryption) according to the 128-bit XOR material stream provided by the ADCP module to obtain an encrypted code stream (i.e., encrypted data block). Table 4 below provides an implementation method for performing XOR between the XOR material and the audio and video data during the encryption process.

[0167] Table 4

[0168] Video and audio data before encryption XOR Material Encrypted code stream 0 0 0 0 1 1 1 0 1 1 1 0

[0169] That is, if the two inputs are the same, the output is 0; if the two inputs are different, the output is 1.

[0170] Table 4 only shows the XOR encryption process for one bit of data. Figure 7 During the encryption process, the source device 31 sequentially matches the 128-bit audio and video data before encryption with each bit in the 128-bit XOR material stream to implement the encryption process of the audio and video data.

[0171] In this embodiment, after each group of 128-bit audio and video data before encryption is encrypted, the payload data includes a payload symbol (payload symbol) related to the audio and video data. For example, a multimedia data message includes four encrypted data blocks, then the correspondence between the four encrypted data blocks and the XOR material stream is shown in Table 5 below.

[0172] Table 5

[0173] XOR Material Flow Payload Symbol [127：96] payload_symbol0[31:0] [95：64] payload_symbol1[31:0] [63：32] payload_symbol2[31:0] [31：0] payload_symbol3[31:0]

[0174] Step ②: The UMI TX in the source device 31 sends the encrypted code stream to the sink device 32 via the UMI bus.

[0175] Step ③, the UMI RX in the sink device 32 receives the encrypted code stream (code stream before decryption) through the UMI bus, determines the XOR material stream to be used according to the payload symbol in the encrypted code stream, decrypts the encrypted code stream according to the determined XOR material stream, and outputs 128-bit decrypted audio and video data.

[0176] Based on Table 4 and Table 5, the following Table 6 provides an implementation method of performing XOR between the XOR material and the encrypted code stream during the decryption process.

[0177] Table 6

[0178] Encrypted code stream XOR Material Decrypted audio and video data 0 0 0 0 1 1 1 0 1 1 1 0

[0179] That is, if the two inputs are the same, the output is 0; if the two inputs are different, the output is 1. Table 6 only shows the XOR decryption process of one bit of data. Figure 7 During the encryption process, the sink device 32 sequentially matches the 128-bit encrypted code stream with each bit in the 128-bit XOR material stream to implement the decryption process of the audio and video data.

[0180] Optionally, the data that needs to be encrypted between the EDPs of two consecutive frames in the multimedia data stream is not necessarily an integer multiple of 16 bytes. Therefore, the last 16-byte XOR material stream generated by ADCP may be left over. The remaining XOR material stream needs to be discarded. When the next EDP starts, the newly generated XOR material stream is used to start encrypting or decrypting new data.

[0181] In summary, in the multimedia data transmission method provided in the above embodiments, when the multimedia data belongs to high-value content (such as 4K video, 8K video or paid video) or the multimedia service provider requires content protection, the source device encrypts the multimedia data transmitted in the UMI bus with an EDP message, so that when the sink device does not support parsing of the EDP message, it is impossible to parse and obtain the payload data in the multimedia data message, thereby effectively protecting the payload data in these multimedia data messages and improving the data security between devices connected through the bus.

[0182] It is understandable that, in order to implement the functions in the above embodiments, the source device and the sink device include hardware structures and / or software modules corresponding to the execution of each function. It should be easily appreciated by those skilled in the art that, in combination with the units and method steps of each example described in the embodiments disclosed in this application, the present application can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is executed in the form of hardware or computer software driving hardware depends on the specific application scenario and design constraints of the technical solution.

[0183] Combined with the above Figures 1 to 7 , describes in detail the multimedia data transmission method provided by this embodiment, and will now be combined with Figure 8 , describing the multimedia transmission device provided according to this embodiment.

[0184] Figure 8 Schematic diagram of the structure of a multimedia transmission device provided by the present application. The multimedia transmission device 800 can be used to implement the functions of any one of the devices in the above method embodiments, and thus can also achieve the beneficial effects of the above method embodiments. In this embodiment, the multimedia transmission device 800 can be as follows: Figure 1The set-top box 110, smart TV 120 or any display device shown may also be Figure 2 The source device 210 or sink device 220 shown may also be a source device or display device provided in subsequent embodiments. It should be understood that the multimedia transmission apparatus 800 may also be a module (such as a chip) applied to any of the aforementioned devices.

[0185] like Figure 8 As shown, the multimedia transmission device 800 includes: a transceiver module 810 and a processing module 820. The transceiver module 810 and the processing module 820 can cooperate to implement each step in the above method embodiment. A more detailed description of the above transceiver module 810 and the processing module 820 can be directly obtained by referring to the relevant description of the device in the method embodiment shown in the above figures, and no further description is given here.

[0186] When the multimedia transmission device implements the multimedia data transmission method shown in any of the above-mentioned figures through software, the multimedia transmission device and its various units can also be software modules. The above-mentioned multimedia data transmission method is implemented by calling the software module through a processor. The processor can be a central processing unit (CPU), an application-specific integrated circuit (ASIC), or a programmable logic device (PLD), and the above-mentioned PLD can be a complex programmable logical device (CPLD), a field programmable gate array (FPGA), a generic array logic (GAL) or any combination thereof.

[0187] Understandably, Figure 8 The multimedia transmission device shown is only an example provided in this embodiment. The multimedia transmission device may include more or fewer units according to different video transmission processes, and this application is not limited to this.

[0188] When the multimedia transmission device is implemented by hardware, the hardware can be implemented by a processor or a chip system. The chip system includes one or more chips, each chip including an interface circuit and a control circuit. The interface circuit is used to receive data from other devices outside the chip and transmit it to the control circuit, or send data from the control circuit to other devices outside the chip. The control circuit and the interface circuit are used to implement any possible implementation method in the above embodiments through a logic circuit or execution code instructions. The beneficial effects can be found in the description of any aspect of the above embodiments, which will not be repeated here.

[0189] It is understood that the processor in the embodiments of the present application may be a CPU, or other general-purpose processors, digital signal processors (DSP), ASICs, FPGAs or other programmable logic devices, transistor logic devices, hardware components or any combination thereof. The general-purpose processor may be a microprocessor or any conventional processor.

[0190] in addition, Figure 8 The multimedia transmission device 800 shown can also be implemented by a multimedia device, such as Fig. 9 As shown, Fig. 9 A schematic diagram of the structure of a multimedia device provided in the present application. The multimedia device includes: a processor 210, an external memory interface 220, an internal memory 221, a universal serial bus (USB) interface 230, a UMI interface 231, an antenna 1, an antenna 2, a mobile communication module 250, a wireless communication module 260, an audio module 270, a speaker 270A, a receiver 270B, a microphone 270C, a sensor module 280, a button 290, an indicator 292, a camera 293, a display screen 294, a subscriber identification module (SIM) card interface 1-N 295, etc.

[0191] Among them, the above-mentioned sensor module 280 may include sensors such as pressure sensor, gyroscope sensor, air pressure sensor, magnetic sensor, acceleration sensor, distance sensor, proximity light sensor, fingerprint sensor, temperature sensor, touch sensor, ambient light sensor and bone conduction sensor.

[0192] It is to be understood that the structure illustrated in this embodiment does not constitute a specific limitation on the multimedia device. In other embodiments, the multimedia device may include more or fewer components than shown in the figure, or combine some components, or split some components, or arrange the components differently. The components shown in the figure may be implemented in hardware, software, or a combination of software and hardware.

[0193] Fig. 9 The multimedia device shown may be Figure 1 Any device in, or a source device or a sink device in subsequent embodiments, etc.

[0194] The processor 210 may include one or more processing units, for example, the processor 210 may include an application processor (AP), a modem processor, a graphics processor (GPU), an image signal processor (ISP), a controller, a memory, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), etc. Different processing units may be independent devices or integrated into one or more processors.

[0195] The controller can be the nerve center and command center of the multimedia device. The controller can generate operation control signals according to the instruction operation code and timing signal to complete the control of fetching and executing instructions.

[0196] The processor 210 may also be provided with a memory for storing instructions and data. In some embodiments, the memory in the processor 210 is a cache memory. The memory may store instructions or data that the processor 210 has just used or cyclically used. If the processor 210 needs to use the instruction or data again, it may be directly called from the memory. This avoids repeated access, reduces the waiting time of the processor 210, and thus improves the efficiency of the system.

[0197] In some embodiments, the processor 210 may include one or more interfaces. The interface may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a USB interface, a UMI interface, etc.

[0198] It is understandable that the interface connection relationship between the modules illustrated in this embodiment is only a schematic illustration and does not constitute a structural limitation on the multimedia device. In other embodiments, the multimedia device may also adopt different interface connection methods in the above embodiments, or a combination of multiple interface connection methods.

[0199] The wireless communication function of the multimedia device can be implemented through antenna 1, antenna 2, mobile communication module 250, wireless communication module 260, modem processor and baseband processor, etc. In some embodiments, antenna 1 and mobile communication module 250 of the multimedia device are coupled, and antenna 2 and wireless communication module 260 are coupled, so that the multimedia device can communicate with the network and other devices through wireless communication technology.

[0200] The wired communication function of the multimedia device can be implemented through the USB interface 230 or the UMI interface 231. For example, the multimedia device receives or sends video streams and AVP messages through the bus connected to the UMI interface 231.

[0201] The multimedia device implements display functions through a GPU, a display screen 294, and an application processor. The GPU is a microprocessor for image processing, connecting the display screen 294 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. The processor 210 may include one or more GPUs that execute program instructions to generate or change display information.

[0202] The display screen 294 is used to display images, videos, etc. The display screen 294 includes a display panel.

[0203] The multimedia device can realize the shooting function through ISP, camera 293, video codec, GPU, display screen 294 and application processor. ISP is used to process the data fed back by camera 293. Camera 293 is used to capture static images or videos. In some embodiments, the multimedia device may include 1 or N cameras 293, where N is a positive integer greater than 1.

[0204] In this embodiment, the above display screen 294, video codec, GPU, display screen 294 and application processor may also be collectively referred to as a display unit of the multimedia device 32, which is used to process and display the received multimedia data stream (such as a video stream).

[0205] The external memory interface 220 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the multimedia device. The external memory card communicates with the processor 210 through the external memory interface 220 to implement a data storage function. For example, files such as music and videos can be stored in the external memory card.

[0206] The internal memory 221 may be used to store computer executable program codes, which include instructions. The processor 210 executes various functional applications and data processing of the multimedia device by running the instructions stored in the internal memory 221. For example, in an embodiment of the present application, the processor 210 may execute the instructions stored in the internal memory 221, and the internal memory 221 may include a program storage area and a data storage area.

[0207] The program storage area may store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc. The data storage area may store data created during the use of the multimedia device (such as video and audio data, a phone book, etc.), etc. In addition, the internal memory 221 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, a universal flash storage (UFS), etc.

[0208] The multimedia device can implement audio functions such as music playing and recording through the audio module 270, the speaker 270A, the receiver 270B, the microphone 270C, and the application processor.

[0209] The button 290 includes a power button, a volume button, etc. The button 290 may be a mechanical button or a touch button. The indicator 292 may be an indicator light, which may be used to indicate the charging status, the change of the power, or may be used to indicate messages, missed calls, notifications, etc.

[0210] The method steps in the embodiments of the present application can also be implemented by a processor executing software instructions. The software instructions can be composed of corresponding software modules, and the software modules can be stored in random access memory (RAM), flash memory, read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), registers, hard disks, mobile hard disks, CD-ROMs or any other form of storage medium known in the art. An exemplary storage medium is coupled to the processor so that the processor can read information from the storage medium and write information to the storage medium. Of course, the storage medium can also be a component of the processor. The processor and the storage medium can be located in an ASIC. In addition, the ASIC can be located in a network device or a terminal device. Of course, the processor and the storage medium can also be present in a video processing device and a multimedia device as discrete components.

[0211] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented by software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer programs or instructions. When the computer program or instruction is loaded and executed on a computer, the process or function described in the embodiment of the present application is executed in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, a network device, a user device or other programmable device. The computer program or instruction may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer program or instruction may be transmitted from one website site, computer, server or data center to another website site, computer, server or data center by wired or wireless means. The computer-readable storage medium may be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium may be a magnetic medium, for example, a floppy disk, a hard disk, a tape; it may also be an optical medium, for example, a digital video disc (DVD); it may also be a semiconductor medium, for example, a solid state drive (SSD).

[0212] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed in the present application, and these modifications or replacements should be included in the protection scope of the present application. Therefore, the protection scope of the present application shall be based on the protection scope of the claims.

Claims

1. A multimedia data transmission method, characterized in that: The method is performed by a source device, the source device is connected to a sink device via a unified multimedia interconnect (UMI) bus, and the method includes: The source device enables a content protection function; The source device sends a multimedia data stream to the sink device, the multimedia data stream includes one or more data packets, the data packets include: one or more encryption description messages and multimedia data messages, the encryption description messages are used to transmit encryption parameters of the multimedia data messages.

2. The method according to claim 1, characterized in that The row blanking area of ​​the data packet includes: one or more vertical blanking messages VBP, and the encrypted description message is located after the VBP.

3. The method according to claim 1 or 2, characterized in that: The encrypted description message includes a first tunnel message header, the first tunnel message header includes a content protection flag, the value of the content protection flag is a first value or a second value, the first value indicates that the encrypted description message is not encrypted, and the second value indicates that the encrypted description message is encrypted.

4. The method according to claim 3, characterized in that The first tunnel message header also includes: a first message length and a first type, the value of the first message length is 28 bytes, and the value of the first type is a third value.

5. The method according to any one of claims 1 to 4, characterized in that The line blanking area of ​​the data packet also includes: one or more key distribution messages, the key distribution message is located after the encryption description message, and the key distribution message is used to transmit the encryption key of the multimedia data message.

6. The method according to claim 5, characterized in that The key distribution message includes a second tunnel message header, which includes: a second message length, a second type and a label, the value of the second message length is 48 bytes, the value of the second type is a fourth value, and the value of the content protection flag in the label is 0.

7. The method according to any one of claims 1 to 6, characterized in that The multimedia data message includes: one or both of an audio sampling message and a valid video message.

8. The method according to any one of claims 1 to 7, characterized in that The multimedia data message includes a tunnel message header and payload data. The value of the content protection flag in the tunnel message is a fifth value, and the fifth value indicates that the payload data has been encrypted.

9. The method according to claim 8, characterized in that The payload data includes a plurality of encrypted data blocks, wherein the encrypted i-th data block is determined according to the XOR material and the i-th data block to be encrypted; The XOR material is generated based on the counting flag of the encrypted description message and the counting flag of the i-th data block, the counting flag of the encrypted description message is the high 64 bits of the counter, and the counting flag of the i-th data block is the low 64 bits of the counter.

10. The method according to claim 9, characterized in that The count mark of the encrypted i-th data block is the sixth value, and the count mark of the i+1-th data block among the encrypted multiple data blocks is the seventh value; If the sixth value is less than the first maximum value, the seventh value is the sixth value plus one; If the sixth value is equal to the first maximum value, the seventh value is 0.

11. The method according to claim 9 or 10, characterized in that: The count flag of the j-th encrypted description message in the multimedia data stream is an eighth value, and the count flag of the j+1-th encrypted description message in the multimedia data stream is a ninth value; If the eighth value is less than the second maximum value, the ninth value is the eighth value plus one; If the eighth value is equal to the second maximum value, the ninth value is 0.

12. A multimedia data transmission method, characterized in that: The method is performed by a sink device, the sink device is connected to a source device via a first bus, and the method includes: The sink device receives a multimedia data stream sent by the source device; the multimedia data stream includes one or more data packets, and the data packets include: one or more encryption description messages and multimedia data messages, and the encryption description messages are used to transmit encryption parameters of the multimedia data messages; The sink device parses the multimedia data message according to the encrypted description message in the multimedia data stream.

13. The method according to claim 12, characterized in that The row blanking area of ​​the data packet includes: one or more vertical blanking messages VBP, and the encrypted description message is located after the VBP.

14. The method according to claim 12 or 13, characterized in that The encrypted description message includes a first tunnel message header, the first tunnel message header includes a content protection flag, the value of the content protection flag is a first value or a second value, the first value indicates that the encrypted description message is not encrypted, and the second value indicates that the encrypted description message is encrypted.

15. The method according to claim 14, characterized in that The first tunnel message header also includes: a first message length and a first type, the value of the first message length is 28 bytes, and the value of the first type is a third value.

16. The method according to any one of claims 12 to 15, characterized in that: The line blanking area of ​​the data packet also includes: one or more key distribution messages, the key distribution message is located after the encryption description message, and the key distribution message is used to transmit the encryption key of the multimedia data message.

17. The method according to claim 16, characterized in that The key distribution message includes a second tunnel message header, which includes: a second message length, a second type and a label, the value of the second message length is 48 bytes, the value of the second type is a fourth value, and the value of the content protection flag in the label is 0.

18. The method according to any one of claims 12 to 17, characterized in that: The multimedia data message includes: one or both of an audio sampling message and a valid video message.

19. The method according to any one of claims 12 to 18, characterized in that: The multimedia data message includes a tunnel message header and payload data. The value of the content protection flag in the tunnel message is a fifth value, and the fifth value indicates that the payload data has been encrypted.

20. The method according to claim 19, characterized in that The payload data includes a plurality of encrypted data blocks, wherein the encrypted i-th data block is determined according to the XOR material and the i-th data block to be encrypted; The XOR material is generated based on the counting flag of the encrypted description message and the counting flag of the i-th data block, the counting flag of the encrypted description message is the high 64 bits of the counter, and the counting flag of the i-th data block is the low 64 bits of the counter.

21. The method according to claim 20, characterized in that The count mark of the encrypted i-th data block is the sixth value, and the count mark of the i+1-th data block among the encrypted multiple data blocks is the seventh value; If the sixth value is less than the first maximum value, the seventh value is the sixth value plus one; If the sixth value is equal to the first maximum value, the seventh value is 0.

22. The method according to claim 20 or 21, characterized in that The count flag of the j-th encrypted description message in the multimedia data stream is an eighth value, and the count flag of the j+1-th encrypted description message in the multimedia data stream is a ninth value; If the eighth value is less than the second maximum value, the ninth value is the eighth value plus one; If the eighth value is equal to the second maximum value, the ninth value is 0.

23. A multimedia device, characterized in that: include: processors and transceivers; The processor is used to enable a content protection function and cooperate with the transceiver to execute the method according to any one of claims 1 to 11; And / or, the transceiver is used to receive a video stream and cooperate with the processor to execute the method described in any one of claims 12-22.

Images