Audio and video data playback method, device, equipment, storage medium and program product

Abstract

The present application discloses a method, apparatus, device, storage medium, and program product for playing audio and video data. The method includes: a user terminal sends a play instruction to a cloud server; the cloud server, in response to the play instruction, sends a key derivation vector, a key derivation vector number, and a derivation rule to a video surveillance device; the video surveillance device encrypts the audio and video code stream according to the key derivation vector number, the derivation rule, and a preset encryption rule to obtain an encrypted audio and video code stream, and assembles the encrypted audio and video code stream to obtain a key information frame; the encrypted audio and video code stream and the key information frame are then assembled to obtain a transport stream code stream; the transport stream code stream is sent to the user terminal; the user terminal obtains a key derivation vector corresponding to the key derivation vector number according to the key information frame in the transport stream code stream; the encrypted audio and video code stream is decrypted according to the key derivation vector, the derivation rule in the key information frame, and the encryption rule identifier to obtain the audio and video code stream and play it.

Description

Technical Field

[0001] The present application belongs to the field of communication technology, and in particular relates to a method, device, equipment, storage medium and program product for playing audio and video data. Background Art

[0002] With the advancement of audio and video technology, a growing number of video surveillance products (e.g., security cameras, remote video conferencing cameras, etc.) are available on the market. Since the audio and video being monitored are user privacy, users are increasingly demanding the security of their audio and video data. Existing technologies can encrypt the audio and video streams from cameras using a pre-negotiated key during the transmission of audio and video data for playback, thereby protecting data security. However, if the key of such encrypted audio and video streams is leaked during transmission, an attacker could use the leaked key to impersonate a legitimate user and gain unauthorized access, preventing the user terminal from securely accessing and playing the audio and video streams. Summary of the Invention

[0003] The embodiments of the present application provide a method, apparatus, device, storage medium, and program product for playing audio and video data, which can ensure the security of audio and video data, thereby enabling user terminals to securely obtain and play audio and video streams.

[0004] In a first aspect, an embodiment of the present application provides a method for playing audio and video data, which is applied to a video surveillance device, and the method includes:

[0005] receiving a key derivation vector, a key derivation vector number, and a derivation rule sent by a cloud server, wherein the key derivation vector, the key derivation vector number, and the derivation rule are sent by the cloud server in response to a play instruction sent by a user terminal, wherein the play instruction is used to instruct to play an audio and video stream in the video surveillance device;

[0006] Encrypting the audio and video stream according to the key derivation vector number, the derivation rule, and a preset encryption rule to obtain an encrypted audio and video stream, and assembling the encrypted audio and video stream to obtain a key information frame, wherein the key information frame includes the key derivation vector number, the derivation rule, and an encryption rule identifier, wherein the encryption rule identifier is used to indicate the encryption rule;

[0007] Assembling the encrypted audio and video code stream and the key information frame to obtain a transport stream code stream, wherein the transport stream code stream includes the encrypted audio and video code stream and the key information frame;

[0008] The transport stream is sent to the user terminal, so that the user terminal obtains the key derivation vector corresponding to the key derivation vector number according to the key information frame in the transport stream; and decrypts the encrypted audio and video stream according to the key derivation vector, the derivation rule in the key information frame, and the encryption rule identifier to obtain the audio and video stream and play it.

[0009] In a second aspect, an embodiment of the present application provides a method for playing audio and video data, applied to a user terminal, the method comprising:

[0010] Sending a play instruction to the cloud server, the play instruction being used to instruct the playback of the audio and video code stream in the video surveillance device, so that the cloud server responds to the play instruction and sends the key derivation vector, the key derivation vector number, and the derivation rule to the video surveillance device; the video surveillance device encrypts the audio and video code stream according to the key derivation vector number, the derivation rule, and a preset encryption rule to obtain an encrypted audio and video code stream, and assembles the encrypted audio and video code stream into a key information frame; assembling the encrypted audio and video code stream and the key information frame to obtain a transport stream code stream; and sending the transport stream code stream to the user terminal;

[0011] receiving the transport stream code stream sent by the video surveillance device, the transport stream code stream including the encrypted audio and video code stream and the key information frame, the key information frame including the key derivation vector number, the derivation rule, and an encryption rule identifier, the encryption rule identifier being used to indicate the encryption rule;

[0012] Acquire the key derivation vector corresponding to the key derivation vector number according to the key information frame in the transport stream code stream;

[0013] The encrypted audio and video code stream is decrypted according to the key derivation vector, the derivation rule in the key information frame, and the encryption rule identifier to obtain the audio and video code stream and play it.

[0014] In a third aspect, an embodiment of the present application provides a method for playing audio and video data, which is applied to a cloud server, and the method includes:

[0015] Receive a play instruction sent by a user terminal, wherein the play instruction is used to instruct the playback of an audio and video stream in a video surveillance device;

[0016] In response to the play instruction, the key derivation vector, the key derivation vector number and the derivation rule are sent to the video surveillance device,

[0017] The video surveillance device encrypts the audio and video code stream according to the key derivation vector number, the derivation rule, and the preset encryption rule to obtain an encrypted audio and video code stream, and assembles it to obtain a key information frame, where the key information frame includes the key derivation vector number, the derivation rule, and an encryption rule identifier, and the encryption rule identifier is used to indicate the encryption rule; assembles the encrypted audio and video code stream and the key information frame to obtain a transport stream code stream, where the transport stream code stream includes the encrypted audio and video code stream and the key information frame; sends the transport stream code stream to the user terminal; the user terminal obtains the key derivation vector corresponding to the key derivation vector number according to the key information frame in the transport stream code stream; decrypts the encrypted audio and video code stream according to the key derivation vector, the derivation rule, and the encryption rule identifier in the key information frame to obtain the audio and video code stream and play it.

[0018] In a fourth aspect, an embodiment of the present application provides an audio and video data playback device, which is applied to a video surveillance device, and the device includes:

[0019] a first receiving module, configured to receive a key derivation vector, a key derivation vector number, and a derivation rule sent by a cloud server, wherein the key derivation vector, the key derivation vector number, and the derivation rule are sent by the cloud server in response to a play instruction sent by a user terminal, wherein the play instruction is used to instruct the playback of an audio and video stream in the video surveillance device;

[0020] an encryption module, configured to encrypt the audio and video code stream according to the key derivation vector number, the derivation rule, and a preset encryption rule to obtain an encrypted audio and video code stream, and assemble the encrypted audio and video code stream to obtain a key information frame, wherein the key information frame includes the key derivation vector number, the derivation rule, and an encryption rule identifier, wherein the encryption rule identifier is used to indicate the encryption rule;

[0021] an assembling module, configured to assemble the encrypted audio and video code stream and the key information frame to obtain a transport stream code stream, wherein the transport stream code stream includes the encrypted audio and video code stream and the key information frame;

[0022] A first sending module is configured to send the transport stream code stream to the user terminal, so that the user terminal obtains the key derivation vector corresponding to the key derivation vector number based on the key information frame in the transport stream code stream; and decrypt the encrypted audio and video code stream based on the key derivation vector, the derivation rule in the key information frame, and the encryption rule identifier to obtain and play the audio and video code stream.

[0023] In a fifth aspect, an embodiment of the present application provides an audio and video data playback device, applied to a user terminal, the device comprising:

[0024] a second sending module, configured to send a play instruction to the cloud server, wherein the play instruction is used to instruct the playback of the audio and video code stream in the video surveillance device, so that the cloud server, in response to the play instruction, sends the key derivation vector, the key derivation vector number, and the derivation rule to the video surveillance device; the video surveillance device encrypts the audio and video code stream according to the key derivation vector number, the derivation rule, and a preset encryption rule to obtain an encrypted audio and video code stream, and assembles the encrypted audio and video code stream into a key information frame; assembles the encrypted audio and video code stream and the key information frame into a transport stream; and sends the transport stream to the user terminal;

[0025] a second receiving module, configured to receive the transport stream code stream sent by the video surveillance device, the transport stream code stream including the encrypted audio and video code stream and the key information frame, the key information frame including the key derivation vector number, the derivation rule, and an encryption rule identifier, the encryption rule identifier being used to indicate the encryption rule;

[0026] an acquisition module, configured to acquire the key derivation vector corresponding to the key derivation vector number according to the key information frame in the transport stream code stream;

[0027] The decryption and playback module is used to decrypt the encrypted audio and video code stream according to the key derivation vector, the derivation rule in the key information frame and the encryption rule identifier, obtain the audio and video code stream and play it.

[0028] In a sixth aspect, an embodiment of the present application provides an audio and video data playback device, which is applied to a cloud server, and the device includes:

[0029] A third receiving module is used to receive a play instruction sent by a user terminal, wherein the play instruction is used to instruct the playback of the audio and video stream in the video surveillance device;

[0030] a third sending module, configured to send the key derivation vector, the key derivation vector number, and the derivation rule to the video surveillance device in response to the play instruction;

[0031] A third receiving module is used to receive a play instruction sent by a user terminal, wherein the play instruction is used to instruct the playback of the audio and video stream in the video surveillance device;

[0032] a third sending module, configured to send the key derivation vector, the key derivation vector number, and the derivation rule to the video surveillance device in response to the play instruction;

[0033] The video surveillance device encrypts the audio and video code stream according to the key derivation vector number, the derivation rule, and the preset encryption rule to obtain an encrypted audio and video code stream, and assembles it to obtain a key information frame, where the key information frame includes the key derivation vector number, the derivation rule, and an encryption rule identifier, and the encryption rule identifier is used to indicate the encryption rule; assembles the encrypted audio and video code stream and the key information frame to obtain a transport stream code stream, where the transport stream code stream includes the encrypted audio and video code stream and the key information frame; sends the transport stream code stream to the user terminal; the user terminal obtains the key derivation vector corresponding to the key derivation vector number according to the key information frame in the transport stream code stream; decrypts the encrypted audio and video code stream according to the key derivation vector, the derivation rule, and the encryption rule identifier in the key information frame to obtain the audio and video code stream and play it.

[0034] In a seventh aspect, an embodiment of the present application provides an electronic device, comprising: a processor and a memory storing computer program instructions; when the processor executes the computer program instructions, the audio and video data playback method as described in any one of the above items is implemented.

[0035] In an eighth aspect, an embodiment of the present application provides a computer-readable storage medium, on which computer program instructions are stored. When the computer program instructions are executed by a processor, the audio and video data playback method as described in any one of the above items is implemented.

[0036] In a ninth aspect, an embodiment of the present application provides a computer program product, which includes computer program instructions, and when the computer program instructions are executed by a processor, implements the audio and video data playback method as described in any one of the above.

[0037] The audio and video data playback method, apparatus, device, storage medium and program product of the embodiment of the present application can enable a user terminal to send a playback instruction to a cloud server, where the playback instruction is used to instruct the playback of the audio and video code stream in the video surveillance device; the cloud server responds to the playback instruction by sending a key derivation vector, a key derivation vector number and a derivation rule to the video surveillance device; the video surveillance device encrypts the audio and video code stream according to the key derivation vector number, the derivation rule and the preset encryption rule to obtain an encrypted audio and video code stream, and assembles a key information frame, where the key information frame includes the key derivation vector number, the derivation rule and the preset encryption rule. The derivation rule and encryption rule identifier are used to indicate the encryption rule; the video surveillance device then assembles the encrypted audio and video code stream and the key information frame to obtain a transport stream code stream, which includes the encrypted audio and video code stream and the key information frame; the video surveillance device sends the transport stream code stream to the user terminal; the user terminal obtains the key derivation vector corresponding to the key derivation vector number based on the key information frame in the transport stream code stream; the user terminal decrypts the encrypted audio and video code stream based on the key derivation vector, the derivation rule in the key information frame, and the encryption rule identifier to obtain the audio and video code stream and play it. In this way, in the embodiment of the present application, during the transmission and playback of audio and video data, the key information frame does not include the encryption key. Therefore, even if the key information frame is obtained, the encryption key will not be leaked, which can ensure the security of the audio and video data, thereby enabling the user terminal to safely obtain the audio and video code stream and play it. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following is a brief introduction to the drawings required for use in the embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0039] Figure 1 This is an architectural diagram of the audio and video data playback system provided by an embodiment of the present application;

[0040] Figure 2 Schematic diagram of the flow of the audio and video data playback method provided in the embodiment of the present application;

[0041] Figure 3 This is a flow chart of an application scenario provided by an embodiment of the present application;

[0042] Figure 4 This is a structural diagram of the key information frame content and audio and video frame encryption offset information provided by an embodiment of the present application;

[0043] Figure 5 This is a schematic diagram of the position of the key information frame in the TS code stream provided by an embodiment of the present application;

[0044] Figure 6 This is a structural diagram of an audio and video data playback device provided in an embodiment of the present application;

[0045] Figure 7 is a structural diagram of another audio and video data playback device provided in an embodiment of the present application;

[0046] Figure 8 This is a structural diagram of another audio and video data playback device provided in an embodiment of the present application;

[0047] Figure 9 It is a structural diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0048] The features and exemplary embodiments of various aspects of the present application will be described in detail below. In order to make the purpose, technical solutions and advantages of the present application clearer, the present application will be further described in detail below in conjunction with the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are only intended to explain the present application, rather than to limit the present application. For those skilled in the art, the present application can be implemented without the need for some of these specific details. The following description of the embodiments is merely to provide a better understanding of the present application by illustrating the examples of the present application.

[0049] It should be noted that, in this document, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article, or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or device. In the absence of further limitations, the elements defined by the phrase "comprising..." do not exclude the presence of other identical elements in the process, method, article, or device comprising the elements.

[0050] With the advancement of audio and video technology, a growing number of video surveillance products (e.g., security cameras, remote video conferencing cameras, etc.) are available on the market. Since the audio and video being monitored are user privacy, users are increasingly demanding the security of their audio and video data. Existing technologies can encrypt the audio and video streams from cameras using a pre-negotiated key during the transmission of audio and video data for playback, thereby protecting data security. However, if the key of such encrypted audio and video streams is leaked during transmission, an attacker could use the leaked key to impersonate a legitimate user and gain unauthorized access, preventing the user terminal from securely accessing and playing the audio and video streams.

[0051] In order to solve the problems of the prior art, the embodiments of the present application provide a method, apparatus, device, storage medium and program product for playing audio and video data. The following first introduces the audio and video data playing system provided by the embodiments of the present application.

[0052] Figure 1 FIG1 shows an architecture diagram of an audio and video data playback system provided by an embodiment of the present application. Figure 1 As shown, the audio and video data playback system 100 may include: a user terminal 101 , a cloud server 102 and a video surveillance device 103 .

[0053] The user terminal 101 may refer to a device used by a user to operate a computer system for access, such as a computer, smartphone, or tablet computer. A user terminal typically runs a user operating system, and a user can use the user terminal to access and use server resources, such as files and applications. User terminals are typically located at the edge of a network and are used to receive and send data.

[0054] The cloud server 102 described above can be a virtualized service based on cloud computing technology. It provides computing, storage, and network resources to users, enabling them to create, deploy, and manage applications and services in the cloud. Cloud servers offer flexible resource allocation, high availability, flexibility, and scalability, enabling users to quickly access required computing resources without investing in extensive hardware. Users can use cloud servers to host websites, run applications, store data, and more.

[0055] The video surveillance device 103 can be a device used to monitor and record specific areas or locations, allowing for real-time monitoring and recording of events. It typically includes components such as a camera, a video recorder, a storage device, and monitoring software. In some embodiments, the video surveillance device can have a motion detection function that can trigger recording or an alarm by sensing changes in moving objects. Many network cameras and advanced video recorders have motion detection functions. In this embodiment, the video surveillance device can specifically be a network camera that transmits video and provides remote access via a network connection. It has a built-in network interface that can directly connect to a local area network or the Internet, allowing users to remotely view real-time video and recordings through a web browser or mobile device.

[0056] The user terminal 101, the cloud server 102, and the video surveillance device 103 are connected to each other in a communication manner. Specifically, a wired or wireless network connection can be used for communication. Of course, the communication manner is not limited to this, and other network connection communication manners can also be used, which are not specifically limited here.

[0057] In some embodiments, in order to further improve the security of information transmission, a certificate system can be used to provide an identity authentication-based transmission method to securely transmit encryption-related information between video surveillance equipment and cloud servers. In addition, the user terminal and cloud server have also been activated and bound to verify their identities.

[0058] The following is an introduction to the audio and video data playback method provided in the embodiment of the present application.

[0059] Figure 2 The flowchart of the method for playing audio and video data provided by an embodiment of the present application is shown. Figure 1 The user terminal 101, cloud server 102 and video monitoring device 103 are shown. Figure 2 As shown, a method 200 for playing audio and video data may include the following steps S201 to S210:

[0060] S201: The user terminal sends a play instruction to the cloud server, where the play instruction is used to instruct the playback of the audio and video stream in the video surveillance device;

[0061] S202: The cloud server receives a playback instruction sent by the user terminal;

[0062] S203. The cloud server sends the key derivation vector, the key derivation vector number, and the derivation rule to the video surveillance device in response to the play instruction.

[0063] S204. The video surveillance device receives the key derivation vector, key derivation vector number, and derivation rule sent by the cloud server.

[0064] S205. The video surveillance device encrypts the audio and video stream according to the key derivation vector number, the derivation rule, and the preset encryption rule to obtain an encrypted audio and video stream, and assembles the encrypted audio and video stream into a key information frame. The key information frame includes the key derivation vector number, the derivation rule, and an encryption rule identifier, where the encryption rule identifier is used to indicate the encryption rule.

[0065] S206. The video surveillance device assembles the encrypted audio and video code stream and the key information frame to obtain a transport stream code stream, which includes the encrypted audio and video code stream and the key information frame;

[0066] S207, the video surveillance device sends the transport stream code stream to the user terminal;

[0067] S208. The user terminal receives the transport stream sent by the video surveillance device;

[0068] S209. The user terminal obtains a key derivation vector corresponding to the key derivation vector number according to the key information frame in the transport stream code stream;

[0069] S210: The user terminal decrypts the encrypted audio and video stream according to the key derivation vector, the derivation rule in the key information frame, and the encryption rule identifier to obtain the audio and video stream and play it.

[0070] The audio and video data playback method of the embodiment of the present application can enable a user terminal to send a playback instruction to a cloud server, where the playback instruction is used to instruct the playback of the audio and video code stream in the video surveillance device; the cloud server responds to the playback instruction and sends a key derivation vector, a key derivation vector number, and a derivation rule to the video surveillance device; the video surveillance device encrypts the audio and video code stream according to the key derivation vector number, the derivation rule, and a preset encryption rule to obtain an encrypted audio and video code stream, and assembles it to obtain a key information frame, where the key information frame includes the key derivation vector number, the derivation rule, and an encryption rule identifier, where the encryption rule identifier is used to indicate the encryption rule; the video surveillance device then assembles the encrypted audio and video code stream and the key information frame to obtain a transport stream code stream, where the transport stream code stream includes the encrypted audio and video code stream and the key information frame; the video surveillance device sends the transport stream code stream to the user terminal; the user terminal obtains the key derivation vector corresponding to the key derivation vector number based on the key information frame in the transport stream code stream; the user terminal decrypts the encrypted audio and video code stream according to the key derivation vector, the derivation rule, and the encryption rule identifier in the key information frame to obtain the audio and video code stream and plays it. In this way, in the embodiment of the present application, during the transmission and playback of audio and video data, the encryption key is not included in the key information frame. Therefore, even if the key information frame is obtained, the encryption key will not be leaked, which can ensure the security of the audio and video data, thereby enabling the user terminal to safely obtain and play the audio and video code stream.

[0071] The specific implementation methods of the above steps are introduced below.

[0072] In S201, the play instruction may be used to instruct the playing of the audio and video streams in the video surveillance device.

[0073] The user terminal sends the play instruction to the cloud server. For example, the user terminal may send the play instruction to the cloud server via wireless communication.

[0074] In S202, the cloud server receives the play instruction sent by the user terminal. For example, the cloud server may receive the play instruction sent by the user terminal via wireless communication.

[0075] In S203, the key derivation vector is a technology for deriving other keys from a master key. It is a randomly generated value, usually used in combination with the master key to calculate other keys through a specific algorithm.

[0076] The key derivation vector number can be used to uniquely identify the key derivation vector. It can be a string, a number, or another identifier to distinguish different key derivation vectors.

[0077] The above-mentioned derivation rule may be a derivation algorithm, which is an algorithm for generating a derived key based on an input key or key material. Exemplarily, a specific key derivation function (KDF) is used to perform the derivation operation.

[0078] The aforementioned key derivation vector, key derivation vector number, and derivation rule can be different each time the key derivation vector, key derivation vector number, and derivation rule are transmitted in an application scenario where different audio and video streams are played. Therefore, there is no need to worry about the encryption key being out of sync with the audio and video streams.

[0079] In S204, the video surveillance device receives the key derivation vector, key derivation vector number, and derivation rule sent by the cloud server. For example, the video surveillance device receives the key derivation vector, key derivation vector number, and derivation rule sent by the cloud server via wireless communication.

[0080] In S205, the key information frame may include a key derivation vector number, a derivation rule, and an encryption rule identifier, wherein the encryption rule identifier may be used to indicate the encryption rule, and the encryption rule identifier uniquely corresponds to the encryption rule.

[0081] The above-mentioned encryption of the audio and video code stream according to the key derivation vector number, derivation rule, and preset encryption rule to obtain an encrypted audio and video code stream and assembling the key information frame can illustratively include generating an encryption key according to the key derivation vector and derivation rule; assembling the key information frame according to the key derivation vector number, derivation rule, and preset encryption rule identifier, the key information frame including the key derivation vector number, derivation rule, and encryption rule identifier; and encrypting the audio and video code stream according to the encryption key and the encryption rule indicated by the encryption rule identifier to obtain the encrypted audio and video code stream.

[0082] In S206, the transport stream may include an encrypted audio and video stream and a key information frame.

[0083] The above-mentioned video surveillance device assembles the encrypted audio and video code stream and the key information frame to obtain the transport stream code stream. For example, the video surveillance device may set the key information frame at a preset position in the encrypted audio and video code stream to assemble the transport stream code stream.

[0084] In S207, the video surveillance device sends the transport stream code stream to the user terminal. For example, the video surveillance device may first send the transport stream code stream to the cloud server via the TLS protocol, and the cloud server then sends the transport stream code stream to the user terminal via the TLS protocol. Alternatively, the video surveillance device may directly send the transport stream code stream to the user terminal via wireless communication.

[0085] In S208, the user terminal receives the transport stream transmitted by the video surveillance device. For example, the user terminal may receive the transport stream transmitted by the cloud server, which in turn is transmitted by the video surveillance device to the cloud server. Alternatively, the user terminal may directly receive the transport stream transmitted by the video surveillance device via wireless communication.

[0086] In S209, the user terminal obtains the key derivation vector corresponding to the key derivation vector number based on the key information frame in the transport stream code stream. Exemplarily, the user terminal may extract the key derivation vector number based on the key information frame in the transport stream code stream; generate an acquisition request based on the key derivation vector number, the acquisition request being used to request acquisition of the key derivation vector corresponding to the key derivation vector number; send the acquisition request to the cloud server; and in response to the acquisition request, the cloud server sends the key derivation vector corresponding to the key derivation vector number to the user terminal.

[0087] In S210, the user terminal decrypts the encrypted audio and video code stream according to the key derivation vector, the derivation rule in the key information frame, and the encryption rule identifier, obtains the audio and video code stream, and plays it. Exemplarily, the user terminal may first generate a decryption key according to the key derivation vector and the derivation rule in the key information frame; then decrypts the encrypted audio and video code stream according to the decryption key and the encryption rule indicated by the encryption rule identifier in the key information frame, obtains the audio and video code stream, and plays it.

[0088] In some embodiments, the above S205 may specifically include:

[0089] The video surveillance device generates an encryption key based on the key derivation vector and the derivation rule;

[0090] The video surveillance device assembles a key information frame according to the key derivation vector number, the derivation rule, and the preset encryption rule identifier, where the key information frame includes the key derivation vector number, the derivation rule, and the encryption rule identifier;

[0091] The video surveillance device encrypts the audio and video code stream according to the encryption key and the encryption rule indicated by the encryption rule identifier to obtain an encrypted audio and video code stream.

[0092] The above S210 may specifically include:

[0093] The user terminal generates a decryption key based on the key derivation vector and the derivation rule in the key information frame;

[0094] The user terminal decrypts the encrypted audio and video code stream according to the decryption key and the encryption rule indicated by the encryption rule identifier in the key information frame, obtains the audio and video code stream, and plays it.

[0095] The encryption key can be used to encrypt the audio and video code stream to obtain an encrypted audio and video code stream. The decryption key can be a key corresponding to the encryption key and can be used to decrypt the encrypted audio and video code stream to obtain an audio and video code stream.

[0096] The key information frame may include a key derivation vector number, a derivation rule, and an encryption rule identifier.

[0097] The aforementioned encryption of the audio and video bitstream according to the encryption key and the encryption rule indicated by the encryption rule identifier to obtain the encrypted audio and video bitstream may, for example, be performed by encrypting the entire audio and video bitstream according to the encryption key and the encryption rule indicated by the encryption rule identifier to obtain the encrypted audio and video bitstream. Alternatively, the encryption may be performed by encrypting only the bitstream content information in the audio and video bitstream according to the encryption key and the encryption rule indicated by the encryption rule identifier to obtain the encrypted bitstream content information, where the encrypted audio and video bitstream includes the video encoding parameter set and the encrypted bitstream content information.

[0098] The aforementioned decryption of the encrypted audio and video stream according to the encryption rule indicated by the decryption key and the encryption rule identifier in the key information frame to obtain and play the audio and video stream may, for example, be performed by decrypting the entire encrypted audio and video stream according to the encryption rule indicated by the decryption key and the encryption rule identifier in the key information frame to obtain and play the audio and video stream. Alternatively, the encrypted code stream content information in the encrypted audio and video stream may be decrypted according to the encryption rule indicated by the decryption key and the encryption rule identifier in the key information frame to obtain and play the audio and video stream.

[0099] In this embodiment, the audio and video stream can be encrypted using an encryption key generated by a key derivation vector and a derivation rule, and the encrypted audio and video stream and the key information frame are transmitted to the user terminal. Since the key information frame does not include the encryption key, the encryption key will not be leaked during the process of the video surveillance device transmitting the key information frame to the user terminal, thereby further ensuring that the user can securely receive and play the audio and video stream.

[0100] In some embodiments, the above-mentioned audio and video code stream may include a video encoding parameter set and code stream content information;

[0101] The video surveillance device encrypts the audio and video stream according to the encryption key and the encryption rule indicated by the encryption rule identifier to obtain the encrypted audio and video stream, which may specifically include:

[0102] The video surveillance device encrypts the code stream content information according to the encryption key and the encryption rule indicated by the encryption rule identifier to obtain the encrypted code stream content information.

[0103] The encrypted audio and video code stream includes a video encoding parameter set and encrypted code stream content information.

[0104] The user terminal decrypts the encrypted audio and video stream according to the decryption key and the encryption rule indicated by the encryption rule identifier in the key information frame to obtain the audio and video stream and play it, which may specifically include:

[0105] The user terminal decrypts the encrypted code stream content information according to the decryption key and the encryption rule indicated by the encryption rule identifier in the key information frame to obtain the code stream content information. The audio and video code stream includes a video encoding parameter set and code stream content information.

[0106] The user terminal plays the audio and video streams.

[0107] In this embodiment, the video surveillance device only encrypts the code stream content information in the audio and video code stream to obtain the encrypted code stream content information. This partial encryption method not only does not destroy the original code stream header structure, thereby improving encryption efficiency, but also reduces the decryption workload when the user terminal decrypts the encrypted code stream content information, thereby improving the user's experience of playing the audio and video code stream.

[0108] As an implementation of the present application, in order to further reduce the amount of data for encryption and decryption, the above-mentioned code stream content information may include key frames and predicted frames;

[0109] Before the video surveillance device encrypts the code stream content information according to the encryption key and the encryption rule indicated by the encryption rule identifier to obtain the encrypted code stream content information, the method may further include:

[0110] The video surveillance device obtains motion detection information, which is used to indicate whether the corresponding monitoring screen of the audio and video stream in the video surveillance device has motion;

[0111] The video surveillance device determines the frame encryption type of the audio and video stream in the video surveillance device based on the motion detection information. The frame encryption type is one of the full frame encryption type and the key frame encryption type.

[0112] The key information frame may further include a frame encryption type;

[0113] The video surveillance device encrypts the code stream content information according to the encryption key and the encryption rule indicated by the encryption rule identifier to obtain the encrypted code stream content information, which may specifically include:

[0114] When the frame encryption type is key frame encryption, the video surveillance device encrypts the key frame in the code stream content information according to the encryption key and the encryption rule indicated by the encryption rule identifier to obtain an encrypted key frame.

[0115] The encrypted code stream content information includes encrypted key frames and predicted frames.

[0116] The user terminal decrypts the encrypted code stream content information according to the decryption key and the encryption rule indicated by the encryption rule identifier in the key information frame to obtain the code stream content information, which may specifically include:

[0117] The user terminal decrypts the encrypted key frame according to the decryption key and the encryption rule indicated by the encryption rule identifier in the key information frame to obtain the key frame.

[0118] The code stream content information includes key frames and predicted frames.

[0119] The above motion detection information can be used to indicate whether the audio and video streams in the video surveillance device correspond to the monitoring screen and whether there is any motion.

[0120] The full-frame encryption type instructs encryption of key frames and predicted frames in the stream content information when motion detection information indicates motion in the corresponding monitoring image of the network camera's audio and video stream. If the stream content information also includes bidirectionally predicted frames, the full-frame encryption type also instructs encryption of the bidirectionally predicted frames.

[0121] The key frame encryption type is to instruct to encrypt the key frames in the code stream content information when the motion detection information indicates that there is no motion in the corresponding monitoring screen of the audio and video code stream in the network camera.

[0122] The key information frame may also include a frame encryption type.

[0123] In this embodiment, when the frame encryption type is key frame encryption, only the key frames in the code stream content information are encrypted to obtain encrypted code stream content information including encrypted key frames and predicted frames. When the user terminal decrypts the encrypted code stream content information, only the encrypted key frames need to be decrypted, thereby further reducing the amount of encrypted and decrypted data and improving the user experience of playing audio and video code streams.

[0124] In some embodiments, the above S209 may specifically include:

[0125] The user terminal extracts the key derivation vector number according to the key information frame in the transport stream code stream;

[0126] The user terminal generates an acquisition request according to the key derivation vector number, where the acquisition request is used to request to obtain the key derivation vector corresponding to the key derivation vector number;

[0127] The user terminal sends the acquisition request to the cloud server;

[0128] The above method may further include:

[0129] The cloud server receives the acquisition request sent by the user terminal;

[0130] In response to the acquisition request, the cloud server sends the key derivation vector corresponding to the key derivation vector number to the user terminal.

[0131] The user terminal extracts the key derivation vector number according to the key information frame in the transport stream code stream. For example, it can be

[0132] The above acquisition request can be used to request to obtain the key derivation vector corresponding to the key derivation vector number.

[0133] The user terminal extracts the key derivation vector number from the key information frame in the transport stream. Exemplarily, the user terminal may extract the key derivation vector number from the key information frame at a preset position in the transport stream.

[0134] In this embodiment, the user terminal extracts the key derivation vector number from the key information frame in the transport stream, generates an acquisition request, and sends it to the cloud server. In response to the acquisition request, the cloud server sends the key derivation vector corresponding to the key derivation vector number to the user terminal. The resulting key derivation vector is consistent with the vector used by the video surveillance device to encrypt the audio and video stream, thereby enabling the user terminal to securely decrypt the encrypted audio and video stream, obtain the audio and video stream, and play it.

[0135] To facilitate understanding of the audio and video data playback method in the embodiment of the present application, the actual application process of this audio and video data playback method is described as follows:

[0136] This embodiment uses transport stream (TS) code stream encapsulation to demonstrate this process. Figure 3 As shown, the main structure of this embodiment includes three parts: a mobile app (i.e., the above-mentioned user terminal), a cloud platform (i.e., the above-mentioned cloud server), and a network camera (i.e., the above-mentioned video surveillance device). The cloud platform in the flowchart represents a non-local cloud service, but is not limited to only one or one type of cloud platform. Among them, the mobile app and the cloud platform have been activated and bound, and their identities have been verified. The interaction protocol between the mobile app and the cloud platform is over the Transport Layer Security (TLS) protocol; in addition, the signaling transmission protocol between the cloud platform and the network camera is based on TLS transmission. The network camera has a built-in public key certificate for verifying the cloud platform, and its identity is verified every time it connects to the cloud platform.

[0137] like Figure 3 As shown, the specific steps may include:

[0138] 1. The user sends the live broadcast instruction (i.e. the playback instruction mentioned above) to the cloud platform through the mobile app via a secure encrypted channel (all messages thereafter are sent through a secure encrypted channel, so I will not repeat them here);

[0139] 2. In response to the live broadcast instruction, the cloud platform sends the key derivation vector, key derivation vector ID (i.e., the key derivation vector number), and derivation algorithm (i.e., the derivation rule) to the network camera via a secure encrypted channel (all subsequent messages are sent via a secure encrypted channel, which will not be repeated here);

[0140] 3. The network camera generates an encryption key based on the key derivation vector and derivation algorithm. This key is used to encrypt the audio and video streams.

[0141] 4. Before the transmission of each Group of Pictures (GOP) in the audio and video stream begins, a key information frame is assembled based on the motion detection situation. The key information frame is used by the decryption end to obtain relevant information from the encryption end (note: the key information frame does not contain the encryption key). The content of the key information frame includes: key derivation vector ID, derivation algorithm, encryption algorithm identifier, frame encryption type (full frame encryption or key frame encryption), etc. And according to the encryption information (including encryption key and encryption algorithm), the audio and video stream data is encrypted (the encryption algorithm, that is, the above encryption rules, can select the current mainstream algorithm such as: AES, 3DES, etc., or it can be customized. The present invention does not limit the encryption algorithm). The encryption process does not destroy the original data structure of the stream, but only encrypts the stream content (that is, the above stream content information). For example, for H264 stream, the sequence parameter set (Sequence Parameter Set, sps) and picture parameter set (Picture Parameter Set, pps) frames are not encrypted; for H265 stream, the video parameter set (Video Parameter Set, vps), sps, and pps frames are not encrypted; for aac audio frames with advanced audio transport stream (Advanced Audio Transport Stream, adts) headers, the adts header is skipped and only the audio frames are encrypted. For detailed encryption frame types and position offsets of the encryption process, see Figure 4 The encrypted audio and video data (i.e. the above-mentioned encrypted audio and video code stream) is assembled into a TS code stream together with the key information frame. The TS code stream structure is as follows: Figure 5 As shown;

[0142] 5. Send the TS stream to the cloud platform;

[0143] 6. The cloud platform forwards the TS stream to the corresponding mobile app;

[0144] 7. The mobile app parses the key information frame from the TS code stream and obtains encryption-related information such as the key derivation vector ID;

[0145] 8. Based on the key derivation vector ID, request the corresponding key derivation vector from the cloud platform;

[0146] 9. The cloud platform returns the key derivation vector;

[0147] 10. The mobile app generates a decryption key using the key derivation vector and key derivation algorithm, and uses the decryption key in combination with the encryption algorithm identifier and frame encryption type in the key information frame to decrypt the audio and video stream data and play it.

[0148] Regarding the assembly of the key information frame based on the motion detection situation in step 4 above, there are two situations:

[0149] 11. When motion detection occurs, indicating that the image is moving, the frame encryption type in the key information frame is changed to full-frame encryption, and the key frames (i.e., I frames) and predicted frames (i.e., P frames) in the audio and video stream are encrypted (if there are bidirectionally predicted frames, i.e., B frames, they are also encrypted) and assembled into a TS stream;

[0150] 12. It will be sent to the cloud storage platform;

[0151] 13. The cloud platform forwards the TS code stream to the corresponding mobile app. The subsequent decryption and decrypted playback process is the same as steps 7 to 10.

[0152] 14. When no motion detection occurs, indicating that the picture is still, the frame encryption type in the key information frame is changed to key frame encryption, and the key frames (i.e., I frames) in the audio and video streams are encrypted and assembled into a TS stream;

[0153] 15. Will be sent to the cloud storage platform;

[0154] 16. The cloud platform forwards the TS code stream to the corresponding mobile app. The subsequent decryption and decrypted playback process is the same as steps 7 to 10.

[0155] In this embodiment, the key derivation vector is cloud-based, and the key generation algorithm and encryption algorithm are platform-specific. Once a key leak is discovered, a more secure key or algorithm can be upgraded without upgrading the device firmware, improving the ability to handle and recover from abnormal issues. Furthermore, a key information frame is added to the TS stream. This key information frame contains only encryption-related information and no key information. Even if it is obtained, it does not affect security. This allows for precise encryption of audio and video stream content without destroying the stream structure. This not only reduces the amount of encrypted content but also solves the problem of encrypting the entire stream, which destroys the stream structure and makes it unrecognizable without decryption. Furthermore, this solution supports both video and audio encryption. Furthermore, by combining motion detection to encrypt I-frames and P-frames in motion scenarios and only I-frames in static scenarios, this not only reduces the performance requirements for video surveillance equipment but also reduces the amount of encrypted and decrypted data, improving encryption and decryption efficiency and enhancing the user's audio and video playback experience.

[0156] Based on the audio and video data playback method provided in the above embodiment, the present application also provides a specific implementation of an audio and video data playback device. It is understood that the relevant descriptions in the following device embodiments can refer to the above method embodiments, and for the sake of brevity, they are not repeated here. Please refer to the following embodiments.

[0157] See Figure 6, is a structural diagram of an audio and video data playback device 600 provided in an embodiment of the present application, which is applied to video surveillance equipment. The device 600 may include: a first receiving module 601, an encryption module 602, an assembly module 603 and a first sending module 604.

[0158] A first receiving module 601 is configured to receive a key derivation vector, a key derivation vector number, and a derivation rule sent by a cloud server. The key derivation vector, the key derivation vector number, and the derivation rule are sent by the cloud server in response to a play instruction sent by a user terminal. The play instruction is used to instruct the playback of an audio and video stream in the video surveillance device.

[0159] An encryption module 602 is configured to encrypt the audio and video stream based on the key derivation vector number, derivation rule, and preset encryption rule to obtain an encrypted audio and video stream, and assemble the encrypted audio and video stream into a key information frame. The key information frame includes the key derivation vector number, derivation rule, and encryption rule identifier, where the encryption rule identifier indicates the encryption rule.

[0160] An assembling module 603 is configured to assemble the encrypted audio and video code stream and the key information frame to obtain a transport stream code stream, wherein the transport stream code stream includes the encrypted audio and video code stream and the key information frame;

[0161] The first sending module 604 is configured to send the transport stream code stream to the user terminal, so that the user terminal obtains the key derivation vector corresponding to the key derivation vector number based on the key information frame in the transport stream code stream; decrypts the encrypted audio and video code stream based on the key derivation vector, the derivation rule in the key information frame, and the encryption rule identifier, to obtain the audio and video code stream and play it.

[0162] The audio and video data playback device of the embodiment of the present application is capable of sending a playback instruction from a user terminal to a cloud server, where the playback instruction is used to instruct the playback of the audio and video code stream in the video surveillance device; the cloud server responds to the playback instruction and sends a key derivation vector, a key derivation vector number, and a derivation rule to the video surveillance device; the video surveillance device encrypts the audio and video code stream according to the key derivation vector number, the derivation rule, and a preset encryption rule to obtain an encrypted audio and video code stream, and assembles it to obtain a key information frame, where the key information frame includes the key derivation vector number, the derivation rule, and an encryption rule identifier, where the encryption rule identifier is used to indicate the encryption rule; the video surveillance device then assembles the encrypted audio and video code stream and the key information frame to obtain a transport stream code stream, which includes the encrypted audio and video code stream and the key information frame; the video surveillance device sends the transport stream code stream to the user terminal; the user terminal obtains the key derivation vector corresponding to the key derivation vector number based on the key information frame in the transport stream code stream; the user terminal decrypts the encrypted audio and video code stream according to the key derivation vector, the derivation rule, and the encryption rule identifier in the key information frame to obtain the audio and video code stream and plays it. In this way, in the embodiment of the present application, during the transmission and playback of audio and video data, the encryption key is not included in the key information frame. Therefore, even if the key information frame is obtained, the encryption key will not be leaked, which can ensure the security of the audio and video data, thereby enabling the user terminal to safely obtain and play the audio and video code stream.

[0163] In some embodiments, the encryption module 602 may specifically include:

[0164] A first generating unit, configured to generate an encryption key according to the key derivation vector and a derivation rule;

[0165] an assembling unit, configured to assemble a key information frame according to the key derivation vector number, the derivation rule, and the preset encryption rule identifier, wherein the key information frame includes the key derivation vector number, the derivation rule, and the encryption rule identifier;

[0166] The encryption unit is used to encrypt the audio and video code stream according to the encryption key and the encryption rule indicated by the encryption rule identifier to obtain an encrypted audio and video code stream.

[0167] In some embodiments, the above-mentioned audio and video code stream may include a video encoding parameter set and code stream content information;

[0168] The encryption unit can be used to encrypt the code stream content information according to the encryption key and the encryption rule indicated by the encryption rule identifier to obtain the encrypted code stream content information.

[0169] The encrypted audio and video code stream includes a video encoding parameter set and encrypted code stream content information.

[0170] As an implementation of the present application, in order to further reduce the amount of data for encryption and decryption, the above-mentioned code stream content information may include key frames and predicted frames;

[0171] The encryption module 602 may further include:

[0172] An acquisition unit, configured to acquire motion detection information, wherein the motion detection information is used to indicate whether motion occurs in the monitoring screen corresponding to the audio and video stream in the video surveillance device;

[0173] The determination unit is used to determine the frame encryption type of the audio and video code stream in the video surveillance device according to the motion detection information, the frame encryption type being one of the full frame encryption type and the key frame encryption type.

[0174] The key information frame also includes a frame encryption type;

[0175] The encryption unit may also be configured to encrypt the key frames in the code stream content information according to the encryption key and the encryption rule indicated by the encryption rule identifier when the frame encryption type is key frame encryption, thereby obtaining encrypted key frames.

[0176] The encrypted code stream content information includes encrypted key frames and predicted frames.

[0177] See Figure 7 , is a structural diagram of an audio and video data playback device 700 provided in an embodiment of the present application, which is applied to a user terminal. The device 700 may include: a second sending module 701, a second receiving module 702, an acquisition module 703 and a decryption and playback module 704.

[0178] The second sending module 701 is configured to send a play instruction to the cloud server, where the play instruction is used to instruct the cloud server to play the audio and video stream in the video surveillance device, so that the cloud server responds to the play instruction and sends the key derivation vector, the key derivation vector number, and the derivation rule to the video surveillance device; the video surveillance device encrypts the audio and video stream according to the key derivation vector number, the derivation rule, and a preset encryption rule to obtain an encrypted audio and video stream, and assembles the encrypted audio and video stream into a key information frame; assembles the encrypted audio and video stream and the key information frame into a transport stream; and sends the transport stream to the user terminal.

[0179] A second receiving module 702 is configured to receive a transport stream sent by a video surveillance device, the transport stream including an encrypted audio and video stream and a key information frame, the key information frame including a key derivation vector number, a derivation rule, and an encryption rule identifier, the encryption rule identifier being used to indicate the encryption rule;

[0180] An acquisition module 703 is configured to acquire a key derivation vector corresponding to a key derivation vector number according to a key information frame in a transport stream code stream;

[0181] The decryption and playback module 704 is used to decrypt the encrypted audio and video code stream according to the key derivation vector, the derivation rule in the key information frame, and the encryption rule identifier, to obtain the audio and video code stream and play it.

[0182] The audio and video data playback device of the embodiment of the present application is capable of sending a playback instruction from a user terminal to a cloud server, where the playback instruction is used to instruct the playback of the audio and video code stream in the video surveillance device; the cloud server responds to the playback instruction and sends a key derivation vector, a key derivation vector number, and a derivation rule to the video surveillance device; the video surveillance device encrypts the audio and video code stream according to the key derivation vector number, the derivation rule, and a preset encryption rule to obtain an encrypted audio and video code stream, and assembles it to obtain a key information frame, where the key information frame includes the key derivation vector number, the derivation rule, and an encryption rule identifier, where the encryption rule identifier is used to indicate the encryption rule; the video surveillance device then assembles the encrypted audio and video code stream and the key information frame to obtain a transport stream code stream, which includes the encrypted audio and video code stream and the key information frame; the video surveillance device sends the transport stream code stream to the user terminal; the user terminal obtains the key derivation vector corresponding to the key derivation vector number based on the key information frame in the transport stream code stream; the user terminal decrypts the encrypted audio and video code stream according to the key derivation vector, the derivation rule, and the encryption rule identifier in the key information frame to obtain the audio and video code stream and plays it. In this way, in the embodiment of the present application, during the transmission and playback of audio and video data, the encryption key is not included in the key information frame. Therefore, even if the key information frame is obtained, the encryption key will not be leaked, which can ensure the security of the audio and video data, thereby enabling the user terminal to safely obtain and play the audio and video code stream.

[0183] In some embodiments, the acquisition module 703 may specifically include:

[0184] An extraction unit, configured to extract a key derivation vector number according to a key information frame in a transport stream code stream;

[0185] a second generating unit, configured to generate an acquisition request according to the key derivation vector number, where the acquisition request is used to request to obtain the key derivation vector corresponding to the key derivation vector number;

[0186] The sending unit is configured to send an acquisition request to the cloud server, so that the cloud server sends a key derivation vector corresponding to the key derivation vector number to the user terminal in response to the acquisition request.

[0187] In some embodiments, the decryption and playback module 704 may specifically include:

[0188] a third generating unit, configured to generate a decryption key according to the key derivation vector and the derivation rule in the key information frame;

[0189] The decryption and playback unit is used to decrypt the encrypted audio and video code stream according to the decryption key and the encryption rule indicated by the encryption rule identifier in the key information frame, obtain the audio and video code stream and play it.

[0190] In some embodiments, the encrypted audio and video code stream may include a video encoding parameter set and encrypted code stream content information;

[0191] The above-mentioned decryption and playback unit may specifically include:

[0192] The decryption subunit is used to decrypt the encrypted code stream content information according to the decryption key and the encryption rule indicated by the encryption rule identifier in the key information frame to obtain the code stream content information. The audio and video code stream includes the video encoding parameter set and the code stream content information;

[0193] The playback subunit is used to play audio and video streams.

[0194] As an implementation of the present application, in order to further reduce the amount of data encrypted and decrypted, the key information frame may further include a frame encryption type. When the frame encryption type is a key frame encryption type, the encrypted code stream content information includes an encrypted key frame and a predicted frame.

[0195] The decryption subunit may be specifically configured to decrypt the encrypted key frame to obtain the key frame according to the decryption key and the encryption rule indicated by the encryption rule identifier in the key information frame. The code stream content information includes the key frame and the predicted frame.

[0196] See Figure 8 , is a structural diagram of an audio and video data playback device 800 provided in an embodiment of the present application, which is applied to a cloud server. The device 800 may include: a third receiving module 801 and a third sending module 802.

[0197] The third receiving module 801 is used to receive a play instruction sent by a user terminal, where the play instruction is used to instruct the playback of the audio and video stream in the video surveillance device;

[0198] The third sending module 802 is configured to send the key derivation vector, the key derivation vector number and the derivation rule to the video surveillance device in response to the play instruction.

[0199] The video surveillance device encrypts the audio and video code stream according to the key derivation vector number, derivation rule and preset encryption rule to obtain an encrypted audio and video code stream, and assembles it to obtain a key information frame, which includes the key derivation vector number, derivation rule and encryption rule identifier, and the encryption rule identifier is used to indicate the encryption rule; assembles the encrypted audio and video code stream and the key information frame to obtain a transport stream code stream, which includes the encrypted audio and video code stream and the key information frame; sends the transport stream code stream to the user terminal; the user terminal obtains the key derivation vector corresponding to the key derivation vector number according to the key information frame in the transport stream code stream; decrypts the encrypted audio and video code stream according to the key derivation vector, the derivation rule and encryption rule identifier in the key information frame, obtains the audio and video code stream and plays it.

[0200] The audio and video data playback device of the embodiment of the present application is capable of sending a playback instruction from a user terminal to a cloud server, where the playback instruction is used to instruct the playback of the audio and video code stream in the video surveillance device; the cloud server responds to the playback instruction and sends a key derivation vector, a key derivation vector number, and a derivation rule to the video surveillance device; the video surveillance device encrypts the audio and video code stream according to the key derivation vector number, the derivation rule, and a preset encryption rule to obtain an encrypted audio and video code stream, and assembles it to obtain a key information frame, where the key information frame includes the key derivation vector number, the derivation rule, and an encryption rule identifier, where the encryption rule identifier is used to indicate the encryption rule; the video surveillance device then assembles the encrypted audio and video code stream and the key information frame to obtain a transport stream code stream, which includes the encrypted audio and video code stream and the key information frame; the video surveillance device sends the transport stream code stream to the user terminal; the user terminal obtains the key derivation vector corresponding to the key derivation vector number based on the key information frame in the transport stream code stream; the user terminal decrypts the encrypted audio and video code stream according to the key derivation vector, the derivation rule, and the encryption rule identifier in the key information frame to obtain the audio and video code stream and plays it. In this way, in the embodiment of the present application, during the transmission and playback of audio and video data, the encryption key is not included in the key information frame. Therefore, even if the key information frame is obtained, the encryption key will not be leaked, which can ensure the security of the audio and video data, thereby enabling the user terminal to safely obtain and play the audio and video code stream.

[0201] In some embodiments, the apparatus 800 may further include:

[0202] a fourth receiving module, configured to receive an acquisition request sent by a user terminal, where the acquisition request is used to request to obtain a key derivation vector corresponding to the key derivation vector number;

[0203] The fourth sending module is configured to send the key derivation vector corresponding to the key derivation vector number to the user terminal in response to the acquisition request.

[0204] Figure 9 A schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present application is shown.

[0205] The electronic device may include a processor 901 and a memory 902 storing computer program instructions.

[0206] Specifically, the processor 901 may include a central processing unit (CPU), or an application-specific integrated circuit (ASIC), or may be configured to implement one or more integrated circuits of the embodiments of the present application.

[0207] The memory 902 may include a large capacity memory for data or instructions. By way of example and not limitation, the memory 902 may include a hard disk drive (HDD), a floppy disk drive, a flash memory, an optical disk, a magneto-optical disk, a magnetic tape, or a universal serial bus (USB) drive, or a combination of two or more of these. Where appropriate, the memory 902 may include removable or non-removable (or fixed) media. Where appropriate, the memory 902 may be inside or outside the integrated gateway disaster recovery device. In a specific embodiment, the memory 902 is a non-volatile solid-state memory.

[0208] In certain embodiments, the memory 902 may include read-only memory (ROM), random access memory (RAM), magnetic disk storage media devices, optical storage media devices, flash memory devices, electrical, optical, or other physical / tangible memory storage devices. Thus, in general, the memory includes one or more tangible (non-transitory) computer-readable storage media (e.g., memory devices) encoded with software including computer-executable instructions, and when the software is executed (e.g., by one or more processors), it is operable to perform the operations described with reference to the method according to an aspect of the present disclosure.

[0209] The processor 901 reads and executes computer program instructions stored in the memory 902 to implement any one of the audio and video data playback methods in the above embodiments.

[0210] In one example, the electronic device may further include a communication interface 903 and a bus 910. Figure 9 As shown, the processor 901 , the memory 902 , and the communication interface 903 are connected via a bus 910 and communicate with each other.

[0211] The communication interface 903 is mainly used to implement communication between various modules, devices, units and / or equipment in the embodiments of the present application.

[0212] Bus 910 comprises hardware, software or both, couples the parts of electronic equipment to each other.For example, and not limitation, bus can comprise accelerated graphics port (AGP) or other graphics bus, enhanced industry standard architecture (EISA) bus, front side bus (FSB), hypertransport (HT) interconnection, industry standard architecture (ISA) bus, infinite bandwidth interconnection, low pin count (LPC) bus, memory bus, micro channel architecture (MCA) bus, peripheral component interconnection (PCI) bus, PCI-Express (PCI-X) bus, serial advanced technology attachment (SATA) bus, video electronics standard association local (VLB) bus or other suitable bus or two or more of these combinations.In suitable cases, bus 910 can comprise one or more buses.Although the present application embodiment describes and shows specific bus, the application considers any suitable bus or interconnection.

[0213] The electronic device can execute the audio and video data playing method in the embodiment of the present application, thereby realizing the combination Figure 1 、 Figures 6 to 8 The invention relates to a method and device for playing audio and video data.

[0214] In addition, in conjunction with the audio and video data playback method in the above embodiments, the present application embodiment may provide a computer-readable storage medium for implementation. The computer-readable storage medium stores computer program instructions; when the computer program instructions are executed by a processor, any of the audio and video data playback methods in the above embodiments is implemented.

[0215] An embodiment of the present application provides a computer program product, which includes computer program instructions, and when the computer program instructions are executed by a processor, implements the audio and video data playback method described in any one of the above. It should be clear that the present application is not limited to the specific configuration and processing described above and shown in the figures. For the sake of simplicity, a detailed description of the known methods is omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of the present application is not limited to the specific steps described and shown. After understanding the spirit of the present application, those skilled in the art can make various changes, modifications and additions, or change the order between the steps.

[0216] The functional blocks shown in the above-described block diagram can be implemented as hardware, software, firmware or a combination thereof. When implemented in hardware, they can be, for example, electronic circuits, application specific integrated circuits (ASICs), appropriate firmware, plug-ins, function cards, etc. When implemented in software, the elements of the present application are programs or code segments that are used to perform the required tasks. The program or code segment can be stored in a machine-readable medium, or transmitted on a transmission medium or a communication link by a data signal carried in a carrier wave. "Machine-readable medium" can include any medium that can store or transmit information. Examples of machine-readable media include electronic circuits, semiconductor memory devices, ROMs, flash memories, erasable ROMs (EROMs), floppy disks, CD-ROMs, optical disks, hard disks, optical fiber media, radio frequency (RF) links, etc. The code segment can be downloaded via a computer network such as the Internet, an intranet, etc.

[0217] It should also be noted that the exemplary embodiments mentioned in this application describe some methods or systems based on a series of steps or devices. However, this application is not limited to the order of the above steps. In other words, the steps can be performed in the order mentioned in the embodiments, or in a different order, or several steps can be performed simultaneously.

[0218] Aspects of the present disclosure have been described above with reference to the flowcharts and / or block diagrams of the methods, devices (systems) and computer program products according to the embodiments of the present disclosure. It should be understood that each box in the flowchart and / or block diagram and the combination of each box in the flowchart and / or block diagram can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer or other programmable data processing device to produce a machine so that these instructions executed by the processor of the computer or other programmable data processing device enable the implementation of the function / action specified in one or more boxes of the flowchart and / or block diagram. Such a processor can be, but is not limited to, a general-purpose processor, a special-purpose processor, a special application processor or a field programmable logic circuit. It is also understood that each box in the block diagram and / or flowchart and the combination of the boxes in the block diagram and / or flowchart can also be implemented by dedicated hardware that performs the specified function or action, or can be implemented by a combination of dedicated hardware and computer instructions.

[0219] The above description is only a specific embodiment of the present application. Those skilled in the art will clearly understand that for the convenience and brevity of description, the specific working processes of the systems, modules and units described above can refer to the corresponding processes in the aforementioned method embodiments, and will not be repeated here. It should be understood that the scope of protection of the present application is not limited thereto. Any person skilled in the art can easily think of various equivalent modifications or replacements within the technical scope disclosed in the present application, and these modifications or replacements should be included in the scope of protection of the present application.

Claims

1. A method for playing audio and video data, characterized in that: Applied to video surveillance equipment, the method includes: receiving a key derivation vector, a key derivation vector number, and a derivation rule sent by a cloud server, wherein the key derivation vector, the key derivation vector number, and the derivation rule are sent by the cloud server in response to a play instruction sent by a user terminal, wherein the play instruction is used to instruct to play an audio and video stream in the video surveillance device; Encrypting the audio and video stream according to the key derivation vector number, the derivation rule, and a preset encryption rule to obtain an encrypted audio and video stream, and assembling the encrypted audio and video stream to obtain a key information frame, wherein the key information frame includes the key derivation vector number, the derivation rule, and an encryption rule identifier, wherein the encryption rule identifier is used to indicate the encryption rule; Assembling the encrypted audio and video code stream and the key information frame to obtain a transport stream code stream, wherein the transport stream code stream includes the encrypted audio and video code stream and the key information frame; The transport stream is sent to the user terminal, so that the user terminal obtains the key derivation vector corresponding to the key derivation vector number according to the key information frame in the transport stream; and decrypts the encrypted audio and video stream according to the key derivation vector, the derivation rule in the key information frame, and the encryption rule identifier to obtain the audio and video stream and play it.

2. The method according to claim 1, characterized in that The step of encrypting the audio and video code stream according to the key derivation vector number, the derivation rule, and the preset encryption rule to obtain an encrypted audio and video code stream, and assembling the key information frame includes: generating an encryption key according to the key derivation vector and the derivation rule; Assembling a key information frame according to the key derivation vector number, the derivation rule, and a preset encryption rule identifier, wherein the key information frame includes the key derivation vector number, the derivation rule, and the encryption rule identifier; The audio and video code stream is encrypted according to the encryption key and the encryption rule indicated by the encryption rule identifier to obtain an encrypted audio and video code stream.

3. The method according to claim 2, characterized in that The audio and video code stream includes a video encoding parameter set and code stream content information; The step of encrypting the audio and video stream according to the encryption key and the encryption rule indicated by the encryption rule identifier to obtain the encrypted audio and video stream comprises: Encrypting the code stream content information according to the encryption key and the encryption rule indicated by the encryption rule identifier to obtain encrypted code stream content information, The encrypted audio and video code stream includes the video encoding parameter set and the encrypted code stream content information.

4. The method according to claim 3, characterized in that The code stream content information includes key frames and predicted frames; Before encrypting the code stream content information according to the encryption key and the encryption rule indicated by the encryption rule identifier to obtain the encrypted code stream content information, the method further includes: Acquiring motion detection information, wherein the motion detection information is used to indicate whether a monitoring screen corresponding to an audio and video stream in the video surveillance device moves; Determine the frame encryption type of the audio and video code stream in the video surveillance device according to the motion detection information, wherein the frame encryption type is one of a full frame encryption type and a key frame encryption type. Wherein, the key information frame also includes the frame encryption type; The step of encrypting the code stream content information according to the encryption key and the encryption rule indicated by the encryption rule identifier to obtain the encrypted code stream content information includes: In the case where the frame encryption type is the key frame encryption, the key frame in the code stream content information is encrypted according to the encryption key and the encryption rule indicated by the encryption rule identifier to obtain an encrypted key frame. The encrypted code stream content information includes the encrypted key frame and the predicted frame.

5. A method for playing audio and video data, characterized in that: Applied to a user terminal, the method includes: Sending a play instruction to the cloud server, the play instruction being used to instruct the playback of the audio and video code stream in the video surveillance device, so that the cloud server responds to the play instruction and sends the key derivation vector, the key derivation vector number, and the derivation rule to the video surveillance device; the video surveillance device encrypts the audio and video code stream according to the key derivation vector number, the derivation rule, and a preset encryption rule to obtain an encrypted audio and video code stream, and assembles the encrypted audio and video code stream into a key information frame; assembling the encrypted audio and video code stream and the key information frame to obtain a transport stream code stream; and sending the transport stream code stream to the user terminal; receiving the transport stream code stream sent by the video surveillance device, the transport stream code stream including the encrypted audio and video code stream and the key information frame, the key information frame including the key derivation vector number, the derivation rule, and an encryption rule identifier, the encryption rule identifier being used to indicate the encryption rule; Acquire the key derivation vector corresponding to the key derivation vector number according to the key information frame in the transport stream code stream; The encrypted audio and video code stream is decrypted according to the key derivation vector, the derivation rule in the key information frame, and the encryption rule identifier to obtain the audio and video code stream and play it.

6. The method according to claim 5, characterized in that The obtaining, according to the key information frame in the transport stream code stream, the key derivation vector corresponding to the key derivation vector number, includes: Extracting the key derivation vector number according to the key information frame in the transport stream code stream; generating an acquisition request according to the key derivation vector number, the acquisition request being used to request acquisition of the key derivation vector corresponding to the key derivation vector number; The acquisition request is sent to the cloud server, so that the cloud server sends the key derivation vector corresponding to the key derivation vector number to the user terminal in response to the acquisition request.

7. The method according to claim 5, characterized in that The decrypting the encrypted audio and video code stream according to the key derivation vector, the derivation rule in the key information frame, and the encryption rule identifier to obtain the audio and video code stream and play it includes: generating a decryption key according to the key derivation vector and the derivation rule in the key information frame; The encrypted audio and video code stream is decrypted according to the decryption key and the encryption rule indicated by the encryption rule identifier in the key information frame to obtain the audio and video code stream and play it.

8. The method according to claim 7, characterized in that The encrypted audio and video code stream includes a video encoding parameter set and encrypted code stream content information; The decrypting the encrypted audio and video code stream according to the decryption key and the encryption rule indicated by the encryption rule identifier in the key information frame to obtain the audio and video code stream and play it includes: decrypting the encrypted code stream content information according to the decryption key and the encryption rule indicated by the encryption rule identifier in the key information frame to obtain code stream content information, wherein the audio and video code stream includes the video encoding parameter set and the code stream content information; Play the audio and video stream.

9. The method according to claim 8, characterized in that The key information frame further includes a frame encryption type. When the frame encryption type is a key frame encryption type, the encrypted code stream content information includes an encrypted key frame and a predicted frame. The decrypting the encrypted code stream content information according to the decryption key and the encryption rule indicated by the encryption rule identifier in the key information frame to obtain the code stream content information includes: Decrypting the encrypted key frame to obtain a key frame according to the decryption key and the encryption rule indicated by the encryption rule identifier in the key information frame, The code stream content information includes the key frame and the predicted frame.

10. A method for playing audio and video data, characterized in that: Applied to a cloud server, the method includes: Receive a play instruction sent by a user terminal, wherein the play instruction is used to instruct the playback of an audio and video stream in a video surveillance device; In response to the play instruction, the key derivation vector, the key derivation vector number and the derivation rule are sent to the video surveillance device, The video surveillance device encrypts the audio and video code stream according to the key derivation vector number, the derivation rule, and the preset encryption rule to obtain an encrypted audio and video code stream, and assembles it to obtain a key information frame, where the key information frame includes the key derivation vector number, the derivation rule, and an encryption rule identifier, and the encryption rule identifier is used to indicate the encryption rule; assembles the encrypted audio and video code stream and the key information frame to obtain a transport stream code stream, where the transport stream code stream includes the encrypted audio and video code stream and the key information frame; sends the transport stream code stream to the user terminal; the user terminal obtains the key derivation vector corresponding to the key derivation vector number according to the key information frame in the transport stream code stream; decrypts the encrypted audio and video code stream according to the key derivation vector, the derivation rule, and the encryption rule identifier in the key information frame to obtain the audio and video code stream and play it.

11. The method according to claim 10, characterized in that The method further comprises: receiving an acquisition request sent by the user terminal, where the acquisition request is used to request acquisition of the key derivation vector corresponding to the key derivation vector number; In response to the acquisition request, the key derivation vector corresponding to the key derivation vector number is sent to the user terminal.

12. An audio and video data playback device, characterized in that: Applied to video surveillance equipment, the device includes: a first receiving module, configured to receive a key derivation vector, a key derivation vector number, and a derivation rule sent by a cloud server, wherein the key derivation vector, the key derivation vector number, and the derivation rule are sent by the cloud server in response to a play instruction sent by a user terminal, wherein the play instruction is used to instruct the playback of an audio and video stream in the video surveillance device; an encryption module, configured to encrypt the audio and video code stream according to the key derivation vector number, the derivation rule, and a preset encryption rule to obtain an encrypted audio and video code stream, and assemble the encrypted audio and video code stream to obtain a key information frame, wherein the key information frame includes the key derivation vector number, the derivation rule, and an encryption rule identifier, wherein the encryption rule identifier is used to indicate the encryption rule; an assembling module, configured to assemble the encrypted audio and video code stream and the key information frame to obtain a transport stream code stream, wherein the transport stream code stream includes the encrypted audio and video code stream and the key information frame; A first sending module is configured to send the transport stream code stream to the user terminal, so that the user terminal obtains the key derivation vector corresponding to the key derivation vector number based on the key information frame in the transport stream code stream; and decrypt the encrypted audio and video code stream based on the key derivation vector, the derivation rule in the key information frame, and the encryption rule identifier to obtain and play the audio and video code stream.

13. An audio and video data playback device, characterized in that: Applied to a user terminal, the device includes: a second sending module, configured to send a play instruction to the cloud server, wherein the play instruction is used to instruct the playback of the audio and video code stream in the video surveillance device, so that the cloud server, in response to the play instruction, sends the key derivation vector, the key derivation vector number, and the derivation rule to the video surveillance device; the video surveillance device encrypts the audio and video code stream according to the key derivation vector number, the derivation rule, and a preset encryption rule to obtain an encrypted audio and video code stream, and assembles the encrypted audio and video code stream into a key information frame; assembles the encrypted audio and video code stream and the key information frame into a transport stream; and sends the transport stream to the user terminal; a second receiving module, configured to receive the transport stream code stream sent by the video surveillance device, the transport stream code stream including the encrypted audio and video code stream and the key information frame, the key information frame including the key derivation vector number, the derivation rule, and an encryption rule identifier, the encryption rule identifier being used to indicate the encryption rule; an acquisition module, configured to acquire the key derivation vector corresponding to the key derivation vector number according to the key information frame in the transport stream code stream; The decryption and playback module is used to decrypt the encrypted audio and video code stream according to the key derivation vector, the derivation rule in the key information frame and the encryption rule identifier, obtain the audio and video code stream and play it.

14. An audio and video data playback device, characterized in that: Applied to a cloud server, the device includes: A third receiving module is used to receive a play instruction sent by a user terminal, wherein the play instruction is used to instruct the playback of the audio and video stream in the video surveillance device; a third sending module, configured to send the key derivation vector, the key derivation vector number, and the derivation rule to the video surveillance device in response to the play instruction; The video surveillance device encrypts the audio and video code stream according to the key derivation vector number, the derivation rule, and the preset encryption rule to obtain an encrypted audio and video code stream, and assembles it to obtain a key information frame, where the key information frame includes the key derivation vector number, the derivation rule, and an encryption rule identifier, and the encryption rule identifier is used to indicate the encryption rule; assembles the encrypted audio and video code stream and the key information frame to obtain a transport stream code stream, where the transport stream code stream includes the encrypted audio and video code stream and the key information frame; sends the transport stream code stream to the user terminal; the user terminal obtains the key derivation vector corresponding to the key derivation vector number according to the key information frame in the transport stream code stream; decrypts the encrypted audio and video code stream according to the key derivation vector, the derivation rule, and the encryption rule identifier in the key information frame to obtain the audio and video code stream and play it.

15. An electronic device, characterized in that: The device includes: a processor and a memory storing computer program instructions; when the processor executes the computer program instructions, the audio and video data playback method according to any one of claims 1 to 11 is implemented.

16. A computer-readable storage medium, characterized in that The computer-readable storage medium stores computer program instructions, and when the computer program instructions are executed by the processor, the audio and video data playback method according to any one of claims 1 to 11 is implemented.

17. A computer program product, characterized in that The computer program product includes computer program instructions, and when the computer program instructions are executed by a processor, the audio and video data playback method according to any one of claims 1 to 11 is implemented.

Images