Video stream processing method and device, electronic device and storage medium

Abstract

The embodiment of the present application provides a video stream processing method and device, an electronic device and a storage medium, which belongs to the field of video processing technology. The method includes: obtaining a target video stream from the cloud, and the target video stream includes a first video stream and a second video stream. According to the first video stream, the target data volume of the first ciphertext block is obtained. If the target data volume is less than the target threshold, the first ciphertext block is software decrypted to obtain a second plaintext block. Otherwise, the first ciphertext block is hardware decrypted to obtain a second plaintext block. A second plaintext frame is constructed according to the first plaintext block and the second plaintext block. A decrypted video stream is constructed according to the first plaintext frame and the second plaintext frame, and the decrypted video stream or the second video stream is decoded. Soft decryption is faster for decrypting ciphertext blocks with a small amount of data, and hard decryption is faster for decrypting ciphertext blocks with a large amount of data. The ciphertext blocks in the same encrypted video stream are decrypted by soft decryption or hard decryption according to the amount of data, thereby improving the decryption efficiency.

Description

Technical Field

[0001] The present invention relates to the field of video processing technology, and in particular to a video stream processing method and device, electronic equipment and storage medium. Background Art

[0002] Digital Rights Management (DRM) technology refers to the technology for protecting the rights, controlling the use and managing the digital content, such as audio and video program content, documents, and e-books, during the production, dissemination, sales and use. The working principle of DRM technology is to establish a digital program authorization center, encode and compress the digital program content, and use keys for encryption protection. When users play on demand, they can send the relevant key decryption after verification and authorization by the digital program authorization center, and the program can be played. In related technologies, the encrypted video stream can be decrypted by software modules or hardware engines, but the decryption efficiency of software decryption or hardware decryption alone is low. Therefore, how to improve the decryption efficiency of encrypted video streams has become a technical problem that needs to be solved urgently. Summary of the invention

[0003] The main purpose of the embodiments of the present application is to provide a video stream processing method and device, an electronic device and a storage medium, the purpose of which is to improve the decryption efficiency of the encrypted video stream.

[0004] To achieve the above object, an embodiment of the present application provides a video stream processing method, which is applied to a terminal device. The method includes:

[0005] Obtain a target video stream from the cloud; wherein the target video stream includes a first video stream and a second video stream, the first video stream is a video stream that encrypts a preset video stream, the second video stream is a non-encrypted video stream, the first video stream includes a first plaintext frame and a ciphertext frame, and the ciphertext frame includes a first ciphertext block and a first plaintext block;

[0006] According to the first video stream, obtaining the data amount of the first ciphertext block to obtain a target data amount;

[0007] If the target data amount is less than the target threshold, performing software decryption on the first ciphertext block to obtain a second plaintext block;

[0008] If the target data amount is greater than or equal to the target threshold, performing hardware decryption on the first ciphertext block to obtain a second plaintext block;

[0009] constructing a second plaintext frame according to the first plaintext block and the second plaintext block;

[0010] Constructing a decrypted video stream according to the first plaintext frame and the second plaintext frame;

[0011] The decrypted video stream or the second video stream is decoded.

[0012] In some embodiments, the terminal device is provided with a secure area and a non-secure area, the terminal device includes a secure memory provided in the secure area and a non-secure memory provided in the non-secure area, and after obtaining the target video stream from the cloud, the method further includes:

[0013] If the target video stream is the first video stream, saving the first video stream to the non-secure memory, and obtaining the first video stream from the non-secure memory;

[0014] A decryption operation is performed on the first video stream obtained from the non-secure memory in the secure area to obtain the decrypted video stream, and the decrypted video stream is saved in the secure memory.

[0015] In some embodiments, the terminal device further includes a security decoder disposed in the security area, and the decoding of the decrypted video stream or the second video stream includes:

[0016] The security decoder is called, and the decrypted video stream is decoded by the called security decoder.

[0017] In some embodiments, if the target data amount is less than a target threshold, performing software decryption on the first ciphertext block to obtain a second plaintext block includes:

[0018] If the target data volume is less than the target threshold, acquiring a first central processor;

[0019] The performance mode of the first central processor is adjusted to a first mode, and the first central processor and the first core are bound to obtain a second central processor; wherein the first core is a high-performance core;

[0020] The first ciphertext block is decrypted by software based on the second central processing unit to obtain the second plaintext block.

[0021] In some embodiments, decoding the decrypted video stream or the second video stream comprises:

[0022] adjusting the performance mode of the second central processor from the first mode to a second mode, and unbinding the second central processor from the first core to restore the second central processor to the first central processor; wherein the processing speed of the second central processor in the first mode is higher than the processing speed of the first central processor in the second mode;

[0023] The second video stream is decoded by a decoder and the first central processing unit.

[0024] In some embodiments, the terminal device further includes a non-secure decoder disposed in the non-secure area, and the decoding of the second video stream by the decoder and the first central processor includes:

[0025] The non-secure decoder is called, and the second video stream is decoded by the non-secure decoder and the first central processor.

[0026] In some embodiments, before obtaining the target video stream from the cloud, the method further includes:

[0027] Obtaining a third video stream from the cloud; wherein the third video stream is a video stream that encrypts the preset video stream, and the third video stream includes a second ciphertext block;

[0028] Acquire the data amount of the second ciphertext block to obtain the selected data amount;

[0029] If the selected data amount is less than a preset threshold, the second ciphertext block is decrypted by software; if the selected data amount is greater than or equal to the preset threshold, the second ciphertext block is decrypted by hardware to obtain a decryption time;

[0030] The decryption time corresponding to each of the preset thresholds is obtained, and the preset threshold corresponding to the minimum decryption time is used as the target threshold.

[0031] In some embodiments, if the target data amount is greater than or equal to the target threshold, performing hardware decryption on the first ciphertext block to obtain a second plaintext block includes:

[0032] If the target data volume is greater than or equal to the target threshold, and the first video stream is a video stream encrypted in a first preset mode, setting the hardware engine to set the address pointer of the second plaintext block to be the same as the address pointer of the first ciphertext block, and setting the data length of the second plaintext block to be the same as the data length of the first ciphertext block;

[0033] The first ciphertext block is hardware decrypted by the hardware engine to obtain the second plaintext block.

[0034] In some embodiments, if the target data amount is greater than or equal to the target threshold, performing hardware decryption on the first ciphertext block to obtain a second plaintext block includes:

[0035] If the target data volume is greater than or equal to the target threshold, and the first video stream is a video stream encrypted in the second preset mode, setting the hardware engine to save the block offset; wherein the block offset is the offset of the first ciphertext block in the ciphertext frame;

[0036] The first ciphertext block is decrypted by the hardware engine and the block offset to obtain the second plaintext block; wherein the hardware engine locates the first ciphertext block based on the block offset.

[0037] To achieve the above-mentioned purpose, a second aspect of an embodiment of the present application provides a video stream processing device, the device comprising:

[0038] A first acquisition module is used to acquire a target video stream from the cloud; wherein the target video stream includes a first video stream and a second video stream, the first video stream is a video stream that encrypts a preset video stream, the second video stream is a non-encrypted video stream, the first video stream includes a first plaintext frame and a ciphertext frame, and the ciphertext frame includes a first ciphertext block and a first plaintext block;

[0039] A second acquisition module, configured to acquire the data volume of the first ciphertext block according to the first video stream to obtain a target data volume;

[0040] a first decryption module, wherein if the target data volume is less than a target threshold, the first decryption module is used to perform software decryption on the first ciphertext block to obtain a second plaintext block;

[0041] a second decryption module, wherein if the target data volume is greater than or equal to the target threshold, the second decryption module is used to perform hardware decryption on the first ciphertext block to obtain a second plaintext block;

[0042] A first constructing module, configured to construct a second plaintext frame according to the first plaintext block and the second plaintext block;

[0043] A second construction module, configured to construct a decrypted video stream according to the first plaintext frame and the second plaintext frame;

[0044] A decoding module is used to decode the decrypted video stream or the second video stream.

[0045] To achieve the above objectives, a third aspect of an embodiment of the present application proposes an electronic device, which includes a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the method described in the first aspect is implemented.

[0046] To achieve the above-mentioned purpose, the fourth aspect of an embodiment of the present application proposes a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method described in the first aspect is implemented.

[0047] The video stream processing method and device, electronic device and storage medium proposed in the present application obtain a target video stream from the cloud, wherein the target video stream includes a first video stream and a second video stream, and obtains a target data volume of a first ciphertext block according to the first video stream. If the target data volume is less than the target threshold, the first ciphertext block is decrypted by software to obtain a second plaintext block. Otherwise, the first ciphertext block is decrypted by hardware to obtain a second plaintext block. A second plaintext frame is constructed according to the first plaintext block and the second plaintext block. A decrypted video stream is constructed according to the first plaintext frame and the second plaintext frame, and the decrypted video stream or the second video stream is decoded. Soft decryption is faster for decrypting ciphertext blocks with a small amount of data, and hard decryption is faster for decrypting ciphertext blocks with a large amount of data. The ciphertext blocks in the same encrypted video stream are decrypted by soft decryption or hard decryption according to the amount of data, thereby improving the decryption efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] Figure 1 is an optional flowchart of the video stream processing method provided in an embodiment of the present application;

[0049] Figure 2 is a flowchart of a video stream processing method provided by another embodiment of the present application;

[0050] Figure 3 yes Figure 1 Flow chart of step S103 in FIG.

[0051] Figure 4 yes Figure 1 Flow chart of step S107 in FIG.

[0052] Figure 5 is a flowchart of a video stream processing method provided by a third embodiment of the present application;

[0053] Figure 6 yes Figure 1 Flow chart of step S104 in FIG.

[0054] Figure 7 yes Figure 1 Another flow chart of step S104 in ;

[0055] Figure 8 is a structural diagram of a video stream processing device provided in an embodiment of the present application;

[0056] Fig. 9 It is a schematic diagram of the hardware structure of the electronic device provided in the embodiment of the present application. DETAILED DESCRIPTION

[0057] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0058] It should be noted that, although the functional modules are divided in the device schematic diagram and the logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than the module division in the device or the order in the flowchart. The terms "first", "second", etc. in the specification, claims and the above drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence.

[0059] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein are only for the purpose of describing the embodiments of this application and are not intended to limit this application.

[0060] First, some nouns involved in this application are analyzed:

[0061] TrustZone: TrustZone is a hardware architecture designed by ARM for terminal devices. Its purpose is to build a security framework for terminal devices to resist various possible attacks. TrustZone conceptually divides the hardware and software resources on the terminal device into a secure area and a non-secure area. All operations that need to be kept confidential are performed in the secure area (such as fingerprint recognition, password processing, data encryption and decryption, security authentication, etc.), which can be protected from many potential attacks. The rest of the operations are performed in the non-secure area (such as user operating systems, various applications, etc.).

[0062] DRM video stream: DRM video stream refers to the encryption and protection of video content through a digital rights management mechanism to restrict access, storage and dissemination by unauthorized users.

[0063] Digital Rights Management (DRM) technology refers to the technology for protecting the rights, controlling the use and managing the production, dissemination, sales and use of digital content, such as audio and video program content, documents, and e-books. The working principle of DRM technology is to establish a digital program authorization center, encode and compress digital program content, and use keys for encryption protection. When users play on demand, they can send the relevant key decryption after verification and authorization by the digital program authorization center, and the program can be played. In related technologies, encrypted video streams can be decrypted by software modules or hardware engines, but the decryption efficiency of separate software decryption or hardware decryption is low.

[0064] Based on this, the embodiments of the present application provide a video stream processing method and device, an electronic device and a storage medium, which are intended to obtain the data amount of the first ciphertext block and obtain the target data amount. If the target data amount is less than the target threshold, the first ciphertext block is software decrypted to obtain the second plaintext block. If the target data amount is greater than or equal to the target threshold, the first ciphertext block is hardware decrypted. Software decryption is faster for ciphertext blocks with small data amounts, and hardware decryption is faster for ciphertext blocks with large data amounts. The ciphertext blocks in the same encrypted video stream are decrypted using soft or hard decryption according to the amount of data, thereby improving the decryption efficiency.

[0065] The video stream processing method and device, electronic device and storage medium provided in the embodiments of the present application are specifically described through the following embodiments. First, the video stream processing method in the embodiments of the present application is described.

[0066] The embodiments of the present application can acquire and process relevant data based on artificial intelligence technology. Artificial Intelligence (AI) is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results.

[0067] AI basic technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing technology, operation / interaction systems, mechatronics, etc. AI software technologies mainly include computer vision technology, robotics technology, biometrics technology, speech processing technology, natural language processing technology, and machine learning / deep learning.

[0068] The video stream processing method provided in the embodiment of the present application can be applied to the terminal device, can also be applied to the server side, and can also be software running in the terminal device or the server side. In some embodiments, the terminal device can be a smart phone, a tablet computer, a laptop computer, a desktop computer, etc.; the server side can be configured as an independent physical server, or a server cluster or distributed system composed of multiple physical servers, and can also be configured as a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms; the software can be an application that implements the video stream processing method, etc., but is not limited to the above forms.

[0069] The present application can be used in many general or special computer system environments or configurations. For example: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, etc. The present application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The present application can also be practiced in distributed computing environments, in which tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules can be located in local and remote computer storage media including storage devices.

[0070] Please refer to Figure 1 , Figure 1 It is an optional flowchart of the video stream processing method provided in an embodiment of the present application, and the video stream processing method is applied to a terminal device. Figure 1 The method may include but is not limited to steps S101 to S107.

[0071] Step S101, obtaining a target video stream from the cloud; wherein the target video stream includes a first video stream and a second video stream, the first video stream is a video stream that encrypts a preset video stream, the second video stream is a non-encrypted video stream, the first video stream includes a first plaintext frame and a ciphertext frame, and the ciphertext frame includes a first ciphertext block and a first plaintext block;

[0072] Step S102, obtaining the data volume of the first ciphertext block according to the first video stream to obtain the target data volume;

[0073] Step S103, if the target data volume is less than the target threshold, software decryption is performed on the first ciphertext block to obtain a second plaintext block;

[0074] Step S104, if the target data volume is greater than or equal to the target threshold, hardware decryption is performed on the first ciphertext block to obtain a second plaintext block;

[0075] Step S105, constructing a second plaintext frame according to the first plaintext block and the second plaintext block;

[0076] Step S106, constructing a decrypted video stream according to the first plaintext frame and the second plaintext frame;

[0077] Step S107: decode the decrypted video stream or the second video stream.

[0078] In step S101 of some embodiments, the target video stream can be obtained from the cloud through a media player for viewing. If the target video stream is an encrypted video stream, the encrypted video stream needs to be decrypted, and after obtaining the decrypted video stream, it is decoded and then displayed and played. If the target video stream is an unencrypted video stream, it does not need to be decrypted, and only needs to be decoded for display and playback. Specifically, the target video stream includes a first video stream and a second video stream, the first video stream is a video stream that encrypts the preset video stream, and the second video stream is an unencrypted video stream.

[0079] In one example, the first video stream may be a DRM video stream, which includes multiple first plaintext frames and multiple ciphertext frames, and each ciphertext frame includes multiple first ciphertext blocks and multiple first plaintext blocks. Each first ciphertext block in each ciphertext frame needs to be decrypted to obtain the decrypted video stream. Among them, the DRM video stream is a variable code stream, and the data amount of the ciphertext blocks in different ciphertext frames is generally different, but the data amount of all ciphertext blocks in the same ciphertext frame is the same. For example, the DRM video stream includes ciphertext frame 1 and ciphertext frame 2. Ciphertext frame 1 includes the first ciphertext block a, the first ciphertext block b and the first ciphertext block c. Ciphertext frame 2 includes the first ciphertext block d, the first ciphertext block e and the first ciphertext block f, then the data amount of the first ciphertext block a, the first ciphertext block b and the first ciphertext block c are all n bytes, and the data amount of the first ciphertext block d, the first ciphertext block e and the first ciphertext block f are all m bytes. Among them, n and m are not the same, n can be greater than m, and n can also be less than m.

[0080] In step S102 of some embodiments, when the target video stream is the first video stream, the first video stream needs to be decrypted first. In order to improve the decryption speed, the ciphertext block needs to be decrypted using soft decoding or hard decoding according to the data volume of the ciphertext block. Therefore, the data volume of the first ciphertext block is obtained to obtain the target data volume.

[0081] In step S103 of some embodiments, software decryption is performed using a software module, which requires certain computing resources and time to complete the decryption process, and decryption is faster for ciphertext blocks with small data volumes. Therefore, when the target data volume is less than the target threshold, the first ciphertext block is software decrypted by the software module to obtain the second plaintext block.

[0082] In step S104 of some embodiments, hardware decryption is performed using a hardware engine, and decryption is faster for ciphertext blocks with large data volumes. Therefore, when the target data volume is greater than or equal to the target threshold, the hardware engine performs hardware decryption on the first ciphertext block to obtain a second plaintext block.

[0083] In step S105 of some embodiments, when the ciphertext frame includes multiple first ciphertext blocks and multiple first plaintext blocks, the data amounts of the multiple first ciphertext blocks are the same. Therefore, the same soft solution or hard solution is used for decryption to obtain multiple first plaintext blocks. The multiple first plaintext blocks and the multiple second plaintext blocks constitute the second plaintext frame, that is, the second plaintext frame is the decrypted ciphertext frame.

[0084] In step S106 of some embodiments, when the first video stream includes multiple first plaintext frames and multiple ciphertext frames, multiple ciphertext frames are decrypted to obtain multiple second plaintext frames. Multiple first plaintext frames and multiple second plaintext frames constitute a decrypted video stream, that is, after the first video stream is decrypted, a decrypted video stream is obtained.

[0085] In step S107 of some embodiments, in order to reduce storage space and transmission bandwidth, the video stream is also encoded when it is encrypted. Therefore, after the first video stream is decrypted and the decrypted video stream is obtained, the decrypted video stream must be decoded before it can be displayed and played. The second video stream is a non-encrypted video stream. If the target video stream is the second video stream, the second video stream can be directly decoded so that it can be displayed and played.

[0086] Steps S101 to S107 shown in the embodiment of the present application are obtained by obtaining a target video stream from the cloud, and the target video stream includes a first video stream and a second video stream. According to the first video stream, the data amount of the first ciphertext block is obtained to obtain the target data amount. If the target data amount is less than the target threshold, the first ciphertext block is software decrypted to obtain the second plaintext block. If the target data amount is greater than or equal to the target threshold, the first ciphertext block is hardware decrypted to obtain the second plaintext block. A second plaintext frame is constructed based on the first plaintext block and the second plaintext block. A decrypted video stream is constructed based on the first plaintext frame and the second plaintext frame, and the decrypted video stream or the second video stream is decoded. Software decryption is faster for decrypting ciphertext blocks with a small amount of data, and hardware decryption is faster for decrypting ciphertext blocks with a large amount of data. The ciphertext blocks in the same encrypted video stream are decrypted by soft or hard decryption according to the amount of data, thereby improving the decryption efficiency.

[0087] See also Figure 2 In some embodiments, the terminal device is provided with a secure area and a non-secure area, and the terminal device includes a secure memory provided in the secure area and a non-secure memory provided in the non-secure area. After step S101, the video stream processing method may further include but is not limited to steps S201 to S202:

[0088] Step S201, if the target video stream is the first video stream, save the first video stream to the non-secure memory, and obtain the first video stream from the non-secure memory;

[0089] Step S202: performing a decryption operation on the first video stream obtained from the non-secure memory in the secure area to obtain a decrypted video stream, and saving the decrypted video stream in the secure memory.

[0090] In steps S201 to S202 illustrated in this embodiment, the terminal device is applied in a hardware security environment (TrustZone), and TrustZone divides the hardware and software resources on the terminal device into a secure area and a non-secure area. Therefore, the terminal device includes a secure memory arranged in a secure area and a non-secure memory arranged in a non-secure area. Since the operation of obtaining the target video stream from the cloud is performed on the non-secure area, after obtaining the target video stream from the cloud, the target video stream can only be saved in the non-secure memory. If the target video stream is the first video stream, if the decryption operation is continued to be performed on the first video stream in the non-secure area, the decrypted video stream will be illegally obtained, resulting in the leakage of the decrypted video stream. Therefore, a decryption operation is performed on the first video stream obtained from the non-secure memory in the secure area to obtain a decrypted video stream. The decrypted video stream is saved in the secure memory, and the decrypted video stream is decoded and played in the secure memory, thereby improving the security of the decrypted video stream.

[0091] In some embodiments, the terminal device further includes a security decoder disposed in a secure area, and therefore, step S107 may include calling the security decoder, and decoding the decrypted video stream through the called security decoder.

[0092] Specifically, the Video Processing Unit (VPU) is a core engine of a video processing platform, which has a hard decoding function and the ability to reduce CPU load. In addition, the VPU can reduce server load and network bandwidth consumption. The secure decoder can be a VPU Secure decoder, which is called to decode the decrypted video stream, further improving the security of the decrypted video stream.

[0093] See also Figure 3 In some embodiments, step S103 may include but is not limited to steps S301 to S303:

[0094] Step S301, if the target data volume is less than the target threshold, obtain the first central processor;

[0095] Step S302, adjusting the performance mode of the first central processor to the first mode, and binding the first central processor and the first core to obtain a second central processor; wherein the first core is a high-performance core;

[0096] Step S303: performing software decryption on the first ciphertext block based on the second central processing unit to obtain a second plaintext block.

[0097] In step S301 of some embodiments, the first central processor is a CPU, and software decryption requires the use of the CPU's computing power. If the target data volume is less than the target threshold, it means that software decryption is required, and therefore, the first central processor is acquired.

[0098] In step S302 of some embodiments, during the operation of the CPU, in order to protect the stability and security of the CPU, the CPU will generally run in a low power consumption mode, at which time the CPU's main frequency and performance will be subject to certain restrictions. In order to better utilize the CPU's computing power to improve decryption efficiency, the CPU needs to be set to a high-performance mode. In the high-performance mode, the CPU's main frequency and performance will be improved to a certain extent, so that various computing tasks can be completed faster.

[0099] Specifically, the first mode is a high-performance mode, and the performance mode of the first central processing unit is adjusted to a high-performance mode. In order to further improve the computing power of the CPU, thereby improving the decryption efficiency, the first central processing unit and the first core can also be bound to obtain a second central processing unit. Among them, the first core is a high-performance core, which is also called a large core, and refers to a core with higher performance and higher power consumption in the processor core of the CPU. In conventional multi-core CPUs, due to power consumption and heat generation issues, multiple cores are generally designed, including some low-power small cores and some high-performance large cores.

[0100] In step S303 of some embodiments, the second CPU is a CPU that enters high-performance mode and is bound to a high-performance core. The second CPU performs software decryption on the first ciphertext block to obtain the second plaintext block, thereby increasing the speed of software decryption.

[0101] In steps S301 to S303 of this embodiment, if the target data volume is less than the target threshold, it is determined to be software decryption. The performance mode of the first central processor is adjusted to a high-performance mode, and a high-performance core is bound to obtain a second central processor. The first ciphertext block is software decrypted based on the second central processor to improve the speed of software decryption.

[0102] See also Figure 4 In some embodiments, step S107 includes but is not limited to steps S401 to S402:

[0103] Step S401, adjusting the performance mode of the second central processor from the first mode to the second mode, and unbinding the second central processor and the first core to restore the second central processor to the first central processor; wherein the processing speed of the second central processor in the first mode is higher than the processing speed of the first central processor in the second mode;

[0104] Step S402: decode the second video stream by using a decoder and a first central processing unit.

[0105] In steps S401 to S402 illustrated in the present embodiment, the second mode may be a low power consumption mode, and the processing speed of the second central processing unit in the high performance mode is higher than the processing speed of the first central processing unit in the low power consumption mode. Since the second video stream is a non-encrypted video stream, only the second video stream needs to be decoded. During decoding, the requirements for the CPU's computing power are not high. In order to prevent the waste of the CPU's computing resources, the performance mode of the second central processing unit is adjusted from the high performance mode to the low power consumption mode, and the high performance core is untied to restore the second central processing unit to the first central processing unit. The decoder then decodes the second video stream based on the first central processing unit. Through this embodiment, it is achieved that when the requirements for the CPU's computing power are not high, the use of the CPU's computing resources is reduced.

[0106] In some embodiments, the terminal device further includes a non-secure decoder located in a non-secure area. Therefore, step S402 may include but is not limited to calling the non-secure decoder and decoding the second video stream through the non-secure decoder and the first central processor.

[0107] Specifically, the non-secure decoder may be a VPU Normal decoder, which is called to decode the second video stream. In this embodiment, since the secure decoder usually performs some additional authentication operations to ensure the security of the decoding process, the decoding speed may be slower than that of the non-secure decoder. The second video stream is a non-encrypted video stream and does not need to be protected from illegal acquisition. Therefore, the non-secure decoder is used for decoding, which can improve the decoding speed.

[0108] Please refer to Figure 5 In some embodiments, before step S101, the video stream processing method may further include but is not limited to steps S501 to S504:

[0109] Step S501, obtaining a third video stream from the cloud; wherein the third video stream is a video stream that encrypts the preset video stream, and the third video stream includes a second ciphertext block;

[0110] Step S502, obtaining the data amount of the second ciphertext block to obtain the selected data amount;

[0111] Step S503, if the selected data amount is less than the preset threshold, the second ciphertext block is decrypted by software; if the selected data amount is greater than or equal to the preset threshold, the second ciphertext block is decrypted by hardware to obtain the decryption time;

[0112] Step S504, obtaining the decryption time corresponding to each preset threshold, and taking the preset threshold corresponding to the minimum decryption time as the target threshold.

[0113] In step S501 of some embodiments, it is necessary to pre-measure the target threshold value at which the decryption time required for the combination of soft decryption and hard decryption is the minimum. Therefore, a third video stream is obtained from the cloud for testing. The third video stream is a video stream that encrypts the preset video stream, and the third video stream includes a third plaintext frame and a first ciphertext frame, and the first ciphertext frame includes a second ciphertext block and a second plaintext block.

[0114] In steps S502 to S503 of some embodiments, since the second ciphertext block needs to be decrypted, the data amount of the second ciphertext block is obtained to obtain the selected data amount. If the selected data amount is less than the preset threshold, the second ciphertext block is decrypted by software; if the selected data amount is greater than or equal to the preset threshold, the second ciphertext block is decrypted by hardware to obtain the decryption time.

[0115] In step S504 of some embodiments, if four preset thresholds are set, the decryption durations corresponding to the four preset thresholds are obtained, and the preset threshold corresponding to the minimum decryption duration is used as the target threshold. In one example, multiple preset thresholds may be set to make the value of the target threshold more accurate.

[0116] In steps S501 to S504 shown in this embodiment, the preset threshold corresponding to the minimum decryption duration is used as the target threshold, thereby improving the decryption speed of the encrypted video stream.

[0117] See also Figure 6 In some embodiments, step S104 includes but is not limited to steps S601 to S602:

[0118] Step S601, if the target data volume is greater than or equal to the target threshold, and the first video stream is a video stream encrypted in the first preset mode, setting the hardware engine to set the address pointer of the second plaintext block to be the same as the address pointer of the first ciphertext block, and setting the data length of the second plaintext block to be the same as the data length of the first ciphertext block;

[0119] Step S602: Perform hardware decryption on the first ciphertext block through a hardware engine to obtain a second plaintext block.

[0120] In the steps S601 to S602 shown in this embodiment, the first preset mode is the CBCS (Cipher BlockChaining with Segmenting) mode, which is a mode in the symmetric encryption algorithm. The encryption method of the CBCS mode for the ciphertext frame can be that there is a plaintext block behind each ciphertext block. As a result, it is difficult for ordinary hardware engines to decrypt the video stream encrypted by the CBCS mode, that is, when using ordinary hardware engines for decryption, the software module needs to call the hardware engine multiple times to complete the decryption of a frame of data. The decryption operation is complicated and takes a long time. Therefore, in this embodiment, by enabling the hardware engine to support the hash mode (DDT mode), and enabling the SRC DDT Table to include the address pointers and data lengths of all first ciphertext blocks; the DST DDT Table includes the address pointers and data lengths of all second plaintext blocks. The hardware engine performs hardware decryption on the first ciphertext block to obtain the second plaintext block, and the hardware engine sets the address pointer of the second plaintext block to be the same as the address pointer of the first ciphertext block, so that the position of the second plaintext block after decryption can be clearly defined. The data length of the second plaintext block is set to be the same as the data length of the first ciphertext block, thereby ensuring that the second plaintext block and the first plaintext block are continuous plaintext data. The second plaintext frame is constructed according to the first plaintext block and the second plaintext block, and the decrypted video stream is constructed according to the first plaintext frame and the second plaintext frame, so that the hardware engine can decrypt the video stream encrypted in the CBCS mode, and the operation is simple and the time required is short.

[0121] See also Figure 7 In some embodiments, step S104 includes but is not limited to steps S701 to S702:

[0122] Step S701, if the target data volume is greater than or equal to the target threshold, and the first video stream is a video stream encrypted in the second preset mode, then the hardware engine is set to save the block offset; wherein the block offset is the offset of the first ciphertext block in the ciphertext frame;

[0123] Step S702, decrypting the first ciphertext block through the hardware engine and the block offset to obtain a second plaintext block; wherein the hardware engine locates the first ciphertext block based on the block offset.

[0124] In the steps S701 to S702 shown in this embodiment, the second preset mode is the CENC (Common Encryption specification) mode. The CENC mode is a common encryption mode. When the CENC mode encrypts the ciphertext frame, the first ciphertext block will contain a block offset. The block offset is the offset of the first ciphertext block in the ciphertext frame. The offset is not fixed, sometimes 0, and sometimes not 0. As a result, the hardware engine cannot accurately locate the position of the first ciphertext block in the ciphertext frame and cannot decrypt it. Therefore, the hardware engine can support the Data Package format (data packet format). The Data Package format contains a PRE_AAD area (pre-auxiliary area). The PRE_AAD area is used to store block_offset, which facilitates the hardware engine to quickly locate the position of the first ciphertext block in the ciphertext frame based on the block offset, decrypt the first ciphertext block, and obtain the second plaintext block. Thereby, the hardware engine is enabled to decrypt the ciphertext frame encrypted by the CENC mode.

[0125] See also Figure 8 The embodiment of the present application further provides a video stream processing device, which can implement the above-mentioned video stream processing method, and the device includes:

[0126] The first acquisition module 801 is used to acquire a target video stream from the cloud; wherein the target video stream includes a first video stream and a second video stream, the first video stream is a video stream that encrypts a preset video stream, the second video stream is a non-encrypted video stream, the first video stream includes a first plaintext frame and a ciphertext frame, and the ciphertext frame includes a first ciphertext block and a first plaintext block;

[0127] The second acquisition module 802 is used to acquire the data amount of the first ciphertext block according to the first video stream to obtain the target data amount;

[0128] A first decryption module 803, if the target data volume is less than the target threshold, the first decryption module 803 is used to perform software decryption on the first ciphertext block to obtain a second plaintext block;

[0129] A second decryption module 804, if the target data volume is greater than or equal to the target threshold, the second decryption module 804 is used to perform hardware decryption on the first ciphertext block to obtain a second plaintext block;

[0130] A first constructing module 805, configured to construct a second plaintext frame according to the first plaintext block and the second plaintext block;

[0131] A second construction module 806, configured to construct a decrypted video stream according to the first plaintext frame and the second plaintext frame;

[0132] The decoding module 807 is used to decode the decrypted video stream or the second video stream.

[0133] The specific implementation of the video stream processing device is basically the same as the specific implementation of the above-mentioned video stream processing method, and will not be repeated here.

[0134] The embodiment of the present application also provides an electronic device, the electronic device includes a memory and a processor, the memory stores a computer program, and the processor implements the above-mentioned video stream processing method when executing the computer program. The electronic device can be any intelligent terminal including a tablet computer, a car computer, etc.

[0135] See also Fig. 9 , Fig. 9 The hardware structure of an electronic device of another embodiment is illustrated, and the electronic device includes:

[0136] The processor 901 may be implemented by a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of the present application;

[0137] The memory 902 can be implemented in the form of a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 902 can store an operating system and other application programs. When the technical solution provided in the embodiment of this specification is implemented by software or firmware, the relevant program code is stored in the memory 902, and the processor 901 calls and executes the video stream processing method of the embodiment of this application;

[0138] Input / output interface 903, used to implement information input and output;

[0139] Communication interface 904, used to realize communication interaction between the device and other devices, which can be realized through wired mode (such as USB, network cable, etc.) or wireless mode (such as mobile network, WIFI, Bluetooth, etc.);

[0140] A bus 905 that transmits information between various components of the device (e.g., the processor 901, the memory 902, the input / output interface 903, and the communication interface 904);

[0141] The processor 901 , the memory 902 , the input / output interface 903 and the communication interface 904 are connected to each other in communication within the device via a bus 905 .

[0142] An embodiment of the present application further provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the above-mentioned video stream processing method is implemented.

[0143] The memory, as a non-transient computer-readable storage medium, can be used to store non-transient software programs and non-transient computer executable programs. In addition, the memory may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some embodiments, the memory may optionally include a memory remotely disposed relative to the processor, and these remote memories may be connected to the processor via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0144] The embodiments described in the embodiments of the present application are intended to more clearly illustrate the technical solutions of the embodiments of the present application and do not constitute a limitation on the technical solutions provided in the embodiments of the present application. Those skilled in the art will appreciate that with the evolution of technology and the emergence of new application scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.

[0145] Those skilled in the art will appreciate that the technical solutions shown in the figures do not constitute a limitation on the embodiments of the present application, and may include more or fewer steps than shown in the figures, or a combination of certain steps, or different steps.

[0146] The device embodiments described above are merely illustrative, and the units described as separate components may or may not be physically separated, that is, they may be located in one place or distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0147] Those skilled in the art will appreciate that all or some of the steps in the methods disclosed above, and the functional modules / units in the systems and devices may be implemented as software, firmware, hardware, or a suitable combination thereof.

[0148] The terms "first", "second", "third", "fourth", etc. (if any) in the specification of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0149] It should be understood that in the present application, "at least one (item)" means one or more, and "plurality" means two or more. "And / or" is used to describe the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" can mean: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally indicates that the objects associated before and after are in an "or" relationship. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.

[0150] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the above units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0151] The units described above as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0152] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0153] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including multiple instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of various embodiments of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, referred to as ROM), random access memory (Random Access Memory, referred to as RAM), disk or optical disk and other media that can store programs.

[0154] The preferred embodiments of the present invention are described above with reference to the accompanying drawings, but the scope of the rights of the present invention is not limited thereto. Any modification, equivalent substitution and improvement made by a person skilled in the art without departing from the scope and essence of the present invention should be within the scope of the rights of the present invention.

Claims

1. A video stream processing method, characterized in that: Applied to a terminal device, the method comprises: Obtain a target video stream from the cloud; wherein the target video stream includes a first video stream and a second video stream, the first video stream is a video stream that encrypts a preset video stream, the second video stream is a non-encrypted video stream, the first video stream includes a first plaintext frame and a ciphertext frame, and the ciphertext frame includes a first ciphertext block and a first plaintext block; According to the first video stream, obtaining the data amount of the first ciphertext block to obtain a target data amount; If the target data volume is less than the target threshold, a first central processor is obtained, a performance mode of the first central processor is adjusted to a high-performance mode, and the first central processor and a first core are bound to obtain a second central processor, wherein the first core is a high-performance core, and software decryption is performed on the first ciphertext block based on the second central processor to obtain a second plaintext block; wherein a processing speed of the second central processor in the high-performance mode is higher than a processing speed of the first central processor in the low-power mode; If the target data volume is greater than or equal to the target threshold, and the first video stream is a video stream encrypted in a first preset mode, the hardware engine is set to set the address pointer of the second plaintext block to be the same as the address pointer of the first ciphertext block, and the data length of the second plaintext block to be the same as the data length of the first ciphertext block, and the hardware engine is used to perform hardware decryption on the first ciphertext block to obtain a second plaintext block; wherein the first preset mode is a mode in a symmetric encryption algorithm; constructing a second plaintext frame according to the first plaintext block and the second plaintext block; Constructing a decrypted video stream according to the first plaintext frame and the second plaintext frame; The decrypted video stream or the second video stream is decoded.

2. The video stream processing method according to claim 1, characterized in that: The terminal device is provided with a secure area and a non-secure area, and the terminal device includes a secure memory provided in the secure area and a non-secure memory provided in the non-secure area. After acquiring the target video stream from the cloud, the method further includes: If the target video stream is the first video stream, saving the first video stream to the non-secure memory, and obtaining the first video stream from the non-secure memory; A decryption operation is performed on the first video stream obtained from the non-secure memory in the secure area to obtain the decrypted video stream, and the decrypted video stream is saved in the secure memory.

3. The video stream processing method according to claim 2, characterized in that: The terminal device further includes a security decoder disposed in the security area, and the decoding of the decrypted video stream or the second video stream includes: The security decoder is called, and the decrypted video stream is decoded by the called security decoder.

4. The video stream processing method according to claim 1, characterized in that: The decoding of the decrypted video stream or the second video stream comprises: Adjusting the performance mode of the second central processor from the high-performance mode to the low-power mode, and unbinding the second central processor from the first core to restore the second central processor to the first central processor; The second video stream is decoded by a decoder and the first central processing unit.

5. The video stream processing method according to claim 4, characterized in that: The terminal device further includes a non-secure decoder disposed in a non-secure area, and the decoding of the second video stream by the decoder and the first central processor includes: The non-secure decoder is called, and the second video stream is decoded by the non-secure decoder and the first central processor.

6. The video stream processing method according to any one of claims 1 to 5, characterized in that: Before obtaining the target video stream from the cloud, the method further includes: Obtaining a third video stream from the cloud; wherein the third video stream is a video stream that encrypts the preset video stream, and the third video stream includes a second ciphertext block; Acquire the data amount of the second ciphertext block to obtain the selected data amount; If the selected data amount is less than a preset threshold, the second ciphertext block is decrypted by software; if the selected data amount is greater than or equal to the preset threshold, the second ciphertext block is decrypted by hardware to obtain a decryption time; The decryption time corresponding to each of the preset thresholds is obtained, and the preset threshold corresponding to the minimum decryption time is used as the target threshold.

7. The video stream processing method according to any one of claims 1 to 5, characterized in that: If the target data volume is greater than or equal to the target threshold, and the first video stream is a video stream encrypted in the second preset mode, setting the hardware engine to save the block offset; wherein the block offset is the offset of the first ciphertext block in the ciphertext frame; The first ciphertext block is decrypted by the hardware engine and the block offset to obtain the second plaintext block; wherein the hardware engine locates the first ciphertext block based on the block offset.

8. A video stream processing device, characterized in that: The device comprises: A first acquisition module is used to acquire a target video stream from the cloud; wherein the target video stream includes a first video stream and a second video stream, the first video stream is a video stream that encrypts a preset video stream, the second video stream is a non-encrypted video stream, the first video stream includes a first plaintext frame and a ciphertext frame, and the ciphertext frame includes a first ciphertext block and a first plaintext block; A second acquisition module, configured to acquire the data volume of the first ciphertext block according to the first video stream to obtain a target data volume; The first decryption module, if the target data volume is less than the target threshold, obtains a first central processor, adjusts the performance mode of the first central processor to a high-performance mode, binds the first central processor and the first core to obtain a second central processor, wherein the first core is a high-performance core, and performs software decryption on the first ciphertext block based on the second central processor to obtain a second plaintext block; wherein the processing speed of the second central processor in the high-performance mode is higher than the processing speed of the first central processor in the low-power mode; A second decryption module, if the target data volume is greater than or equal to the target threshold value, and the first video stream is a video stream encrypted in a first preset mode, then setting a hardware engine to set the address pointer of the second plaintext block to be the same as the address pointer of the first ciphertext block, setting the data length of the second plaintext block to be the same as the data length of the first ciphertext block, and performing hardware decryption on the first ciphertext block through the hardware engine to obtain a second plaintext block; wherein the first preset mode is a mode in a symmetric encryption algorithm; A first constructing module, configured to construct a second plaintext frame according to the first plaintext block and the second plaintext block; A second construction module, configured to construct a decrypted video stream according to the first plaintext frame and the second plaintext frame; A decoding module is used to decode the decrypted video stream or the second video stream.

9. An electronic device, characterized in that: The electronic device comprises a memory and a processor, the memory stores a computer program, and the processor implements the video stream processing method according to any one of claims 1 to 7 when executing the computer program.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the video stream processing method according to any one of claims 1 to 7 is implemented.

Images