Safety output method and electronic device

Abstract

The present application relates to the field of multimedia technology, and discloses a secure output method and an electronic device, the secure output method comprising: when the electronic device decodes a video code stream to obtain a first decoded video frame, the electronic device generates frame-level protection information of the first decoded video frame according to an output control strategy and a resolution of the first decoded video frame. Then, the electronic device determines whether to output a first display video frame according to the frame-level protection information, and the first display video frame is obtained according to the first decoded video frame. It can be seen that each time the electronic device outputs a video frame, it can output the video frame according to the frame-level protection information corresponding to the video frame, thereby providing more accurate output protection and improving the protection performance.

Description

Technical Field

[0001] The present application relates to the field of multimedia technology, and in particular to a secure output method and electronic device. Background Art

[0002] In order to protect media content that needs protection, electronic devices can process media content in the transmission process according to media content usage rules (CUR) from decoding the code stream to transmitting the decoded video stream to the output port. CUR can include protection strategies such as high bandwidth digital content protection (HDCP) strategy and video watermark protection strategy.

[0003] However, the existing secure output mechanism protects media content at the bitstream level, which may cause problems such as video error display and has poor protection performance for media content. Summary of the invention

[0004] The present application provides a secure output method and electronic device to solve the problem of poor protection performance of existing media content protection mechanisms.

[0005] In a first aspect, the present application provides a secure output method, the method comprising: in a stage of decoding a video code stream to obtain a first decoded video frame, generating frame-level protection information of the first decoded video frame according to an output control strategy and a resolution of the first decoded video frame; determining whether to output a first display video frame according to the frame-level protection information, the first display video frame being obtained based on the first decoded video frame.

[0006] Among them, the electronic devices involved in this field can support a trusted execution environment (TEE) and a rich execution environment (REE). TEE is used to provide a protected execution environment for protected application software, and REE is used to provide an execution environment for unprotected application software. The media content transmitted under the TEE can, for example, be processed according to the media content usage rules (CUR) to achieve protection of the media content. CUR may include HDCP policies, video watermark protection policies, prohibition of transcoding, prohibition of recording and other protection policies. This application collectively refers to HDCP policies and video watermark protection policies as "output protection policies" or "output control policies."

[0007] In the technical solution of the present application, the electronic device decodes the video code stream to obtain the first decoded video frame, and generates frame-level protection information of the first decoded video frame. Furthermore, the electronic device can determine whether to output the first display video frame based on the frame-level protection information. It can be seen that the protection information in the embodiment of the present application is at the frame level, and each video frame has its own corresponding frame-level protection information. Each output video frame can be output protected according to the frame-level protection information corresponding to the video frame, which has higher security and better flexibility.

[0008] In a possible implementation, after decoding the video code stream to obtain a first decoded video frame, the method further includes: processing the first decoded video frame to obtain the first display video frame; and updating the frame-level protection information according to the resolution of the first display video frame.

[0009] In some embodiments, the electronic device may process the first decoded video frame to obtain an output first display video frame. Accordingly, after the electronic device processes the first decoded video frame, it is possible that the resolution of the first decoded video frame changes accordingly, that is, the resolution of the first display video frame may be different from the resolution of the first decoded video frame. Based on this, the electronic device may update the frame-level protection information according to the resolution of the first display video frame. It can be seen that by adopting this implementation method, the electronic device can update the frame-level protection information of the video frame according to the change of the video frame resolution, so that each video frame corresponds to the protection information of the video frame.

[0010] In a possible implementation, the processing the first decoded video frame to obtain the first display video frame includes: enlarging the first decoded video frame to obtain the first display video frame; or reducing the first decoded video frame to obtain the first display video frame.

[0011] In a possible implementation, the frame-level protection information includes the resolution and minimum resolution of the first decoded video frame, the minimum resolution being the minimum resolution corresponding to the video code stream in the entire media path, and updating the frame-level protection information according to the resolution of the first displayed video frame includes: when the resolution of the first displayed video frame is less than the minimum resolution, modifying the minimum resolution in the frame-level protection information to the resolution of the first displayed video frame. The frame-level protection information includes the minimum resolution, and the minimum resolution is the minimum resolution corresponding to the video code stream in the entire media path.

[0012] It should be understood that the minimum resolution is usually set in the initial information of the frame-level protection information. In the frame-level protection information of the first decoded video frame, if the resolution of the first decoded video frame is less than the minimum resolution initially set, the minimum resolution in the frame-level protection information is modified to the resolution of the first decoded video frame. Similarly, when the resolution of the first displayed video frame is less than the minimum resolution in the frame-level protection information (i.e., the resolution of the first decoded video frame), the minimum resolution in the frame-level protection information is modified to the resolution of the first displayed video frame.

[0013] In one possible implementation, the determination of whether to output the first display video frame is based on the frame-level protection information, including: obtaining the output protection status allowed by the output port; determining a target output control policy that matches the output protection status; judging whether the minimum resolution in the frame-level protection information is less than the resolution allowed by the target output control policy; when the minimum resolution in the frame-level protection information is less than the resolution allowed by the target output control policy, outputting the first display video frame.

[0014] High Definition Multimedia Interface (HDMI) is a hardware interface that supports HDCP policy. The HDMI of an electronic device can determine the output protection state (also called HDCP protection state) of the output video frame according to the HDCP support of the connected display device. For example, if the HDMI of an electronic device supports HDCP2.2 and HDCP1.4, and the interface connected to the HDMI only supports HDCP1.4, then the HDCP protection state of the output video frame determined by the HDMI is HDCP1.4.

[0015] Before outputting the first display video, the electronic device can obtain the HDCP protection state of HDMI, and then determine the target output control strategy that matches the HDCP protection state from the frame-level protection information of the first display video frame. Then, when the minimum resolution in the frame-level protection information is less than the resolution allowed by the target output control strategy, the electronic device outputs the first display video frame. It can be seen that each time the electronic device outputs a video frame, it can output the video frame according to the frame-level protection information corresponding to the video frame, so that the security is higher. When the resolution of the video frame exceeds the resolution allowed by the output control strategy, the output of the video frame will be prohibited. It can be seen that this implementation method can effectively protect the output of the high-definition video frame. The protection information in the embodiment of the present application is at the frame level, and each video frame has its own corresponding frame-level protection information. Each output video frame can be output protected according to the frame-level protection information corresponding to the video frame, which is more secure and more flexible.

[0016] In a possible implementation, before decoding the video stream to obtain the first decoded video frame, it also includes: a trusted application TA configures the output control policy for the video stream. The software system that performs output protection on the video content includes a conditional access system (CAS) or a digital rights management (DRM) system. In this application, CAS or DRM is expressed as "CAS / DRM". The TEE software application layer may include a CAS / DRM trusted application (CAS / DRM TA). After receiving the encrypted video stream, TA decrypts the encrypted video stream to obtain the video stream and the protection requirements corresponding to the video stream. Afterwards, TA can configure the output control policy according to the protection requirements. Using this implementation, information for generating frame-level protection information can be provided.

[0017] In a possible implementation, the method further includes: determining whether the video code stream and the first decoded video frame correspond to the same media path; determining whether the first decoded video frame and the first display video frame correspond to the same media path. In the technical field of the present invention, the path for transmitting video content is a media path. The media path includes a series of hardware modules and memory used for video processing. In the present application, the hardware decoding module (video decoder, VDEC) is used to decode the video code stream to obtain the first decoded video frame. The video processor (video processor, VPSS) is used to process the first decoded video frame to obtain the first display video frame. Before outputting the first decoded video frame, the VDEC shall determine whether the video code stream and the first decoded video frame correspond to the same media path. In this way, the problem of accessing data of other media paths when the VDEC outputs the first decoded video frame can be avoided, thereby improving the security of video frame transmission. Similarly, before outputting the first display video frame, the VPSS shall determine whether the first decoded video frame and the first display video frame correspond to the same media path. In this way, the problem of accessing data of other media paths when the VPSS outputs the first display video frame can be avoided, thereby improving the security of video frame transmission.

[0018] In a possible implementation, the determination of whether the video code stream and the first decoded video frame correspond to the same media channel includes: obtaining the first channel identifier corresponding to the video code stream and the second channel identifier in the frame-level protection information of the first decoded video frame; when the first channel identifier and the second channel identifier are the same, determining that the video code stream and the frame-level protection information of the first decoded video frame correspond to the same media channel; the determination of whether the first decoded video frame and the first displayed video frame correspond to the same media channel includes: obtaining the third channel identifier in the frame-level protection information of the first decoded video frame, and the fourth channel identifier in the updated frame-level protection information; when the third channel identifier and the fourth channel identifier are the same, determining that the first decoded video frame and the first displayed video frame correspond to the same media channel. Wherein, when the electronic device creates a media channel, a channel identifier is set for the corresponding media channel, and the channel identifier is used to identify the media channel. Afterwards, the electronic device can maintain the corresponding relationship between the channel identifier and each data memory on the media channel, and write the channel identifier as initial information into the frame-level protection information. Furthermore, in the present application, the electronic device can determine whether two video frames correspond to the same media path by comparing whether the path identifiers are the same, thereby avoiding the problem of the hardware module accessing data of other media paths, thereby improving the security of video frame transmission.

[0019] In a possible implementation, before determining whether to output the first display video frame according to the frame-level protection information, and before updating the frame-level protection information according to the resolution of the first display video frame, it also includes: detecting whether the frame-level protection information is valid. Among them, any hardware module on the media path needs to detect the validity of the corresponding frame-level protection information before reading the frame-level protection information and before writing the relevant information of the frame-level protection information. For example, before writing the output control strategy and the resolution of the first decoded video frame, the VDEC should detect whether the corresponding initial information of the frame-level protection information is valid. Before the VPSS reads the frame-level protection information of the first decoded video frame, it needs to detect whether the frame-level protection information of the first decoded video frame is valid. Before the VPSS updates the minimum resolution in the frame-level protection information, it needs to detect whether the frame-level protection information to be updated is valid. Before the VDP reads the frame-level protection information of the first display video frame, it needs to detect whether the frame-level protection information of the first display video frame is valid. In this way, it can be ensured that the corresponding video frame is output protected according to the correct protection information.

[0020] In a possible implementation, the detection of whether the frame-level protection information is valid includes: detecting at least one of the following: detecting whether the type of the frame-level protection information matches the preconfigured type, the type being used to indicate whether the frame-level protection information is bound to a video frame; determining whether the check value of the frame-level protection information is the same as the initial check value; or detecting whether the corresponding relationship between the storage address of the first display video frame and the length of the storage address in the frame-level protection information is correct. Among them, detecting whether the frame-level protection information is valid is essentially detecting at least one initial protection information in the frame-level protection information, and determining that the corresponding frame-level protection information is valid under the condition that at least one initial protection information detected is valid.

[0021] Taking VDP as an example, VDP can detect whether the type (tag) of the frame-level protection information of the first display video frame matches the preconfigured tag. If the tag of the frame-level protection information of the first display video frame matches the preconfigured tag, it means that the tag information is valid. Among them, the tag is used to indicate whether the frame-level protection information is bound to the video frame. If the frame-level protection information is bound to the video frame, then VDP shall perform operations on the first display video frame according to the resolution of the video frame in the frame-level protection information. If the frame-level protection information is not bound to the video frame, then the frame-level protection information is global protection information, and VDP shall perform operations on the first display video frame according to the resolution in the extended output policy. Using this implementation method, VDP can process the first display video frame according to the matching processing logic based on the frame-level protection information after determining the correctness of the frame-level protection information type.

[0022] VDP can also detect whether the correspondence between the storage address of the first display video frame and the length of the storage address in the frame-level protection information is correct. If the correspondence between the storage address of the first display video frame and the length of the storage address is correct, it means that the frame-level protection information is the frame-level protection information corresponding to the first display video frame. With this implementation, VDP can ensure whether the frame-level protection information is associated with the video frame, thereby ensuring that the protection information of the corresponding video frame is read from the frame-level protection information, and then, accurate output protection is achieved for the video frame.

[0023] VDP can also determine whether the check value of the frame-level protection information is the same as the initial check value. If the check value of the frame-level protection information is the same as the initial check value, it means that the frame-level protection information has not been tampered with. If the check value of the frame-level protection information is different from the initial check value, it means that the frame-level protection information has been tampered with. Using this implementation method, VDP can perform output protection on the first display video frame according to the frame-level protection information while ensuring that the frame-level protection information has not been destroyed, forged or tampered with, thereby achieving correct output protection for the first display video frame and optimizing the performance of output protection.

[0024] It should be understood that the detection order of the above three information items can be arbitrary, and this application does not limit this. In addition, the detection of the validity of the frame-level protection information by VDEC and the detection of the validity of the frame-level protection information by VPSS are similar to the detection process of the above VDP, and will not be described in detail here.

[0025] In one possible implementation, the determination of whether the check value of the frame-level protection information is the same as the initial check value includes: performing an XOR operation on all information in the frame-level protection information to obtain a first result; performing an XOR operation on the first result and a secure random number to obtain the check value of the frame-level protection information; and comparing whether the check value of the frame-level protection information is the same as the initial check value.

[0026] The electronic device is provided with a mask register, which is, for example, a 32-bit register. When the electronic device is powered on, the TEE side can read a 32-bit random number from the hardware random number module, write it into the mask register and latch it. The REE side has no access to the mask register. Based on this, the 32-bit random number in the mask register is relatively safe and can be called a "secure random number". Further, the electronic device can, for example, determine the initial checksum in the frame-level protection information according to the algorithm checksum = (Word0^Word1^...Wordn)^Mask, where n is an integer greater than or equal to 1, Word0 to Wordn refers to each information in the frame-level protection information, Mask refers to a secure random number, and ^ refers to an exclusive OR operation. When detecting the checksum, the corresponding hardware module can calculate the check value of the current frame-level protection information according to the same algorithm. Since the secure random number will not be tampered with, if the calculated check value is the same as the checksum, it means that the frame-level protection information has not been tampered with. If the calculated check value is different from the checksum, it means that the frame-level protection information has been tampered with by the REE side.

[0027] In a possible implementation manner, the method further includes: adding a watermark to the first display video frame according to the frame-level protection information and the position and size of the first display video frame.

[0028] Among them, the frame-level protection information includes the storage address of the watermark information. The watermark information may include the content of the watermark information, the position added to the video frame, the relative size with the video frame, etc. The display position and size of the first display video frame are controlled by the user through the REE side input control instructions. Therefore, if the first display video frame requires video watermark protection, the electronic device can also obtain the position and size of the first display video frame from the REE driver. Then, the watermark information is obtained from the storage address of the watermark information, and then the electronic device can determine the position and size of the watermark information to be added according to the position and size of the first display video frame, the relative position of the watermark information and the first display video frame, and the relative size of the watermark information and the first display video frame. With this implementation, the electronic device can determine the position and size of the watermark to be added according to the position and size of the video frame when it is displayed, so as to ensure that the watermark is accurately added and displayed as the video frame changes. In addition, the electronic device directly corresponds the video frame to the watermark information of the video frame through the frame-level protection information, which can avoid the problem of failure to add watermarks due to the display level problem of the video frame in the scene where multiple videos are displayed simultaneously.

[0029] A possible implementation method also includes: if the frame-level protection information of the first display video frame is lost, determining whether to output the first display video frame according to the resolution in the global protection information, and the resolution in the global protection information is the minimum resolution in the output strategy corresponding to at least one code stream.

[0030] In a possible implementation, before decoding the video code stream to obtain the first decoded video frame, it also includes: obtaining a request to create a media channel, the media channel is used to transmit the video code stream; generating a channel identifier of the media channel; configuring the type of protection information in the frame-level protection information, the corresponding relationship between the storage address of the video frame corresponding to the frame-level protection information and the length of the storage address, and the initial check value. With this implementation, the electronic device can pre-configure and maintain the software, hardware, related information, and the corresponding relationship between the above factors involved in the video transmission process to achieve the function of safely outputting the video.

[0031] In a second aspect, the present application provides a safe output device, comprising: a decoding module, used to generate frame-level protection information of a first decoded video frame according to an output control strategy and a resolution of the first decoded video frame, wherein the first decoded video frame is obtained by the decoding module decoding a video code stream; a display control module, used to determine whether to output a first display video frame according to the frame-level protection information, wherein the first display video frame is obtained based on the first decoded video frame.

[0032] In a possible implementation, the device further includes: a processing module, configured to process the first decoded video frame to obtain the first display video frame; and an updating module, configured to update the frame-level protection information according to a resolution of the first display video frame.

[0033] In a possible implementation, the processing module is specifically used to enlarge the first decoded video frame to obtain the first display video frame; or, the processing module is specifically used to reduce the first decoded video frame to obtain the first display video frame.

[0034] In one possible implementation, the frame-level protection information includes the resolution and minimum resolution of the first decoded video frame, the minimum resolution being the minimum resolution corresponding to the video code stream in the entire media channel, and the update module is specifically used to modify the minimum resolution in the frame-level protection information to the resolution of the first display video frame when the resolution of the first display video frame is less than the minimum resolution.

[0035] In one possible implementation, the display control module is specifically used to: obtain the output protection status allowed by the output port, and determine the target output control strategy that matches the output protection status; determine whether the minimum resolution in the frame-level protection information is less than the resolution allowed by the target output control strategy; when the minimum resolution in the frame-level protection information is less than the resolution allowed by the target output control strategy, output the first display video frame.

[0036] In a possible implementation, the device further includes a trusted application TA, which is used to configure the output control policy for the video code stream.

[0037] In a possible implementation, the decoding module is further used to determine whether the video code stream and the first decoded video frame correspond to the same media path; and the processing module is further used to determine whether the first decoded video frame and the first displayed video frame correspond to the same media path.

[0038] In a possible implementation, the decoding module is specifically used to: obtain a first path identifier corresponding to the video code stream and a second path identifier in the frame-level protection information of the first decoded video frame; when the first path identifier and the second path identifier are the same, determine that the video code stream and the frame-level protection information of the first decoded video frame correspond to the same media path; and the processing module is specifically used to: obtain a third path identifier in the frame-level protection information of the first decoded video frame, and a fourth path identifier in the updated frame-level protection information; when the third path identifier and the fourth path identifier are the same, determine that the first decoded video frame and the first displayed video frame correspond to the same media path.

[0039] In a possible implementation, the device further includes a detection module, and the detection module is used to detect whether the frame-level protection information is valid.

[0040] In one possible implementation, the detection module is specifically used to perform at least one of the following detections: detecting whether the type of the frame-level protection information matches a preconfigured type, which type is used to indicate whether the frame-level protection information is bound to a video frame; determining whether the check value of the frame-level protection information is the same as the initial check value; or detecting whether the correspondence between the storage address of the first display video frame in the frame-level protection information and the length of the storage address is correct.

[0041] In one possible implementation, the detection module is specifically used to: perform an XOR operation on all information in the frame-level protection information to obtain a first result; perform an XOR operation on the first result and a secure random number to obtain a verification value of the frame-level protection information; and compare whether the verification value of the frame-level protection information is the same as the initial verification value.

[0042] In a possible implementation manner, the device further includes a watermark adding module, which is used to add a watermark to the first display video frame according to the frame-level protection information and the position and size of the first display video frame.

[0043] In one possible implementation, the display control module is also used to determine whether to output the first display video frame based on the resolution in the global protection information when the frame-level protection information of the first display video frame is lost, and the resolution in the global protection information is the minimum resolution in the output strategy corresponding to at least one code stream.

[0044] In a possible implementation, the device also includes an acquisition module, which is used to obtain a request for creating a media channel, and the media channel is used to transmit the video code stream; a session management module, which is used to generate a channel identifier of the media channel; and configure the type of protection information in the frame-level protection information, the corresponding relationship between the storage address of the video frame corresponding to the frame-level protection information and the length of the storage address, and the initial check value.

[0045] Among them, the technical effects produced by the second aspect and each implementation method of the second aspect are the same as the technical effects produced by the first aspect and each implementation method of the first aspect, and will not be repeated here.

[0046] In a third aspect, the present application provides an electronic device, comprising a processor and a transmission interface, wherein the processor is configured to call software instructions stored in a memory to implement: decoding a video stream; in the stage of decoding the video stream to obtain a first decoded video frame, generating frame-level protection information of the first decoded video frame according to an output control strategy and a resolution of the first decoded video frame; and determining whether to output a first display video frame according to the frame-level protection information, the first display video frame being obtained based on the first decoded video frame.

[0047] In a possible implementation, the processor is further configured to process the first decoded video frame to obtain the first display video frame; and update the frame-level protection information according to a resolution of the first display video frame.

[0048] In a possible implementation, the processor is specifically configured to enlarge the first decoded video frame to obtain the first display video frame; or to reduce the first decoded video frame to obtain the first display video frame.

[0049] In a possible implementation, the frame-level protection information includes the resolution and minimum resolution of the first decoded video frame, the minimum resolution being the minimum resolution corresponding to the video code stream in the entire media path, and the processor is further configured to modify the minimum resolution in the frame-level protection information to the resolution of the first display video frame when the resolution of the first display video frame is less than the minimum resolution. The frame-level protection information includes the minimum resolution, and the minimum resolution is the minimum resolution corresponding to the video code stream in the entire media path.

[0050] In one possible implementation, the processor specifically obtains the output protection status allowed by the output port, determines the target output control strategy that matches the output protection status; determines whether the minimum resolution in the frame-level protection information is less than the resolution allowed by the target output control strategy; when the minimum resolution in the frame-level protection information is less than the resolution allowed by the target output control strategy, outputs the first display video frame.

[0051] In a possible implementation, the processor is further used to configure the output control strategy for the video code stream.

[0052] In a possible implementation, the processor is further used to determine whether the video code stream and the first decoded video frame correspond to the same media path; the processor is further used to determine whether the first decoded video frame and the first displayed video frame correspond to the same media path.

[0053] In a possible implementation, the processor is specifically used to obtain a first path identifier corresponding to the video code stream and a second path identifier in the frame-level protection information of the first decoded video frame; when the first path identifier and the second path identifier are the same, it is determined that the video code stream and the frame-level protection information of the first decoded video frame correspond to the same media path; the processor is specifically used to obtain a third path identifier in the frame-level protection information of the first decoded video frame, and a fourth path identifier in the updated frame-level protection information; when the third path identifier and the fourth path identifier are the same, it is determined that the first decoded video frame and the first displayed video frame correspond to the same media path.

[0054] In a possible implementation, the processor is further configured to detect whether the frame-level protection information is valid.

[0055] In one possible implementation, the processor is specifically used to perform at least one of the following detections: detecting whether the type of the frame-level protection information matches a preconfigured type, which type is used to indicate whether the frame-level protection information is bound to a video frame; determining whether a check value of the frame-level protection information is the same as an initial check value; or detecting whether the correspondence between the storage address of the first display video frame in the frame-level protection information and the length of the storage address is correct.

[0056] In one possible implementation, the processor is specifically used to perform an XOR operation on all information in the frame-level protection information to obtain a first result; perform an XOR operation on the first result and a secure random number to obtain a check value of the frame-level protection information; and compare whether the check value of the frame-level protection information is the same as the initial check value.

[0057] In a possible implementation, the processor is further configured to add a watermark to the first display video frame according to the frame-level protection information and the position and size of the first display video frame.

[0058] In one possible implementation, the processor is also used to determine whether to output the first display video frame based on the resolution in the global protection information when the frame-level protection information of the first display video frame is lost, and the resolution in the global protection information is the minimum resolution in the output strategy corresponding to at least one code stream.

[0059] In one possible implementation, the processor is further used to obtain a request to create a media path, where the media path is used to transmit the video code stream; generate a path identifier for the media path; configure the type of protection information in the frame-level protection information, the corresponding relationship between the storage address of the video frame corresponding to the frame-level protection information and the length of the storage address, and the initial check value.

[0060] Among them, the technical effects produced by the third aspect and each implementation method of the third aspect are the same as the technical effects produced by the first aspect and each implementation method of the first aspect, and will not be repeated here.

[0061] In a fourth aspect, the present application provides an electronic device, the electronic device comprising a processor and a memory, wherein the memory is used to store programs, instructions or codes, and the processor is used to execute the programs, instructions or codes in the memory to complete the first aspect, or the method in any possible design of the first aspect. It should be understood that the electronic device can be a processor chip, in which case the processor in the electronic device is a processor core or a central processing unit in the processor chip.

[0062] In a fifth aspect, the present application provides a computer-readable storage medium, which stores instructions, and when the computer-readable storage medium is run on a computer or a processor, it enables the computer or the processor to execute the method in the first aspect or any possible design of the first aspect.

[0063] In a sixth aspect, the present application provides a computer program product comprising instructions, which, when executed on a computer or a processor, enables the computer or the processor to execute the method in the first aspect or any possible design of the first aspect.

[0064] By adopting the technical solution of the present application, after the electronic device decodes the video code stream to obtain the video frame, the frame-level protection information is generated for the corresponding video frame. Afterwards, the electronic device updates the frame-level protection information of the video frame according to the change in the resolution of the video frame. Therefore, before outputting the video frame to be displayed, the electronic device can perform output protection on the video frame to be displayed according to the frame-level protection information corresponding to the video frame to be displayed. In this way, protection is performed at the level of the video frame. When the video frame changes, the electronic device can timely update the output protection information corresponding to the video frame, so that the electronic device can provide more accurate output protection, avoid protection failure problems caused by the asynchrony between the video frame and the output protection strategy, and improve the protection performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0065] In order to more clearly illustrate the technical solution of the present application, the drawings required for use in the embodiments are briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0066] Figure 1A It is a system architecture diagram of a typical electronic device provided by this application;

[0067] Figure 1B This is a schematic diagram of a first exemplary application scenario of output protection provided by the present application;

[0068] Figure 1C is a schematic diagram of a second exemplary application scenario of output protection provided by the present application;

[0069] Figure 2 is a schematic diagram of the system architecture of the electronic device 10 provided in this application;

[0070] Figure 3 is a schematic diagram of an exemplary structure of frame-level protection information provided by this application;

[0071] Figure 4 is an exemplary method flow chart of the safety output method 100 provided in the present application;

[0072] Figure 5 is a schematic diagram of an exemplary application scenario of the safety output method 100 provided in the present application;

[0073] Fig. 6A is a schematic diagram of an exemplary system architecture of an electronic device 20 provided in the present application;

[0074] Figure 6B It is a schematic diagram of the transmission process of frame-level protection information provided by this application;

[0075] Fig. 7A is an exemplary structural diagram of a safety output device 70 provided in the present application;

[0076] Figure 7B It is an exemplary structural diagram of the safety output device 71 provided in the present application. DETAILED DESCRIPTION

[0077] The technical solution in this application will be clearly described below in conjunction with the drawings in this application.

[0078] The terms used in the following embodiments of the present application are only for the purpose of describing specific embodiments, and are not intended to be used as limitations on the present application. As used in the specification and the appended claims of the present application, the singular expressions "one", "a kind", "said", "above", "the" and "this" are intended to also include plural expressions, unless there is a clear contrary indication in the context. It should also be understood that although the terms first, second, etc. may be used to describe a certain class of objects in the following embodiments, the objects should not be limited to these terms. These terms are only used to distinguish specific objects of this class of objects. For example, the terms first, second, etc. may be used in the following embodiments to describe channel identifiers, but channel identifiers should not be limited to these terms. These terms are only used to distinguish the identifiers of different media channels. The same is true for other classes of objects that may be described by the terms first, second, etc. in the following embodiments, which will not be repeated here.

[0079] The architecture and business scenarios described in this application are intended to more clearly illustrate the technical solutions of the embodiments of this application, and do not constitute a limitation on the technical solutions provided by this application. A person of ordinary skill in the art will appreciate that with the evolution of the architecture and the emergence of new business scenarios, the technical solutions provided by the embodiments of this application are also applicable to similar technical problems.

[0080] The following is an introduction to the implementation scenario of this application.

[0081] The present application can be applied to electronic devices that support audio and video input / output, such as smart phones, smart set-top boxes, smart TVs, surveillance, computers, tablet computers, etc.

[0082] Figure 1A A typical system architecture of an electronic device is shown. The system architecture includes: an application layer, a driver layer, an operating system layer, a hardware module and a storage module. The application layer is used to run application software, such as Tencent, Youku, etc. The driver layer includes a driver program written for each hardware module, which is used to drive the corresponding hardware module to access the memory block. The operating system layer is responsible for memory management, stack management, task scheduling management, etc. When the application software of the application layer is running, it can apply to the operating system layer for allocation of memory blocks and obtain the virtual address of the allocated memory blocks. The allocated memory blocks are used to store data during the operation of the application software, such as the video frames and other data described below. The driver layer can drive the hardware module to access the corresponding memory block according to the virtual address of the memory block. The "access" involved in this application includes read operations and write operations. For example, the hardware module can respond to the instructions of the driver program to process the data read from the input memory block, and then write the processed data to the output memory block.

[0083] Before transmitting media content, the application software can create a media path according to the intention of the application software and the processing process of the media content. The media path is the path for the application software to process the media content on the electronic device. The processing process includes but is not limited to playing, recording, transcoding and forwarding, etc. Accordingly, the media path may include, for example, a recording path, a playing path and a transcoding path. The media path includes resources used for audio and video processing, such as a series of hardware modules and memory.

[0084] Combination Figure 1A, exemplarily, the application software may apply to the operating system layer to occupy hardware module 1, hardware module 2 and hardware module 3, and to the operating system layer to allocate memory block 1 and memory block 2. Hardware module 1, hardware module 2 and hardware module 3, as well as memory block 1 and memory block 2, may form a media path, and the transmission process of the media stream on the media path may be, for example: hardware module 1 to memory block 1, memory block 1 to hardware module 2, hardware module 2 to memory block 2, memory block 2 to hardware module 3. In actual operation, the driver layer receives the instruction of the application software, and then drives the corresponding hardware module in the media path to perform access operations on the corresponding memory block, and process the corresponding media data.

[0085] Figure 1A The electronic device shown can, for example, support a trusted execution environment (TEE) and a rich execution environment (REE). TEE corresponds to REE, TEE is used to provide a protected execution environment for protected application software, and REE is used to provide an execution environment for unprotected application software. The media content transmitted under the TEE can, for example, be processed according to the content usage rules (CUR) to achieve the protection of the media content. Accordingly, the hardware modules and memory blocks of the media path under the TEE should process and transmit relevant data according to the CUR. Exemplarily, CUR may include protection policies such as HDCP policy, video watermark protection policy, prohibition of transcoding and prohibition of recording.

[0086] The following is an explanation using the HDCP policy and video watermark protection policy as an example. Among them, the HDCP policy and video watermark protection policy act on the video output stage. Therefore, this application collectively refers to the HDCP policy and video watermark protection policy as "output protection policy" or "output control policy." For example, taking video output as an example, the electronic device first decrypts the video source, that is, the encrypted video stream, to obtain a video stream to be decoded. Then, the electronic device decodes the video stream to be decoded to obtain a video frame. Afterwards, the electronic device can output the video frame through the hardware interface. Among them, the electronic device can use the HDCP policy to provide protection for the decoded video frame, and the electronic device can add a watermark to the video frame to achieve watermark protection for the video frame.

[0087] In actual implementation, the software system that performs output protection on video content includes a conditional access system (CAS) or a digital rights management (DRM) system. In this application, CAS or DRM is expressed as "CAS / DRM", where " / " means "or" in this application. "CAS / DRM" mentioned in this specification means CAS or DRM.

[0088] Among them, the HDCP policy is a technology that prevents high-definition digital media content from being illegally recorded. The HDCP policy may include the HDCP1.4 protection level and the HDCP2.2 protection level, wherein the resolutions allowed by the HDCP1.4 protection level and the HDCP2.2 protection level are different. For example, the resolution supported by the HDCP1.4 protection level is usually 1080 progressive scan (P), that is, 1080P, for example, indicates that the number of pixels per frame is 1920*1080, and the resolution supported by the HDCP2.2 protection level is usually 4K, 4K, for example, indicates that the number of pixels per frame is 4096*2160, and "K" is used to represent 2 to the power of 10*4. When the HDCP policy is actually used for protection, the software and hardware for processing video frames, displaying video frames, etc. are required to support the HDCP policy, otherwise, the electronic device is prohibited from outputting the corresponding video frames, or outputs the corresponding video frames at a reduced resolution.

[0089] High Definition Multimedia Interface (HDMI) is a commonly used hardware interface that supports HDCP policy. At present, commonly used HDMIs include HDMI 1.0, HDMI 1.3, HDMI 1.4 and HDMI 2.0, etc. Different versions of HDMI support different HDCP levels. For example, HDMI 1.3 supports HDCP 1.4 protection level, and HDMI 2.0 supports HDCP 2.2 protection level. Before outputting a video frame, the HDMI of an electronic device can determine the HDCP protection status of the output video frame based on the support of the connected interface for HDCP. For example, the HDMI of an electronic device supports HDCP 2.2 and HDCP 1.4, and the interface to which the HDMI is connected only supports HDCP 1.4. Then, the electronic device can output the video frame at the resolution corresponding to the HDCP 1.4 protection level. Accordingly, the HDCP protection status of the output video frame determined by the HDMI of the electronic device is HDCP 1.4.

[0090] The video watermark protection strategy refers to adding the required watermark information to the output video frame to identify the source of the video frame, etc. The watermark information may include the address of the video source to which the video frame belongs, the content identifier, the timestamp, etc. Among them, the electronic device usually adds watermark information to the video frame according to the display position and size of the video frame, etc.

[0091] like Figure 1B As shown, the TEE application layer decrypts the encrypted video stream, and then writes the decrypted video stream into the memory block accessed by the decoding module in the corresponding media path. Based on this, in an existing implementation method, before the TEE application layer decrypts, the HDMI slave determines the supported HDCP protection state according to the connected device and sends it to the TEE application layer, and then the TEE application layer determines whether the HDCP protection state matches the HDCP protection level corresponding to the video stream. If the HDCP protection state does not match the HDCP protection level of the video stream, for example, the HDCP protection level of the video stream is HDCP2.2, and the HDCP protection state supports HDCP1.4 level protection. Then, the TEE application layer will not decrypt the encrypted video stream, and then the electronic device outputs a still frame signal (mute). Among them, outputting mute refers to outputting pure color video frames such as all-black video frames or all-blue video frames. If the HDCP protection state matches the HDCP protection level of the stream, the TEE application layer decrypts the encrypted stream. Then, the electronic device outputs the video frame corresponding to the video stream. It can be seen that in this solution, the electronic device has only two states for the encrypted video code stream: decryption and non-decryption. It does not adaptively adjust the output protection strategy of the video code stream, and has poor applicability.

[0092] like Figure 1CAs shown, before the electronic device outputs the video frame, the video display module in the corresponding media channel reads the video frame to be displayed from the memory block m, and then configures the video frame to be displayed according to the output protection strategy. The output protection strategy includes, for example, the output resolution corresponding to the video frame to be displayed, the location of adding the watermark, and the size of the watermark. In some embodiments, the output protection strategy in the video display module is configured by the TEE application layer, and after the output protection strategy is configured in the video display module, the corresponding output protection strategy is immediately configured for the read video frame to be displayed. Based on this, on the one hand, the memory block m can cache part of the video frame to be displayed. If the output protection strategy configured in the video display module changes, the video frame already cached in the memory block m corresponds to the output protection strategy before the change, but the video display module will configure the video frame to be displayed already cached in the memory block m according to the output protection strategy after the change, thereby causing the output protection to fail. On the other hand, the video frame to be displayed stored in the memory block m can be a processed video frame, and the processing mechanism of the video frame is usually controlled by the REE application layer. Therefore, if the position, size, etc. of the video frame changes, the TEE application layer cannot perceive it, and thus the TEE application layer cannot update the output protection policy in the video display module in time, which will also cause the output protection to fail. For example, the video frame is not output according to the resolution allowed by HDCP, the watermark is added incorrectly, etc.

[0093] Based on the above, the present application provides a safe output method and electronic device, which generates frame-level protection information corresponding to a video frame, so that before outputting the video frame to be displayed, the video frame to be displayed can be output protected according to the frame-level protection information corresponding to the video frame to be displayed. In this way, protection is performed at the level of the video frame, which can provide more accurate output protection, and when the video frame changes, the corresponding output protection information can be updated in time, thereby improving the protection performance.

[0094] The following is an introduction to the software and hardware structure of the electronic device involved in this application.

[0095] Figure 2 The system architecture diagram of an exemplary application environment of the electronic device 10 is shown. The electronic device 10 supports TEE. Figure 2 The REE system architecture and TEE system architecture are introduced respectively.

[0096] The REE system architecture includes a REE software application layer, a REE software interface adaptation layer, a REE software driver layer, a REE software operating system layer, a REE hardware module, and a REE storage module. The software application in the REE software application layer can control the REE hardware module through the REE software interface adaptation layer, the REE software driver layer, and the REE software operating system layer, and perform access operations on the memory blocks in the REE storage module. It should be understood that the REE software operating system layer, the REE software driver layer, the REE software interface adaptation layer, and the REE software application layer are implemented by software codes, and illustratively, these software codes can be stored in a memory and run on a processor.

[0097] The REE software application layer is used to provide an operating environment for the REE software application, and is also used to apply to the REE software operating system layer for allocation of REE hardware modules and REE memory blocks when the REE software application is running, and to store the virtual addresses of the REE memory blocks.

[0098] The REE software interface adaptation layer is used to match the driver in the REE software driver layer according to the virtual address accessed by the REE software application layer.

[0099] The REE software driver layer includes a driver program written for each hardware module, and the driver program drives the corresponding hardware module to access the corresponding REE memory block according to the virtual address.

[0100] The REE software operating system layer is used to perform REE hardware module resource management, and is also used for stack management and task scheduling. For example, the REE software operating system layer can respond to the REE software application layer instructions to configure the REE hardware module to create a media path, which is used to transmit media data without security requirements. In addition, the REE software operating system layer is also used to maintain one or more computer programs and data. When the one or more computer programs are running, the functions of each software layer on the REE side can be realized. The data is used to provide support for the operation of the one or more computer programs.

[0101] The REE storage module may include but is not limited to double data rate (DDR) memory, flash memory, static random access memory (SRAM), etc., which are not limited in this application. The REE storage module includes multiple memory blocks, each of which has a different physical address.

[0102] The TEE system architecture includes a TEE software application layer, a TEE software interface adaptation layer, a TEE software driver layer, a TEE software operating system layer, a TEE hardware module, and a TEE storage module. Among them, the TEE software operating system layer, the TEE software driver layer, the TEE software interface adaptation layer, and the TEE software application layer are implemented by software codes. For example, these software codes can be stored in a memory and run on a processor.

[0103] Among them, the software application in the TEE software application layer can control the TEE hardware module through the TEE software interface adaptation layer, the TEE software driver layer and the TEE software operating system layer, and access the memory blocks in the TEE storage module. It should be understood that the basic interaction process between the various software layers is similar to that on the REE side and will not be described in detail here.

[0104] In this application, the TEE software application layer may include a CAS / DRM trusted application (CAS / DRM trustedapplication, CAS / DRM TA), which may also be referred to as TA. TA may be used to configure the output protection strategy of the video stream.

[0105] In addition, in the present application, the TEE system architecture also includes a session management module, which is implemented by software code and runs on a processor. In some embodiments, the session management module runs, for example, in the TEE software driver layer. In other embodiments, the session management module runs, for example, in the TEE software operating system layer. This is not limited here.

[0106] Among them, the session management module can be used to configure a protection information memory block during the process of creating a media channel in the TEE system architecture. The protection information memory block is used to store frame-level protection information corresponding to the video frames transmitted in the media channel. Frame-level protection information includes output protection information corresponding to the video frames. The session management module can also be used to configure the initial information of the frame-level protection information in each protection information memory block. The initial information may include the type of frame-level protection information, the correspondence between the memory block virtual address of the video frame corresponding to the frame-level protection information and the length of the virtual address, and the initial check value of the frame-level protection information. The embodiments of the frame-level protection information are described in detail below and are not described in detail here.

[0107] In addition, the session management module can also be used to configure a channel identifier for the media channel created by the TEE system architecture, and store the correspondence between the virtual address of the memory block belonging to the media channel and the channel identifier.

[0108] The TEE storage module may include multiple secure memory blocks, each of which may be identified by a physical address. Exemplarily, in the process of creating a media path for video output, some of the secure memory blocks may be configured as memory blocks for storing video frames, and another portion of the secure memory blocks may be configured as memory blocks for storing frame-level protection information, and the initial information of the frame-level protection information may be written into the memory blocks. In the present application, the memory blocks used to store video frames may be referred to as data memory blocks, and the memory blocks used to store frame-level protection information may be referred to as protection information memory blocks.

[0109] In the present application, the hardware module can respond to the instruction of the TEE software driver layer or the TEE software operating system layer, and detect the validity of the frame-level protection information in the protection information memory block before accessing the protection information memory block. The hardware module can also detect whether the protection information memory block corresponds to the same media path as the memory block corresponding to the read operation before performing a write operation on the protection information memory block.

[0110] It is understandable that in some embodiments, the processor running each software layer of REE and the processor running each software layer of TEE can be physically the same processor. When the processor runs in REE mode, the processor implements the functions of each software layer of REE, and when the processor runs in TEE mode, the processor implements the functions of each software layer of TEE. The processor can be, for example, a system-level chip control logic unit, a microprocessor, a microcontroller (MCU), a central processing unit (CPU), a digital signal processor (DSP), a graphics processing unit (GPU), a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), etc., and this application does not limit it.

[0111] Figure 2The REE hardware module and TEE hardware module shown in the figure may include, for example, a multiplexing module (demux), a hardware decryption module, a hardware encryption module, a hardware decoding module (decoder), a video decoding module (video decoder, VDEC), a hardware communication module, a hardware video processing module (video processor, VPSS), a hardware display module (video display, VDP), an analog to digital converter (analog to digital converter, ADC), a digital to analog converter (digital to analog converter, DAC), a communication interface, a radio frequency unit, and a micro-electromechanical module, etc., which are not limited in this application. Among them, the REE hardware module and the TEE hardware module can be implemented by two processing channels in the same hardware module.

[0112] Both the REE storage module and the TEE storage module may include but are not limited to DDR memory, flash memory (Flash), SRAM, etc., which are not limited in this application. The REE storage module and the TEE storage module are isolated from each other, so that the ordinary memory block and the secure memory block are isolated from each other.

[0113] Figure 2 The physical address described in the illustrated embodiment can be: in the storage module, information is stored in bytes. In order to correctly store or obtain information, each byte unit has a unique memory block address. The physical address can also be called an actual address or an absolute address. The physical address can be addressed in the storage module through the address bus and is the address where the data is actually stored.

[0114] Figure 2 The virtual address described in the illustrated embodiment may be: a logical address used by application software to access a memory block. A virtual address does not actually store data, but needs to be mapped to an actual physical address to obtain data.

[0115] In some embodiments, Figure 2 The hardware and software of the illustrated REE system architecture and the hardware and software of the TEE system architecture may be located in the same system on chip (SOC). Figure 2 The illustrated REE software application layer, REE software interface adaptation layer, REE software driver layer, REE software operating system layer, REE hardware module, and TEE software application layer, TEE software interface adaptation layer, TEE software driver layer, TEE software operating system layer and TEE hardware module are located in the same SOC, and the REE storage module and TEE storage module can be independent of the SOC.

[0116] Understandably, Figure 2 This is only an illustrative description and does not constitute a specific limitation on the electronic device 10. In other embodiments of the present application, the electronic device 10 may include more or fewer components than shown in the figure, or combine some components, or separate some components, or arrange the components differently. The components shown in the figure may be implemented in hardware, software, or a combination of software and hardware.

[0117] The following describes the safety output method of the present application in conjunction with the electronic device 10 .

[0118] It is understandable that the secure output method described in the present application is implemented in the TEE. Accordingly, the secure output method described in the present application is executed by at least one of software, hardware, and a combination of software and hardware on the TEE side of the electronic device 10.

[0119] First, the frame-level protection information involved in the embodiments of the present application is introduced. Figure 3 An exemplary structure of frame-level protection information is shown. Figure 3 As shown, the frame-level protection information may include type (tag) information, memory block information (frame buffer information) of the video frame, resolution information of the video frame, HDCP protection policy (HDCP policy) information, watermark control information (watermark control), extended output policy (output control extend) information and initial checksum. Among them, the memory block information of the video frame may, for example, specifically include the storage address of the video frame and the length information and path identifier of the storage address. The resolution information of the video frame may, for example, specifically include the current resolution and the minimum resolution. The HDCP protection policy may, for example, specifically include the protection policy of no HDCP protection (NoHDCP), the protection policy of HDCP1.4 and the protection policy of HDCP2.2. The watermark control information may, for example, specifically include watermark parameter information and memory block information of the watermark parameter.

[0120] Figure 3 Table 1 shows the various types of information included in the frame-level protection information, the specific information included in each type of information, and the information description of each specific information.

[0121] Table 1

[0122]

[0123] Among them, the type of frame-level protection information is used to indicate whether the frame-level protection information is bound to the video frame. Exemplarily, if the type of frame-level protection information in Table 1 is frame, it means that the frame-level protection information is bound to the video frame, and the frame-level protection information contains the memory block information and resolution information of the bound video frame. If the type of frame-level protection information in Table 1 is global, it means that the frame-level protection information is not bound to the video frame. Accordingly, the memory block information and resolution information of the video frame in the frame-level protection information can be empty. In addition, when the type of frame-level protection information is global, the frame-level protection information can be applied to all video frames that have lost frame-level protection information. For ease of distinction, this application refers to this type of protection information as "global protection information." The output resolution in the global protection information can be included in the extended output policy information.

[0124] According to the above description of the secure memory block, the TEE system architecture is configured with a data memory block and a protection information memory block when creating a media channel. Accordingly, the memory block information of the video frame in Table 1 is used to establish the corresponding relationship between the frame-level protection information and the video frame bound to the frame-level protection information.

[0125] The resolution information of the video frame in Table 1 can be updated as the video frame is updated. See the description of the following embodiment for details, which will not be described in detail here.

[0126] The watermark control information in Table 1 is used to maintain information such as the watermark type and the storage address of the watermark when the bound video frame needs to be watermarked. The storage address of the watermark is used to indicate the memory block storing the watermark information. The watermark information includes, for example, the content of the watermark information, the position added to the video frame, the relative size to the video frame, etc. For example, the content of the watermark information is, for example, the address of the video source, the position added to the video frame is, for example, from the lower left direction to the upper right direction of the video frame, and the relative size to the video frame includes, for example, that the watermark length is 80% of the vertical length of the video frame, and the width is 10% of the vertical length of the video frame.

[0127] The checksum in Table 1 is used to check whether the frame-level protection information has been tampered with. See the description of the following embodiment for details, which will not be described in detail here.

[0128] According to the description of the above embodiment, the tag, the storage address of the video frame, the corresponding relationship of the length of the storage address, the path identifier and the checksum in Table 1 may be added to the protection information memory block by the session management module when configuring the protection information memory block. The tag, the storage address of the video frame, the corresponding relationship of the length of the storage address and the checksum are the initial information of the frame-level protection information.

[0129] It is understandable that Table 1 is only a schematic description and does not constitute a limitation on the frame-level protection information involved in this application. In some other embodiments, the frame-level protection information may also include more or less information.

[0130] See also Figure 4 , Figure 4 A safety output method 100 (hereinafter referred to as method 100) is shown. Method 100 includes the following steps:

[0131] Step S101 , in the stage of decoding a video code stream to obtain a first decoded video frame, frame-level protection information of the first decoded video frame is generated according to an output control strategy and a resolution of the first decoded video frame.

[0132] The output control strategy is a protection strategy that matches the protection requirements of the video code stream. Before decoding the video code stream, the electronic device can obtain the resolution of the first decoded video frame. For example, the electronic device can obtain the resolution of the first decoded video frame based on the video code stream.

[0133] Exemplarily, after the TEE software application layer of the electronic device receives the encrypted video stream, it decrypts the encrypted video stream to obtain the video stream and the protection requirements corresponding to the video stream. The protection requirements are, for example, 1080 progressive scan (1080p) to output the video corresponding to the video stream. Afterwards, the TEE software application layer can configure the output control policy according to the protection requirements. The output control policy includes the video width and height allowed at the NoHDCP protection level, the video width and height allowed at the HDCP1.4 protection level, and the video width and height allowed at the HDCP2.2 protection level. Afterwards, the electronic device decodes the video stream to obtain a first decoded video frame, and then the electronic device writes the resolution of the first decoded video frame into the protection information memory block corresponding to the first decoded video frame to obtain the frame-level protection information of the first decoded video frame.

[0134] The frame-level protection information of the first decoded video frame includes the resolution and minimum resolution of the first decoded video frame, and the minimum resolution is the minimum resolution corresponding to the video code stream in the entire media channel. It should be understood that the minimum resolution is usually set in the initial information of the frame-level protection information, and the resolution of the first decoded video frame is the resolution of the current decoded video frame. If the resolution of the current decoded video frame is less than the minimum resolution initially set, the minimum resolution in the frame-level protection information is modified to the resolution of the current decoded video frame (that is, the resolution of the first decoded video frame).

[0135] Step S102: determining whether to output a first display video frame according to the frame-level protection information.

[0136] The first display video frame is obtained according to the first decoded video frame. In some embodiments, the first decoded video frame obtained by decoding can be directly used as output, and the first display video frame is the first decoded video frame. In other embodiments, the electronic device can process the first decoded video frame and then obtain the output first display video frame.

[0137] For example, in some embodiments, the electronic device may enlarge the first decoded video frame to obtain the first display video frame. In other embodiments, the electronic device may reduce the first decoded video frame to obtain the first display video frame.

[0138] It should be noted that when the electronic device zooms in or out on the first decoded video frame, the resolution of the first decoded video frame will also change accordingly. Based on this, the resolution of the first display video frame is different from the resolution of the first decoded video frame. Furthermore, after obtaining the first display video frame, the electronic device can update the frame-level protection information according to the resolution of the first display video frame. Exemplarily, when the resolution of the first display video frame is less than the minimum resolution in the frame-level protection information, the minimum resolution in the frame-level protection information is modified to the resolution of the first display video frame, and the frame-level protection information of the first display video frame is obtained.

[0139] Furthermore, the electronic device can obtain the output protection state allowed by the output port, and then determine the target output control policy that matches the output protection state from the frame-level protection information of the first display video frame. Afterwards, the electronic device can determine whether the minimum resolution in the frame-level protection information is less than the resolution allowed by the target output control policy. When the minimum resolution in the frame-level protection information is less than the resolution allowed by the target output control policy, the electronic device outputs the first display video frame. When the minimum resolution in the frame-level protection information is greater than the resolution allowed by the target output control policy, the electronic device outputs mute. Among them, the output protection state of the output port is, for example, an HDCP protection state. For detailed embodiments of the output protection state of the output port, as well as the embodiments of mute, please refer to the description of the above embodiments, which will not be described in detail here.

[0140] With this implementation, when the minimum resolution in the frame-level protection information is less than the resolution allowed by the target output control policy, the electronic device can output the first display video frame according to the resolution allowed by the target output control policy. It can be seen that each time the electronic device outputs a video frame, it can output the video frame according to the frame-level protection information corresponding to the video frame, so that the security is higher. When the minimum resolution in the frame-level protection information is greater than the resolution allowed by the target output control policy, it means that the resolution of the first display video frame does not comply with the output control policy, and the electronic device outputs mute. That is, when the resolution of the video frame exceeds the resolution allowed by the output control policy, the output of the video frame will be prohibited. It can be seen that this implementation can effectively protect the output of high-definition video frames. The protection information in the embodiment of the present application is frame-level, and each video frame has its own corresponding frame-level protection information. Each video frame output can be output protected according to the frame-level protection information corresponding to the video frame, which is more secure and more flexible.

[0141] The above describes an embodiment of a video frame with frame-level protection information. In some other embodiments, if the frame-level protection information of the first display video frame is lost, the electronic device may determine whether to output the first display video frame based on the resolution in the global protection information. The resolution in the global protection information is the minimum resolution in the output strategy corresponding to at least one bitstream. The global protection information may be generated by the electronic device based on at least one bitstream while generating the frame-level protection information, and the at least one bitstream may include the video bitstream. The global protection information may be generated based on at least one bitstream, and the at least one bitstream may include the video bitstream. Figure 3 The exemplary embodiments are described in detail and will not be described in detail here.

[0142] In addition, the display position and size of the first display video frame are controlled by the user through the REE side input control instructions. Based on this, in some other embodiments, if the first display video frame requires video watermark protection, before outputting the first display video frame, the electronic device can also obtain the position and size of the first display video frame from the REE driver. Then, the electronic device can determine the memory block storing the watermark information based on the storage address information of the watermark in the frame-level protection information, and then read the watermark information from the corresponding memory block. Afterwards, the electronic device can determine the position and size of the watermark information to be added based on the position and size of the first display video frame, the relative position of the watermark information and the first display video frame, and the relative size of the watermark information and the first display video frame.

[0143] It can be seen that by adopting this implementation, the electronic device can determine the position and size of the watermark to be added according to the position and size of the video frame when it is displayed, so as to ensure that the watermark is accurately added and displayed as the video frame changes. In addition, the electronic device directly corresponds the video frame and the watermark information of the video frame through the frame-level protection information, which can avoid the problem of failure to add watermarks due to the display level problem of the video frame in the scene where multiple videos are displayed simultaneously.

[0144] In summary, the secure output method provided by the present application is that after the electronic device decodes the video code stream to obtain the video frame, it generates frame-level protection information corresponding to the video frame. Afterwards, the electronic device updates the frame-level protection information of the video frame according to the change in the resolution of the video frame. Therefore, before outputting the video frame to be displayed, the electronic device can perform output protection on the video frame to be displayed according to the frame-level protection information corresponding to the video frame to be displayed. In this way, protection is performed at the level of the video frame. When the video frame changes, the electronic device can update the output protection information corresponding to the video frame in time, so that the electronic device can provide more accurate output protection, avoid protection failure problems caused by the asynchrony between the video frame and the output protection strategy, and improve the protection performance.

[0145] The method 100 is exemplarily introduced below.

[0146] Before executing the above step S101, Figure 2 The TEE software application layer in the can request the TEE software operating system layer and the session management module to create a media path through the TEE software interface adaptation layer. The created media path can be used to output protected video content. Accordingly, the media path includes, for example, VDEC, VDP, and output ports, etc., and also includes at least one data memory block and at least one protection information memory block. The at least one data memory block is used to store video-related data transmitted by the media path, and the at least one protection information memory block is used to store frame-level protection information corresponding to the video frame transmitted by the media path.

[0147] In this embodiment, the session management module sets a channel identifier A for the media channel, and the channel identifier A is used to indicate the media channel. The session management module can also maintain the correspondence between at least one data memory block and the channel identifier A. In addition, the session management module can configure the initial information of the frame-level protection information in the at least one protection information memory block. In this embodiment, the initial information includes, for example, the type of the frame-level protection information is frame, the channel identifier A, and the correspondence between the address and length of the data memory block of the video frame bound to the frame-level protection information.

[0148] Further, such as Figure 5As shown, the TA with CAS / DRM running in the TEE software application layer can receive the encrypted video stream, and then decrypt the encrypted video stream to obtain the video stream. Then, the TA can configure the output control policy according to the protection requirements corresponding to the video stream. After that, the TA can write the video stream into the data memory block 01. In addition, in some embodiments, the TA can configure the output control policy to the VDEC. In other embodiments, the TA can configure the output control policy to the stream output control policy memory block (stream output control policy buffer). Among them, VDEC has read operation permission on the stream output control policy memory block.

[0149] Furthermore, VDEC can respond to the instruction of the TEE software driver layer, read the video code stream from the data memory block 01, and then decode the video code stream to obtain the first decoded video frame. After that, VDEC can write the first decoded video frame into the data memory block 02. VDEC can also read the output control strategy from the code stream output control strategy memory block, and then write the output control strategy and the resolution of the first decoded video frame into the protection information memory block 01 to obtain the frame-level protection information of the first decoded video frame. The frame-level protection information in the protection information memory block 01 includes the resolution and minimum resolution of the first decoded video frame. VDEC learns the resolution of the first decoded video frame before performing the decoding operation.

[0150] In the present application, before the VDEC writes the first decoded video frame into the data memory block 02, it can detect whether the video code stream and the first decoded video frame correspond to the same media path. After determining that the video code stream and the first decoded video frame correspond to the same media path, the first decoded video frame is written into the data memory block 02. In this way, the problem of accessing data of other media paths when the VDEC outputs the first decoded video frame can be avoided, thereby improving the security of video frame transmission.

[0151] Exemplarily, the VDEC may obtain the first path identifier corresponding to the data memory block 01 and the second path identifier in the frame-level protection information of the first decoded video frame. When the first path identifier and the second path identifier are the same, it may be determined that the video code stream and the frame-level protection information of the first decoded video frame correspond to the same media path.

[0152] For example, data memory block 01 corresponds to channel identifier A, and the channel identifier in the frame-level protection information of the first decoded video frame is preconfigured by the session management module, that is, the second channel identifier is channel identifier A. Based on this, the first channel identifier should be the same as the second channel identifier.

[0153] In addition, before writing the output control strategy and the resolution of the first decoded video frame into the protection information memory block 01, the VDEC may detect whether the initial information in the protection information memory block 01 is valid. After determining that the initial information in the protection information memory block 01 is valid, the VDEC writes the output control strategy and the resolution of the first decoded video frame into the protection information memory block 01. The embodiment of the VDEC detecting whether the initial information is valid is described below and will not be described in detail here.

[0154] In some embodiments, the first decoded video frame is the first display video frame. Further, VDP can respond to instructions from the TEE software driver layer, read frame-level protection information from the protection information memory block 01, and obtain the HDCP protection status from the output port. Then, VDP can determine the target output control policy that matches the HDCP protection status from the frame-level protection information. In this embodiment, the target output control policy is, for example, the HDCP1.4 output control policy. Then, VDP determines whether the minimum resolution in the frame-level protection information is less than the resolution allowed by the HDCP1.4 output control policy. When the minimum resolution in the frame-level protection information is less than the resolution allowed by the HDCP1.4 output control policy, the first display video frame is output through the output port.

[0155] In this embodiment, before reading the frame-level protection information from the protection information memory block 01, the VDP needs to detect whether the frame-level protection information in the protection information memory block 01 is valid, and then, after determining that the frame-level protection information in the protection information memory block 01 is valid, read the frame-level protection information from the protection information memory block 01. The process of VDP detecting whether the frame-level protection information in the protection information memory block 01 is valid is described below and is not described in detail here.

[0156] In some other embodiments, the media path may also include VPSS ( Figure 5 After VDEC writes the first decoded video frame into data memory block 02 and writes the output control strategy and the resolution of the first decoded video frame into protection information memory block 01, VPSS can respond to the instruction of the TEE software driver layer, read the first decoded video frame from data memory block 02 and process the first decoded video frame, which can be, for example, a reduction process or an enlargement process, for example, reducing the first decoded video frame to obtain the first display video frame and the resolution of the first display video frame. Afterwards, VPSS can, for example, write the first display video frame into data memory block 03 ( Figure 5 ), and updating the protection information memory block 02 ( Figure 5VPSS can write the resolution of the first display video frame into the protection information memory block 02, and update the minimum resolution of the frame-level protection information in the protection information memory block 02 to the resolution of the first display video frame, to obtain the frame-level protection information of the first display video frame. Correspondingly, in this embodiment, VDP reads the frame-level protection information of the first display video frame from the protection information memory block 02 to determine whether to output the first display video frame. This will not be described in detail here.

[0157] In this embodiment, before writing the first display video frame into the data memory block 03, the VPSS can detect whether the first decoded video frame and the first display video frame correspond to the same media path. After determining that the first decoded video frame and the first display video frame correspond to the same media path, the first display video frame is written into the data memory block 03. This can avoid the problem of accessing data of other media paths when the VPSS outputs the first display video frame, thereby improving the security of video frame transmission.

[0158] Exemplarily, the VPSS may obtain the third channel identifier in the level protection information in the protection information memory block 01 and the fourth channel identifier in the level protection information in the protection information memory block 02. When the third channel identifier and the fourth channel identifier are the same, it is determined that the first decoded video frame and the first displayed video frame correspond to the same media channel.

[0159] For example, the third path identifier is pre-configured by the session management module, that is, the third path identifier should be path identifier A. The fourth path identifier is pre-configured by the session management module, that is, the fourth path identifier should also be path identifier A, so the third path identifier should be the same as the fourth path identifier.

[0160] In addition, before VPSS reads the frame-level protection information from the protection information memory block 01, it is necessary to detect whether the frame-level protection information in the protection information memory block 01 is valid. After determining that the frame-level protection information in the protection information memory block 01 is valid, VPSS reads the frame-level protection information from the protection information memory block 01. In addition, before VPSS updates the frame-level protection information in the protection information memory block 02, it is necessary to detect whether the frame-level protection information in the protection information memory block 02 is valid. After determining that the frame-level protection information in the protection information memory block 02 is valid, VPSS updates the frame-level protection information in the protection information memory block 02.

[0161] It should be pointed out that in the present application, VDEC, VPSS and VDP detect whether the frame-level protection information is valid, which is essentially to detect at least one initial protection information in the frame-level protection information, and determine that the corresponding frame-level protection information is valid under the condition that at least one initial protection information detected is valid. In actual operation, for example, the tag in the frame-level protection information, the storage address of the video frame, the correspondence between the length of the storage address and at least one of the information in the checksum are detected. In combination with the above three exemplary information, the operation of detecting validity is exemplified by taking VDP as an example.

[0162] Detect tag: VDP can detect whether the tag of the frame-level protection information of the first display video frame matches the pre-configured tag. If the tag of the frame-level protection information of the first display video frame matches the pre-configured tag, VDP can read the frame-level protection information according to the processing logic indicated by the instruction and perform display processing and other operations.

[0163] according to Figure 3 As described in the illustrated embodiment, the tag is used to indicate whether the frame-level protection information is bound to the video frame. If the frame-level protection information is bound to the video frame, then the VDP shall perform operations on the first display video frame according to the resolution of the video frame in the frame-level protection information. If the frame-level protection information is not bound to the video frame, then the frame-level protection information is global protection information, and the VDP shall perform operations on the first display video frame according to the resolution in the extended output policy. It can be seen that the information read by VDP and the operation logic are different when corresponding to different types of frame-level protection information.

[0164] Based on this, by adopting this implementation, after determining the correctness of the frame-level protection information type, the VDP can process the first display video frame according to the matching processing logic based on the frame-level protection information.

[0165] Detect the correspondence between the storage address of the video frame and the length of the storage address: VDP can detect whether the correspondence between the storage address of the first display video frame and the length of the storage address in the frame-level protection information is correct. If the correspondence between the storage address of the first display video frame and the length of the storage address is correct, it means that the frame-level protection information is the frame-level protection information corresponding to the first display video frame. If the correspondence between the storage address of the first display video frame and the length of the storage address is incorrect, it means that the frame-level protection information is not the frame-level protection information corresponding to the first display video frame.

[0166] By adopting this implementation method, VDP can ensure whether the frame-level protection information is associated with the video frame, thereby ensuring that the protection information of the corresponding video frame is read from the frame-level protection information, and then achieving accurate output protection for the video frame.

[0167] Checksum detection: VDP can determine whether the checksum value of the frame-level protection information is the same as the initial checksum value. If the checksum value of the frame-level protection information is the same as the initial checksum value, it means that the frame-level protection information has not been tampered with. If the checksum value of the frame-level protection information is different from the initial checksum value, it means that the frame-level protection information has been tampered with.

[0168] Exemplarily, the VDP may perform an XOR operation on all information in the frame-level protection information to obtain a first result. Then, the VDP may perform an XOR operation on the first result and the secure random number to obtain a check value. Thereafter, the VDP compares whether the check value is the same as the initial check value.

[0169] Combination Figure 2 The architecture of the electronic device shown in the figure, REE driver and TEE driver both have driving effects on hardware modules. Although the frame-level protection information is stored in the secure memory block, the REE driver can control the hardware module with access rights to the secure memory block to tamper with and forge the frame-level protection information. Based on this, the present application sets up a checksum detection mechanism.

[0170] This embodiment relates to the feature of "secure random numbers", which are random numbers in the mask register of an electronic device. The mask register is, for example, a 32-bit register. When the electronic device is powered on, the TEE side can read a 32-bit random number from the hardware random number module, write it into the mask register and latch it. The REE side has no access to the mask register. Based on this, the 32-bit random number in the mask register is relatively secure and can be called a "secure random number".

[0171] Furthermore, the session management module can determine the initial checksum in the frame-level protection information according to the algorithm checksum = (Word0^Word1^...Wordn)^Mask, where n is an integer greater than or equal to 1, Word0 to Wordn refer to each information in the frame-level protection information, and Mask refers to a secure random number. When detecting the checksum, VDP can calculate the check value of the current frame-level protection information according to the same algorithm. Since the secure random number cannot be tampered with, if the calculated check value is the same as the checksum, it means that the frame-level protection information has not been tampered with. If the calculated check value is different from the checksum, it means that the frame-level protection information has been tampered with by the REE side, and VDP can output mute.

[0172] It can be seen that by adopting this implementation method, VDP can perform output protection on the first display video frame according to the frame-level protection information while ensuring that the frame-level protection information has not been destroyed, forged or tampered with, thereby achieving correct output protection for the first display video frame and optimizing the output protection performance.

[0173] It is understandable that the above algorithm for calculating the check value is only a schematic description and does not constitute a specific limitation on determining the check value. In other embodiments of the present application, the electronic device session management module and the hardware module may use other mechanisms and other algorithms to determine the check value of the frame-level protection information.

[0174] In some embodiments, VDP may determine whether the frame-level protection information is valid only through the above-mentioned checksum. For example, when the checksum is the same as the verification value, VDP determines that the frame-level protection information is valid. In other embodiments, VDP may determine whether the frame-level protection information is valid through any two of the above-mentioned three items of information. When both detected two pieces of information meet the conditions, VDP determines that the frame-level protection information is valid. In some other embodiments, for example, when the detection results of the above-mentioned three items of information all meet the conditions, VDP determines that the frame-level protection information is valid. In addition, the detection result of any one of the above-mentioned three items of information has no effect on the detection of other information. Accordingly, the detection order of the above-mentioned three items of information can be arbitrary, and the present application does not limit this.

[0175] In addition, in the present application, the detection of the validity of the frame-level protection information by VDEC and the detection of the validity of the frame-level protection information by VPSS are similar to the detection process of the above-mentioned VDP, except that in the scenario of detecting the corresponding relationship between the storage address of the video frame and the length of the storage address, the video frame and the storage address of the video frame involved may be different from the VDP. For example, VDEC should detect whether the corresponding relationship between the storage address of the first decoded video frame and the length of the storage address is correct. This will not be described in detail here.

[0176] It is understandable that the above operation of detecting the validity of the frame-level protection information is only a schematic description and does not constitute a limitation on the detection of the validity of the frame-level protection information in this application. In other embodiments of this application, the information related to the validity of the frame-level protection information can be more or less. This is not limited here.

[0177] The technical solution of the present application is introduced below with reference to examples.

[0178] Fig. 6A An architecture diagram of an electronic device 20 is provided, and the electronic device 20 supports TEE. The TEE side of the electronic device 20 includes: a software part and a hardware part. The software part includes a TEE application, a session manager module, and a TEE driver module. The TEE application includes a TA. It should be understood that the software part is a functional module implemented by software instructions or software codes, which run on a processor to implement corresponding functions. The hardware part includes VDEC, VPSS, VDP, and HDMI. The hardware part also includes a TEE storage module.

[0179] In this embodiment, the TEE application runs in the TEE software application layer in the electronic device 10. The session management module and the TEE driver module run in the TEE software driver layer of the electronic device 10, for example.

[0180] Understandably, Fig. 6A This is only an exemplary description of the electronic device of the present application, and does not constitute any limitation to the electronic device involved in the present application. In some other embodiments, the electronic device involved in the present application may include more or fewer hardware modules, and correspondingly, the electronic device involved in the present application may include hardware modules with other functions. In addition, the functional software of the electronic device may also adopt other forms of expression. No further details are given here.

[0181] Understandably, Fig. 6A This is only an illustrative description and does not constitute a specific limitation on the electronic device 20. In other embodiments of the present application, the electronic device 20 may include more or fewer components than shown in the figure, or combine some components, or split some components, or arrange the components differently. The components shown in the figure may be implemented in hardware, software, or a combination of software and hardware.

[0182] Further, combined with Fig. 6A , an exemplary introduction to the operation process of this application is given.

[0183] Create a target media channel

[0184] The TEE application may send a request to the session management module to create a target media path, where the target media path is used to transmit a video stream, for example. Furthermore, the session management module may create the target media path according to the purpose of the target media path, and configure a path identifier Z for the target media path.

[0185] Exemplarily, the target media channel includes VDEC, VPSS, VDP and HDMI, as well as secure memory block A, secure memory block B1, secure memory block B2, secure memory block C1 and secure memory block C2. Among them, secure memory block A, secure memory block B1 and secure memory block C1 are used to store video-related data. Secure memory block B2 and secure memory block C2 are used to store frame-level protection information. The frame-level protection information in secure memory block B2 corresponds to the video frame data in secure memory block B1. The frame-level protection information in secure memory block C2 corresponds to the video frame data in secure memory block C1.

[0186] In this embodiment, the session management module can maintain the correspondence between the secure memory block A, the secure memory block B1, and the secure memory block C1 and the channel identifier Z. In addition, the session management module can also configure the initial information of the frame-level protection information in the secure memory block B2, and the initial information of the frame-level protection information in the secure memory block C2. Exemplarily, the initial information of the frame-level protection information in the secure memory block B2 includes the tag is frame, the virtual address of the secure memory block B1 and the correspondence of the length of the virtual address, and the initial check value of the frame-level protection information in the secure memory block B2. The initial information of the frame-level protection information in the secure memory block C2 includes the tag is frame, the virtual address of the secure memory block C1 and the correspondence of the length of the virtual address, and the initial check value of the frame-level protection information in the secure memory block C2. Exemplarily, the secure memory block B2 is used to store the frame-level protection information 01, and the secure memory block C2 is used to store the frame-level protection information 02.

[0187] Combination Fig. 6A Indicative media channels, Figure 6B An exemplary transmission process of frame-level protection information is shown in FIG. 1 . The control process on the TEE side is as follows: Figure 6B As shown by the dotted arrow on the TEE side, the data transmission process on the TEE side is as follows Figure 6B As shown by the solid arrow on the TEE side, the control flow on the REE side is as follows Figure 6B Indicated by the solid arrow on the REE side.

[0188] Get video stream

[0189] After creating the target media channel, the REE application obtains the video stream source. After determining that the video stream source is a protected video stream source, the REE application transmits the protected video stream source to the TA. The video stream source is the encrypted video stream. The TA decrypts the video stream source to obtain the video stream and the protection requirements corresponding to the video stream. Afterwards, the TA configures the output control policy corresponding to the video stream according to the protection requirements. Among them, the output control policy is as described in the above embodiment and is not described in detail here. In this embodiment, the display control driver on the TEE side configures the output control policy configured by the TA to the VDEC, and the TA writes the video stream into the secure memory block A. Among them, the display control driver is a type of TEE driver module.

[0190] Decode the video code stream to get the decoded video frame

[0191] VDEC responds to the instruction of the TEE driver module and reads the video code stream from the secure memory block A. Then, VDEC decodes the video code stream to obtain a decoded video frame. In addition, VDEC can obtain the resolution of the decoded video frame before decoding, and the resolution of the decoded video frame is, for example, resolution 01.

[0192] Afterwards, VDEC obtains the channel identifier corresponding to secure memory block A and obtains channel identifier Z. VDEC also obtains the channel identifier in the initial information of secure memory block B2 and also obtains channel identifier Z. Based on the fact that the channel identifier corresponding to secure memory block A is the same as the channel identifier in the initial information of secure memory block B2, it is determined that both the video code stream and the decoded video frame correspond to the target media channel, and VDEC writes the decoded video frame into secure memory block B1. The process from VDEC reading the video code stream to writing the decoded video frame into secure memory block B1 is as follows: Figure 6B Not shown.

[0193] In addition, combined Figure 6B , VDEC detects whether the initial information in the secure memory block B2 is valid. Exemplarily, VDEC detects whether the tag of the initial information in the secure memory block B2 is frame, whether the correspondence between the virtual address of the secure memory block B1 and the length of the virtual address is correct, and whether the check value of the initial information in the secure memory block B2 is the same as the initial check value. In an optional case, when the detection results of the above three items of information are all "yes", indicating that the initial information in the secure memory block B2 is valid, VDEC writes the resolution 01 into the information bits corresponding to the "current resolution" and "minimum resolution" in the secure memory block B2, and obtains the frame-level protection information 01 of the decoded video frame.

[0194] Process the decoded video frame to get the displayed video frame

[0195] like Figure 6B As shown, the user can input instructions for processing the decoded video frame through the REE application layer. In this embodiment, the REE application layer can call the VPSS driver to send specific processing instructions to the VPSS. Accordingly, the VPSS reads the decoded video frame from the secure memory block B1, and then performs a reduction operation on the decoded video frame to obtain a display video frame ( Figure 6B not shown).

[0196] In this embodiment, combined with Figure 6B As shown, VPSS also needs to detect whether the frame-level protection information 01 in the secure memory block B2 is valid. If the frame-level protection information 01 in the secure memory block B2 is valid, VPSS can obtain the resolution 01 from the frame-level protection information 01 in the secure memory block B2, and then determine the resolution 02 of the displayed video frame according to the resolution 01 and the multiple by which the displayed video frame is reduced relative to the decoded video frame. In this embodiment, the resolution 02 is, for example, smaller than the resolution 01. The operation process of VPSS detecting whether the frame-level protection information 01 in the secure memory block B2 is valid is similar to the operation process of VDEC detecting whether the initial information in the secure memory block B2 is valid, and will not be repeated here.

[0197] Afterwards, VPSS can obtain the path identifier of the frame-level protection information 01 in the secure memory block B2 and obtain the path identifier Z. VPSS can also obtain the path identifier of the initial information in the secure memory block C2 and obtain the path identifier Z. According to the fact that the path identifier of the frame-level protection information 01 in the secure memory block B2 is the same as the path identifier of the initial information in the secure memory block C2, it is determined that the decoded video frame and the displayed video frame both correspond to the target media path, and VPSS writes the displayed video frame into the secure memory block C1.

[0198] In addition, VPSS detects whether the initial information in the secure memory block C2 is valid. Exemplarily, VPSS can detect whether the tag of the initial information in the secure memory block C2 is frame, whether the correspondence between the virtual address of the secure memory block C1 and the length of the virtual address is correct, and whether the check value of the frame-level protection information 02 in the secure memory block C2 is the same as the initial check value. After determining that the initial information in the secure memory block C2 is valid, VPSS writes the resolution 02 to the information bit corresponding to the "current resolution" in the secure memory block C2. Since the resolution 02 is less than the resolution 01, VPSS also needs to update the information bit corresponding to the "minimum resolution" in the secure memory block C2 to the resolution 02, and obtain the frame-level protection information 02 for the displayed video frame.

[0199] Output display video frame

[0200] VDP reads display video frames from secure memory block C1.

[0201] VDP also needs to detect whether the frame-level protection information 02 in the secure memory block C2 is valid. If the frame-level protection information 02 in the secure memory block C2 is valid, VDP reads the frame-level protection information 02 from the secure memory block C2. Exemplarily, in this embodiment, the HDCP monitoring module (HDCP monitor) in the VDP may read the frame-level protection information 02. In this embodiment, VDP also obtains the HDCP protection status supported by HDMI from HDMI. The HDCP protection status includes the HDCP protection level allowed by HDMI. In this embodiment, the operation process of VDP detecting whether the frame-level protection information 02 in the secure memory block C2 is valid is similar to the operation process of VPSS detecting whether the initial information in the secure memory block C2 is valid, and will not be repeated here.

[0202] It should be noted that the operation of VDP reading frame-level protection information 02 and the operation of VDP obtaining the HDCP protection status supported by HDMI are two independent operation processes. Therefore, the execution order of the two operation processes can be arbitrary, and this application does not limit this.

[0203] Furthermore, VDP determines the target output protection policy that matches the HDCP protection state from the frame-level protection information 02. The target output protection policy is, for example, HDCP1.4. Then, VDP detects whether the resolution 02 in the frame-level protection information 02 is less than the resolution allowed by HDCP1.4. If the resolution 02 is less than the resolution allowed by HDCP1.4, VDP can output and display the video frame through HDMI at the resolution allowed by HDCP1.4. If the resolution 02 is greater than the resolution allowed by HDCP1.4, VDP outputs mute through HDMI.

[0204] Combination Figure 6B In other implementation scenarios, if the frame-level protection information 02 also includes watermark control information, after determining that the display video frame can be output, the VDP can also obtain the position and size of the display video frame from the video output driver on the REE side, and then read the watermark information from the watermark storage address in the frame-level protection information 02, and then add the watermark information to the display video frame according to the position and size of the display video frame. The video output driver on the REE side is one of the software driver modules on the REE side.

[0205] Understandably, Fig. 6A and Figure 6B The embodiments involved are only schematic descriptions and do not limit the technical solutions of the present application. In some other embodiments, the target media path may also include more or fewer hardware modules and memory blocks. The operation process of each hardware module may also be different from the above description. The protection information in each memory block may also be different from the above description. No further details are given here.

[0206] In addition, this specification does not show all implementation scenarios applicable to the present application. In other implementation scenarios, other implementation means based on the technical ideas of the present application are also within the protection scope of the present application.

[0207] In summary, the technical solution provided by the present application is that after the electronic device decodes the video code stream to obtain the video frame, the frame-level protection information is generated for the corresponding video frame. Afterwards, the electronic device updates the frame-level protection information of the video frame according to the change in the resolution of the video frame. Therefore, before outputting the video frame to be displayed, the electronic device can perform output protection on the video frame to be displayed according to the frame-level protection information corresponding to the video frame to be displayed. In this way, protection is performed at the level of the video frame. When the video frame changes, the electronic device can update the output protection information corresponding to the video frame in time, so that the electronic device can provide more accurate output protection, avoid protection failure problems caused by the asynchrony between the video frame and the output protection strategy, and improve the protection performance.

[0208] In the above embodiments, the various schemes of the safe output method provided by the present application are introduced from the perspective of the hardware entity structure of the electronic device, the software architecture, and the actions performed by each software and hardware. Those skilled in the art should easily realize that, in combination with the establishment of the corresponding relationship described in the embodiments disclosed herein and the processing steps for executing the output according to the corresponding relationship, the present application can not only be implemented in the form of hardware or a combination of hardware and computer software. Whether certain functions are executed in the form of hardware or computer software driving hardware depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each of the above-mentioned specific applications, but such implementation should not be considered to exceed the scope of the embodiments of the present application.

[0209] For example, the electronic device 10 and the electronic device 20 may implement the functions of the above parts in the form of functional modules. Fig. 7A As shown, the safety output device 70 may include a decoding module 701 and a display control module 702. The safety output device 70 may be used to perform the above Figure 4 Part or all of the embodiments of the safety output method in any of the illustrated embodiments.

[0210] For example, the decoding module 701 is used to generate frame-level protection information of the first decoded video frame according to the output control strategy and the resolution of the first decoded video frame, and the first decoded video frame is obtained by the decoding module decoding the video code stream. The display control module 702 is used to determine whether to output the first display video frame according to the frame-level protection information, and the first display video frame is obtained according to the first decoded video frame.

[0211] It can be seen that the security output device 70 provided in the present application can output protection for each video frame according to the frame-level protection information corresponding to the video frame, thereby providing higher security.

[0212] Optionally, the safety output device 70 may also include a processing module, an updating module, a TA, a detection module, a watermark adding module, an acquisition module and a session management module. In different embodiments, the above modules are used to implement different functions.

[0213] For example, in some embodiments, the processing module can be used to process the first decoded video frame to obtain the first display video frame. The update module is used to update the frame-level protection information according to the resolution of the first display video frame. The processing module is specifically used to enlarge the first decoded video frame to obtain the first display video frame; or, it is specifically used to reduce the first decoded video frame to obtain the first display video frame. The frame-level protection information includes the resolution and minimum resolution of the first decoded video frame, and the minimum resolution is the minimum resolution corresponding to the video code stream in the entire media channel. In this embodiment, the update module is specifically used to modify the minimum resolution in the frame-level protection information to the resolution of the first display video frame when the resolution of the first display video frame is less than the minimum resolution.

[0214] Optionally, the display control module 702 is specifically used to obtain the output protection status allowed by the output port, determine the target output control strategy that matches the output protection status; determine whether the minimum resolution in the frame-level protection information is less than the resolution allowed by the target output control strategy; and when the minimum resolution in the frame-level protection information is less than the resolution allowed by the target output control strategy, output the first display video frame.

[0215] Optionally, TA may be used to configure the output control strategy for the video code stream.

[0216] Optionally, the decoding module 701 is also used to determine whether the video code stream and the first decoded video frame correspond to the same media path. The decoding module 701 is specifically used to obtain the first path identifier corresponding to the video code stream and the second path identifier in the frame-level protection information of the first decoded video frame. When the first path identifier and the second path identifier are the same, it is determined that the video code stream and the frame-level protection information of the first decoded video frame correspond to the same media path. The processing module is also used to determine whether the first decoded video frame and the first display video frame correspond to the same media path. The processing module is used to obtain the third path identifier in the frame-level protection information of the first decoded video frame and the fourth path identifier in the updated frame-level protection information. When the third path identifier and the fourth path identifier are the same, it is determined that the first decoded video frame and the first display video frame correspond to the same media path.

[0217] Optionally, the detection module is used to detect whether the frame-level protection information is valid. The detection module is specifically used to perform at least one of the following detections: detecting whether the type of the frame-level protection information matches a preconfigured type, the type is used to indicate whether the frame-level protection information is bound to a video frame, determining whether the check value of the frame-level protection information is the same as the initial check value, and detecting whether the corresponding relationship between the storage address of the first display video frame in the frame-level protection information and the length of the storage address is correct.

[0218] Optionally, the detection module is specifically used to perform an XOR operation on all the information in the frame-level protection information to obtain a first result, perform an XOR operation on the first result and a secure random number to obtain a verification value of the frame-level protection information, and compare whether the verification value of the frame-level protection information is the same as the initial verification value.

[0219] Optionally, the watermark adding module is used to add a watermark to the first display video frame according to the frame-level protection information and the position and size of the first display video frame.

[0220] Optionally, the display control module 702 is also used to determine whether to output the first display video frame according to the resolution in the global protection information when the frame-level protection information of the first display video frame is lost, and the resolution in the global protection information is the minimum resolution in the output strategy corresponding to at least one code stream.

[0221] Optionally, the acquisition module is used to obtain a request to create a media path, and the media path is used to transmit the video code stream. In this embodiment, the session management module can be used to generate a path identifier of the media path. The configuration module is also used to configure the type of protection information in the frame-level protection information, the corresponding relationship between the storage address of the video frame corresponding to the frame-level protection information and the length of the storage address, and the initial check value.

[0222] For details, please refer to method 100 and Figure 6B The relevant descriptions in the corresponding embodiments are not repeated here.

[0223] It is understood that in some embodiments, Fig. 7A The decoding module 701 shown in FIG. Figure 2 , Figure 5 as well as Fig. 6A The function of VDEC in it. Fig. 7A The display control module 702 shown in FIG. Figure 2 , Figure 5 as well as Fig. 6A The function of VDP in. Fig. 7A The display processing module shown in FIG. Figure 2 , Figure 5as well as Fig. 6A The function of VPSS in. Fig. 7A Other functional modules shown in the figure can be realized Figure 2 as well as Fig. 6A For example, the acquisition module can implement the functions of the TEE software operating system layer, etc. See the description of the electronic device 10 or the electronic device 20 in the above embodiment for details, which will not be described in detail here.

[0224] It can be understood that the division of the above modules is only a division of logical functions. In actual implementation, the functions of the above modules can be integrated into the processor for implementation. Figure 7B As shown, the safety output device 71 includes a processor 711, a transmission interface 712 and a memory 713. It should be understood that the transmission interface 712 may include an input interface and an output interface, or the transmission interface 712 has the functions of both an input interface and an output interface, which is not limited in the embodiments of the present application. Among them, the memory 713 can be used to store programs / codes pre-installed in the safety output device 71, and can also store codes for execution by the processor 711, etc. The transmission interface 712 can execute the operations of determining the HDCP status and outputting the first display video frame in method 100. The processor 711 can execute the operations in method 100 except for determining the HDCP status and outputting the first display video frame.

[0225] For details, please refer to method 100 and Figure 6B The relevant descriptions in the corresponding embodiments are not repeated here.

[0226] In a specific implementation, the present application also provides a computer storage medium corresponding to the electronic device, wherein the computer storage medium set in any device may store a program, and when the program is executed, some or all of the steps in each embodiment of the secure output method provided by method 100 may be implemented. The storage medium in any device may be a disk, an optical disk, a read-only memory (ROM) or a random access memory (RAM), etc.

[0227] It is also known to those skilled in the art that the various illustrative logical blocks and steps listed in the embodiments of the present application can be implemented by electronic hardware, computer software, or a combination of the two. Whether such functions are implemented by hardware or software depends on the specific application and the design requirements of the entire system. Those skilled in the art can use various methods to implement the functions for each specific application, but such implementation should not be understood as exceeding the scope of protection of the embodiments of the present application.

[0228] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function according to the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions can be transmitted from a website site, computer, server or message center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, server or message center. The computer-readable storage medium can be any available medium that a computer can access or a message storage device such as a server, message center, etc. that contains one or more available media integrations. The available medium may be a magnetic medium (eg, a floppy disk, a hard disk, a magnetic tape), an optical medium (eg, a DVD), or a semiconductor medium (eg, a solid state disk (SSD)).

[0229] It should be understood that in various embodiments of the present application, the size of the sequence number of each process does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiment.

[0230] Each part of this specification is described in a progressive manner, and the same or similar parts between the various embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the device and system embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the description of the method embodiment part.

[0231] Although the preferred embodiments of the present application have been described, those skilled in the art may make other changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications falling within the scope of the present application.

[0232] Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.

Claims

1. A safe output method, characterized in that: The method comprises: In a stage of decoding a video code stream to obtain a first decoded video frame, frame-level protection information of the first decoded video frame is generated according to an output control strategy and a resolution of the first decoded video frame, wherein the output control strategy is a protection strategy that matches a protection requirement of the video code stream; Enlarging the first decoded video frame to obtain a first display video frame; or reducing the first decoded video frame to obtain the first display video frame; updating the frame-level protection information according to a resolution of the first display video frame; Get the output protection status of the output port; Determining a target output control strategy that matches the output protection state; Determine whether to output the first display video frame according to the frame-level protection information and the target output control strategy.

2. The method according to claim 1, characterized in that The frame-level protection information includes a resolution and a minimum resolution of the first decoded video frame, where the minimum resolution is a minimum resolution corresponding to the video code stream in the entire media path, and updating the frame-level protection information according to the resolution of the first displayed video frame includes: When the resolution of the first display video frame is smaller than the minimum resolution, the minimum resolution in the frame-level protection information is modified to the resolution of the first display video frame.

3. The method according to claim 1 or 2, characterized in that The determining whether to output the first display video frame according to the frame-level protection information and the target output control strategy includes: Determining whether the minimum resolution in the frame-level protection information is less than the resolution allowed by the target output control strategy; When the minimum resolution in the frame-level protection information is smaller than the resolution allowed by the target output control policy, the first display video frame is output.

4. The method according to claim 1 or 2, characterized in that: Before decoding the video code stream to obtain the first decoded video frame, the method further includes: The trusted application TA configures the output control policy for the video code stream.

5. The method according to claim 1 or 2, characterized in that: The method further comprises: Determining whether the video code stream and the first decoded video frame correspond to the same media path; Determine whether the first decoded video frame and the first displayed video frame correspond to the same media path.

6. The method according to claim 5, characterized in that The determining whether the video code stream and the first decoded video frame correspond to the same media path comprises: Obtaining a first path identifier corresponding to the video code stream and a second path identifier in the frame-level protection information of the first decoded video frame; When the first path identifier and the second path identifier are the same, determining that the frame-level protection information of the video code stream and the first decoded video frame corresponds to the same media path; The determining whether the first decoded video frame and the first displayed video frame correspond to the same media path comprises: Acquire a third path identifier in the frame-level protection information of the first decoded video frame, and a fourth path identifier in the updated frame-level protection information; When the third path identifier and the fourth path identifier are the same, it is determined that the first decoded video frame and the first displayed video frame correspond to the same media path.

7. The method according to claim 1 or 2, characterized in that: Before determining whether to output the first display video frame according to the frame-level protection information, and before updating the frame-level protection information according to the resolution of the first display video frame, the method further includes: Check whether the frame level protection information is valid.

8. The method according to claim 7, characterized in that The detecting whether the frame-level protection information is valid comprises: Test for at least one of the following: Detecting whether a type of the frame-level protection information matches a preconfigured type, the type being used to indicate whether the frame-level protection information is bound to a video frame; Determining whether the check value of the frame-level protection information is the same as the initial check value; or It is detected whether the corresponding relationship between the storage address of the first display video frame and the storage address length in the frame-level protection information is correct.

9. The method according to claim 8, characterized in that The determining whether the check value of the frame-level protection information is the same as the initial check value includes: Performing an XOR operation on all information in the frame-level protection information to obtain a first result; Performing an XOR operation on the first result and the secure random number to obtain a check value of the frame-level protection information; Compare the check value of the frame-level protection information with the initial check value to see if they are the same.

10. The method according to claim 8, characterized in that Before decoding the video code stream to obtain the first decoded video frame, the method further includes: Obtaining a request for creating a media channel, where the media channel is used to transmit the video code stream; generating a channel identifier of the media channel; The type of protection information in the frame-level protection information, the corresponding relationship between the storage address of the video frame corresponding to the frame-level protection information and the length of the storage address, and the initial check value are configured.

11. The method according to claim 1 or 2, characterized in that: The method further comprises: A watermark is added to the first display video frame according to the frame-level protection information, the position and the size of the first display video frame.

12. The method according to claim 1, characterized in that Also includes: If the frame-level protection information of the first display video frame is lost, whether to output the first display video frame is determined according to the resolution in the global protection information, where the resolution in the global protection information is the minimum resolution in the output strategy corresponding to at least one code stream.

13. A safety output device, characterized in that: The device comprises: a decoding module, configured to generate frame-level protection information of a first decoded video frame according to an output control strategy and a resolution of the first decoded video frame, wherein the first decoded video frame is obtained by decoding a video code stream by the decoding module, and the output control strategy is a protection strategy matching a protection requirement of the video code stream; The device also includes: A processing module, configured to amplify the first decoded video frame to obtain a first display video frame; or to reduce the first decoded video frame to obtain the first display video frame; An updating module, configured to update the frame-level protection information according to a resolution of the first display video frame; A display control module, used for obtaining an output protection state of an output port and determining a target output control strategy matching the output protection state; Determine whether to output the first display video frame according to the frame-level protection information and the target output control strategy.

14. The device according to claim 13, characterized in that The frame-level protection information includes the resolution and minimum resolution of the first decoded video frame, wherein the minimum resolution is the minimum resolution corresponding to the video code stream in the entire media path. The updating module is specifically configured to modify the minimum resolution in the frame-level protection information to the resolution of the first display video frame when the resolution of the first display video frame is smaller than the minimum resolution.

15. The device according to claim 13 or 14, characterized in that The display control module is specifically used for: Determining whether the minimum resolution in the frame-level protection information is less than the resolution allowed by the target output control strategy; When the minimum resolution in the frame-level protection information is smaller than the resolution allowed by the target output control policy, the first display video frame is output.

16. The device according to claim 13 or 14, characterized in that The device also includes a trusted application TA, which is used to configure the output control policy for the video code stream.

17. The device according to claim 13 or 14, characterized in that The decoding module is further used to determine whether the video code stream and the first decoded video frame correspond to the same media channel; and The processing module is further configured to determine whether the first decoded video frame and the first displayed video frame correspond to the same media channel.

18. The device according to claim 17, characterized in that The decoding module is specifically used for: Obtaining a first path identifier corresponding to the video code stream and a second path identifier in the frame-level protection information of the first decoded video frame; When the first path identifier and the second path identifier are the same, determining that the frame-level protection information of the video code stream and the first decoded video frame corresponds to the same media path; and The processing module is further specifically used for: Acquire a third path identifier in the frame-level protection information of the first decoded video frame, and a fourth path identifier in the updated frame-level protection information; When the third path identifier and the fourth path identifier are the same, it is determined that the first decoded video frame and the first displayed video frame correspond to the same media path.

19. The device according to claim 13 or 14, characterized in that The device also includes a detection module, The detection module is used to detect whether the frame-level protection information is valid.

20. The device according to claim 19, characterized in that The detection module is specifically used to perform at least one of the following detections: Detecting whether a type of the frame-level protection information matches a preconfigured type, the type being used to indicate whether the frame-level protection information is bound to a video frame; Determining whether the check value of the frame-level protection information is the same as the initial check value; or It is detected whether the corresponding relationship between the storage address of the first display video frame and the storage address length in the frame-level protection information is correct.

21. The device according to claim 20, characterized in that The detection module is specifically used for: Performing an XOR operation on all information in the frame-level protection information to obtain a first result; Performing an XOR operation on the first result and the secure random number to obtain a check value of the frame-level protection information; Compare the check value of the frame level protection information with the initial check value to see if they are the same.

22. The device according to claim 20, characterized in that The device also includes: An acquisition module, used for acquiring a request for creating a media channel, where the media channel is used for transmitting the video code stream; The session management module is used to generate a path identifier of the media path; configure the type of protection information in the frame-level protection information, the corresponding relationship between the storage address of the video frame corresponding to the frame-level protection information and the length of the storage address, and the initial check value.

23. The device according to claim 13 or 14, characterized in that The device also includes a watermark adding module, which is used to add a watermark to the first display video frame according to the frame-level protection information, the position and the size of the first display video frame.

24. The device according to claim 13, characterized in that The display control module is also used to determine whether to output the first display video frame according to the resolution in the global protection information when the frame-level protection information of the first display video frame is lost, and the resolution in the global protection information is the minimum resolution in the output strategy corresponding to at least one code stream.

25. A safety output device, characterized in that: The device comprises a processor and a transmission interface, wherein: The processor is configured to call program instructions stored in a memory to execute the safety output method according to any one of claims 1 to 12.

Images