Color matrix processing method, system, medium and terminal

Abstract

The present invention provides a color matrix processing method, system, medium and terminal applied to a terminal. The color matrix processing method includes: when the color inversion function is turned on, the display system transmits the original color matrix to the hardware synthesizer; the hardware synthesizer encodes the original color matrix to obtain an encoded intermediate color matrix; the hardware synthesizer transmits the intermediate color matrix to the platform display hardware processor; the platform display hardware processor decodes the intermediate color matrix to obtain a target color matrix; the platform display hardware processor performs color inversion processing on the target color matrix. This method enables the color matrix to be processed by the platform display hardware processor during the color inversion process, eliminating the need for the GPU to process each layer that requires color inversion, thereby reducing the load on the GPU.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular, to a color matrix processing method, system, medium, and terminal applied to a terminal. Background Art

[0002] With the gradual popularization of portable terminals such as mobile phones and tablet computers, users' dependence on portable terminals is getting higher and higher, and users often need to use portable terminals in different usage scenarios. Therefore, portable terminals usually set a color inversion function to meet the usage needs of customers in some scenarios.

[0003] In the prior art, the processing of the color inversion function for terminals with a GPU (Graphics Processing Unit) mainly involves processing the color matrix for each layer. This processing method has the following disadvantages: Since the GPU processes each layer, it will increase the GPU load; there are special scenarios in the use of portable terminals that do not support GPU synthesis, and for these scenarios, the color inversion cannot be processed by the GPU processing method; if the user records a video of the content currently displayed on the screen after processing in the blue light eye protection mode, etc., the display effect after color matrix processing will be recorded, which is not unified with the screen recording function of some terminals (such as Google), and the recording effect cannot be kept consistent, and there will be different display effects for the screen recording content between different devices, which cannot meet the user's requirements for the recording effect when sharing the screen recording. Therefore, it is necessary to propose a new color matrix processing method applied to terminals to solve the above problems. Summary of the Invention

[0004] The purpose of the present invention is to provide a color matrix processing method, system, medium, and terminal to improve the problem of increased load caused by the GPU processing color inversion.

[0005] In a first aspect, the present invention provides a color matrix processing method applied to a terminal, including: when the color inversion function is turned on, the display system transmits the original color matrix to the hardware synthesizer; the hardware synthesizer encodes the original color matrix to obtain an encoded intermediate color matrix; the hardware synthesizer transmits the intermediate color matrix to the platform display hardware processor; the platform display hardware processor decodes the intermediate color matrix to obtain a target color matrix; the platform display hardware processor performs color inversion processing on the target color matrix.

[0006] The beneficial effects of the color matrix processing method provided by the present invention are as follows: In the color inversion process, the color matrix to be processed is processed by the platform display hardware processor, eliminating the need for the GPU to process each layer that requires color inversion. This reduces the GPU load and also avoids the situation where color inversion cannot be achieved in some special scenarios where the GPU does not support composition. Further, performing color inversion processing using the platform display hardware is carried out on the synthesized layer, which can greatly reduce the number of layers to be processed. Applying this method for color matrix processing also enables the display effects obtained by various terminals during screen recording to be those without color matrix processing, ensuring unified recording effects and thus meeting the user's requirements for recording effects when sharing screen recordings.

[0007] In a possible embodiment, the method further includes: The platform display hardware processor also receives a color adjustment matrix from the image quality enhancement service; The platform display hardware processor performs color adjustment processing on the color adjustment matrix.

[0008] In another possible embodiment, the platform display hardware processor performing color adjustment processing on the color adjustment matrix includes: The platform display hardware processor performs blue light eye protection processing or color temperature adjustment processing on the color adjustment matrix.

[0009] In other possible embodiments, the hardware synthesizer encodes the original color matrix to obtain an encoded intermediate color matrix, including:

[0010] Converting the original color matrix to an integer, and then combining the original color matrix in a pairwise combination manner to form the intermediate color matrix;

[0011] Among them, the intermediate color matrix includes a flag bit, and the flag bit is used to indicate whether the color matrix needs to be reconfigured.

[0012] The platform display hardware processor decodes the intermediate color matrix to obtain a target color matrix, including:

[0013] The platform display hardware processor decodes the intermediate color matrix to obtain a candidate color matrix;

[0014] If it is determined that the flag bit indicates that the color matrix needs to be reconfigured, then the candidate color matrix and the color adjustment matrix from the image quality enhancement service are multiplied to obtain the target color matrix.

[0015] The format of the original color matrix is different from the format of the target color matrix.

[0016] In a second aspect, the present invention also provides a color matrix processing system, which is applied to a terminal. The system includes a display system, a hardware synthesizer, and a platform display hardware processor. The display system and the platform display hardware processor are both communicatively connected to the hardware synthesizer;

[0017] The display system is configured to transfer the original color matrix to the hardware synthesizer when the color inversion function is turned on;

[0018] The hardware synthesizer is configured to encode the original color matrix to obtain an encoded intermediate color matrix;

[0019] The hardware synthesizer is configured to transfer the intermediate color matrix to the platform display hardware processor;

[0020] The platform display hardware processor is configured to decode the intermediate color matrix to obtain a target color matrix;

[0021] The platform display hardware processor is configured to perform color inversion processing on the target color matrix.

[0022] In the color matrix processing system, the platform display hardware processor is further configured to receive a color adjustment matrix from an image quality enhancement service;

[0023] The platform display hardware processor is further configured to perform color adjustment processing on the color adjustment matrix.

[0024] The platform display hardware processor performs color adjustment processing on the color adjustment matrix, specifically:

[0025] The platform display hardware processor performs blue light eye protection processing or color temperature adjustment processing on the color adjustment matrix.

[0026] The hardware synthesizer encodes the original color matrix to obtain an encoded intermediate color matrix, specifically:

[0027] Convert the original color matrix to an integer, and then combine the original color matrix in a pairwise combination manner to form the intermediate color matrix;

[0028] Among them, the intermediate color matrix includes a flag bit, and the flag bit is used to indicate whether the color matrix needs to be reconfigured.

[0029] The platform display hardware processor decodes the intermediate color matrix to obtain a target color matrix, specifically:

[0030] The platform display hardware processor decodes the intermediate color matrix to obtain a candidate color matrix;

[0031] If it is determined that the flag bit indicates that the color matrix needs to be reconfigured, multiply the candidate color matrix by the color adjustment matrix from the image quality enhancement service to obtain the target color matrix.

[0032] The format of the original color matrix is different from that of the target color matrix.

[0033] In a third aspect, the present invention further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the above-described color matrix processing method is implemented.

[0034] In a fourth aspect, the present invention further provides a terminal, including: a processor and a memory; the memory is used to store a computer program; the processor is used to execute the computer program stored in the memory so that the terminal executes the above-described color matrix processing method.

[0035] For the beneficial effects of the above second to fourth aspects, reference can be made to the description of the first aspect above. Description of the Drawings

[0036] Figure 1 It is a schematic flowchart of a color matrix processing method provided by an embodiment of the present invention;

[0037] Figure 2 It is a schematic working flowchart of a color matrix processing method provided by an embodiment of the present invention;

[0038] Figure 3 It is a schematic diagram of a color matrix processing device provided by an embodiment of the present invention;

[0039] Figure 4 It is a schematic diagram of a terminal structure provided by an embodiment of the present invention. Detailed Embodiments

[0040] To make the objectives, technical solutions, and advantages of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings of the present invention. Obviously, the described embodiments are some, but not all, of the embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention. Unless otherwise defined, the technical terms or scientific terms used herein shall have the ordinary meaning as understood by those of ordinary skill in the art in the field to which the present invention belongs. The words such as "including" used herein are intended to mean that the elements or items appearing before the word cover the elements or items listed after the word and their equivalents, without excluding other elements or items.

[0041] In view of the problems existing in the prior art, embodiments of the present invention provide a method, a system, a medium and a terminal for color matrix processing applied to a terminal.

[0042] The present embodiment provides a color matrix processing method. Refer to the accompanying drawings of the specification Figure 1 The method includes:

[0043] S101: When the color inversion function is turned on, the display system transmits the original color matrix to the hardware composer.

[0044] In S101, in a possible embodiment, the display system transmits the original color matrix to the Hardware Composer (HWC). Among them, the original color matrix is the color matrix defined by Google.

[0045] S102: The hardware composer encodes the original color matrix to obtain an encoded intermediate color matrix.

[0046] In S102, in a possible embodiment, the intermediate color matrix is a color conversion matrix defined by the Direct Rendering Manager (DRM).

[0047] In a specific embodiment, the hardware composer encodes the original color matrix to obtain an encoded intermediate color matrix, including: converting the original color matrix into an integer, and then combining the original color matrix in a pairwise combination manner to form the intermediate color matrix. Specifically, when combining the original color matrix in a pairwise combination manner to form the intermediate color matrix, every two 16-bit original color matrices are combined into a 64-bit matrix.

[0048] In a possible embodiment, the intermediate color matrix includes a flag bit, and the flag bit is used to indicate whether the color matrix needs to be reconfigured.

[0049] S103: The hardware composer transmits the intermediate color matrix to the platform display hardware processor.

[0050] S104: The platform display hardware processor decodes the intermediate color matrix to obtain a target color matrix.

[0051] In S104, in a possible embodiment, what the platform display hardware processor decodes from the intermediate color matrix is a color matrix defined by the Cathode Ray Tube Controller (CRTC) hardware algorithm, and the target color matrix is a color matrix defined by the CRTC hardware algorithm.

[0052] In a possible embodiment, the platform display hardware processor decodes the intermediate color matrix to obtain the target color matrix, including: the platform display hardware processor decodes the intermediate color matrix to obtain a candidate color matrix; if the judgment flag indicates that the color matrix needs to be reconfigured, the candidate color matrix is multiplied by the color adjustment matrix from the image quality enhancement service to obtain the target color matrix. Among them, the color adjustment matrix from the image quality enhancement service is the color matrix defined by the CRTC hardware algorithm.

[0053] In a specific embodiment, when only the color inversion function is enabled, after the platform display hardware processor obtains the intermediate color matrix, it decodes the intermediate color matrix and can directly obtain the target color matrix after decoding.

[0054] In a specific embodiment, when functions such as blue light eye protection and color temperature adjustment, which are ultimately processed by the hardware display unit, and the color inversion function are enabled at the same time, since they are all processed by the hardware display unit but the color inversion function and other functions follow different processes, once both take effect simultaneously, the problem of effect superposition needs to be considered. Specifically, after the platform display hardware processor obtains the intermediate color matrix, since the intermediate color matrix is encoded color matrix data, it is first decoded and converted into the struct cm_cfg form defined by the CRTC hardware algorithm, and then the flag bit of matrix[0] in the decoded color matrix is obtained to determine whether the color matrix needs to be reconfigured. If it is determined that the color matrix needs to be reconfigured, it needs to be multiplied by the matrix transmitted by the image quality enhancement service, and the result after multiplication is the target color matrix.

[0055] S105: The platform display hardware processor performs color inversion processing on the target color matrix.

[0056] See the attached drawings of the specification Figure 2 In a possible embodiment, the color matrix processing method of the present invention further includes: the platform display hardware processor also receives the color adjustment matrix from the image quality enhancement service; the platform display hardware processor performs color adjustment processing on the color adjustment matrix.

[0057] In a specific embodiment, the platform display hardware processor performs color adjustment processing on the color adjustment matrix, including: the platform display hardware processor performs blue light eye protection processing or color temperature adjustment processing on the color adjustment matrix.

[0058] In a possible embodiment, the format of the original color matrix is different from that of the target color matrix. Therefore, when the hardware synthesizer obtains the original color matrix and wants to transfer it to the platform display hardware processor, it is necessary to encode the original color matrix to obtain an intermediate color matrix, transfer the intermediate color matrix to the platform display hardware processor, and then the platform display hardware processor decodes the intermediate color matrix to obtain the target color matrix.

[0059] In a specific embodiment, the color matrix defined by Google is as follows:

[0060] float ColorTransformMatrix

[16] = {1.0, 0.0, 0.0, 0.0

[0061] 0.0, 1.0, 0.0, 0.0,

[0062] 0.0, 0.0, 1.0, 0.0,

[0063] 0.0, 0.0, 0.0, 1.0};

[0064] The structure originally defined by DRM is as follows:

[0065] struct drm_color_ctm {

[0066] __s64 matrix[9];

[0067] };

[0068] The color matrix defined by the CRTC hardware algorithm is as follows:

[0069] struct cm_cfg {

[0070] u16 cm00;

[0071] u16 cm01;

[0072] u16 cm02;

[0073] u16 cm03;

[0074] u16 cm10;

[0075] u16 cm11;

[0076] u16 cm12;

[0077] u16 cm13;

[0078] u16 cm20;

[0079] u16 cm21;

[0080] u16 cm22;

[0081] u16 cm23;

[0082] };

[0083] Therefore, encoding the color matrix defined by Google, the encoded intermediate color matrix specifically includes:

[0084] int ctm_low = ColorTransformMatrix[i] * 10000;

[0085] int ctm_high = ColorTransformMatrix[i + 1] * 10000;

[0086] matraix[i] = (__s64)((ctm_low & 0xffff) | ((ctm_high << 16) & 0xffff0000));

[0087] In addition, matrix[0] can be used as a flag bit, and the flags that can be judged include: whether the passed color matrix is an identity matrix, and whether the current matrix has changed compared with the previous matrix.

[0088] In a possible embodiment, when the matrix changes in the hardware synthesizer, a blob (Binary Large Object) needs to be created, and then this attribute is added, provided that it depends on the CRTC having already bound the DRM native ctmproperty (color transformation matrix property). The specific process includes: defining DRM ctm and assigning a value to ctm (color transformation matrix); when turning on or off color inversion, the hardware synthesizer creates ctmproperty, applies to the underlying layer for ctm_blob_id (color transformation matrix object identifier), and binds the already encoded drm_color_ctm (DRM custom color transformation matrix structure); adding ctm_blob_id; submitting the above application.

[0089] When the matrix changes in the hardware synthesizer, binding and creating ctmproperty can associate with the native interface provided by libDrm (DRM interface library), and pass DRM ctm to the platform display hardware processor.

[0090] The method of the present invention proposes a color matrix processing method to improve the problem of increased load caused by the GPU processing color inversion. When using this method for processing, the color matrix that needs to be processed during the color inversion process is processed by the platform display hardware processor, and there is no need for the GPU to process each layer that needs to be color-inverted, thereby reducing the load on the GPU and also avoiding the situation where color inversion processing cannot be achieved in some special scenarios where the GPU does not support composition. For example, in some security video scenarios, when playing HDR (High Dynamic Range Imaging) videos, etc., in any scenario where the GPU does not participate in composition, the color matrix processing by the platform display hardware processor can solve this problem, regardless of the scenario. Further, using the platform display hardware to perform color inversion processing is for the processed layer after composition, which can greatly reduce the number of processed layers. It can also make the display effects obtained by screen recording on various terminals be those without color matrix processing, and the recording effects are unified, which can solve the problem that when recording the content displayed on the current screen in the case of processing such as the blue light eye protection mode, the processed display effect after color matrix processing will be recorded and affect the recording effect.

[0091] See the accompanying Figure 3 In this embodiment, a color matrix processing system is further provided. This system is used to implement the above method embodiment. The system includes: a display system 301, a hardware synthesizer 302, and a platform display hardware processor 303. Both the display system 301 and the platform display hardware processor 303 are communicatively connected to the hardware synthesizer 302.

[0092] The display system 301 is used to transfer the original color matrix to the hardware synthesizer 302 when the color inversion function is enabled.

[0093] The hardware synthesizer 302 is used to encode the original color matrix to obtain an encoded intermediate color matrix.

[0094] The hardware synthesizer 302 is used to transfer the intermediate color matrix to the platform display hardware processor.

[0095] The platform display hardware processor 303 is used to decode the intermediate color matrix to obtain a target color matrix.

[0096] The platform display hardware processor 303 is used to perform color inversion processing on the target color matrix.

[0097] In some other embodiments of the present application, embodiments of the present application disclose a terminal, such as Figure 4As shown, the terminal 400 may include: one or more processors 401; a memory 402; a display 403; one or more applications (not shown); and one or more computer programs 404. Each of the above components may be connected via one or more communication buses 405. The one or more computer programs 404 are stored in the memory 402 and configured to be executed by the one or more processors 401. The one or more computer programs 404 include instructions that can be used to execute the steps in Figure 1 the corresponding embodiments.

[0098] Through the description of the above embodiments, those skilled in the art can clearly understand that, for the convenience and brevity of description, only the division of the above functional modules is used as an example. In actual applications, the above functions can be allocated to different functional modules as needed, that is, the internal structure of the system can be divided into different functional modules to complete all or part of the functions described above. For the specific working processes of the above-described system, system, and unit, reference may be made to the corresponding processes in the foregoing method embodiments, and details are not described herein again.

[0099] In each embodiment of this application, the functional units may be integrated into one processing unit, or each unit may exist physically alone, or two or more units may be integrated into one unit. The above integrated unit may be implemented in the form of hardware or in the form of a software functional unit.

[0100] If the above integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it may be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application embodiment, in essence, or the part that contributes to the prior art, or all or part of this technical solution, may be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to enable a computer device (which may be a personal computer, a server, or a network device, etc.) or a processor to execute all or part of the steps of the methods described in each embodiment of this application. The foregoing storage medium includes: various media such as flash memory, mobile hard disk, read-only memory, random access memory, magnetic disk, or optical disk that can store program codes.

[0101] The above is only the specific implementation manner of this application embodiment, but the protection scope of this application embodiment is not limited thereto. Any changes or substitutions within the technical scope disclosed in this application embodiment should be covered by the protection scope of this application embodiment. Therefore, the protection scope of this application embodiment should be subject to the protection scope of the claims.

Claims

1. A color matrix processing method, applied to a terminal, characterized in that Including: When the color inversion function is enabled, the display system transmits the original color matrix to the hardware synthesizer; The hardware synthesizer encodes the original color matrix to obtain an encoded intermediate color matrix; The hardware synthesizer transmits the intermediate color matrix to the platform display hardware processor; The platform display hardware processor decodes the intermediate color matrix to obtain a target color matrix; The platform display hardware processor performs color inversion processing on the target color matrix.

2. The method according to claim 1, wherein Also including: The platform display hardware processor also receives a color adjustment matrix from the picture quality enhancement service; The platform display hardware processor performs color adjustment processing on the color adjustment matrix.

3. The method according to claim 2, wherein The platform display hardware processor performing color adjustment processing on the color adjustment matrix includes: The platform display hardware processor performs blue light eye protection processing or color temperature adjustment processing on the color adjustment matrix.

4. The method according to claim 2, wherein The hardware synthesizer encoding the original color matrix to obtain an encoded intermediate color matrix includes: Converting the original color matrix to an integer, and then combining the original color matrix in a pairwise combination manner to form the intermediate color matrix; Wherein, the intermediate color matrix includes a flag bit, and the flag bit is used to indicate whether the color matrix needs to be reconfigured.

5. The method according to claim 4, wherein The platform display hardware processor decoding the intermediate color matrix to obtain a target color matrix includes: The platform display hardware processor decodes the intermediate color matrix to obtain a candidate color matrix; If it is determined that the flag bit indicates that the color matrix needs to be reconfigured, then multiply the candidate color matrix by the color adjustment matrix from the picture quality enhancement service to obtain the target color matrix.

6. The method according to claim 1, wherein The format of the original color matrix is different from the format of the target color matrix.

7. A color matrix processing system, applied to a terminal, characterized in that Including a display system, a hardware synthesizer, and a platform display hardware processor, the display system and the platform display hardware processor are both communicatively connected to the hardware synthesizer; The display system is used to transmit the original color matrix to the hardware synthesizer when the color inversion function is enabled; The hardware synthesizer is used to encode the original color matrix to obtain an encoded intermediate color matrix; The hardware synthesizer is used to transmit the intermediate color matrix to the platform display hardware processor; The platform display hardware processor is used to decode the intermediate color matrix to obtain a target color matrix; The platform display hardware processor is used to perform color inversion processing on the target color matrix.

8. The system according to claim 7, wherein The platform display hardware processor is also used to receive a color adjustment matrix from the picture quality enhancement service; The platform display hardware processor is also used to perform color adjustment processing on the color adjustment matrix.

9. The system according to claim 8, wherein The platform display hardware processor performing color adjustment processing on the color adjustment matrix specifically includes: The platform display hardware processor performs blue light eye protection processing or color temperature adjustment processing on the color adjustment matrix.

10. The system according to claim 8, wherein The hardware synthesizer encoding the original color matrix to obtain an encoded intermediate color matrix specifically includes: Convert the original color matrix into an integer, and then combine the original color matrix into the intermediate color matrix in a pairwise combination manner; Among them, the intermediate color matrix includes a flag bit, and the flag bit is used to indicate whether the color matrix needs to be reconfigured.

11. The system according to claim 10, wherein The platform display hardware processor decodes the intermediate color matrix to obtain a target color matrix, specifically for: The platform display hardware processor decodes the intermediate color matrix to obtain a candidate color matrix; If it is determined that the flag bit indicates that the color matrix needs to be reconfigured, then multiply the candidate color matrix by the color adjustment matrix from the image quality enhancement service to obtain the target color matrix.

12. The system according to claim 7, wherein The format of the original color matrix is different from the format of the target color matrix.

13. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the color matrix processing method according to any one of claims 1 to 6.

14. A terminal, characterized in that, Including: A processor and a memory; The memory is used to store a computer program; The processor is used to execute the computer program stored in the memory, so that the terminal executes the color matrix processing method according to any one of claims 1 to 6.

Images