A mixed layer processing method and device

Abstract

The present application provides a mixed layer processing method and device, which relates to the field of image processing and is used to solve the problem of display errors in images obtained by mixed layer processing in the prior art. The method includes: obtaining a first layer, a second layer and a third layer, the first layer and the second layer have a first dynamic range, the third layer has a second dynamic range, the first layer is a transparent layer, and the second layer is a non-transparent layer; converting the dynamic range of the first layer into a second dynamic range, and determining a first target area in the second layer; merging the second layer, the third layer and the converted first layer to obtain a first image; converting the dynamic range of the second target area in the first image into the second dynamic range to obtain a second image, and the second target area is the corresponding area of ​​the first target area in the first image.

Description

Technical Field

[0001] The present application relates to the field of image processing, and in particular to a mixed layer processing method and device. Background Art

[0002] Dynamic range (DR) is used in many fields to represent the ratio of the maximum and minimum values ​​of each variable. In digital images, dynamic range represents the ratio between the maximum grayscale value and the minimum grayscale value within the displayable range of the image, that is, the number of grayscale levels from "brightest" to "darkest" in the image. The larger the dynamic range of an image, the richer the brightness levels it can represent, and the more realistic the visual effect of the image. Since the dynamic range of natural scenes in the real world is between 10 -3 to 10 6 The dynamic range is very large, so it is called high dynamic range (HDR). Compared with high dynamic range images, the dynamic range of ordinary images is low dynamic range (LDR), which is sometimes also called standard dynamic range (SDR).

[0003] In the prior art, there are usually scenarios where HDR images and SDR images are displayed in a mixed manner, such as an SDR image advertisement that pops up when a user uses a mobile phone to play an HDR video. In this scenario, the current solution does not distinguish between HDR videos and SDR images, but directly merges the images of multiple layers in the video, and then adaptively adjusts the resolution, hue, dynamic range, etc. in the image after obtaining a unified image, and then adapts the display according to the screen characteristics of the mobile phone. However, during the processing process, the above solution cannot distinguish between HDR images and SDR images, but processes the two as the same type of images, which will cause the final displayed image to have multiple problems such as color confusion and invisible text. Summary of the invention

[0004] The present application provides a mixed layer processing method and device, which are used to solve the problem of display errors in images obtained by mixed layer processing in the prior art.

[0005] To achieve the above objectives, the embodiments of the present application adopt the following technical solutions:

[0006] In a first aspect, a mixed layer processing method is provided, the method comprising: obtaining a first layer, a second layer and a third layer, wherein the first layer and the second layer have a first dynamic range, the third layer has a second dynamic range, the first layer is a transparent layer, and the second layer is a non-transparent layer; converting the dynamic range of the first layer into a second dynamic range, and determining a first target area in the second layer; merging the second layer, the third layer and the converted first layer to obtain a first image; converting the dynamic range of the second target area in the first image into the second dynamic range to obtain a second image, wherein the second target area is a corresponding area of ​​the first target area in the first image.

[0007] In the above technical solution, by converting the dynamic range of the first layer into the second dynamic range, and converting the dynamic range of the second target area in the first image into the second dynamic range after merging, it is ensured that the dynamic ranges of different image areas in the second image are consistent, so that the second image will not have problems such as color confusion and invisible text, thereby improving the performance and user experience of the image processing device in processing mixed layers; in addition, the above solution does not convert the dynamic range of each layer into the same dynamic range before merging multiple layers, but only converts the dynamic range of the first layer into the second dynamic range. After the layers are merged, the dynamic range of the second target area in the first image is converted into the second dynamic range to unify the image content. Compared with converting the dynamic range of each layer into the same dynamic range before merging multiple layers, it can greatly reduce the hardware cost while obtaining the same image quality.

[0008] In a possible implementation of the first aspect, the first dynamic range is a standard dynamic range SDR, and the second dynamic range is a high dynamic range HDR; or, the first dynamic range is HDR, and the second dynamic range is SDR. In the above possible implementations, when the dynamic range of the mixed layer includes SDR and HDR, the dynamic range of the final second image can be SDR or HDR, thereby improving the flexibility of mixed layer processing.

[0009] In a possible implementation of the first aspect, before obtaining the first layer, the second layer, and the third layer, the method further includes: determining that the first layer has a first dynamic range according to the dynamic range identifier of the first layer; determining that the second layer has a first dynamic range according to the dynamic range identifier of the second layer; and determining that the third layer has a second dynamic range according to the dynamic range identifier of the third layer. In the above possible implementation, the dynamic range of each layer can be simply and effectively determined according to the dynamic range identifier of each layer, thereby quickly distinguishing layers with different dynamic ranges, thereby improving the efficiency of mixed layer processing.

[0010] In a possible implementation of the first aspect, before obtaining the first layer, the second layer, and the third layer, the method further includes: determining that the first layer is a transparent layer according to the layer transparency of the first layer; and determining that the second layer is a non-transparent layer according to the layer transparency of the second layer. In the above possible implementation, each layer can be simply and effectively determined to be a transparent layer or a non-transparent layer according to the layer transparency of each layer, thereby quickly distinguishing transparent layers and non-transparent layers from layers with the first dynamic range, thereby improving the efficiency of mixed layer processing.

[0011] In a possible implementation of the first aspect, determining the first target area in the second layer includes: determining the first target area by identifying a region of interest (ROI). The above possible implementation can simply and effectively determine the first target area in the second layer, thereby improving the efficiency of mixed layer processing.

[0012] In a possible implementation of the first aspect, merging the second layer, the third layer, and the converted first layer to obtain a first image includes: performing weighted calculation on the converted pixel values ​​of the first layer, the second layer, and the third layer according to the layer transparency of the first layer, the layer transparency of the second layer, and the layer transparency of the third layer to obtain the first image. In the above possible implementation, the merged first image can include the image of each layer, and the images of different layers will not be affected by the merge, thereby improving the performance of mixed layer processing compared with the prior art.

[0013] In a possible implementation of the first aspect, the pixel value of the area outside the first target area in the second layer is 0. In the above possible implementation, the influence of the area outside the first target area in the second layer on other layers during the layer merging process can be avoided.

[0014] In a possible implementation of the first aspect, converting the dynamic range of the first layer into the second dynamic range includes: when the first dynamic range is SDR and the second dynamic range is HDR, converting the dynamic range of the first layer into HDR through inverse tone mapping; when the first dynamic range is HDR and the second dynamic range is SDR, converting the dynamic range of the first layer into SDR through tone mapping. In the above possible implementations, the dynamic range of the first layer can be converted into HDR or the dynamic range of the first layer can be converted into SDR according to different dynamic ranges, thereby improving the flexibility of mixed layer processing.

[0015] In a possible implementation of the first aspect, the dynamic range of the second target area in the first image is converted into the second dynamic range to obtain the second image, including: when the dynamic range of the second target area is SDR and the second dynamic range is HDR, the dynamic range of the second target area is converted into HDR by inverse tone mapping to obtain the second image; when the dynamic range of the second target area is HDR and the second dynamic range is SDR, the dynamic range of the second target area is converted into SDR by tone mapping to obtain the second image. In the above possible implementations, the dynamic range of the second target area can be converted into HDR or SDR according to different dynamic ranges, thereby improving the flexibility of mixed layer processing.

[0016] In a possible implementation of the first aspect, the method further includes: displaying the second image. In the above possible implementation, the displayed second image can be free from various display errors such as color confusion and invisible text.

[0017] In a second aspect, a hybrid layer processing system is provided, comprising: an image reading interface for acquiring multiple input layers to be processed; an image type detector for receiving the input layer in the image reading interface, and determining the dynamic range and transparent layer attributes of the input layer according to the dynamic range identifier and / or layer transparency of the input layer, wherein the transparent layer attributes include transparent layers and non-transparent layers; a first computing engine for receiving the first layer when the image type detector determines that the input layer is the first layer, and converting the dynamic range of the first layer into a target dynamic range, wherein the first layer is a transparent layer, and the dynamic range of the first layer is different from the target dynamic range; an image area calibrator for receiving the second layer when the image type detector determines that the input layer is the second layer. layer, and determining a first target area of ​​a second layer, wherein the second layer is a non-transparent layer, and the dynamic range of the second layer is different from the target dynamic range; a weighted calculator, used to receive the first layer processed by the first computing engine, the second layer determined by the image type detector, and the input layer as the third layer determined by the image type detector, and merge the first layer, the second layer and the third layer into a first image, and the third layer has a target dynamic range; a second computing engine, used to receive the first image processed by the weighted calculator and the first target area determined by the image area calibrator, and convert the dynamic range of the second target area in the first image into the target dynamic range to obtain a second image, wherein the second target area is the corresponding area of ​​the first target area in the first image.

[0018] In the above technical solution, by converting the dynamic range of the first layer into the second dynamic range, and converting the dynamic range of the second target area in the first image into the second dynamic range after merging, it is ensured that the dynamic ranges of different image areas in the second image are consistent, so that the second image will not have problems such as color confusion and invisible text, thereby improving the performance and user experience of the image processing device in processing mixed layers; in addition, the above solution does not convert the dynamic range of each layer into the same dynamic range before merging multiple layers, but only converts the dynamic range of the first layer into the second dynamic range. After the layers are merged, the dynamic range of the second target area in the first image is converted into the second dynamic range to unify the image content. Compared with converting the dynamic range of each layer into the same dynamic range before merging multiple layers, it can greatly reduce the hardware cost while obtaining the same image quality.

[0019] In a possible implementation of the second aspect, the weighted calculator is specifically used to perform weighted calculation on the pixel values ​​of the first layer processed by the first computing engine, the pixel values ​​of the second layer determined by the image type detector, and the pixel values ​​of the third layer determined by the image type detector according to the layer transparency of the first layer, the layer transparency of the second layer, and the layer transparency of the third layer to obtain a first image.

[0020] In a possible implementation manner of the second aspect, a pixel value of an area outside the first target area in the second layer is 0.

[0021] In a third aspect, a mixed layer processing device is provided, which includes: an acquisition unit, used to acquire a first layer, a second layer and a third layer, wherein the first layer and the second layer have a first dynamic range, the third layer has a second dynamic range, the first layer is a transparent layer, and the second layer is a non-transparent layer; a conversion unit, used to convert the dynamic range of the first layer into a second dynamic range; a determination unit, used to determine a first target area in the second layer; a merging unit, used to merge the second layer, the third layer and the converted first layer to obtain a first image; the conversion unit is also used to convert the dynamic range of the second target area in the first image into the second dynamic range to obtain a second image, wherein the second target area is a corresponding area of ​​the first target area in the first image.

[0022] In a possible implementation manner of the third aspect, the first dynamic range is a standard dynamic range SDR, and the second dynamic range is a high dynamic range HDR; or, the first dynamic range is HDR, and the second dynamic range is SDR.

[0023] In a possible implementation of the third aspect, the determination unit is also used to: determine that the first layer has a first dynamic range based on the dynamic range identifier of the first layer; determine that the second layer has a first dynamic range based on the dynamic range identifier of the second layer; and determine that the third layer has a second dynamic range based on the dynamic range identifier of the third layer.

[0024] In a possible implementation manner of the third aspect, the determination unit is further used to: determine that the first layer is a transparent layer according to the layer transparency of the first layer; and determine that the second layer is a non-transparent layer according to the layer transparency of the second layer.

[0025] In a possible implementation manner of the third aspect, the determining unit is further configured to: determine the first target region through region of interest ROI identification.

[0026] In a possible implementation of the third aspect, the merging unit is specifically used to perform weighted calculation on the converted pixel values ​​of the first layer, the second layer, and the third layer according to the layer transparency of the first layer, the layer transparency of the second layer, and the layer transparency of the third layer to obtain a first image.

[0027] In a possible implementation manner of the third aspect, a pixel value of an area outside the first target area in the second layer is 0.

[0028] In a possible implementation of the third aspect, the conversion unit is specifically used to: when the first dynamic range is SDR and the second dynamic range is HDR, convert the dynamic range of the first layer into HDR through inverse tone mapping; when the first dynamic range is HDR and the second dynamic range is SDR, convert the dynamic range of the first layer into SDR through tone mapping.

[0029] In a possible implementation of the third aspect, the conversion unit is specifically used to: when the dynamic range of the second target area is SDR and the second dynamic range is HDR, convert the dynamic range of the second target area into HDR through inverse tone mapping to obtain a second image; when the dynamic range of the second target area is HDR and the second dynamic range is SDR, convert the dynamic range of the second target area into SDR through tone mapping to obtain a second image.

[0030] In a possible implementation manner of the third aspect, the device further includes: a display unit, configured to display the second image.

[0031] Optionally, the present application also provides a mixed layer processing device, which includes: a memory, and a processor coupled to the memory, the memory stores instructions and data, and the processor runs the instructions in the memory so that the processor executes the mixed layer processing method provided by the above-mentioned first aspect or any possible implementation of the first aspect.

[0032] In a fourth aspect, a mixed layer processing method is provided, the method comprising: obtaining a first layer and a second layer, wherein the first layer has a first dynamic range, the second layer has a second dynamic range, and the first layer is a transparent layer; converting the dynamic range of the first layer into the second dynamic range; merging the second layer and the third layer with the converted first layer to obtain a first image.

[0033] In a possible implementation manner of the fourth aspect, the first dynamic range is SDR and the second dynamic range is HDR; or, the first dynamic range is HDR and the second dynamic range is SDR.

[0034] In a possible implementation of the fourth aspect, before obtaining the first layer and the second layer, it also includes: determining that the first layer has a first dynamic range according to the dynamic range identifier of the first layer; and determining that the second layer has a second dynamic range according to the dynamic range identifier of the second layer.

[0035] In a possible implementation manner of the fourth aspect, before acquiring the first layer and the second layer, the method further includes: determining that the first layer is a transparent layer according to the layer transparency of the first layer.

[0036] In a possible implementation of the fourth aspect, merging the second layer and the converted first layer includes: performing weighted calculation on pixel values ​​of the converted first layer and pixel values ​​of the second layer according to layer transparency of the first layer and layer transparency of the second layer to obtain a first image.

[0037] In a possible implementation of the fourth aspect, the dynamic range of the first layer is converted into the second dynamic range, including: when the first dynamic range is SDR and the second dynamic range is HDR, converting the dynamic range of the first layer into HDR through inverse tone mapping; when the first dynamic range is HDR and the second dynamic range is SDR, converting the dynamic range of the first layer into SDR through tone mapping.

[0038] In a possible implementation manner of the fourth aspect, the method further includes: displaying the first image.

[0039] In a fifth aspect, a hybrid layer processing device is provided, which can implement the function of the hybrid layer processing method provided by any possible implementation of the fourth aspect to the fourth aspect. The function can be implemented by hardware, or by hardware executing corresponding software. The hardware or software includes one or more units corresponding to the above functions. Exemplarily, the hybrid layer processing device may include an acquisition unit, a conversion unit, and a merging unit.

[0040] In a possible implementation of the fifth aspect, the structure of the mixed layer processing device includes a processor, a memory, a communication interface and a bus, the memory is used to store program code, and the processor, the memory and the communication interface are connected via the bus. When the program code is executed by the processor, the information transmission device executes the steps of the mixed layer processing method provided in any possible implementation of the fourth aspect to the fourth aspect.

[0041] In a sixth aspect, a mixed layer processing method is provided, the method comprising: obtaining a first layer and a second layer, wherein the first layer has a first dynamic range, the second layer has a second dynamic range, and the first layer is a non-transparent layer; determining a first target area in the first layer; merging the first layer and the second layer to obtain a first image; converting the dynamic range of the second target area in the first image into a second dynamic range to obtain a second image, wherein the second target area is a corresponding area of ​​the first target area in the first image.

[0042] In a possible implementation manner of the sixth aspect, the first dynamic range is SDR and the second dynamic range is HDR; or, the first dynamic range is HDR and the second dynamic range layer is SDR.

[0043] In a possible implementation of the sixth aspect, before obtaining the first layer, the second layer, and the third layer, it also includes: determining that the first layer has a first dynamic range according to the dynamic range identifier of the first layer; and determining that the second layer has a second dynamic range according to the dynamic range identifier of the second layer.

[0044] In a possible implementation manner of the sixth aspect, before obtaining the first layer, the second layer, and the third layer, it also includes: determining that the first layer is a non-transparent layer according to the layer transparency of the first layer.

[0045] In a possible implementation manner of the sixth aspect, determining the first target area in the first layer includes: determining the first target area through region of interest (ROI) identification.

[0046] In a possible implementation of the sixth aspect, merging a first layer and a second layer to obtain a first image includes: performing weighted calculation on pixel values ​​of the first layer and pixel values ​​of the second layer according to layer transparency of the first layer and layer transparency of the second layer to obtain the first image.

[0047] In a possible implementation manner of the sixth aspect, the pixel value of the area outside the first target area in the first layer is 0.

[0048] In a possible implementation of the sixth aspect, the dynamic range of the second target area in the first image is converted into a second dynamic range to obtain a second image, including: when the dynamic range of the second target area is SDR and the second dynamic range is HDR, converting the dynamic range of the second target area into HDR through inverse tone mapping to obtain the second image; when the dynamic range of the second target area is HDR and the second dynamic range is SDR, converting the dynamic range of the second target area into SDR through tone mapping to obtain the second image.

[0049] In a possible implementation manner of the sixth aspect, the method further includes: displaying a second image.

[0050] In the seventh aspect, a mixed layer processing device is provided, which can implement the function of the mixed layer processing method provided by any possible implementation method of the sixth aspect to the sixth aspect. The function can be implemented by hardware, or by hardware executing corresponding software. The hardware or software includes one or more units corresponding to the above functions. Exemplarily, the mixed layer processing device may include an acquisition unit, a conversion unit, and a merging unit.

[0051] In a possible implementation of the seventh aspect, the structure of the mixed layer processing device includes a processor, a memory, a communication interface and a bus, the memory is used to store program code, and the processor, the memory and the communication interface are connected via the bus. When the program code is executed by the processor, the information transmission device executes the steps of the mixed layer processing method provided in any possible implementation of the sixth aspect to the sixth aspect.

[0052] In another aspect of the present application, a computer-readable storage medium is provided, wherein instructions are stored in the computer-readable storage medium, and when the computer-readable storage medium is executed on a computer, the computer executes the mixed layer processing method provided by the first aspect or any possible implementation of the first aspect.

[0053] In another aspect of the present application, a computer-readable storage medium is provided, wherein instructions are stored in the computer-readable storage medium, and when the computer-readable storage medium is run on a computer, the computer executes the mixed layer processing method provided by the fourth aspect or any possible implementation of the fourth aspect.

[0054] In another aspect of the present application, a computer-readable storage medium is provided, wherein instructions are stored in the computer-readable storage medium, and when the computer-readable storage medium is run on a computer, the computer executes the mixed layer processing method provided by the sixth aspect or any possible implementation of the sixth aspect.

[0055] In another aspect of the present application, a computer program product is provided. When the computer program product is run on a computer, the computer is enabled to execute the mixed layer processing method provided by the first aspect or any possible implementation of the first aspect.

[0056] In another aspect of the present application, a computer program product is provided. When the computer program product is run on a computer, the computer is enabled to execute the mixed layer processing method provided by the fourth aspect or any possible implementation of the fourth aspect.

[0057] In another aspect of the present application, a computer program product is provided. When the computer program product runs on a computer, the computer is enabled to execute the mixed layer processing method provided by the sixth aspect or any possible implementation of the sixth aspect.

[0058] It can be understood that the system, device, computer storage medium or computer program product of any of the mixed layer processing methods provided above are used to execute the corresponding methods provided above. Therefore, the beneficial effects that can be achieved can refer to the beneficial effects in the corresponding methods provided above and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] Figure 1 A schematic diagram of the structure of an image processing device provided in an embodiment of the present application;

[0060] Figure 2 A flow chart of a mixed layer processing method provided in an embodiment of the present application Figure 1 ;

[0061] Figure 3 A curve diagram of a PQ photoelectric conversion function provided in an embodiment of the present application;

[0062] Figure 4 A graph of an HLG photoelectric conversion function provided in an embodiment of the present application;

[0063] Figure 5A graph of a SLG photoelectric conversion function provided in an embodiment of the present application;

[0064] Figure 6 A flow chart of a mixed layer processing method provided in an embodiment of the present application Figure 2 ;

[0065] Figure 7 A flow chart of a mixed layer processing method provided in an embodiment of the present application Figure 3 ;

[0066] Figure 8 A schematic diagram of layers with different dynamic ranges to be merged provided in an embodiment of the present application;

[0067] Fig. 9 A schematic diagram of a display error in an image after layer merging provided in an embodiment of the present application;

[0068] Fig.10 A schematic diagram of the hardware implementation structure of a hybrid layer processing system provided in an embodiment of the present application;

[0069] Fig.11 A schematic diagram of the structure of a hybrid layer processing device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0070] In the present application, "at least one" means one or more, and "more" means two or more. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone, where A and B can be singular or plural. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single items or plural items. For example, at least one of a, b or c can represent: a, b, c, ab, ac, bc or abc, where a, b and c can be single or multiple. The character " / " generally indicates that the objects associated before and after are in an "or" relationship. In addition, in the embodiments of the present application, the words "first", "second" and the like do not limit the quantity and execution order.

[0071] It should be noted that, in this application, words such as "exemplary" or "for example" are used to indicate examples, illustrations or descriptions. Any embodiment or design described as "exemplary" or "for example" in this application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary" or "for example" is intended to present related concepts in a specific way.

[0072] Figure 1A schematic diagram of the structure of an image processing device provided in an embodiment of the present application, the image processing device can be a mobile phone, a tablet computer, a computer, a laptop computer, a video camera, a camera, a wearable device, a vehicle-mounted device, or a terminal device, etc. For the convenience of description, the above-mentioned devices are collectively referred to as image processing devices in this application. The embodiment of the present application is described by taking the image processing device as a mobile phone as an example, and the mobile phone includes: a memory 101, a processor 102, a sensor component 103, a multimedia component 104, an audio component 105, and a power component 106, etc.

[0073] Combine the following Figure 1 A detailed introduction to the various components of the mobile phone:

[0074] The memory 101 can be used to store data, software programs and modules; it mainly includes a program storage area and a data storage area, wherein the program storage area can store an operating system and at least one application required for a function, such as a sound playback function, an image playback function, etc.; the data storage area can store data created according to the use of the mobile phone, such as audio data, image data, phone book, etc. In addition, the mobile phone may include a high-speed random access memory and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other volatile solid-state storage devices.

[0075] The processor 102 is the control center of the mobile phone. It uses various interfaces and lines to connect various parts of the entire device. By running or executing software programs and / or modules stored in the memory 101, and calling data stored in the memory 101, it executes various functions of the mobile phone and processes data, thereby monitoring the mobile phone as a whole. In some feasible embodiments, the processor 102 can be a single processor structure, a multi-processor structure, a single-threaded processor, and a multi-threaded processor; in some feasible embodiments, the processor 102 can include a central processing unit, a general processor, a digital signal processor, a digital signal processor, a microcontroller or a microprocessor. In addition, the processor 102 can further include other hardware circuits or accelerators, such as application-specific integrated circuits, field programmable gate arrays or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. It can implement or execute various exemplary logic blocks, modules, and circuits described in conjunction with the disclosure of this application. The processor 102 can also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of a digital signal processor and a microprocessor, etc.

[0076] The sensor component 103 includes one or more sensors for providing various aspects of status evaluation for the mobile phone. The sensor component 103 may include an optical sensor, such as a CMOS or CCD image sensor, for detecting the distance between an external object and the mobile phone, or used in imaging applications, i.e., as a component of a camera or a camera head. In addition, the sensor component 103 may also include an acceleration sensor, a gyroscope sensor, a magnetic sensor, a pressure sensor or a temperature sensor, and the acceleration / deceleration, orientation, opening / closing state of the mobile phone, the relative positioning of components, or the temperature change of the mobile phone, etc., can be detected by the sensor component 103.

[0077] The multimedia component 104 provides a screen of an output interface between the mobile phone and the user, and the screen can be a touch panel, and when the screen is a touch panel, the screen can be implemented as a touch screen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touch, slide and gestures on the touch panel. The touch sensor can not only sense the boundary of the touch or slide action, but also detect the duration and pressure associated with the touch or slide operation. In addition, the multimedia component 104 also includes at least one camera, for example, the multimedia component 104 includes a front camera and / or a rear camera. When the mobile phone is in an operating mode, such as a shooting mode or a video mode, the front camera and / or the rear camera can receive external multimedia data. Each front camera and the rear camera can be a fixed optical lens system or have a focal length and optical zoom capability.

[0078] The audio component 105 can provide an audio interface between the user and the mobile phone. For example, the audio component 105 can include an audio circuit, a speaker, and a microphone. The audio circuit can convert the received audio data into an electrical signal and transmit it to the speaker, which converts it into a sound signal for output; on the other hand, the microphone converts the collected sound signal into an electrical signal, which is received by the audio circuit and converted into audio data, and then outputs the audio data to be sent to, for example, another mobile phone, or outputs the audio data to the processor 102 for further processing.

[0079] The power supply component 106 is used to provide power to various components of the mobile phone. The power supply component 106 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power in the mobile phone.

[0080] Although not shown, the mobile phone may also include a wireless fidelity (WiFi) module, a Bluetooth module, etc., which will not be described in detail in the present embodiment. Those skilled in the art will understand that Figure 1The mobile phone structure shown in the figure does not constitute a limitation on the mobile phone, and may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently.

[0081] Figure 2 A flow chart of a mixed layer processing method provided in an embodiment of the present application, which can be applied to Figure 1 The image processing device shown, see Figure 2 , the method includes the following steps.

[0082] S201: Acquire a first layer, a second layer, and a third layer, wherein the first layer and the second layer have a first dynamic range, the third layer has a second dynamic range, the first layer is a transparent layer, and the second layer is a non-transparent layer.

[0083] The dynamic range can be used to characterize the ratio between the maximum brightness and the minimum brightness within the displayable range of the image, that is, the number of gray levels from the "brightest" to the "darkest" of the image, and its unit is candela per square meter (cd / m 2 ), which can also be expressed as nits.

[0084] In the embodiment of the present application, the first dynamic range may be a standard dynamic range (SDR) and the second dynamic range may be a high dynamic range (HDR); or the first dynamic range may be HDR and the second dynamic range may be SDR. The mixed layer may include at least one SDR layer and at least one HDR layer. The SDR layer may be divided into a transparent layer and a non-transparent layer, and the HDR layer may also be divided into a transparent layer and a non-transparent layer.

[0085] It should be noted that SDR here can refer to the dynamic range of ordinary images, for example, the dynamic range of images taken by ordinary cameras. Each color channel in SDR images can be represented by 256 levels of grayscale. HDR here can refer to the dynamic range of natural scenes in the real world. HDR images are a type of image that can represent a wide range of brightness changes in actual scenes, and can better represent the optical properties of bright and dark areas in scenes. The range of pixel values ​​to be represented by HDR images is usually large, sometimes reaching hundreds of thousands or even millions. Compared with SDR images, HDR images require more data bits per color channel. HDR images can be synthesized using multiple SDR images with different exposure times, or they can be obtained by shooting with HDR professional shooting equipment. HDR images can better reflect the visual effects in real environments.

[0086] Optionally, before acquiring the first layer, the second layer, and the third layer, the method may further include: determining that the first layer has a first dynamic range according to the dynamic range identifier of the first layer; determining that the second layer has a first dynamic range according to the dynamic range identifier of the second layer; and determining that the third layer has a second dynamic range according to the dynamic range identifier of the third layer. Optionally, the method may further include: determining that the first layer is a transparent layer according to the layer transparency of the first layer; and determining that the second layer is a non-transparent layer according to the layer transparency of the second layer.

[0087] Among them, each layer may correspond to multiple layer parameters, and the multiple layer parameters may include a dynamic range identifier and layer transparency. The dynamic range identifier can be used to distinguish between SDR layers and HDR layers, and the layer transparency can be used to distinguish between transparent layers and non-transparent layers. Optionally, the dynamic range identifier can be a photoelectric conversion function. In practical applications, the photoelectric conversion functions of SDR layers and HDR layers are different, so SDR layers and HDR layers can be distinguished by the photoelectric conversion functions of different layers. For the relevant description of the photoelectric conversion functions of different layers, please refer to the description below.

[0088] Exemplarily, the photoelectric conversion function of the SDR layer is a gamma photoelectric conversion function, and the photoelectric conversion function of the HDR layer can be any one of a perceptual quantizer (PQ) photoelectric conversion function, a hybrid log-gamma (HLG) photoelectric conversion function, and a scene luminance fidelity (SLF) photoelectric conversion function. Then, it can be determined whether it is an SDR layer or an HDR layer according to the type of the photoelectric conversion function of each layer in the mixed multiple layers.

[0089] Exemplarily, transparent layers and non-transparent layers are distinguished by the transparency of the image. Each pixel of the layer corresponds to an alphy value to characterize the transparency of the pixel. The value range of the alphy value can be [0, 1]. When the alphy value is equal to 1, it indicates non-transparency. When the alphy value is not equal to 1, it indicates transparency. Different alpht values ​​between 0 and 1 correspond to different degrees of transparency. When the alphy values ​​of all pixels of the layer are 1, it indicates that the layer is a non-transparent layer. In other cases, it indicates that the layer is a transparent layer. It should be understood that different transparent layers can also have different transparencies.

[0090] Exemplarily, the first dynamic range layer is an SDR layer, and the second dynamic range layer is an HDR layer. It is assumed that the mixed multiple layers include 9 layers, and the 9 layers include 2 transparent SDR layers, 1 non-transparent SDR layer and 6 HDR layers.

[0091] S202: Convert the dynamic range of the first layer into a second dynamic range, and determine a first target area in the second layer.

[0092] Among them, when the first dynamic range is SDR and the second dynamic range is HDR, the dynamic range of the first layer can be converted to HDR by inverse tone mapping. The inverse tone mapping here may refer to the process of mapping a transparent SDR layer to a transparent HDR layer. The detailed description of inverse tone mapping is not specifically limited in the embodiment of the present application. For example, the mixed layer includes 9 layers and represents layers 1 to 9 respectively. If layers 1 to 3 are SDR layers, layers 1 and 2 are transparent layers, layer 3 is a non-transparent layer, and layers 4 to 9 are HDR layers, then the dynamic range of layers 1 and 2 can be converted to HDR by inverse tone mapping, that is, layers 1 and 2 are converted into HDR layers.

[0093] When the first dynamic range is HDR and the second dynamic range is SDR, the dynamic range of the first layer can be converted to SDR through tone mapping. The tone mapping here may refer to the process of mapping a transparent HDR layer to a transparent SDR layer. The detailed description of tone mapping is not specifically limited in the embodiments of the present application. For example, the mixed layer includes 9 layers and represents layers 1 to 9 respectively. If layers 1 to 3 are HDR layers, layers 1 and 2 are transparent layers, layer 3 is a non-transparent layer, and layers 4 to 9 are SDR layers, the dynamic range of layers 1 and 2 can be converted to SDR through tone mapping, that is, layers 1 and 2 are converted into SDR layers.

[0094] In addition, determining the first target area in the second layer may specifically include: identifying the first target area in the second layer according to a region of interest (ROI) recognition technology. The region of interest ROI here may refer to an area to be processed that is outlined in a box, circle, ellipse, irregular polygon, etc. from the processed image, and in the embodiment of the present application, it may specifically refer to an area to be processed that is outlined in a box, circle, ellipse, irregular polygon, etc. from the second layer. Optionally, the first target area identified by the ROI recognition technology may be represented by its position information in the second layer.

[0095] It should be noted that the specific process of identifying the first target area in the second layer by using the ROI identification technology can refer to the detailed description of the ROI identification technology in the prior art, and the embodiment of the present application does not make any specific limitation on this.

[0096] S203: Merge the second layer, the third layer, and the converted first layer to obtain a first image.

[0097] Among them, merging the second layer, the third layer and the converted first layer may include: performing weighted calculation on the pixel values ​​of the converted first layer, the pixel values ​​of the second layer and the pixel values ​​of the third layer according to the layer transparency of the first layer, the layer transparency of the second layer and the layer transparency of the third layer to obtain the first image.

[0098] It should be understood that, generally, the conversion of the dynamic range does not affect the layer transparency, so the layer transparency of the first layer is the same as the layer transparency of the first layer after the conversion.

[0099] Specifically, each layer may include multiple pixel values, each pixel value corresponds to a layer transparency, and for pixel values ​​at the same position in different layers, the sum of the layer transparency corresponding to the pixel values ​​in the different layers is 1. Assuming that each layer includes m×n pixels, the pixel value of the pixel point (i, j) of the first layer after conversion is represented by A(i, j), and the corresponding layer transparency is represented by X(i, j), the pixel value of the pixel point (i, j) of the second layer is represented by B(i, j), and the corresponding layer transparency is represented by Y(i, j), the pixel value of the pixel point (i, j) of the third layer is represented by C(i, j), and the corresponding layer transparency is represented by Z(i, j), where the value range of i is 1, 2, ..., m, and the value range of j is 1, 2, ..., n, then the pixel value D(i, j) of the pixel point (i, j) in the merged first image can be represented by the following formula (0):

[0100] D(i,j)=A(i,j)×X(i,j)+B(i,j)×Y(i,j)+C(i,j)×Z(i,j) (0)

[0101] Among them, for any pixel point (i, j), X(i, j)+Y(i, j)+Z(i, j)=1 is satisfied, that is, the sum of the layer transparency corresponding to the pixel values ​​at the same position in different layers is 1.

[0102] According to the above formula, we can further understand the relationship between the transparency corresponding to the pixel, the pixel value of the pixel and the visually perceived transparency. Specifically, as mentioned above, each pixel corresponds to a transparency, and the layer with pixels whose transparency is not 1 is called a transparent layer. When merging layers, the transparency corresponding to each pixel participates in the weighted calculation of the pixels of each layer as its weight. Among them, for a certain pixel, when the value of the pixel is 0, its product with the transparency value is also 0. At this time, people visually believe that the pixel is transparent, that is, the pixel value of the pixel is not reflected in the merged layer; when the transparency value is 0, its product with the pixel value is also 0. At this time, people also visually believe that the pixel is transparent.

[0103] It should be noted that the above only takes three layers as an example to illustrate the process of merging the second layer, the third layer and the converted first layer. In actual application, two layers, four layers or other different numbers of layers can also be merged in the above manner, and the embodiment of the present application does not impose any specific restrictions on this.

[0104] Optionally, the pixel value of the area outside the first target area in the second layer is 0. Since the second layer is a non-transparent layer, that is, the layer transparency of the second layer is 1, at this time, the pixel value of the area outside the first target area in the second layer is 0, then when the layers are merged in the above manner, the product of the pixel value 0 of the area outside the first target area in the second layer and its layer transparency is 0, so that the merged first image only includes the first target area in the second layer, and does not include the area outside the first target area in the second layer. In other words, merging the second layer, the third layer and the converted first layer to obtain the first image can also be understood as: merging the third layer and the converted first layer to obtain the third image, and replacing the image of the corresponding area of ​​the third image with the image of the first target area to obtain the first image.

[0105] S204: Convert the dynamic range of the second target area in the first image into a second dynamic range to obtain a second image, where the second target area is a corresponding area of ​​the first target area in the first image.

[0106] Among them, the second target area in the first image comes from a non-transparent layer and has a first dynamic range, and the other image areas in the first image are all from a layer with a second dynamic range, that is, the dynamic ranges of different image areas in the first image are inconsistent. Therefore, the dynamic range of the second target area in the first image can be converted into the second dynamic range, so that the dynamic ranges of different image areas in the second image are consistent. Optionally, the position of the second target area in the first image can be determined by the position information of the first target area in the second layer obtained in S202 above.

[0107] Specifically, when the dynamic range of the second target area is SDR and the second dynamic range is HDR, the dynamic range of the second target area is converted to HDR through inverse tone mapping to obtain a second image; when the dynamic range of the second target area is HDR and the second dynamic range is SDR, the dynamic range of the second target area is converted to SDR through tone mapping to obtain a second image.

[0108] It should be noted that the inverse tone mapping and tone mapping here are consistent with the inverse tone mapping and tone mapping in the above S202. For details, please refer to the relevant description in the above S202, and the embodiments of the present application will not be repeated here.

[0109] Optionally, when the shape of the first target area is irregular, the dynamic range of the first target area can be converted into a second dynamic range before merging multiple layers; when the shape of the first target area is regular, the dynamic range of the second target area can be converted into the second dynamic range in S204, and the second target area is the corresponding area of ​​the first target area in the first image.

[0110] Further, after the dynamic range of the second target area in the first image is converted to the second dynamic range to obtain the second image, the second image can also be displayed. Since the dynamic ranges of different image areas in the second image are consistent, there will be no problems such as color confusion and invisible text in the second image that is finally displayed, thereby improving the performance of the image processing device in processing mixed layers, and also improving the user experience. In addition, when unifying multiple layers into the same dynamic range, the embodiment of the present application does not convert the dynamic range of each layer in the mixed layer into the same dynamic range before merging multiple layers, but only converts the dynamic range of the first layer (i.e., the transparent layer, and having the first dynamic range) into the second dynamic range. After the layers are merged, the dynamic range of the second target area in the first image (i.e., the area corresponding to the first target area in the second layer, the second layer is a non-transparent layer, and has the first dynamic range) is converted into the second dynamic range to unify the image content. In this way, compared with converting the dynamic range of each layer in the mixed layer into the same dynamic range before merging multiple layers, the hardware cost can be greatly reduced under the premise of obtaining the same image quality.

[0111] Exemplarily, in order to better distinguish the difference between SDR images and HDR images, the photoelectric conversion functions of SDR images and HDR images are respectively illustrated below.

[0112] The photoelectric conversion function of SDR images is usually the Gamma function. The ITU-R Recommendation BT.1886 standard defines a photoelectric transfer function based on the "Gamma" function, as shown in the following formula (1). In formula (1), L represents the optical signal and V represents the electrical signal. The image quantized into 8 bits as shown in formula (1) is the SDR image. The SDR image and the above transfer function are similar in appearance to those of traditional display devices (with an illumination of 100 cd / m 2 Good performance on the left and right.

[0113]

[0114] With the upgrade of display devices, the illumination range of display devices continues to increase. Existing consumer-level HDR displays can reach 600cd / m 2 , the illumination information of high-end HDR display can reach 2000cd / m 2 There are usually three types of photoelectric conversion functions for HDR images: PQ, HLG and SLF. These three photoelectric conversion functions are conversion functions specified by the AVS standard. The following introduces the three curves respectively.

[0115] The PQ photoelectric transfer function proposes a perceptual quantization transfer function based on the brightness perception model of the human eye, as shown in the following formula (2). Plotting the perceptual quantization transfer function of formula (2) on the image yields Figure 3 The curve shown.

[0116]

[0117] in:

[0118]

[0119]

[0120]

[0121]

[0122]

[0123]

[0124] It should be noted that, in the above formula, L represents an optical signal, V represents an electrical signal, and m1, m2, c1, c2 and c3 are parameters in the formula.

[0125] The HLG transfer function is different from the traditional Gamma transfer function. It is improved on the basis of the traditional Gamma curve. The traditional Gamma is applied in the low segment and the log curve is added in the high segment. The Hybrid Log-Gamma transfer function is proposed. It is shown in the following formula (3). In formula (3), L represents the optical signal, V represents the electrical signal, and a and b are the parameters of formula (3). Plotting the HLG transfer function of formula (3) in the image gives Figure 4 The curve shown.

[0126]

[0127] The SLF photoelectric conversion function first normalizes the input light signal. Here, the maximum brightness is defined as 10000cd / m 2 , and then converted by the conversion function shown in the following formula (4), where L represents the optical signal, V represents the electrical signal, and p, m, a and b are the parameters of formula (4).

[0128]

[0129] The parameter values ​​in formula (4) are selected as follows: p = 2.2, m = 0.14, a = 1.12672, b = -0.12672, and the corresponding curve shape is shown in the figure Figure 5 shown. Figure 5 The curve shown can be obtained based on the brightness characteristics of the human eye and in combination with the brightness distribution of the existing HDR image sequence scene.

[0130] It should be understood that the above embodiment introduces a processing method when a transparent layer with a first dynamic range, an opaque layer with a first dynamic range, and a layer with a second dynamic range exist in a mixed layer at the same time. In some feasible implementations, only a transparent layer with a first dynamic range and a layer with a second dynamic range exist in the mixed layer at the same time, or only an opaque layer with a first dynamic range and a layer with a second dynamic range exist at the same time. Figure 6 and Figure 7 And the corresponding embodiments will be introduced respectively.

[0131] Figure 6 A flow chart of another mixed layer processing method provided in an embodiment of the present application, which can be applied to Figure 1 The image processing device shown, see Figure 6 , the method includes the following steps.

[0132] S601: Acquire a first layer and a second layer, wherein the first layer has a first dynamic range, the second layer has a second dynamic range, and the first layer is a transparent layer.

[0133] The first dynamic range may be SDR and the second dynamic range may be HDR; or the first dynamic range may be HDR and the second dynamic range may be SDR. The mixed layer may include at least one SDR layer and at least one HDR layer. The SDR layer may be divided into a transparent layer and a non-transparent layer, and the HDR layer may also be divided into a transparent layer and a non-transparent layer.

[0134] When the mixed layer only includes a transparent layer with a first dynamic range and a layer with a second dynamic range, the first layer and the second layer can be obtained from the mixed layer. The specific acquisition process is consistent with the process in S201 above. For details, please refer to the relevant description, and the embodiments of the present application will not be repeated here.

[0135] S602: Convert the dynamic range of the first layer into the second dynamic range. The specific process of converting the dynamic range of the first layer into the second dynamic range is consistent with the process in S202 above, and the details can be referred to the relevant description, which will not be repeated in this embodiment of the present application.

[0136] S603: Merge the second layer and the converted first layer to obtain a first image. The specific description of merging the first layer and the second layer to obtain the first image is consistent with the description of merging the second layer, the third layer and the converted first layer to obtain the first image in S203 above. For details, please refer to the description in S203 above, and the present embodiment will not be repeated here.

[0137] Furthermore, after obtaining the first image, the first image can be displayed. Since the first image is obtained by merging multiple layers with the second dynamic range, the dynamic ranges thereof are consistent, so the first image finally displayed will not have problems such as color confusion and invisible text, thereby improving the performance of the image processing device in processing mixed layers and also improving the user experience.

[0138] Figure 7 A flow chart of another mixed layer processing method provided in an embodiment of the present application, which can be applied to Figure 1 The image processing device shown, see Figure 7 , the method includes the following steps.

[0139] S701: Acquire a first layer and a second layer, wherein the first layer has a first dynamic range, the second layer has a second dynamic range, and the first layer is a non-transparent layer.

[0140] The first dynamic range may be SDR and the second dynamic range may be HDR; or the first dynamic range may be HDR and the second dynamic range may be SDR. The mixed layer may include at least one SDR layer and at least one HDR layer. The SDR layer may be divided into a transparent layer and a non-transparent layer, and the HDR layer may also be divided into a transparent layer and a non-transparent layer.

[0141] When the mixed layer only includes a non-transparent layer with a first dynamic range and a layer with a second dynamic range, the first layer and the second layer can be obtained from the mixed layer. The specific acquisition process is consistent with the process in S201 above. For details, please refer to the relevant description, and the embodiments of the present application will not be repeated here.

[0142] S702: Determine the first target area in the first layer. The specific description of determining the first target area in the first layer is consistent with the description of determining the first target area in the second layer in S202 above. For details, please refer to the description in S202 above, and the present embodiment will not be repeated here.

[0143] S703: Merge the first layer and the second layer to obtain a first image. The specific description of merging the first layer and the second layer to obtain the first image is consistent with the description of merging the second layer, the third layer and the converted first layer to obtain the first image in S203 above. For details, please refer to the description in S203 above, and the present embodiment of the application will not be repeated here.

[0144] S704: Convert the dynamic range of the second target area in the first image into a second dynamic range to obtain a second image, where the second target area is a corresponding area of ​​the first target area in the first image.

[0145] It should be noted that the specific process of converting the dynamic range of the second target area in the first image into the second dynamic range is consistent with the process in the above S204. For details, please refer to the relevant description, and the embodiment of the present application will not be repeated here.

[0146] Furthermore, after the dynamic range of the second target area in the first image is converted into the second dynamic range to obtain the second image, the second image can also be displayed. Since the dynamic ranges of different image areas in the second image are consistent, the second image finally displayed will not have problems such as color confusion and invisible text, thereby improving the performance of the image processing device in processing mixed layers and also improving the user experience.

[0147] For ease of understanding, the method provided in an embodiment of the present application is illustrated below by taking an example in which a mixed layer includes an SDR layer and an HDR layer, and the first dynamic range is SDR and the second dynamic range is HDR.

[0148] When the mixed layer includes one or more SDR layers and one or more HDR layers, such as Figure 8 As shown in the figure, for example, it includes an HDR layer (including a layer of a giraffe), an SDR transparent layer (including a layer of thumbnails), and an SDR non-transparent layer (including a layer of a play bar). If the processing method of the prior art is used, without distinguishing between HDR layers and SDR layers, but directly merging multiple layers, it will cause erroneous display due to the inconsistency of image content. For example, the final image display will appear Fig. 9 The error display shown, etc.

[0149] To solve the above problems, the embodiment of the present application converts a transparent SDR layer into a transparent HDR layer, and converts the SDR image area in the non-transparent SDR layer into an HDR image area (that is, converts the SDR image area in the first image obtained after merging into an HDR image area), so that the images in the second image obtained after the conversion are unified, thereby solving the problems existing in the prior art.

[0150] It should be noted that when unifying multiple mixed layers into the same layer (for example, converting an SDR layer into an HDR layer, and converting an SDR layer into a transparent HDR layer), in theory, tone mapping or inverse tone mapping technology can be used for each of the multiple mixed layers to unify the multiple layers into SDR layers or HDR layers. However, in this way, as many conversion units for implementing tone mapping or inverse tone mapping technology need to be added as many layers as there are. Usually, the number of multiple mixed layers is greater than 9, so the hardware cost overhead is relatively large. In the embodiment of the present application, the first dynamic range image area of ​​the transparent first dynamic range layer and the non-transparent first dynamic range layer are distinguished, and the transparent first dynamic range layer is converted into the second dynamic range layer (usually, the number of transparent first dynamic range layers is less than or equal to 3), and then the first dynamic range image area in the image after the layers are merged is passed through a conversion unit (a module for implementing inverse tone mapping or tone mapping technology) to unify the image content and obtain the final image. Compared with the method of adding a conversion unit to each layer, the hardware cost is greatly reduced while obtaining the same image quality.

[0151] The above mainly introduces the mixed layer processing method provided by the embodiment of the present application from the perspective of the image processing device. It can be understood that in order to realize the above functions, the image processing device includes hardware structures and / or software modules corresponding to the execution of each function. Those skilled in the art should easily realize that, in combination with the structures and algorithm steps of each example described in the embodiments disclosed in this article, the present application can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is executed in the form of hardware or computer software driving hardware depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of this application.

[0152] Fig.10 A possible hardware implementation structure diagram of the hybrid layer processing system involved in the above-mentioned embodiment is shown, and the hybrid layer processing system includes: an image reading interface 1001, used to obtain multiple input layers to be processed; an image type detector 1002, used to receive the input layer in the image reading interface 1001, and determine the dynamic range and transparent layer properties of the input layer according to the dynamic range identifier and / or layer transparency of the input layer, and the transparent layer properties include transparent layers and non-transparent layers; a first computing engine 1003, used to receive the first layer when the image type detector 1002 determines that the input layer is the first layer, and convert the dynamic range of the first layer into a target dynamic range, wherein the first layer is a transparent layer, and the dynamic range of the first layer is different from the target dynamic range; an image area calibrator 1004, used to detect the input layer when the image type detector 1002 determines that the input layer is the first layer. When the input layer is the second layer, the second layer is received and the first target area of ​​the second layer is determined, wherein the second layer is a non-transparent layer and the dynamic range of the second layer is different from the target dynamic range; the weighted calculator 1005 is used to receive the first layer processed by the first computing engine 1003, the second layer determined by the image type detector 1002, and the input layer as the third layer determined by the image type detector 1002, and merge the first layer, the second layer and the third layer into a first image, and the third layer has a target dynamic range; the second computing engine 1006 is used to receive the first image processed by the weighted calculator 1005 and the first target area determined by the image area calibrator 1004, and convert the dynamic range of the second target area in the first image into the target dynamic range to obtain a second image, wherein the second target area is the corresponding area of ​​the first target area in the first image.

[0153] In a possible embodiment, the weighted calculator 1005 is specifically used to perform weighted calculation on the pixel values ​​of the first layer processed by the first computing engine 1003, the pixel values ​​of the second layer determined by the image type detector 1002, and the pixel values ​​of the third layer determined by the image type detector 1003 according to the layer transparency of the first layer, the layer transparency of the second layer, and the layer transparency of the third layer to obtain a first image.

[0154] The various components of the above-mentioned hybrid layer processing system provided in the embodiment of the present application are respectively used to implement the functions of the corresponding steps of the above-mentioned hybrid layer processing method. Since each step has been described in detail in the above-mentioned hybrid layer processing method embodiment, it will not be repeated here.

[0155] The hardware structure of the hybrid layer processing system provided in the embodiment of the present application can be implemented by one or more processors such as one or more digital signal processors (DSPs), general-purpose microprocessors, application-specific integrated circuits (ASICs), field programmable logic arrays (FPGAs), or other equivalent integrated or discrete logic circuits. In addition, in some aspects, the functions described by the various illustrative logic boxes, modules, and steps described herein can be provided in dedicated hardware and / or software modules configured for image processing, or incorporated into a combined image processor. Moreover, the technology can be fully implemented in one or more circuits or logic elements.

[0156] The embodiment of the present application can divide the functional modules according to the hybrid layer processing system and hybrid layer processing device corresponding to the above method example. For example, each functional module can be divided according to each function, or two or more functions can be integrated into one processing module. The above integrated module can be implemented in the form of hardware or in the form of software functional modules. It should be noted that the division of modules in the embodiment of the present application is schematic and is only a logical functional division. There may be other division methods in actual implementation.

[0157] In the case of dividing each functional module into corresponding functional modules, Fig.11Another possible structural diagram of the mixed layer processing device involved in the above embodiment is shown. The device can be an image processing device or a chip or system on chip in the image processing device, and can also be a circuit, module or unit in the image processing device for implementing the above method embodiment. The device includes: an acquisition unit 1101, a conversion unit 1102 and a merging unit 1103. Among them, the acquisition unit 1101 is used to support the device to perform S201, S601 or S701 in the method embodiment; the conversion unit 1102 is used to support the device to perform S202, S204, S602, S702 or S704 in the method embodiment; the merging unit 1103 is used to support the device to perform S203, S603, or S703 in the method embodiment. Further, the device also includes: a display unit 1104; the display unit 1104 is used to support the device to perform the steps of displaying the first image or displaying the image in the method embodiment.

[0158] The techniques of the present application may be implemented in a variety of devices or apparatuses, including a wireless handset, an integrated circuit (IC), or a set of ICs (e.g., a chipset). Various components, modules, or units are described in the present application to emphasize the functional aspects of the device for performing the disclosed techniques, but they do not necessarily need to be implemented by different hardware units. In fact, as described above, the various units may be combined in a hardware unit for image processing in conjunction with appropriate software and / or firmware, or provided by interoperating hardware units (including one or more processors as described above).

[0159] A hybrid layer processing device in an embodiment of the present application is described above from the perspective of a modular functional entity. A hybrid layer processing device in an embodiment of the present application is described below from the perspective of hardware processing.

[0160] The embodiment of the present application also provides a mixed layer processing device. The structure of the mixed layer processing device can be as follows: Figure 1 In the embodiment of the present application, the processor 102 is configured to process one or more steps S201-S204, S601-S603, and S701-S704 in the above method embodiment, and / or other technical processes described herein.

[0161] In some feasible embodiments, the memory 101 may be used to store a mixed layer, a first layer, a second layer, a third layer, a first image and / or a second image, etc. The display panel in the multimedia component 104 may be used to display a first image or a second image, etc.

[0162] The various components of the above-mentioned image processing device provided in the embodiment of the present application are respectively used to implement the functions of the corresponding steps of the above-mentioned mixed layer processing method. Since each step has been described in detail in the above-mentioned mixed layer processing method embodiment, it will not be repeated here.

[0163] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of the modules or units is only a logical function division, and there may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed.

[0164] The units described as separate components may or may not be physically separated, and the components shown as units may be one physical unit or multiple physical units, that is, they may be located in one place or distributed in multiple different places. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0165] The embodiment of the present application also provides a computer-readable storage medium, which stores instructions, and when the instructions are executed on a device (for example, the device can be a single-chip microcomputer, a chip, a computer or a processor, etc.), the device executes one or more steps S201-S204, S601-S603, and S701-S704 in the above method embodiment, and / or other technical processes described herein. If the various component modules of the above image processing device are implemented in the form of software functional units and sold or used as independent products, they can be stored in the computer-readable storage medium.

[0166] Based on this understanding, the embodiments of the present application also provide a computer program product comprising instructions. The essence of the technical solution of the present application or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes a number of instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) or a processor therein to execute all or part of the steps of the methods described in the embodiments of the present application.

[0167] Finally, it should be noted that the above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions within the technical scope disclosed in the present application should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A mixed layer processing method, characterized in that: include: Acquire a first layer, a second layer, and a third layer, wherein the first layer and the second layer have a first dynamic range, the third layer has a second dynamic range, the first layer is a transparent layer, and the second layer is a non-transparent layer; Converting the dynamic range of the first layer into the second dynamic range, and determining a first target area in the second layer; Merging the second layer, the third layer and the converted first layer to obtain a first image; The dynamic range of a second target area in the first image is converted into the second dynamic range to obtain a second image, wherein the second target area is a corresponding area of ​​the first target area in the first image.

2. The method according to claim 1, characterized in that The first dynamic range is a standard dynamic range SDR, and the second dynamic range is a high dynamic range HDR; or, the first dynamic range is HDR, and the second dynamic range is SDR.

3. The method according to claim 1 or 2, characterized in that: Before obtaining the first layer, the second layer and the third layer, the method further includes: Determining, according to the dynamic range identifier of the first layer, that the first layer has the first dynamic range; Determining, according to the dynamic range identifier of the second layer, that the second layer has the first dynamic range; According to the dynamic range identifier of the third layer, it is determined that the third layer has the second dynamic range.

4. The method according to claim 1, characterized in that Before obtaining the first layer, the second layer and the third layer, the method further includes: According to the layer transparency of the first layer, determining that the first layer is the transparent layer; According to the layer transparency of the second layer, the second layer is determined to be the non-transparent layer.

5. The method according to claim 1, characterized in that The determining the first target area in the second layer includes: The first target area is determined by identifying a region of interest (ROI).

6. The method according to claim 4 or 5, characterized in that: The merging the second layer, the third layer and the converted first layer to obtain a first image includes: According to the layer transparency of the first layer, the layer transparency of the second layer and the layer transparency of the third layer, the pixel values ​​of the converted first layer, the pixel values ​​of the second layer and the pixel values ​​of the third layer are weightedly calculated to obtain the first image.

7. The method according to claim 6, characterized in that The pixel value of the area outside the first target area in the second layer is 0.

8. The method according to claim 1, characterized in that The converting the dynamic range of the first layer into the second dynamic range comprises: When the first dynamic range is SDR and the second dynamic range is HDR, converting the dynamic range of the first layer to HDR by inverse tone mapping; When the first dynamic range is HDR and the second dynamic range is SDR, the dynamic range of the first layer is converted into SDR through tone mapping.

9. The method according to claim 1, characterized in that: The step of converting the dynamic range of the second target area in the first image into the second dynamic range to obtain the second image includes: When the dynamic range of the second target area is SDR and the second dynamic range is HDR, converting the dynamic range of the second target area into HDR by inverse tone mapping to obtain a second image; When the dynamic range of the second target area is HDR and the second dynamic range is SDR, the dynamic range of the second target area is converted into SDR through tone mapping to obtain a second image.

10. The method according to claim 1, characterized in that Also includes: The second image is displayed.

11. A mixed layer processing system, characterized in that: include: Image reading interface, used to obtain multiple input layers to be processed; An image type detector, configured to receive the input layer in the image reading interface, and determine the dynamic range and transparent layer properties of the input layer according to the dynamic range identifier and / or layer transparency of the input layer, wherein the transparent layer properties include a transparent layer and a non-transparent layer; a first computing engine, configured to receive a first layer when the image type detector determines that the input layer is a first layer, and convert a dynamic range of the first layer into a target dynamic range, wherein the first layer is the transparent layer, and the dynamic range of the first layer is different from the target dynamic range; an image region calibrator, configured to receive the second layer when the image type detector determines that the input layer is the second layer, and determine a first target region of the second layer, wherein the second layer is the non-transparent layer, and a dynamic range of the second layer is different from the target dynamic range; a weighted calculator, configured to receive the first layer processed by the first computing engine, the second layer determined by the image type detector, and an input layer as a third layer determined by the image type detector, and merge the first layer, the second layer, and the third layer into a first image, wherein the third layer has the target dynamic range; A second computing engine is used to receive the first image processed by the weighted calculator and the first target area determined by the image area calibrator, and convert the dynamic range of the second target area in the first image into the target dynamic range to obtain a second image, wherein the second target area is the corresponding area of ​​the first target area in the first image.

12. The system according to claim 11, characterized in that The weighted calculator is specifically used for: According to the layer transparency of the first layer, the layer transparency of the second layer and the layer transparency of the third layer, weighted calculation is performed on the pixel values ​​of the first layer processed by the first computing engine, the pixel values ​​of the second layer determined by the image type detector and the pixel values ​​of the third layer determined by the image type detector to obtain the first image.

13. A mixed layer processing device, characterized in that: The device comprises: an acquisition unit, configured to acquire a first layer, a second layer, and a third layer, wherein the first layer and the second layer have a first dynamic range, the third layer has a second dynamic range, the first layer is a transparent layer, and the second layer is a non-transparent layer; A conversion unit, configured to convert the dynamic range of the first layer into the second dynamic range; A determination unit, configured to determine a first target area in the second layer; a merging unit, configured to merge the second layer, the third layer and the converted first layer to obtain a first image; The conversion unit is further used to convert the dynamic range of a second target area in the first image into the second dynamic range to obtain a second image, wherein the second target area is a corresponding area of ​​the first target area in the first image.

14. The device according to claim 13, characterized in that The first dynamic range is a standard dynamic range SDR, and the second dynamic range is a high dynamic range HDR; or, the first dynamic range is HDR, and the second dynamic range is SDR.

15. The device according to claim 13 or 14, characterized in that The determining unit is further configured to: Determining, according to the dynamic range identifier of the first layer, that the first layer has the first dynamic range; Determining, according to the dynamic range identifier of the second layer, that the second layer has the first dynamic range; According to the dynamic range identifier of the third layer, it is determined that the third layer has the second dynamic range.

16. The device according to claim 13, characterized in that The determining unit is further configured to: According to the layer transparency of the first layer, determining that the first layer is the transparent layer; According to the layer transparency of the second layer, the second layer is determined to be the non-transparent layer.

17. The device according to claim 13, characterized in that The determining unit is specifically configured to: The first target area is determined by identifying a region of interest (ROI).

18. The device according to claim 16 or 17, characterized in that The merging unit is specifically used for: According to the layer transparency of the first layer, the layer transparency of the second layer and the layer transparency of the third layer, the pixel values ​​of the converted first layer, the pixel values ​​of the second layer and the pixel values ​​of the third layer are weightedly calculated to obtain the first image.

19. The device according to claim 18, characterized in that The pixel value of the area outside the first target area in the second layer is 0.

20. The device according to claim 13, characterized in that The conversion unit is specifically used for: When the first dynamic range is SDR and the second dynamic range is HDR, converting the dynamic range of the first layer to HDR by inverse tone mapping; When the first dynamic range is HDR and the second dynamic range is SDR, the dynamic range of the first layer is converted into SDR through tone mapping.

21. The device according to claim 13, characterized in that The conversion unit is specifically used for: When the dynamic range of the second target area is SDR and the second dynamic range is HDR, converting the dynamic range of the second target area into HDR by inverse tone mapping to obtain a second image; When the dynamic range of the second target area is HDR and the second dynamic range is SDR, the dynamic range of the second target area is converted into SDR through tone mapping to obtain a second image.

22. The device according to claim 13, characterized in that The device also includes: A display unit is used to display the second image.

23. A mixed layer processing device, characterized in that: The device includes: a memory, and a processor coupled to the memory, the memory stores instructions and data, and the processor runs the instructions in the memory so that the processor executes the mixed layer processing method as described in any one of claims 1 to 10.

24. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores instructions, which, when executed on a computer, enable the computer to execute the mixed layer processing method as described in any one of claims 1 to 10.

25. A computer program product, characterized in that When the computer program product runs on a computer, the computer is enabled to execute the mixed layer processing method according to any one of claims 1 to 10.

Images