Local tone mapping method, system and device of HDR video and storage medium

Abstract

The invention discloses a local tone mapping method, system and device of an HDR video and a storage medium, and belongs to the technical field of video processing. The method comprises the following steps: calculating a forward optical flow vector and a backward optical flow vector of a target HDR video by adopting an optical flow estimation algorithm; based on the target HDR video and the forward optical flow vector and the backward optical flow vector of the target HDR video, time sequence filtering and space-time filtering are carried out respectively, a covering layer image is obtained through the time sequence filtering, and a base layer image is obtained through the space-time filtering; calculating to obtain a detail layer image according to the covering layer image and the base layer image; and mapping the base layer image by using the tone mapping curve to obtain a base mapping layer image, and multiplying the base mapping layer image and the detail layer image to obtain a local tone mapping result of the target HDR video. According to the method, high-quality space-time edge perception filtering is realized, the input dynamic range can be reduced, and scene details are reserved under the condition of not causing significant visual artifacts.

Description

Technical Field

[0001] The present invention belongs to the technical field of video processing, and in particular relates to a local tone mapping method, system, device and storage medium for HDR video. Background Art

[0002] HDR video is designed to present a wider dynamic range, that is, to show brighter and darker details in the same picture. Compared with standard dynamic range (SDR) video, HDR video can more realistically restore the brightness and color of the real world, providing more vivid and delicate visual effects. However, due to the hardware limitations of display devices, the brightness and contrast capabilities of different devices vary greatly and cannot meet the display requirements of HDR video. HDR video needs to be processed by tone mapping to adapt to different display devices.

[0003] Tone mapping is divided into two categories: global tone mapping and local tone mapping. One is global tone mapping: this method uses a global mapping function and applies it to the entire image. Although the calculation is relatively simple, it cannot preserve the local details in the image very well, especially in scenes with strong contrast, where details are lost or the contrast is insufficient. The other is local tone mapping: unlike the global method, local tone mapping adaptively adjusts the mapping curves of different parts according to the content of different areas in the image. This method can better balance brightness and contrast while retaining more local details, and is especially suitable for high-contrast scenes. However, current tone mapping algorithms either produce interfering temporal artifacts or are limited in their ability to reproduce local contrast. Summary of the invention

[0004] The purpose of the present invention is to overcome the deficiencies in the prior art and provide a local tone mapping method, system, device and storage medium for HDR video, which realizes high-quality spatiotemporal edge-aware filtering, can reduce the dynamic range of the input, and retain scene details without causing significant visual artifacts.

[0005] The present invention provides the following technical solutions:

[0006] In a first aspect, a local tone mapping method for an HDR video is provided, comprising: using an optical flow estimation algorithm to calculate a forward optical flow vector and a backward optical flow vector of a target HDR video; performing temporal filtering and spatiotemporal filtering based on the target HDR video and its forward optical flow vector and backward optical flow vector, respectively, wherein the temporal filtering obtains an overlay layer image, and the spatiotemporal filtering obtains a base layer image; a detail layer image is calculated based on the overlay layer image and the base layer image; the base layer image is mapped using a tone mapping curve to obtain a base mapping layer image, and the base mapping layer image and the detail layer image are multiplied to obtain a local tone mapping result of the target HDR video.

[0007] As an optional technical solution of the present invention, the use of an optical flow estimation algorithm to calculate the forward optical flow vector and the backward optical flow vector of the target HDR video includes: using the optical flow estimation algorithm to calculate the forward frame sequence of the target HDR video to obtain the forward optical flow vector; using the optical flow estimation algorithm to calculate the reverse frame sequence of the target HDR video to obtain the backward optical flow vector.

[0008] As an optional technical solution of the present invention, the temporal filtering and the spatiotemporal filtering are performed based on the target HDR video and its forward optical flow vector and backward optical flow vector, respectively, including:

[0009] The cumulative forward optical flow vector and the cumulative backward optical flow vector are calculated based on the target HDR video and its forward optical flow vector and backward optical flow vector, which are expressed as:

[0010] ;

[0011] ;

[0012] in, Indicates Frame image and The accumulated forward optical flow vector of the frame image, , Indicates Frame image and +1 frame image forward optical flow vector, Indicates Frame image and The accumulated backward optical flow vector of the frame image, Indicates Frame image and +1 frame of image's backward optical flow vector;

[0013] According to the accumulated forward optical flow vector and the accumulated backward optical flow vector, for the first Frame Image , No. Frame Image All pixels are accumulated forward optical flow vector moves in the opposite direction, making the Frame Image All pixels of The corresponding pixels of the frame images are aligned; for the first Frame Image , No. Frame Image All pixels of the accumulated backward optical flow vector moves in the opposite direction, making the Frame Image All pixels of The corresponding pixels of the frame images are aligned;

[0014] The light-constant metric permeability and flow gradient metric permeability between adjacent frames are calculated by the Lorentz edge stopping function, which are expressed as:

[0015] ;

[0016] ;

[0017] in, Indicates the current frame The light-constant metric permeability between its adjacent frames, Indicates the current frame All pixel values, Indicates the current frame Adjacent frames All pixel values, and All represent adjustable parameters; Indicates the current frame The flow gradient between its adjacent frames measures the permeability, Indicates the current frame Same as previous frame The optical flow, Indicates the previous frame Compared with the optical flow of the previous frame, and All represent adjustable parameters;

[0018] The pixel light invariance metric permeability and pixel flow gradient metric permeability between any two frames in the calculation window are expressed as:

[0019] ;

[0020] ;

[0021] in, Represents any two frames in the window The constant metric permeability of light between pixels, Indicates the current frame The light-constant metric permeability between its adjacent frames, Represents any two frames in the window The pixel flow gradient between θ and θ measures the permeability, Indicates the current frame The flow gradient between its adjacent frames measures the permeability;

[0022] The light constancy metric matrix and the flow gradient metric matrix are calculated and expressed as:

[0023] ;

[0024] ;

[0025] in, represents the elements in the light-constant metric matrix, represents the element in the flow gradient metric matrix, w represents the number of frames in the time neighborhood window, represents the pixel light constant metric permeability between the nth frame and the tth frame, represents the pixel flow gradient between the nth frame and the tth frame to measure the permeability;

[0026] The first transition probability matrix is ​​obtained by multiplying the light constancy metric matrix and the flow gradient metric matrix. ;

[0027] For the first transition probability matrix Perform time series filtering, expressed as:

[0028] ;

[0029] Where k represents the number of iterations, Represents the first transition probability matrix The elements in represents the set of all pixels at the same position in the time window, and q represents The elements in the collection, represents the timing filter parameters, represents the kth time series filtering result, represents the pixel value of the current frame p, =1;

[0030] The overlay image is represented as:

[0031] ;

[0032] in, Represents an overlay image, Represents the target HDR video frame sequence Frame pixels are mapped to , express.

[0033] As an optional technical solution of the present invention, the temporal filtering and spatiotemporal filtering are performed based on the target HDR video and its forward optical flow vector and backward optical flow vector, respectively, and further include:

[0034] Calculate the number of adjacent pixels and The permeability between is expressed as:

[0035] ;

[0036] in, Represents pixel and The pixel penetration rate, Represents pixel The pixel value of Represents pixel The pixel value of and All represent adjustable parameters;

[0037] Calculate any two pixels in the same row and The permeability between is expressed as:

[0038] ;

[0039] in, represents the permeability of any two pixels in the same row, and Represents pixel points The horizontal and vertical coordinates of and Represents pixel points The horizontal and vertical coordinates of Indicate point The point on its right The permeability between

[0040] Calculate the permeability between any two pixels in the same column ;

[0041] Calculate the horizontal permeability matrix and the vertical permeability matrix , respectively expressed as:

[0042] ;

[0043] ;

[0044] in, Represents the horizontal permeability matrix The elements in Represents the vertical permeability matrix In the elements, X represents the horizontal image width, and Y represents the vertical image height. Represents pixel and The permeability between Represents pixel and The permeability between

[0045] For the horizontal permeability matrix and the vertical permeability matrix Iterative spatial filtering is performed, and the horizontal permeability matrix is and the vertical permeability matrix Alternately, perform time series filtering after iterative filtering;

[0046] The base layer image is represented as:

[0047] ;

[0048] in, represents the base layer image, Represents the frame sequence after iterative spatial filtering Frame pixels are mapped to .

[0049] As an optional technical solution of the present invention, the step of calculating the detail layer image according to the cover layer image and the base layer image includes:

[0050] The detail layer image is obtained by dividing the overlay layer image by the base layer image, which is expressed as:

[0051] ;

[0052] in, Represents a detail layer image.

[0053] As an optional technical solution of the present invention, the base layer image is mapped by using a tone mapping curve to obtain a base mapping layer image, and the base mapping layer image and the detail layer image are multiplied to obtain a local tone mapping result of the target HDR video, including:

[0054] The base layer image is mapped using a tone mapping curve, expressed as:

[0055] ;

[0056] in, Indicates the brightness of the base map layer image. Represents the brightness of the base layer image, and A, B, C, D, and E all represent tone mapping curve coefficients;

[0057] The local tone mapping result of the target HDR video is obtained by multiplying the base map layer image and the detail layer image, which is expressed as:

[0058] ;

[0059] in, represents the local tone mapping result, Represents a base map layer image.

[0060] In a second aspect, a local tone mapping system for an HDR video is provided, comprising: an optical flow calculation module, configured to calculate a forward optical flow vector and a backward optical flow vector of a target HDR video using an optical flow estimation algorithm;

[0061] A filtering module, used for performing temporal filtering and spatiotemporal filtering based on the target HDR video and its forward optical flow vector and backward optical flow vector, respectively, wherein the temporal filtering obtains the overlay layer image, and the spatiotemporal filtering obtains the base layer image;

[0062] An image calculation module, used for calculating a detail layer image according to an overlay layer image and a base layer image;

[0063] The mapping module is used to map the base layer image using a tone mapping curve to obtain a base mapping layer image, and multiply the base mapping layer image and the detail layer image to obtain a local tone mapping result of the target HDR video.

[0064] In a third aspect, a local tone mapping device for HDR video is provided, including a processor and a storage medium;

[0065] The storage medium is used to store instructions;

[0066] The processor is used to operate according to the instructions to execute the steps of the local tone mapping method for HDR video in the first aspect.

[0067] In a fourth aspect, a computer-readable storage medium is provided, on which a computer program is stored, characterized in that when the program is executed by a processor, the steps of the local tone mapping method of the HDR video described in the first aspect are implemented.

[0068] Compared with the prior art, the present invention has the following beneficial effects:

[0069] The local tone mapping method for HDR video provided by the present invention decomposes the base layer image and the detail layer image respectively through temporal filtering and spatiotemporal filtering, and combines the tone mapping curve to significantly reduce the input dynamic range while maintaining local contrast, and retains scene details without causing significant visual artifacts, thereby realizing high-quality spatiotemporal edge-aware filtering. BRIEF DESCRIPTION OF THE DRAWINGS

[0070] Figure 1 is a flow chart of a local tone mapping method for HDR video in an embodiment of the present invention;

[0071] Figure 2 is a base layer image in an embodiment of the present invention;

[0072] Figure 3 It is the detail layer image in the embodiment of the present invention. DETAILED DESCRIPTION

[0073] The present invention will be further described below in conjunction with the accompanying drawings. The following embodiments are only used to more clearly illustrate the technical solution of the present invention, and cannot be used to limit the protection scope of the present invention.

[0074] Example 1

[0075] This embodiment provides a local tone mapping method for HDR video. Figure 1 As shown, the method comprises the following steps:

[0076] Step 1: Use the optical flow estimation algorithm to calculate the forward optical flow vector and backward optical flow vector of the target HDR video.

[0077] Specifically, an optical flow estimation algorithm is used to calculate a forward 10-frame sequence of the target HDR video to obtain 10 forward optical flow vectors; an optical flow estimation algorithm is used to calculate a reverse 10-frame sequence of the target HDR video to obtain 10 backward optical flow vectors. The optical flow estimation algorithm used in this embodiment is a prior art and will not be described in detail here.

[0078] Step 2: Based on the target HDR video and its forward optical flow vector and backward optical flow vector, temporal filtering and spatiotemporal filtering are performed respectively, wherein the temporal filtering obtains the overlay layer image, and the spatiotemporal filtering obtains the base layer image.

[0079] (1) Timing filtering

[0080] The cumulative forward optical flow vector and the cumulative backward optical flow vector are calculated based on the target HDR video and its forward optical flow vector and backward optical flow vector, which are expressed as:

[0081] ;

[0082] ;

[0083] in, Indicates Frame image and The accumulated forward optical flow vector of the frame image, , Indicates Frame image and +1 frame image forward optical flow vector, Indicates Frame image and The accumulated backward optical flow vector of the frame image, Indicates Frame image and +1 frame of backward optical flow vector.

[0084] According to the accumulated forward optical flow vector and the accumulated backward optical flow vector, for the first Frame Image , No. Frame Image All pixels are accumulated forward optical flow vector moves in the opposite direction, making the Frame Image All pixels of The corresponding pixels of the frame images are aligned; for the first Frame Image , No. Frame Image All pixels of the accumulated backward optical flow vector moves in the opposite direction, making the Frame Image All pixels of The corresponding pixels of the frame images are aligned.

[0085] The light-constant metric permeability and flow gradient metric permeability between adjacent frames are calculated by the Lorentz edge stopping function, which are expressed as:

[0086] ;

[0087] ;

[0088] in, Indicates the current frame The light-constant metric permeability between its adjacent frames, Indicates the current frame All pixel values, Indicates the current frame Adjacent frames All pixel values, and All represent adjustable parameters; Indicates the current frame The flow gradient between its adjacent frames measures the permeability, Indicates the current frame Same as previous frame The optical flow, Indicates the previous frame Compared with the optical flow of the previous frame, and All represent adjustable parameters. In this embodiment, Set to 2, Set to 0.5, Set to 2, Set to 2.

[0089] The pixel light invariance metric permeability and pixel flow gradient metric permeability between any two frames in the calculation window are expressed as:

[0090] ;

[0091] ;

[0092] in, Represents any two frames in the window The constant metric permeability of light between pixels, Indicates the current frame The light-constant metric permeability between its adjacent frames, Represents any two frames in the window The pixel flow gradient between θ and θ measures the permeability, Indicates the current frame The flow gradient between its adjacent frames measures the permeability.

[0093] The light constancy metric matrix and the flow gradient metric matrix are calculated and expressed as:

[0094] ;

[0095] ;

[0096] in, represents the elements in the light-constant metric matrix, represents the element in the flow gradient measurement matrix, w represents the number of frames in the time neighborhood window, which is set to 20 in this embodiment, represents the pixel light constant metric permeability between the nth frame and the tth frame, The pixel flow gradient between the nth frame and the tth frame measures the permeability.

[0097] The first transition probability matrix is ​​obtained by multiplying the light constancy metric matrix and the flow gradient metric matrix. ;

[0098] Calculate the permeability of the 10 frames before and after the current frame to get the first transition probability matrix , for the first transition probability matrix Perform time series filtering, expressed as:

[0099] ;

[0100] Wherein, k represents the number of iterations. In this embodiment, k=1 in the time series filtering. Represents the first transition probability matrix The elements in represents the set of all pixels at the same position in the time window, and q represents The elements in the collection, represents the timing filter parameters, represents the kth time series filtering result, represents the pixel value of the current frame p, =1.

[0101] In time series filtering, the iterative filter degenerates into: .

[0102] The overlay image is represented as:

[0103] ;

[0104] in, Represents an overlay image, Represents the target HDR video frame sequence Frame pixels are mapped to , express.

[0105] (2) Spatiotemporal filtering

[0106] Calculate the number of adjacent pixels and The permeability between is expressed as:

[0107] ;

[0108] in, Represents pixel and The pixel penetration rate, Represents pixel The pixel value of Represents pixel The pixel value of and All represent adjustable parameters. In this embodiment, Set to 2, Set to 0.1~1.

[0109] Calculate any two pixels in the same row and The permeability between is expressed as:

[0110] ;

[0111] in, represents the permeability of any two pixels in the same row, and Represents pixel points The horizontal and vertical coordinates of and Represents pixel points The horizontal and vertical coordinates of Indicate point The point on its right The permeability between .

[0112] Calculate the permeability between any two pixels in the same column The method is the same as that for calculating the permeability between any two pixels in the same row, and will not be repeated here.

[0113] Calculate the horizontal permeability matrix and the vertical permeability matrix , respectively expressed as:

[0114] ;

[0115] ;

[0116] in, Represents the horizontal permeability matrix The elements in Represents the vertical permeability matrix In the elements, X represents the horizontal image width, and Y represents the vertical image height. Represents pixel and The permeability between Represents pixel and The permeability between .

[0117] Calculate the permeability of the 10 frames before and after the current frame to obtain the horizontal permeability matrix and the vertical permeability matrix , for the horizontal permeability matrix and the vertical permeability matrix Iterative spatial filtering is performed, and the horizontal permeability matrix is and the vertical permeability matrix In this embodiment, the number of iterations is set to 5, and the diffusion direction changes after each iteration, so that the image intensity diffuses rapidly in the connected areas of similar colors. After the iterative filtering, the temporal filtering is performed.

[0118] The base layer image is represented as:

[0119] ;

[0120] Among them, Figure 2 As shown, represents the base layer image, Represents the frame sequence after iterative spatial filtering Frame pixels are mapped to .

[0121] Step 3: Calculate the detail layer image based on the cover layer image and the base layer image.

[0122] The detail layer image is obtained by dividing the overlay layer image by the base layer image, which is expressed as:

[0123] ;

[0124] in, Represents a detail layer image, such as Figure 3 shown.

[0125] Step 4: Use the tone mapping curve to map the base layer image to obtain a base mapping layer image, and multiply the base mapping layer image and the detail layer image to obtain the local tone mapping result of the target HDR video.

[0126] The ACES tone mapping curve is used to map the base layer image, which is expressed as:

[0127] ;

[0128] in, Indicates the brightness of the base map layer image. represents the brightness of the base layer image, A, B, C, D, and E all represent tone mapping curve coefficients. In this embodiment, A=2.51, B=0.03, C=2.43, D=0.59, and E=0.14.

[0129] The local tone mapping result of the target HDR video is obtained by multiplying the base map layer image and the detail layer image, which is expressed as:

[0130] ;

[0131] in, represents the local tone mapping result, Represents a base map layer image.

[0132] Example 2

[0133] This embodiment provides a local tone mapping system for HDR video, including:

[0134] An optical flow calculation module, used to calculate a forward optical flow vector and a backward optical flow vector of a target HDR video using an optical flow estimation algorithm;

[0135] A filtering module, used for performing temporal filtering and spatiotemporal filtering based on the target HDR video and its forward optical flow vector and backward optical flow vector, respectively, wherein the temporal filtering obtains the overlay layer image, and the spatiotemporal filtering obtains the base layer image;

[0136] An image calculation module, used for calculating a detail layer image according to an overlay layer image and a base layer image;

[0137] The mapping module is used to map the base layer image using a tone mapping curve to obtain a base mapping layer image, and multiply the base mapping layer image and the detail layer image to obtain a local tone mapping result of the target HDR video.

[0138] Example 3

[0139] This embodiment provides a local tone mapping device for HDR video, including a processor and a storage medium; the storage medium is used to store instructions; the processor is used to operate according to the instructions to execute the steps of the local tone mapping method for HDR video described in Example 1.

[0140] Example 4

[0141] This embodiment provides a computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, the steps of the local tone mapping method for HDR video described in Embodiment 1 are implemented.

[0142] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present application may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0143] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0144] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0145] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0146] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the technical principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A local tone mapping method for HDR video, characterized in that: include: The optical flow estimation algorithm is used to calculate the forward optical flow vector and the backward optical flow vector of the target HDR video; Based on the target HDR video and its forward optical flow vector and backward optical flow vector, temporal filtering and spatiotemporal filtering are performed respectively, wherein the temporal filtering obtains the cover layer image, and the spatiotemporal filtering obtains the base layer image; A detail layer image is obtained by calculating the cover layer image and the base layer image; The base layer image is mapped using a tone mapping curve to obtain a base mapping layer image, and the base mapping layer image is multiplied with the detail layer image to obtain a local tone mapping result of the target HDR video.

2. The local tone mapping method for HDR video according to claim 1, characterized in that: The method of using an optical flow estimation algorithm to calculate a forward optical flow vector and a backward optical flow vector of a target HDR video includes: The optical flow estimation algorithm is used to calculate the forward frame sequence of the target HDR video to obtain the forward optical flow vector; The optical flow estimation algorithm is used to calculate the reverse frame sequence of the target HDR video to obtain the backward optical flow vector.

3. The local tone mapping method for HDR video according to claim 1, characterized in that: The method of performing temporal filtering and spatiotemporal filtering based on the target HDR video and its forward optical flow vector and backward optical flow vector, respectively, includes: The cumulative forward optical flow vector and the cumulative backward optical flow vector are calculated based on the target HDR video and its forward optical flow vector and backward optical flow vector, which are expressed as: ； ； in, Indicates Frame image and The accumulated forward optical flow vector of the frame image, , Indicates Frame image and +1 frame image forward optical flow vector, Indicates Frame image and The accumulated backward optical flow vector of the frame image, Indicates Frame image and +1 frame of image's backward optical flow vector; According to the accumulated forward optical flow vector and the accumulated backward optical flow vector, for the first Frame Image , No. Frame Image All pixels are accumulated forward optical flow vector moves in the opposite direction, making the Frame Image All pixels of The corresponding pixels of the frame images are aligned; for the first Frame Image , No. Frame Image All pixels of the accumulated backward optical flow vector moves in the opposite direction, making the Frame Image All pixels of The corresponding pixels of the frame images are aligned; The light-constant metric permeability and flow gradient metric permeability between adjacent frames are calculated by the Lorentz edge stopping function, which are expressed as: ； ； in, Indicates the current frame The light-constant metric permeability between its adjacent frames, Indicates the current frame All pixel values, Indicates the current frame Adjacent frames All pixel values, and All represent adjustable parameters; Indicates the current frame The flow gradient between its adjacent frames measures the permeability, Indicates the current frame Same as previous frame The optical flow, Indicates the previous frame Compared with the optical flow of the previous frame, and All represent adjustable parameters; The pixel light invariance metric permeability and pixel flow gradient metric permeability between any two frames in the calculation window are expressed as: ； ； in, Represents any two frames in the window The constant metric permeability of light between pixels, Indicates the current frame The light-constant metric permeability between its adjacent frames, Represents any two frames in the window The pixel flow gradient between θ and θ measures the permeability, Indicates the current frame The flow gradient between its adjacent frames measures the permeability; The light constancy metric matrix and the flow gradient metric matrix are calculated and expressed as: ； ； in, represents the elements in the light-constant metric matrix, represents the element in the flow gradient metric matrix, w represents the number of frames in the time neighborhood window, represents the pixel light constant metric permeability between the nth frame and the tth frame, represents the pixel flow gradient between the nth frame and the tth frame to measure the permeability; The first transition probability matrix is ​​obtained by multiplying the light constancy metric matrix and the flow gradient metric matrix. ; For the first transition probability matrix Perform time series filtering, expressed as: ； Where k represents the number of iterations, Represents the first transition probability matrix The elements in represents the set of all pixels at the same position in the time window, and q represents The elements in the collection, represents the timing filter parameters, represents the kth time series filtering result, represents the pixel value of the current frame p, =1; The overlay image is represented as: ； in, Represents an overlay image, Represents the target HDR video frame sequence Frame pixels are mapped to .

4. The local tone mapping method for HDR video according to claim 3, characterized in that: The method of performing temporal filtering and spatiotemporal filtering based on the target HDR video and its forward optical flow vector and backward optical flow vector, respectively, further includes: Calculate the number of adjacent pixels and The permeability between is expressed as: ； in, Represents pixel and The pixel penetration rate, Represents pixel The pixel value of Represents pixel The pixel value of and All represent adjustable parameters; Calculate any two pixels in the same row and The permeability between is expressed as: ； in, represents the permeability of any two pixels in the same row, and Represents pixel points The horizontal and vertical coordinates of and Represents pixel points The horizontal and vertical coordinates of Indicate point The point on its right The permeability between Calculate the permeability between any two pixels in the same column ; Calculate the horizontal permeability matrix and the vertical permeability matrix , respectively expressed as: ； ； in, Represents the horizontal permeability matrix The elements in Represents the vertical permeability matrix In the elements, X represents the horizontal image width, and Y represents the vertical image height. Represents pixel and The permeability between Represents pixel and The permeability between For the horizontal permeability matrix and the vertical permeability matrix Iterative spatial filtering is performed, and the horizontal permeability matrix is and the vertical permeability matrix Alternately, perform time series filtering after iterative filtering; The base layer image is represented as: ； in, represents the base layer image, Represents the frame sequence after iterative spatial filtering Frame pixels are mapped to .

5. The local tone mapping method for HDR video according to claim 4, characterized in that: The step of calculating the detail layer image according to the cover layer image and the base layer image includes: The detail layer image is obtained by dividing the overlay layer image by the base layer image, which is expressed as: ； in, Represents a detail layer image.

6. The local tone mapping method for HDR video according to claim 5, characterized in that: The method of mapping the base layer image by using the tone mapping curve to obtain a base mapping layer image, and multiplying the base mapping layer image and the detail layer image to obtain a local tone mapping result of the target HDR video includes: The base layer image is mapped using a tone mapping curve, expressed as: ； in, Indicates the brightness of the base map layer image. Represents the brightness of the base layer image, and A, B, C, D, and E all represent tone mapping curve coefficients; The local tone mapping result of the target HDR video is obtained by multiplying the base map layer image and the detail layer image, which is expressed as: ； in, represents the local tone mapping result, Represents a base map layer image.

7. A local tone mapping system for HDR video, characterized in that: include: An optical flow calculation module, used to calculate a forward optical flow vector and a backward optical flow vector of a target HDR video using an optical flow estimation algorithm; A filtering module, used for performing temporal filtering and spatiotemporal filtering based on the target HDR video and its forward optical flow vector and backward optical flow vector, respectively, wherein the temporal filtering obtains the overlay layer image, and the spatiotemporal filtering obtains the base layer image; An image calculation module, used for calculating a detail layer image according to an overlay layer image and a base layer image; The mapping module is used to map the base layer image using a tone mapping curve to obtain a base mapping layer image, and multiply the base mapping layer image and the detail layer image to obtain a local tone mapping result of the target HDR video.

8. A local tone mapping device for HDR video, characterized in that: including processor and storage medium; The storage medium is used to store instructions; The processor is used to operate according to the instruction to execute the steps of the local tone mapping method of the HDR video according to any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps of the local tone mapping method for HDR video described in any one of claims 1 to 6 are implemented.

Images