Video processing device and method and related device

Abstract

The invention discloses a video processing device and method and a related device, the video processing device comprises an acquisition unit, a decomposition unit, a determination unit and a reconstruction unit, the acquisition unit is used for acquiring a to-be-processed image, and the to-be-processed image is any frame of image in a target video; the decomposition unit is used for performing multi-scale transformation on the to-be-processed image to obtain a low-frequency sub-band image and K high-frequency sub-band images, and K is a positive integer; the determination unit is used for determining enhancement coefficients of S high-frequency sub-band images in the K high-frequency sub-band images to obtain S enhancement coefficients, and S is a positive integer smaller than or equal to K; and the reconstruction unit is used for performing inverse transformation corresponding to the multi-scale transformation based on the S enhancement coefficients, the low-frequency sub-band image and the K high-frequency sub-band images to obtain a target image. According to the embodiment of the invention, image enhancement processing can be realized, and the image quality can be improved.

Description

[0001] This application is a divisional application of Chinese Patent Application No. 202110496076.8, filed with the Chinese Patent Office on May 7, 2021, entitled "Video Processing Method and Related Products", the entire contents of which are incorporated herein by reference. Technical Field

[0002] This application relates to the field of image processing technology, specifically to a video processing apparatus, method, and related apparatus. Background Technology

[0003] With the rapid development of electronic technology, photography has increasingly become a standard feature of electronic devices (such as mobile phones and tablets). Image quality greatly influences users' judgment of the quality of electronic devices; therefore, the problem of how to enhance images urgently needs to be solved. Summary of the Invention

[0004] This application provides a video processing apparatus, method, and related devices that can perform image enhancement processing and help improve image quality.

[0005] In a first aspect, embodiments of this application provide a video processing method, the method comprising:

[0006] Acquire the image to be processed, which is any frame image in the target video;

[0007] The image to be processed is subjected to multi-scale transformation to obtain a low-frequency sub-band image and K high-frequency sub-band images, where K is a positive integer;

[0008] Determine the enhancement coefficients of S high-frequency sub-band images out of the K high-frequency sub-band images to obtain S enhancement coefficients, where S is a positive integer less than or equal to K;

[0009] Based on the S enhancement coefficients, the low-frequency sub-band image, and the K high-frequency sub-band images, an inverse transform corresponding to the multi-scale transform is performed to obtain the target image.

[0010] Secondly, embodiments of this application provide a video processing apparatus, the apparatus comprising: an acquisition unit, a decomposition unit, a determination unit, and a reconstruction unit, wherein...

[0011] The acquisition unit is used to acquire an image to be processed, wherein the image to be processed is any frame image in the target video;

[0012] The decomposition unit is used to perform multi-scale transformation on the image to be processed to obtain a low-frequency sub-band image and K high-frequency sub-band images, where K is a positive integer.

[0013] The determining unit is used to determine the enhancement coefficients of S high-frequency sub-band images among the K high-frequency sub-band images, thereby obtaining S enhancement coefficients, where S is a positive integer less than or equal to K;

[0014] The reconstruction unit is used to perform an inverse transformation corresponding to the multi-scale transformation based on the S enhancement coefficients, the low-frequency sub-band image, and the K high-frequency sub-band images to obtain the target image.

[0015] Thirdly, embodiments of this application provide an electronic device, including a processor, a memory, a communication interface, and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the processor, and the programs include instructions for performing steps in any method of the first aspect of this application.

[0016] Fourthly, embodiments of this application provide a computer-readable storage medium storing a computer program for electronic data interchange, wherein the computer program causes a computer to perform some or all of the steps described in any method of the first aspect of this application.

[0017] Fifthly, embodiments of this application provide a computer program product, wherein the computer program product includes a non-transitory computer-readable storage medium storing a computer program operable to cause a computer to perform some or all of the steps described in any method of the first aspect of this application. The computer program product may be a software installation package.

[0018] The embodiments of this application have the following beneficial effects:

[0019] As can be seen, the video processing apparatus, method, and related devices described in the embodiments of this application acquire an image to be processed, which is any frame image in the target video; perform a multi-scale transformation on the image to be processed to obtain a low-frequency sub-band image and K high-frequency sub-band images, where K is a positive integer; determine the enhancement coefficients of S high-frequency sub-band images among the K high-frequency sub-band images to obtain S enhancement coefficients, where S is a positive integer less than or equal to K; and perform an inverse transformation corresponding to the multi-scale transformation based on the S enhancement coefficients, the low-frequency sub-band images, and the K high-frequency sub-band images to obtain the target image. Since the high-frequency images reflect image details, enhancement can be performed on image details to improve the salience of detail information, thereby achieving image enhancement processing and helping to improve image quality. Attached Figure Description

[0020] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0021] Figure 1 This is a flowchart illustrating a video processing method provided in an embodiment of this application;

[0022] Figure 2 This is a flowchart illustrating another video processing method provided in an embodiment of this application;

[0023] Figure 3 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application;

[0024] Figure 4 This is a block diagram of the functional units of a video processing device provided in an embodiment of this application. Detailed Implementation

[0025] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present application.

[0026] The terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish different objects, not to describe a specific order. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or apparatus that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to these processes, methods, products, or apparatuses.

[0027] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0028] The electronic devices involved in the embodiments of this application may include various handheld devices (mobile phones, tablets, etc.) with wireless communication functions, vehicle-mounted devices, wearable devices (smart glasses, smart bracelets, smartwatches, etc.), computing devices or other processing devices connected to a wireless modem, as well as various forms of user equipment (UE), mobile station (MS), terminal device, etc.

[0029] Please see Figure 1 , Figure 1 This is a flowchart illustrating a video processing method provided in this application embodiment, applied to an electronic device. As shown in the figure, the video processing method includes the following operations.

[0030] 101. Obtain the image to be processed, wherein the image to be processed is any frame image in the target video.

[0031] The image to be processed can be any frame from the target video, and it can be a grayscale image or a color image. In practice, the electronic device can capture the target video using a camera.

[0032] 102. Perform multi-scale transformation on the image to be processed to obtain a low-frequency sub-band image and K high-frequency sub-band images, where K is a positive integer.

[0033] The multi-scale transform can be at least one of the following: wavelet transform, contourlet transform, non-subsampled contourlet transform, sheared wave transform, ridge transform, pyramid transform, etc., without limitation. For example, in a specific implementation, an electronic device can perform a wavelet transform on the image to be processed to obtain a low-frequency sub-band image and K high-frequency sub-band images, where K is a positive integer.

[0034] In practice, after wavelet transform, an image can be divided into low-frequency and high-frequency signals. For any image I(i,j), its wavelet decomposition can be defined as:

[0035]

[0036] In the formula, C l,n This represents the low-frequency subband (approximate image), namely the LL3 subband; W k,n The high-frequency subbands (detailed images) are represented as LH1′, HL1′ and HH1′, LH2′, HL2′ and HH2′, LH3′, HL3′ and HH3′; l represents the number of wavelet decomposition layers, and n represents the number of wavelet coefficients.

[0037] 103. Determine the enhancement coefficients of S high-frequency sub-band images among the K high-frequency sub-band images to obtain S enhancement coefficients, where S is a positive integer less than or equal to K.

[0038] In a specific implementation, the electronic device can determine the enhancement coefficients of S high-frequency sub-band images out of K high-frequency sub-band images, thus obtaining S enhancement coefficients, where S is a positive integer less than or equal to K. If S = K, it means that image enhancement processing needs to be performed on each high-frequency sub-band image out of the K high-frequency sub-band images. Conversely, if S is less than K, it means that image enhancement processing needs to be performed on some high-frequency sub-band images out of the K high-frequency sub-band images.

[0039] Optionally, step 103 above, determining the enhancement coefficients of S high-frequency sub-band images among the K high-frequency sub-band images to obtain S enhancement coefficients, may include the following steps:

[0040] A31. Determine the target ratio between the resolution i of the high-frequency sub-band image i and the resolution of the image to be processed, wherein the high-frequency sub-band image i is any high-frequency sub-band image among the K high-frequency sub-band images;

[0041] A32. Divide the high-frequency sub-band image i into multiple regions, determine the feature point distribution density of each region in the multiple regions, and obtain multiple feature point distribution densities;

[0042] A33. Determine the target root mean square error corresponding to the distribution density of the plurality of feature points;

[0043] A34. Determine the first enhancement coefficient corresponding to the target ratio according to the preset mapping relationship between the ratio and the enhancement coefficient;

[0044] A35. Determine the target fine-tuning coefficient corresponding to the target mean square error according to the preset mapping relationship between the mean square error and the fine-tuning coefficient;

[0045] A36. Adjust the first enhancement coefficient according to the target fine-tuning coefficient to obtain the enhancement coefficient of the high-frequency sub-band image i.

[0046] In a specific implementation, the electronic device can pre-store the mapping relationship between the preset ratio and the enhancement coefficient, as well as the mapping relationship between the preset mean square error and the fine-tuning coefficient.

[0047] Specifically, taking a high-frequency sub-band image i as an example, i is any one of K high-frequency sub-band images. The electronic device can determine the target ratio between the resolution i of the high-frequency sub-band image i and the resolution of the image to be processed. It can also divide the high-frequency sub-band image i into multiple regions, each region having the same area size or the same regional pattern. Furthermore, it can determine the feature point distribution density of each region, obtaining multiple feature point distribution densities. The feature point distribution density is calculated as: feature point distribution density = number of feature points in the region / area of ​​the region. It can also determine the target mean square error corresponding to the multiple feature point distribution densities.

[0048] Furthermore, the electronic device can determine the first enhancement coefficient corresponding to the target ratio according to a preset mapping relationship between the ratio and the enhancement coefficient, and determine the target fine-tuning coefficient corresponding to the target mean square error according to a preset mapping relationship between the mean square error and the fine-tuning coefficient. The fine-tuning coefficient can be in the range of -1 to 1, for example, -0.1 to 0.1. Then, the first enhancement coefficient is adjusted according to the target fine-tuning coefficient to obtain the enhancement coefficient of the high-frequency sub-band image i, i.e., the specific formula is as follows:

[0049] Enhancement coefficient of high-frequency sub-band image i = (1 + target fine-tuning coefficient) * first enhancement coefficient

[0050] Since the mean square error reflects the correlation between neighborhoods, that is, the consistency between different regions of the image, the enhancement coefficient can be fine-tuned based on the mean square error to make the enhancement effect more in line with the characteristics of the image itself.

[0051] Optionally, step 103 above, determining the enhancement coefficients of S high-frequency sub-band images among the K high-frequency sub-band images to obtain S enhancement coefficients, may include the following steps:

[0052] B31. Determine the first energy value of the low-frequency sub-band image;

[0053] B32. Determine the second energy value of the image to be processed;

[0054] B33. Determine the target energy ratio between the first energy value and the second energy value;

[0055] B34. According to the preset mapping relationship between energy ratio and enhancement coefficient, determine the reference enhancement coefficient corresponding to the target energy ratio, and use the reference enhancement coefficient as the enhancement coefficient of S high-frequency sub-band images in the K high-frequency sub-band images.

[0056] In practice, electronic devices can pre-store the mapping relationship between preset energy ratios and enhancement coefficients.

[0057] Specifically, the electronic device can determine a first energy value of the low-frequency subband image, a second energy value of the image to be processed, and a target energy ratio between the first energy value and the second energy value. The larger the target energy ratio, the less detailed information the high-frequency subband image contains, and vice versa.

[0058] Furthermore, the electronic device can determine the reference enhancement coefficient corresponding to the target energy ratio according to the preset mapping relationship between the energy ratio and the enhancement coefficient, and use the reference enhancement coefficient as the enhancement coefficient of S high-frequency sub-band images out of K high-frequency sub-band images, that is, the enhancement coefficients of the S high-frequency sub-band images can all be the same.

[0059] 104. Based on the S enhancement coefficients, the low-frequency sub-band image, and the K high-frequency sub-band images, perform an inverse transform corresponding to the multi-scale transform to obtain the target image.

[0060] In practical implementation, within the wavelet subband, since the high-frequency subband contains the image's contour and detail information, image enhancement can also be understood as sharpness enhancement. That is, sharpness enhancement aims to enhance these image information. Therefore, a gain coefficient G is introduced into the high-frequency signal. k，n This achieves the purpose of image enhancement. The specific calculation formula is as follows:

[0061]

[0062] Optionally, after obtaining the image to be processed in step 101, which is any frame image in the target video, and before performing multi-scale transformation on the image to be processed in step 102, the following steps may also be included:

[0063] A1. Perform image quality evaluation on the image to be processed to obtain an image quality evaluation value;

[0064] A2. Obtain target environment parameters;

[0065] A3. Obtain target shooting parameters;

[0066] A4. Determine the first reference image quality evaluation value range corresponding to the target environmental parameters according to the preset mapping relationship between environmental parameters and the first image quality evaluation value range;

[0067] A5. Determine the second reference image quality evaluation value range corresponding to the target shooting parameters according to the preset mapping relationship between the shooting parameters and the second image quality evaluation value range;

[0068] A6. Determine the target intersection between the range of the first reference image quality evaluation value and the range of the second reference image quality evaluation value;

[0069] A7. When the image quality evaluation value is not at the target intersection, perform the step of performing multi-scale transformation on the image to be processed.

[0070] In a specific implementation, the electronic device can pre-store the mapping relationship between preset environmental parameters and the first image quality evaluation value range, as well as the mapping relationship between preset shooting parameters and the second image quality evaluation value range.

[0071] In this embodiment, the environmental parameters can be at least one of the following: ambient light intensity, ambient color temperature, ambient temperature, atmospheric pressure, ambient humidity, etc., and are not limited thereto. The shooting parameters can be at least one of the following: ISO, exposure time, white balance value, beauty parameters, etc., and are not limited thereto.

[0072] In a specific implementation, the electronic device can use at least one image quality evaluation index to evaluate the image to be processed and obtain an image quality evaluation value. The image quality evaluation index can be at least one of the following: information entropy, average gradient, sharpness, signal-to-noise ratio, etc., without limitation.

[0073] Furthermore, the electronic device can acquire the target environment parameters corresponding to the image to be processed, as well as the acquisition target shooting parameters corresponding to the image to be processed. Then, according to the preset mapping relationship between the environment parameters and the first image quality evaluation value range, it determines the first reference image quality evaluation value range corresponding to the target environment parameters, and according to the preset mapping relationship between the shooting parameters and the second image quality evaluation value range, it determines the second reference image quality evaluation value range corresponding to the target shooting parameters. It determines the target intersection between the first reference image quality evaluation value range and the second reference image quality evaluation value range. If the image quality evaluation value is not at the target intersection, step 102 is executed; otherwise, it indicates that the image quality is good and image enhancement processing is not required.

[0074] Optionally, step A1 above, which involves evaluating the image quality of the image to be processed to obtain an image quality evaluation value, may include the following steps:

[0075] A11. Extract the target from the image to be processed to obtain the target region;

[0076] A12. Evaluate the image quality of the target region to obtain the image quality evaluation value.

[0077] In a specific implementation, the electronic device can extract the target from the image to be processed to obtain the target region, and can use at least one image quality evaluation index to evaluate the image quality of the target region to obtain an image quality evaluation value.

[0078] As can be seen, the video processing method described in this application involves: acquiring an image to be processed, which is any frame in the target video; performing a multi-scale transformation on the image to be processed to obtain a low-frequency sub-band image and K high-frequency sub-band images, where K is a positive integer; determining the enhancement coefficients of S high-frequency sub-band images among the K high-frequency sub-band images to obtain S enhancement coefficients, where S is a positive integer less than or equal to K; and performing an inverse transformation corresponding to the multi-scale transformation based on the S enhancement coefficients, the low-frequency sub-band image, and the K high-frequency sub-band images to obtain the target image. Since the high-frequency image reflects the image details, enhancement can be performed on the image details to improve the saliency of the detail information, thereby achieving image enhancement processing and helping to improve image quality.

[0079] With the above Figure 1 The embodiments shown are consistent; please refer to [link / reference]. Figure 2 , Figure 2 This is a flowchart illustrating a video processing method provided in an embodiment of this application, applied to an electronic device. As shown in the figure, the video processing method includes the following steps.

[0080] 201. Obtain the image to be processed, wherein the image to be processed is any frame image in the target video.

[0081] 202. Perform image quality evaluation on the image to be processed to obtain an image quality evaluation value.

[0082] 203. Obtain target environment parameters.

[0083] 204. Obtain target shooting parameters.

[0084] 205. Determine the first reference image quality evaluation value range corresponding to the target environmental parameters according to the preset mapping relationship between environmental parameters and the first image quality evaluation value range.

[0085] 206. Determine the second reference image quality evaluation value range corresponding to the target shooting parameters according to the preset mapping relationship between the shooting parameters and the second image quality evaluation value range.

[0086] 207. Determine the target intersection between the range of the first reference image quality evaluation value and the range of the second reference image quality evaluation value.

[0087] 208. When the image quality evaluation value is not in the target intersection, perform multi-scale transformation on the image to be processed to obtain a low-frequency sub-band image and K high-frequency sub-band images, where K is a positive integer.

[0088] 209. Determine the enhancement coefficients of S high-frequency sub-band images among the K high-frequency sub-band images to obtain S enhancement coefficients, where S is a positive integer less than or equal to K.

[0089] 210. Based on the S enhancement coefficients, the low-frequency sub-band image, and the K high-frequency sub-band images, perform the inverse transform corresponding to the multi-scale transform to obtain the target image.

[0090] For a detailed description of steps 201-210 above, please refer to the above. Figure 1 The corresponding steps of the video processing method described herein will not be repeated here.

[0091] As can be seen, the video processing method described in this application involves: acquiring an image to be processed, which is any frame in the target video; performing a multi-scale transformation on the image to be processed to obtain a low-frequency sub-band image and K high-frequency sub-band images, where K is a positive integer; determining the enhancement coefficients of S high-frequency sub-band images among the K high-frequency sub-band images to obtain S enhancement coefficients, where S is a positive integer less than or equal to K; and performing an inverse transformation corresponding to the multi-scale transformation based on the S enhancement coefficients, the low-frequency sub-band image, and the K high-frequency sub-band images to obtain the target image. Since the high-frequency image reflects the image details, enhancement can be performed on the image details to improve the saliency of the detail information, thereby achieving image enhancement processing and helping to improve image quality.

[0092] With the above Figure 1 , Figure 2 The embodiments shown are consistent; please refer to [link / reference]. Figure 3 , Figure 3 This is a schematic diagram of the structure of an electronic device 300 provided in an embodiment of this application. As shown in the figure, the electronic device 300 includes a processor 310, a memory 320, a communication interface 330, and one or more programs 321. The one or more programs 321 are stored in the memory 320 and configured to be executed by the processor 310. The one or more programs 321 include instructions for performing any step in the above method embodiment.

[0093] Acquire the image to be processed, which is any frame image in the target video;

[0094] The image to be processed is subjected to multi-scale transformation to obtain a low-frequency sub-band image and K high-frequency sub-band images, where K is a positive integer;

[0095] Determine the enhancement coefficients of S high-frequency sub-band images out of the K high-frequency sub-band images to obtain S enhancement coefficients, where S is a positive integer less than or equal to K;

[0096] Based on the S enhancement coefficients, the low-frequency sub-band image, and the K high-frequency sub-band images, an inverse transform corresponding to the multi-scale transform is performed to obtain the target image.

[0097] As can be seen, the electronic device described in this application embodiment acquires an image to be processed, which is any frame image in the target video; performs a multi-scale transformation on the image to be processed to obtain a low-frequency sub-band image and K high-frequency sub-band images, where K is a positive integer; determines the enhancement coefficients of S high-frequency sub-band images among the K high-frequency sub-band images to obtain S enhancement coefficients, where S is a positive integer less than or equal to K; and performs an inverse transformation corresponding to the multi-scale transformation based on the S enhancement coefficients, the low-frequency sub-band image, and the K high-frequency sub-band images to obtain the target image. Since the high-frequency image reflects the image details, the image details can be enhanced to improve the saliency of the detail information, thereby achieving image enhancement processing and helping to improve image quality.

[0098] Optionally, determining the enhancement coefficients of S high-frequency sub-band images out of the K high-frequency sub-band images to obtain S enhancement coefficients includes:

[0099] Determine the target ratio between the resolution i of the high-frequency sub-band image i and the resolution of the image to be processed, wherein the high-frequency sub-band image i is any high-frequency sub-band image among the K high-frequency sub-band images;

[0100] The high-frequency sub-band image i is divided into multiple regions, and the feature point distribution density of each region is determined to obtain multiple feature point distribution densities.

[0101] Determine the target mean square error corresponding to the distribution density of the plurality of feature points;

[0102] According to the preset mapping relationship between the ratio and the enhancement coefficient, the first enhancement coefficient corresponding to the target ratio is determined;

[0103] According to the preset mapping relationship between the mean square error and the fine-tuning coefficient, the target fine-tuning coefficient corresponding to the target mean square error is determined;

[0104] The first enhancement coefficient is adjusted according to the target fine-tuning coefficient to obtain the enhancement coefficient of the high-frequency sub-band image i.

[0105] Optionally, determining the enhancement coefficients of S high-frequency sub-band images out of the K high-frequency sub-band images to obtain S enhancement coefficients includes:

[0106] Determine the first energy value of the low-frequency subband image;

[0107] Determine the second energy value of the image to be processed;

[0108] Determine the target energy ratio between the first energy value and the second energy value;

[0109] According to the preset mapping relationship between energy ratio and enhancement coefficient, the reference enhancement coefficient corresponding to the target energy ratio is determined, and the reference enhancement coefficient is used as the enhancement coefficient of S high-frequency sub-band images in the K high-frequency sub-band images.

[0110] Optionally, after acquiring the image to be processed and before performing multi-scale transformation on the image to be processed, the method further includes:

[0111] The image to be processed is subjected to image quality evaluation to obtain an image quality evaluation value;

[0112] Obtain target environment parameters;

[0113] Obtain target shooting parameters;

[0114] According to the preset mapping relationship between environmental parameters and the first image quality evaluation value range, the first reference image quality evaluation value range corresponding to the target environmental parameters is determined;

[0115] According to the preset mapping relationship between shooting parameters and the second image quality evaluation value range, the second reference image quality evaluation value range corresponding to the target shooting parameters is determined;

[0116] Determine the target intersection between the range of quality evaluation values ​​for the first reference image and the range of quality evaluation values ​​for the second reference image;

[0117] When the image quality evaluation value is not within the target intersection, the step of performing multi-scale transformation on the image to be processed is executed.

[0118] Optionally, the step of performing image quality evaluation on the image to be processed to obtain an image quality evaluation value includes:

[0119] The target region is obtained by extracting the target from the image to be processed;

[0120] The target region is subjected to image quality evaluation to obtain the image quality evaluation value.

[0121] The above primarily describes the solutions of the embodiments of this application from the perspective of the method execution process. It is understood that, in order to achieve the above functions, the electronic device includes corresponding hardware structures and / or software modules for executing each function. Those skilled in the art should readily recognize that, in conjunction with the units and algorithm steps of the various examples described in the embodiments provided herein, this application can be implemented in hardware or a combination of hardware and computer software. Whether a function is executed by hardware or by computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0122] This application embodiment can divide the electronic device into functional units according to the above method example. For example, each function can be divided into a separate functional unit, or two or more functions can be integrated into one processing unit. The integrated unit can be implemented in hardware or as a software functional unit. It should be noted that the unit division in this application embodiment is illustrative and only represents one logical functional division. In actual implementation, there may be other division methods.

[0123] Figure 4 This is a functional unit block diagram of the video processing apparatus 400 involved in the embodiments of this application. The video processing apparatus 400 is applied to an electronic device and includes an acquisition unit 401, a decomposition unit 402, a determination unit 403, and a reconstruction unit 404.

[0124] The acquisition unit 401 is used to acquire an image to be processed, wherein the image to be processed is any frame image in the target video;

[0125] The decomposition unit 402 is used to perform multi-scale transformation on the image to be processed to obtain a low-frequency sub-band image and K high-frequency sub-band images, where K is a positive integer.

[0126] The determining unit 403 is used to determine the enhancement coefficients of S high-frequency sub-band images among the K high-frequency sub-band images, and obtain S enhancement coefficients, where S is a positive integer less than or equal to K;

[0127] The reconstruction unit 404 is used to perform an inverse transformation corresponding to the multi-scale transformation based on the S enhancement coefficients, the low-frequency sub-band image, and the K high-frequency sub-band images to obtain the target image.

[0128] As can be seen, the video processing apparatus described in this application acquires an image to be processed, which is any frame image in the target video; performs a multi-scale transformation on the image to be processed to obtain a low-frequency sub-band image and K high-frequency sub-band images, where K is a positive integer; determines the enhancement coefficients of S high-frequency sub-band images among the K high-frequency sub-band images to obtain S enhancement coefficients, where S is a positive integer less than or equal to K; and performs an inverse transformation corresponding to the multi-scale transformation based on the S enhancement coefficients, the low-frequency sub-band image, and the K high-frequency sub-band images to obtain the target image. Since the high-frequency image reflects the image details, the image details can be enhanced to improve the saliency of the detail information, thereby achieving image enhancement processing and helping to improve image quality.

[0129] Optionally, in determining the enhancement coefficients of S high-frequency sub-band images among the K high-frequency sub-band images to obtain S enhancement coefficients, the determining unit 403 is specifically used for:

[0130] Determine the target ratio between the resolution i of the high-frequency sub-band image i and the resolution of the image to be processed, wherein the high-frequency sub-band image i is any high-frequency sub-band image among the K high-frequency sub-band images;

[0131] The high-frequency sub-band image i is divided into multiple regions, and the feature point distribution density of each region is determined to obtain multiple feature point distribution densities.

[0132] Determine the target mean square error corresponding to the distribution density of the plurality of feature points;

[0133] According to the preset mapping relationship between the ratio and the enhancement coefficient, the first enhancement coefficient corresponding to the target ratio is determined;

[0134] According to the preset mapping relationship between the mean square error and the fine-tuning coefficient, the target fine-tuning coefficient corresponding to the target mean square error is determined;

[0135] The first enhancement coefficient is adjusted according to the target fine-tuning coefficient to obtain the enhancement coefficient of the high-frequency sub-band image i.

[0136] Optionally, in determining the enhancement coefficients of S high-frequency sub-band images among the K high-frequency sub-band images to obtain S enhancement coefficients, the determining unit 403 is specifically used for:

[0137] Determine the first energy value of the low-frequency subband image;

[0138] Determine the second energy value of the image to be processed;

[0139] Determine the target energy ratio between the first energy value and the second energy value;

[0140] According to the preset mapping relationship between energy ratio and enhancement coefficient, the reference enhancement coefficient corresponding to the target energy ratio is determined, and the reference enhancement coefficient is used as the enhancement coefficient of S high-frequency sub-band images in the K high-frequency sub-band images.

[0141] Optionally, after acquiring the image to be processed and before performing multi-scale transformation on the image to be processed, the device 400 is further specifically used for:

[0142] The image to be processed is subjected to image quality evaluation to obtain an image quality evaluation value;

[0143] Obtain target environment parameters;

[0144] Obtain target shooting parameters;

[0145] According to the preset mapping relationship between environmental parameters and the first image quality evaluation value range, the first reference image quality evaluation value range corresponding to the target environmental parameters is determined;

[0146] According to the preset mapping relationship between shooting parameters and the second image quality evaluation value range, the second reference image quality evaluation value range corresponding to the target shooting parameters is determined;

[0147] Determine the target intersection between the range of quality evaluation values ​​for the first reference image and the range of quality evaluation values ​​for the second reference image;

[0148] When the image quality evaluation value is not within the target intersection, the step of performing multi-scale transformation on the image to be processed is executed.

[0149] Optionally, in terms of performing image quality evaluation on the image to be processed to obtain an image quality evaluation value, the device 400 is specifically used for:

[0150] The target region is obtained by extracting the target from the image to be processed;

[0151] The target region is subjected to image quality evaluation to obtain the image quality evaluation value.

[0152] This application also provides a computer storage medium storing a computer program for electronic data interchange, which causes a computer to perform some or all of the steps of any of the methods described in the above method embodiments, wherein the computer includes an electronic device.

[0153] This application also provides a computer program product, which includes a non-transitory computer-readable storage medium storing a computer program operable to cause a computer to perform some or all of the steps of any of the methods described in the above method embodiments. The computer program product may be a software installation package, and the computer may include an electronic device.

[0154] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that this application is not limited to the described order of actions, as some steps may be performed in other orders or simultaneously according to this application. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions and modules involved are not necessarily essential to this application.

[0155] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions in other embodiments.

[0156] In the several embodiments provided in this application, it should be understood that the disclosed apparatus can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of the units described above is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical or other forms.

[0157] The units described above as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0158] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0159] If the integrated units described above are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage device (CMD). Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a memory and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned memory includes various media capable of storing program code, such as USB flash drives, read-only memory (ROM), random access memory (RAM), portable hard drives, magnetic disks, or optical disks.

[0160] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, which may include: flash drive, read-only memory (ROM), random access memory (RAM), disk or optical disk, etc.

[0161] The embodiments of this application have been described in detail above. Specific examples have been used to illustrate the principles and implementation methods of this application. The description of the above embodiments is only for the purpose of helping to understand the method and core ideas of this application. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this application. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A video processing apparatus, characterized in that, The video processing device includes: an acquisition unit, a decomposition unit, a determination unit, and a reconstruction unit, wherein, The acquisition unit is used to acquire an image to be processed, wherein the image to be processed is any frame image in the target video; The decomposition unit is used to perform multi-scale transformation on the image to be processed to obtain a low-frequency sub-band image and K high-frequency sub-band images, where K is a positive integer. The determining unit is used to determine the enhancement coefficients of S high-frequency sub-band images among the K high-frequency sub-band images, thereby obtaining S enhancement coefficients, where S is a positive integer less than or equal to K; The reconstruction unit is used to perform an inverse transformation corresponding to the multi-scale transformation based on the S enhancement coefficients, the low-frequency sub-band image, and the K high-frequency sub-band images to obtain the target image.

2. The video processing apparatus according to claim 1, characterized in that, In determining the enhancement coefficients of S high-frequency sub-band images out of the K high-frequency sub-band images, and obtaining S enhancement coefficients, the determining unit is specifically used for: Determine the target ratio between the resolution i of the high-frequency sub-band image i and the resolution of the image to be processed, wherein the high-frequency sub-band image i is any high-frequency sub-band image among the K high-frequency sub-band images; The high-frequency sub-band image i is divided into multiple regions, and the feature point distribution density of each region is determined to obtain multiple feature point distribution densities. Determine the target mean square error corresponding to the distribution density of the plurality of feature points; According to the preset mapping relationship between the ratio and the enhancement coefficient, the first enhancement coefficient corresponding to the target ratio is determined; According to the preset mapping relationship between the mean square error and the fine-tuning coefficient, the target fine-tuning coefficient corresponding to the target mean square error is determined; The first enhancement coefficient is adjusted according to the target fine-tuning coefficient to obtain the enhancement coefficient of the high-frequency sub-band image i.

3. The video processing apparatus according to claim 1, characterized in that, In determining the enhancement coefficients of S high-frequency sub-band images out of the K high-frequency sub-band images, and obtaining S enhancement coefficients, the determining unit is specifically used for: Determine the first energy value of the low-frequency subband image; Determine the second energy value of the image to be processed; Determine the target energy ratio between the first energy value and the second energy value; According to the preset mapping relationship between energy ratio and enhancement coefficient, the reference enhancement coefficient corresponding to the target energy ratio is determined, and the reference enhancement coefficient is used as the enhancement coefficient of S high-frequency sub-band images in the K high-frequency sub-band images.

4. The video processing apparatus according to any one of claims 1-3, characterized in that, After acquiring the image to be processed and before performing multi-scale transformation on the image to be processed, the video processing device is further specifically used for: The image to be processed is subjected to image quality evaluation to obtain an image quality evaluation value; Obtain target environment parameters; Obtain target shooting parameters; According to the preset mapping relationship between environmental parameters and the first image quality evaluation value range, the first reference image quality evaluation value range corresponding to the target environmental parameters is determined; According to the preset mapping relationship between shooting parameters and the second image quality evaluation value range, the second reference image quality evaluation value range corresponding to the target shooting parameters is determined; Determine the target intersection between the range of quality evaluation values ​​for the first reference image and the range of quality evaluation values ​​for the second reference image; When the image quality evaluation value is not within the target intersection, the step of performing multi-scale transformation on the image to be processed is executed.

5. The video processing apparatus according to claim 4, characterized in that, In terms of evaluating the image quality of the image to be processed and obtaining an image quality evaluation value, the video processing device is specifically used for: The target region is obtained by extracting the target from the image to be processed; The target region is subjected to image quality evaluation to obtain the image quality evaluation value.

6. A video processing method, characterized in that, The method includes: Acquire the image to be processed, which is any frame image in the target video; The image to be processed is subjected to multi-scale transformation to obtain a low-frequency sub-band image and K high-frequency sub-band images, where K is a positive integer; Determine the enhancement coefficients of S high-frequency sub-band images out of the K high-frequency sub-band images to obtain S enhancement coefficients, where S is a positive integer less than or equal to K; Based on the S enhancement coefficients, the low-frequency sub-band image, and the K high-frequency sub-band images, an inverse transform corresponding to the multi-scale transform is performed to obtain the target image.

7. The video processing method according to claim 6, characterized in that, The process of determining the enhancement coefficients of S high-frequency sub-band images out of the K high-frequency sub-band images, resulting in S enhancement coefficients, includes: Determine the target ratio between the resolution i of the high-frequency sub-band image i and the resolution of the image to be processed, wherein the high-frequency sub-band image i is any high-frequency sub-band image among the K high-frequency sub-band images; The high-frequency sub-band image i is divided into multiple regions, and the feature point distribution density of each region is determined to obtain multiple feature point distribution densities. Determine the target mean square error corresponding to the distribution density of the plurality of feature points; According to the preset mapping relationship between the ratio and the enhancement coefficient, the first enhancement coefficient corresponding to the target ratio is determined; According to the preset mapping relationship between the mean square error and the fine-tuning coefficient, the target fine-tuning coefficient corresponding to the target mean square error is determined; The first enhancement coefficient is adjusted according to the target fine-tuning coefficient to obtain the enhancement coefficient of the high-frequency sub-band image i.

8. An electronic device, characterized in that, It includes a processor, a memory, a communication interface, and one or more programs, said one or more programs being stored in the memory and configured to be executed by the processor, said programs including instructions for performing the steps of the method as claimed in claim 6 or 7.

9. A computer-readable storage medium, characterized in that, A computer program for electronic data interchange is stored, wherein the computer program causes a computer to perform the method as described in claim 6 or 7.

10. A computer program product, characterized in that, It includes a computer program stored in a computer-readable storage medium; when the processor of the electronic device reads the computer program from the computer-readable storage medium, the processor executes the computer program, causing the electronic device to perform the method of any one of claims 6 or 7.

Images