Image processing method and device
By determining the critical value of encoding distortion in image processing and selecting the appropriate resolution to encode the image data, the problems of redundancy and insufficient code in the prior art are solved, and the effect of high-definition and low code is achieved.
Patent Information
- Application Number
- CN202410967582.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-18
- Publication Date
- 2024-11-29
- Estimated Expiration
- 2044-07-18
AI Technical Summary
The prior art is difficult to encode image data using a lower code rate while ensuring high-definition image quality, resulting in the inability to achieve high-definition and low-code effects.
By encoding the image data with the first resolution, the first code stream information and the first quantization parameters are obtained, and the first decision quantization parameter value of the image data is determined based on the first resolution and the second resolution, and the parameter value is The encoding distortion threshold value of the image data at the first resolution is then processed based on the first quantization parameter and the first decision quantization parameter value, output the second code stream information or encode it with a resolution smaller than the first resolution. The obtained code stream information.
It realizes that the code rate is low during the encoding process while ensuring image quality, avoids redundancy and insufficient code rate, and ensures the high-definition and low-code effect of electronic devices on images.
Smart Images

Figure CN119052572A_ABST
Abstract
Description
Technical Field
[0001] The present application belongs to the technical field of streaming media, and specifically relates to an image processing method and a device thereof. Background Art
[0002] In daily life, users can store or transmit images, such as pictures or videos, through electronic devices. In order to reduce the memory required for image storage or the bandwidth required for image transmission, and to ensure the high-definition quality of the image, electronic devices can encode the image data at a lower bit rate to achieve high-definition and low-code images.
[0003] Usually, electronic devices can set a fixed bit rate to encode image data according to the specifications of the image data (such as resolution or frame rate, etc.). However, since electronic devices directly encode image data using a fixed bit rate, the encoding effect of the image cannot be guaranteed, and thus the current image processing method cannot achieve the effect of high definition and low bit rate. Summary of the invention
[0004] The purpose of the embodiments of the present application is to provide an image processing method and device thereof, which can encode image data at a lower bit rate while ensuring high-definition image quality, thereby achieving a high-definition and low-code effect.
[0005] In a first aspect, an embodiment of the present application provides an image processing method, the method comprising: encoding image data at a first resolution to obtain first code stream information and a first quantization parameter; determining a first decision quantization parameter value of the image data based on the first resolution and the second resolution, the first decision quantization parameter value being a coding distortion critical value of the image data at the first resolution, and the second resolution being smaller than the first resolution; processing the image data based on the first quantization parameter and the first decision quantization parameter value and outputting second code stream information, the second code stream information being the first code stream information, or being code stream information obtained by encoding at a resolution smaller than the first resolution.
[0006] In a second aspect, an embodiment of the present application provides an image processing device, which includes: an encoding module, a determination module, and an output module. The encoding module is used to encode image data using a first resolution to obtain first code stream information and a first quantization parameter; the determination module is used to determine a first decision quantization parameter value of the image data based on the first resolution and the second resolution, the first decision quantization parameter value is a coding distortion critical value of the image data at the first resolution, and the second resolution is smaller than the first resolution; the output module is used to process the image data based on the first quantization parameter obtained by the encoding module and the first decision quantization parameter value determined by the determination module and output second code stream information, the second code stream information is the first code stream information, or is code stream information obtained by encoding at a resolution smaller than the first resolution.
[0007] In a third aspect, an embodiment of the present application provides an electronic device, which includes a processor and a memory, wherein the memory stores programs or instructions that can be run on the processor, and when the program or instructions are executed by the processor, the steps of the method described in the first aspect are implemented.
[0008] In a fourth aspect, an embodiment of the present application provides a readable storage medium, on which a program or instruction is stored, and when the program or instruction is executed by a processor, the steps of the method described in the first aspect are implemented.
[0009] In a fifth aspect, an embodiment of the present application provides a chip, comprising a processor and a communication interface, wherein the communication interface is coupled to the processor, and the processor is used to run a program or instruction to implement the method described in the first aspect.
[0010] In a sixth aspect, an embodiment of the present application provides a computer program product, which is stored in a storage medium and is executed by at least one processor to implement the method described in the first aspect.
[0011] In an embodiment of the present application, the image data can be encoded using a first resolution to obtain first code stream information and a first quantization parameter, and based on the first resolution and a second resolution smaller than the first resolution, a first decision quantization parameter value of the image data can be determined, and the first quantization parameter value is a coding distortion critical value of the image data at the first resolution. Then, based on the first quantization parameter and the first decision quantization parameter value, the image data can be processed to output the first code stream information, or the code stream information obtained by encoding at a resolution smaller than the first resolution can be output. In the present solution, since the electronic device can encode the image data with the first resolution to obtain the first quantization parameter, and the electronic device can determine the first decision quantization parameter value of the image data based on the first resolution and the second resolution, the electronic device can first encode the image data with any one resolution to obtain the first quantization parameter, and then determine the encoding distortion critical value of the image data at the current resolution based on a resolution smaller than the any one resolution, and then judge the output code stream information based on the encoding distortion critical value, that is, output the first code stream information, or the code stream information obtained by encoding with a resolution smaller than the first resolution, that is, based on the first quantization parameter and the encoding distortion critical value, it is possible to determine the code stream information whose encoding distortion degree does not exceed the encoding distortion critical value and whose encoding uses a lower resolution. Therefore, by selecting a suitable resolution to encode the image data, redundancy and insufficiency of the code rate are avoided, so that the image data can achieve a lower code rate while ensuring the image quality during the encoding process, thereby ensuring the high-definition and low-code effect when the electronic device encodes the image. BRIEF DESCRIPTION OF THE DRAWINGS
[0012] Figure 1 It is one of the flow charts of the image processing method provided in the embodiment of the present application;
[0013] Figure 2 This is the second flowchart of the image processing method provided in the embodiment of the present application;
[0014] Figure 3 This is the third flowchart of the image processing method provided in the embodiment of the present application;
[0015] Figure 4 This is the fourth flowchart of the image processing method provided in the embodiment of the present application;
[0016] Figure 5 This is the fifth flowchart of the image processing method provided in the embodiment of the present application;
[0017] Figure 6 is a structural diagram of an image enhancement method provided in an embodiment of the present application;
[0018] Figure 7This is the sixth flowchart of the image processing method provided in the embodiment of the present application;
[0019] Figure 8 is a structural diagram of another image enhancement method provided in an embodiment of the present application;
[0020] Fig. 9 is a schematic diagram of the execution process of the image processing method provided in the embodiment of the present application;
[0021] Fig.10 is a schematic diagram of an image processing device provided in an embodiment of the present application;
[0022] Fig.11 is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application;
[0023] Fig.12 It is a schematic diagram of the hardware structure of the electronic device provided in the embodiment of the present application. DETAILED DESCRIPTION
[0024] The following will be combined with the drawings in the embodiments of the present application to clearly describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments in the present application belong to the scope of protection of this application.
[0025] The terms "first", "second", etc. in the specification and claims of the present application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable under appropriate circumstances, so that the embodiments of the present application can be implemented in an order other than those illustrated or described here, and the objects distinguished by "first", "second", etc. are generally of one type, and the number of objects is not limited. For example, the first object can be one or more. In addition, "and / or" in the specification and claims represents at least one of the connected objects, and the character " / " generally indicates that the objects associated with each other are in an "or" relationship.
[0026] The terms "at least one (item)", "at least one of" and the like in this application refer to any one, any two or a combination of more than two of the objects included therein. For example, at least one (item) of a, b, and c can be represented by: "a", "b", "c", "a and b", "a and c", "b and c" and "a, b and c", where a, b, and c can be single or multiple. Similarly, "at least two (items)" refers to two or more, and its meaning is similar to that of "at least one (item)".
[0027] Some terms and nouns involved in the embodiments of the present application are explained below.
[0028] Coding: refers to the use of technology to remove redundancy within and between coded frames to achieve efficient data compression. Commonly used technologies include predictive coding, transform coding and entropy coding, represented by traditional coding such as H.264 / AVC, H.265 / HEVC, H.266 / VVC, JPEG, etc., and emerging technologies use neural networks for data compression.
[0029] Bit rate: The number of bits output per unit time, usually expressed in kbps. For the same image / video, the higher the encoding bit rate, the better the encoding quality and the lower the compression rate.
[0030] Quantization parameter: The quantization step size used in transform coding, also known as QP (Quantization Parameter). The only source of coding lossy error. The larger the QP, the worse the quality and the higher the compression rate.
[0031] Decompression artifacts: Remove the distortion caused by encoding. Distortion generally includes blurring, blockiness, noise, rings and other defects. Decompression artifacts are a major form of codec enhancement algorithms.
[0032] Enhancement: refers to processing degraded features through algorithms to improve the quality and clarity of images / videos, but is not limited to denoising, sharpening, super-resolution, decompression distortion and other technologies.
[0033] High-definition low-code: also known as narrowband high-definition, refers to the use of a lower bit rate during encoding to achieve high-definition image quality.
[0034] The image processing method provided in the embodiment of the present application is described in detail below through specific embodiments and their application scenarios in conjunction with the accompanying drawings.
[0035] The embodiments of the present application can be applied to a scenario where an image is encoded. Specifically, the electronic device encodes the image data to store or transmit code stream information.
[0036] Usually, in order to reduce the memory required for image storage or the bandwidth required for image transmission, and to ensure the high-definition quality of the image, the electronic device needs to set a fixed bit rate to encode the image data according to the specifications of the image data. However, since the electronic device uses a fixed bit rate to encode the image data, the bit rate will be redundant in some images, resulting in a waste of resources, and the bit rate will be insufficient in some images, resulting in the image being unable to achieve high-definition quality and unable to guarantee the encoding effect of the entire image, so the current image processing method has a poor effect in achieving high-definition and low-code.
[0037] An embodiment of the present application provides an image processing method, which can encode image data at a first resolution to obtain first code stream information and a first quantization parameter, and determine a first decision quantization parameter value of the image data based on the first resolution and a second resolution smaller than the first resolution, where the first quantization parameter value is a coding distortion critical value of the image data at the first resolution, and then process the image data based on the first quantization parameter and the first decision quantization parameter value to output the first code stream information, or output the code stream information obtained by encoding at a resolution smaller than the first resolution. In the present solution, since the electronic device can encode the image data with the first resolution to obtain the first quantization parameter, and the electronic device can determine the first decision quantization parameter value of the image data based on the first resolution and the second resolution, the electronic device can first encode the image data with any one resolution to obtain the first quantization parameter, and then determine the encoding distortion critical value of the image data at the current resolution based on a resolution smaller than the any one resolution, and then judge the output code stream information based on the encoding distortion critical value, that is, output the first code stream information, or the code stream information obtained by encoding with a resolution smaller than the first resolution, that is, based on the first quantization parameter and the encoding distortion critical value, it is possible to determine the code stream information whose encoding distortion degree does not exceed the encoding distortion critical value and whose encoding uses a lower resolution. Therefore, by selecting a suitable resolution to encode the image data, redundancy and insufficiency of the code rate are avoided, so that the image data can achieve a lower code rate while ensuring the image quality during the encoding process, thereby ensuring the high-definition and low-code effect when the electronic device encodes the image.
[0038] The execution subject of the image processing method provided in the embodiment of the present application may be an image processing device, or an electronic device, or a functional module in an electronic device. The following takes an electronic device as an example to illustrate the technical solution provided in the embodiment of the present application.
[0039] Figure 1 A flowchart of an image processing method provided by an embodiment of the present application is shown. Figure 1 As shown, the image processing method provided in the embodiment of the present application may include the following steps 201 to 203.
[0040] Step 201: An electronic device encodes image data using a first resolution to obtain first bit stream information and a first quantization parameter.
[0041] Optionally, in the embodiment of the present application, the first resolution may be any one of a set of preset resolutions, or a user-defined resolution. The set of preset resolutions includes at least two preset resolutions available for encoding image data.
[0042] In an embodiment of the present application, the electronic device can start encoding the image data at any resolution (for example, the maximum resolution in a preset resolution set) to determine whether to reselect other suitable resolutions to encode the image data based on the obtained quantization parameters, thereby storing or transmitting the code stream information obtained by encoding at the suitable resolution.
[0043] In the embodiment of the present application, the above-mentioned image data is a matrix composed of image or video frame pixels.
[0044] Optionally, in an embodiment of the present application, the image data may be unprocessed (e.g., compressed) original picture data or original video frame data, or may be processed picture data or video frame data. Wherein, in the case where the image data is a frame of image data in a video, each frame of image data in the video may be encoded by selecting a suitable resolution according to the solution of the present application to obtain the code stream information of each frame of image data.
[0045] Optionally, in the embodiment of the present application, the above-mentioned image data may be in the form of two-dimensional image data or three-dimensional image data.
[0046] In the embodiment of the present application, encoding is the process of outputting code stream information by calling an encoding tool to encode at a given bit rate and resolution. Encoding is to remove intra-frame and inter-frame redundancy of the encoded frame through technology to achieve efficient data compression.
[0047] Optionally, in an embodiment of the present application, the above-mentioned encoding tools may include but are not limited to at least one of the following: Versatile Video Coding (H.266 / VVC) represented by predictive coding, transform coding and entropy coding, High Efficiency Video Coding (H.265 / HEVC), Advanced Video Coding (H.264 / AVC), Joint Photographic Experts Group (JPEG) and other commonly used encoding methods, and may also be a neural network-based encoding method.
[0048] It should be noted that the above-mentioned bit rate is the number of bits output per unit time. For the same image, the higher the encoding bit rate, the better the encoding quality and the lower the compression rate. When the bit rate is fixed, the resolution is inversely proportional to the image quality (such as clarity). The higher the resolution, the less clear the image, and the lower the resolution, the clearer the image. When the resolution is fixed, the bit rate is directly proportional to the clarity. The higher the bit rate, the clearer the image, and the lower the bit rate, the less clear the image. In the embodiment of the present application, the electronic device selects a suitable resolution so that the image data can achieve a lower bit rate during the encoding process while ensuring better image quality, that is, achieving the effect of high definition and low code.
[0049] In the embodiment of the present application, the first code stream information is the code stream information corresponding to the image data at the first resolution, also known as coded data, which is usually a series of binary sequences or other forms of data, which can effectively reduce the size of the image file and facilitate storage and transmission. Optionally, the coded data of the image can be a compressed image file, such as a file in the format of the Joint Photographic Experts Group, Portable Network Graphics (PNG), Graphics Interchange Format (GIF), etc.; it can also be descriptive data of the image, such as vector graphics data.
[0050] In the embodiment of the present application, the above-mentioned first quantization parameter is the quantization parameter corresponding to the image data at the first resolution, also known as the quantization step size (Quantization Parameter, QP) used in transform coding. Since the coding adopts the form of lossy compression, compression distortion, also known as lossy error, such as blur effect, block effect, noise, ring effect, etc., will be caused during image coding. The quantization parameter is the only source of lossy coding error. The larger the quantization parameter used in image coding, the more serious the compression distortion problem, the worse the image quality, and the higher the image compression rate; the smaller the quantization parameter used in image coding, the milder the compression distortion problem, the better the image quality, and the lower the image compression rate. The embodiment of the present application selects a suitable resolution encoding by judging the quantization parameter, so that the image data can achieve a low bit rate while ensuring good image quality during the encoding process.
[0051] Step 202: The electronic device determines a first decision quantization parameter value of the image data based on the first resolution and the second resolution.
[0052] In the embodiment of the present application, the first decision quantization parameter value is a coding distortion critical value of the image data at a first resolution, and the second resolution is smaller than the first resolution.
[0053] Optionally, in an embodiment of the present application, the second resolution may be a resolution in a preset resolution set, or may be a resolution obtained by reducing the first resolution to a preset ratio (for example, to 90% of the first resolution) by the electronic device.
[0054] In the embodiment of the present application, when the second resolution is a resolution in a preset resolution set, the second resolution may be any resolution in the preset resolution set that is smaller than the first resolution.
[0055] In the embodiment of the present application, the electronic device can use the resolution decision algorithm to process the current frame (i.e., the above-mentioned image data) to obtain the decision QP value of the frame at the current resolution (i.e., the first resolution) and the second resolution, i.e., the coding distortion threshold, which is used to determine whether the lossy error generated by the encoding of the image data at the first resolution is within the allowable limit. The embodiment of the present application can make decisions from the maximum resolution in the preset resolution, and select a suitable and larger resolution for the frame according to the quantization parameter and decision QP value corresponding to each resolution, breaking the fixed resolution encoding method of a given bit rate, effectively removing bit rate redundancy, and obtaining a higher definition effect.
[0056] Optionally, in an embodiment of the present application, the above-mentioned resolution decision algorithm may include but is not limited to at least one of the following: a machine learning support vector regression algorithm, a neural network regression algorithm, etc.
[0057] Step 203: The electronic device processes the image data based on the first quantization parameter and the first decision quantization parameter value and outputs second code stream information.
[0058] In the embodiment of the present application, the second bit stream information is the first bit stream information, or is bit stream information encoded with a resolution smaller than the first resolution.
[0059] In the embodiment of the present application, the electronic device can compare the magnitude relationship between the first quantization parameter and the first decision quantization parameter value to determine whether to output the first code stream information, or to encode the image data at a resolution smaller than the first resolution and output the corresponding code stream information.
[0060] In the embodiment of the present application, when the first quantization parameter is less than or equal to the first decision quantization parameter value, it is considered that the coding distortion degree of the first resolution coding is small and within the acceptable range, that is, the lossy error generated by the coding of the image data is within the allowable limit. Alternatively, when the first quantization parameter is greater than the first decision quantization parameter value, it is considered that the coding distortion degree of the first resolution coding is large and not within the acceptable range, that is, the lossy error generated by the coding of the image data is outside the allowable limit.
[0061] Optionally, in the embodiment of the present application, combined with Figure 1 ,like Figure 2 As shown, the above step 203 can be specifically implemented through the following step 301.
[0062] Step 301: When a first quantization parameter is less than or equal to a first decision quantization parameter value, the electronic device outputs first code stream information.
[0063] In this way, when the first quantization parameter is less than or equal to the first decision quantization parameter value, the electronic device determines that at the first resolution, the lossy error generated by the electronic device encoding the image data is within the allowable limit, so as to trigger the output of the code stream information obtained by the first resolution encoding, so that the electronic device outputs the code stream information in which the lossy error generated by the encoding is within the allowable limit, thereby reducing the lossy error of the encoding and improving the image quality.
[0064] Optionally, in the embodiment of the present application, combined with Figure 1 ,like Figure 3 As shown, the above step 203 can be specifically implemented through the following steps 401 to 403.
[0065] Step 401: When the first quantization parameter is greater than the first decision quantization parameter value, the electronic device encodes the image data using the second resolution to obtain third code stream information and the second quantization parameter.
[0066] Step 402: The electronic device determines a second decision quantization parameter value of the image data based on the second resolution and the third resolution.
[0067] In the embodiment of the present application, the second decision quantization parameter value is a coding distortion critical value of the image data at the second resolution, and the third resolution is smaller than the second resolution.
[0068] Optionally, in an embodiment of the present application, the second resolution may be a resolution in a preset resolution set, or may be a resolution obtained by reducing the first resolution to a preset ratio (for example, to 90% of the first resolution) by the electronic device.
[0069] In the embodiment of the present application, when the second resolution is a resolution in a preset resolution set, the second resolution may be any resolution in the preset resolution set that is smaller than the first resolution.
[0070] Optionally, in an embodiment of the present application, the third resolution may be a resolution in a preset resolution set, or may be a resolution obtained by reducing the second resolution to a preset ratio (for example, to 90% of the second resolution) by the electronic device.
[0071] In the embodiment of the present application, when the third resolution is a resolution in the preset resolution set, the third resolution may be any resolution in the preset resolution set that is smaller than the second resolution.
[0072] Step 403: When the second quantization parameter is less than or equal to the second decision quantization parameter value, the electronic device outputs third code stream information.
[0073] It should be noted that the implementation of steps 401 to 403 is similar to the implementation of steps 201 to 203 described above, and reference may be made to the description of the above embodiment, which will not be described in detail here.
[0074] It can be understood that in the embodiment of the present application, when the first quantization parameter is greater than the first decision quantization parameter value, the electronic device can encode the image data using the i-th resolution to obtain the i-th quantization parameter; then, the electronic device can determine the i-th decision quantization parameter value of the image data based on the i-th resolution and the i+1-th resolution, the i-th decision quantization parameter value is the coding distortion critical value of the image data at the i-th resolution, and the i+1-th resolution is less than the i-th resolution; when the N-th quantization parameter is less than or equal to the N-th decision quantization parameter value, the code stream information obtained by encoding the image data using the N-th resolution is output. Wherein, i and N are positive integers.
[0075] It should be noted that the electronic device can start from any resolution, for example, the electronic device can start from the maximum resolution in the preset resolution set, that is, when i=1, the first resolution is the maximum resolution in the preset resolution set. When the electronic device encodes the image data at a resolution lower than the preset minimum resolution, it can directly output the code stream information obtained at the resolution, and the preset minimum resolution is the minimum resolution in the preset resolution set.
[0076] In an embodiment of the present application, when the second quantization parameter is greater than the second decision quantization parameter value, the electronic device may encode the image data using the third resolution to obtain fourth code stream information and a fourth quantization parameter. The electronic device may then determine the third decision quantization parameter value of the image data based on the third resolution and the fourth resolution, the fourth resolution being less than the third resolution, and when the fourth quantization parameter is less than or equal to the third decision quantization parameter value, the electronic device outputs the fourth code stream information. Similarly, when the electronic device determines that the lossy error generated by encoding the image data using the current resolution exceeds the allowable limit, the electronic device may select a resolution less than the current resolution, compress the image data to the resolution, and repeat the above steps until the electronic device determines that the lossy error generated by encoding the image data is within the allowable limit, and then outputs the code stream information encoded using the resolution.
[0077] Exemplarily, the electronic device encodes the image data using resolution a to obtain quantization parameter a1 and code stream information a2, the electronic device determines a resolution b that is smaller than resolution a, and determines a decision quantization parameter value a3 of the image data based on resolution a and resolution b. If the quantization parameter a1 is greater than the decision quantization parameter value a3, it is considered that the condition for outputting code stream information a2 is not met (i.e., the lossy error generated by encoding is outside the allowable limit), so the electronic device continues to encode the image data using resolution b to obtain quantization parameter b1 and code stream information b2, the electronic device determines a resolution c that is smaller than resolution b, and determines a decision quantization parameter value b3 of the image data based on resolution b and resolution c. If the quantization parameter b1 is greater than the decision quantization parameter value b3, it is considered that the condition for outputting code stream information b2 is not met, so the electronic device continues to encode the image data using resolution c to obtain quantization parameter c1 and code stream information c2, the electronic device determines a resolution d that is smaller than resolution c, and determines a decision quantization parameter value c3 of the image data based on resolution c and resolution d. If the quantization parameter c1 is greater than the decision quantization parameter value c3, it is considered that the condition for outputting the code stream information c2 is not met, so the electronic device continues to encode the image data using the resolution d and determines whether the resolution d meets the condition for outputting the code stream information. Repeat the above steps until the quantization parameter x1 obtained by the electronic device encoding the image data using the resolution x is less than or equal to the decision quantization parameter x3, so that the electronic device can output the code stream information x2.
[0078] Optionally, in the embodiment of the present application, the electronic device outputting the third code stream information may be that the electronic device transmits the third code stream information through a network, or the electronic device stores the third code stream information in a local file.
[0079] In this way, when the lossy error of the code stream information corresponding to the first resolution exceeds the allowable limit, the electronic device can determine whether the next low resolution meets the conditions for outputting the code stream information, and output the code stream information corresponding to the resolution if the conditions are met. As a result, the electronic device outputs the code stream information in which the lossy error generated by encoding is within the allowable limit, thereby reducing the lossy error of encoding and improving image quality.
[0080] The embodiment of the present application provides an image processing method. Since the electronic device can encode the image data with a first resolution to obtain a first quantization parameter, and the electronic device can determine the first decision quantization parameter value of the image data based on the first resolution and the second resolution, the electronic device can first encode the image data with any one resolution to obtain the first quantization parameter, and then determine the encoding distortion critical value of the image data at the current resolution according to a resolution smaller than the any one resolution, and then judge the output code stream information based on the encoding distortion critical value, that is, output the first code stream information, or the code stream information obtained by encoding with a resolution smaller than the first resolution, that is, based on the first quantization parameter and the encoding distortion critical value, it is possible to determine the code stream information whose encoding distortion degree does not exceed the encoding distortion critical value and whose encoding uses a lower resolution. Therefore, by selecting a suitable resolution to encode the image data, redundancy and insufficiency of the code rate are avoided, so that the image data can achieve a lower code rate while ensuring the image quality during the encoding process, thereby ensuring the high-definition and low-code effect when the electronic device encodes the image.
[0081] Optionally, in the embodiment of the present application, combined with Figure 1 ,like Figure 4 As shown, after the above step 203, the image processing method provided in the embodiment of the present application further includes the following steps 501 and 502.
[0082] Step 501: The electronic device decodes the second code stream information to obtain a pixel value matrix and a third quantization parameter corresponding to the image data.
[0083] In the embodiment of the present application, decoding is the process of converting the code stream information obtained by encoding the image data into the original form of the image data.
[0084] In the embodiment of the present application, the above-mentioned pixel value matrix is a way to represent an image, which is a two-dimensional array, in which each element represents the value of the pixel at the corresponding position in the image. For a grayscale image, the value of each pixel is usually between 0 and 255, indicating the grayscale value of the pixel; and for a color image, each pixel is usually composed of three values (red, green, and blue), and the range of each value is also between 0 and 255, indicating the color intensity of the pixel in the three channels of red, green, and blue.
[0085] In the embodiment of the present application, the quantization parameter can be directly obtained from the bitstream information, reflects the degree of coding distortion, and is the only source of coding lossy error.
[0086] Optionally, in an embodiment of the present application, the third quantization parameter may be a single value at the frame QP level (which may be referred to as a frame quantization parameter), or may be a matrix at the QP map level (which may be referred to as a quantization parameter matrix).
[0087] It should be noted that the above frame quantization parameter is a fixed value and applies to the entire frame. The above quantization parameter matrix allows different quantization parameters to be set for different areas within the same frame. In this way, while keeping the overall bit rate unchanged, high image quality can be given to important areas in the picture (such as faces, text, etc.), while the quality of secondary areas such as the background can be appropriately reduced to reduce the bit rate. The quantization parameter matrix can be a two-dimensional array, in which each element represents the quantization parameter value of the corresponding position in the picture.
[0088] Step 502: The electronic device performs image quality enhancement processing on the pixel value matrix based on the third quantization parameter, and outputs an image frame corresponding to the pixel value matrix.
[0089] In the embodiment of the present application, the above-mentioned image quality enhancement processing is to process the degraded features in the image data through an enhancement algorithm, remove artifacts, improve the quality and clarity of the image, and obtain a higher definition effect.
[0090] Optionally, in an embodiment of the present application, the above-mentioned image quality enhancement may include but is not limited to at least one of the following: denoising, sharpening, super-resolution, decompression distortion, etc.
[0091] Optionally, in an embodiment of the present application, the above-mentioned enhancement algorithm may include but is not limited to at least one of the following: a convolutional neural network, a recurrent neural network, and a transformer network.
[0092] Optionally, in the embodiment of the present application, when the image frame corresponding to the above pixel value matrix is a frame image, the electronic device can display the frame image in a picture format. Alternatively, when the image frame corresponding to the above pixel value matrix is a video frame image, the electronic device can play the video frame image through the playback module.
[0093] Optionally, in the embodiment of the present application, combined with Figure 4 ,like Figure 5 As shown, the above step 502 can be specifically implemented through the following steps 601 and 602.
[0094] Step 601: The electronic device converts a third quantization parameter into an intensity coefficient, and uses the intensity coefficient to adjust an intensity parameter in a preset enhancement network.
[0095] Optionally, in the embodiment of the present application, the electronic device may convert the third quantization parameter into an intensity coefficient between 0 and 1, for example, by converting in a piecewise linear manner, so as to adjust the intensity of the preset enhancement network (ie, the above-mentioned enhancement algorithm).
[0096] Step 602: The electronic device performs image quality enhancement processing on the pixel value matrix based on the preset enhancement network after adjusting the intensity parameters, and outputs an image frame corresponding to the pixel value matrix.
[0097] In an embodiment of the present application, the electronic device can input the pixel value matrix into a preset enhancement network after adjusting the intensity parameters to perform enhancement processing on the pixel value matrix to obtain an enhanced image frame, and calculate the difference between the enhanced image frame and the pixel value matrix, and then calculate the product of the difference and the above-mentioned intensity coefficient, and then use the sum of the pixel value matrix and the product as the image frame corresponding to the pixel value matrix, that is, to obtain an enhanced high-definition image frame.
[0098] For example, Figure 6 As shown, a structural diagram of an image enhancement method is shown. The electronic device can input the pixel value matrix into a preset enhancement network (such as an AI neural network) for enhancement processing to obtain an enhanced image frame, and then convert it into a 0-1 intensity coefficient QP_coff according to the QP value (i.e., the third quantization parameter), and adjust the intensity of the preset enhancement network (such as the AI neural network) to output an enhanced high-definition image frame.
[0099] For example, the decoded frame (pixel value matrix) is represented as input_img, the output of the AI neural network is temp_img, the enhanced high-definition image frame is represented as enhance_img, and the above intensity adjustment method is: enhance_img = input_img + (temp_img-input_img)*QP_coff.
[0100] In this way, this method can adjust the strength of the enhancement algorithm according to QP to achieve adaptive adjustment, and obtain better enhancement effect through intensity control. In addition, this method is decoupled from neural network training and can be inserted into other algorithms as a general tool. Therefore, not only the enhancement effect of the neural network enhancement algorithm can be adaptively adjusted, but also when the enhancement network is preset to other enhancement algorithms, adaptive adjustment can be performed to improve image clarity.
[0101] Optionally, in the embodiment of the present application, combined with Figure 4 ,like Figure 7 As shown, the above step 502 can be specifically implemented through the following steps 701 and 702.
[0102] Step 701: The electronic device extracts features from a pixel value matrix based on a third quantization parameter to obtain feature information of the pixel value matrix.
[0103] In the embodiment of the present application, the above-mentioned characteristic information is an attribute, variable or indicator used to describe or characterize the image data. The electronic device can use the third quantization parameter as a priori information, perform feature extraction on the pixel value matrix through network alignment, provide more clues to help the algorithm solve the compression distortion problem, restore image details, and thus improve image clarity.
[0104] It should be noted that the above feature extraction is the process of extracting meaningful information from an image. In machine learning and deep learning, feature extraction is usually performed through well-trained models that can automatically learn features that represent important attributes in the data. The quality of feature extraction directly affects the accuracy of subsequent image processing, such as object detection, scene understanding, and behavior analysis.
[0105] The above network alignment is to align image data from different sources or at different times into a common feature space during the feature extraction process. Alignment can ensure the compatibility and consistency of these data at the feature level, which is crucial for image recognition, classification and analysis. In deep learning, network alignment also involves the alignment of feature maps, which helps capture the intrinsic structure and appearance characteristics of the image.
[0106] Step 702: The electronic device performs image quality enhancement processing on the pixel value matrix based on the feature information, and outputs an image frame corresponding to the pixel value matrix.
[0107] For example, Figure 8 As shown, a structural diagram of another image enhancement method is shown. The electronic device can input the pixel value matrix into a preset enhancement network (such as an AI neural network) for enhancement processing to obtain an enhanced image frame, and then extract features from the pixel value matrix according to the QP value (i.e., the third quantization parameter) to obtain feature information of the pixel value matrix, thereby performing targeted enhancement processing on the pixel value matrix according to the extracted feature information to output an enhanced high-definition image frame.
[0108] In this way, the electronic device can use the third quantization parameter as prior information to perform feature extraction on the pixel value matrix, so that the electronic device can perform targeted enhancement processing on the pixel value matrix based on the extracted feature information, providing more clues to help the algorithm better remove artifacts, thereby better repairing image details, improving image clarity, and achieving higher-definition picture quality effects.
[0109] Optionally, in the embodiment of the present application, the image processing method provided in the embodiment of the present application also includes the following step 801.
[0110] Step 801: The electronic device adjusts the decision quantization parameter value of the image data based on the capability information.
[0111] In the embodiment of the present application, the capability information represents the image quality enhancement capability of the decoded image frame.
[0112] Optionally, in an embodiment of the present application, the electronic device may increase a decision quantization parameter value (such as the first decision quantization parameter value or the second decision quantization parameter value mentioned above) by a certain value, or may set a new decision quantization parameter value, which is specifically determined by the capabilities of the enhanced network and is not specifically limited in the embodiment of the present application.
[0113] In an embodiment of the present application, the electronic device can combine the enhanced algorithm capabilities of the decoding end during the encoding process, appropriately increase the resolution decision algorithm to switch the image to a low-resolution decision quantization parameter value. The numerical critical point, further increase the threshold for reducing the resolution, thereby reducing the bit rate required for high resolution.
[0114] In this way, when the enhancement algorithm capability at the decoding end is strong, more serious compression distortion problems can be solved, and the problem of insufficient bit rate when electronic devices encode image data at high resolution can be compensated, so that electronic devices can consider the enhancement algorithm capability at the decoding end during the encoding process, appropriately increase the numerical critical point of the decision quantization parameter value of the resolution decision algorithm switching to low resolution, and further increase the threshold for reducing resolution, so that electronic devices can encode at high resolution without changing the bit rate, thereby improving image clarity. In other words, by further strengthening the combination of encoding and enhancement, the bit rate required for high resolution can be reduced, and high-definition and low-code can be better achieved. Specifically, from the perspective of bit rate, encoding a higher resolution at the same bit rate effectively reduces the bit rate of high resolution; from the perspective of image quality, encoding a higher resolution retains more image details, making up for the shortcomings of the enhancement network in restoring insufficient realistic high-definition pixels, and obtaining higher-definition image quality.
[0115] Optionally, in an embodiment of the present application, the user can adjust the decision quantization parameter value of the image data. The electronic device includes an adjustment strength selection module and an adjustment QP module. The user adjustment strength selection module allows the user to choose whether to adjust the QP value. If the user believes that the current playback effect meets his requirements, no adjustment is made, otherwise adjustment is made. The QP adjustment module adjusts the specific QP value. If the user believes that the current enhancement effect is too strong, such as the picture quality is too smooth, the QP is reduced to weaken the enhancement effect; on the contrary, if the current enhancement effect is weak, such as the picture quality has obvious artifacts, the QP is increased to increase the enhancement effect.
[0116] In this way, users can adjust the intensity coefficient to trigger the electronic device to adjust the QP value, control the degree of distortion, and adjust the enhancement effect, thereby obtaining a high-definition visual effect that better meets personal needs.
[0117] Fig. 9Schematic diagram of the execution process of the image processing method provided in the embodiment of the present application. Fig. 9 As shown, the image processing method provided in the embodiment of the present application may include the following steps 10 to 18.
[0118] Step 10: The electronic device obtains input image data.
[0119] Step 11: The electronic device encodes the image data using the current resolution to obtain code stream information and quantization parameters.
[0120] Step 12: The electronic device obtains a decision quantization parameter value corresponding to the image data based on the current resolution and the next lower resolution.
[0121] It can be understood that the electronic device can select any resolution smaller than the current resolution as the next low resolution, for example, selected from a preset resolution set, or obtained by reducing the current resolution to a preset ratio.
[0122] Step 13: The electronic device determines whether the quantization parameter is greater than the decision quantization parameter value.
[0123] If the quantization parameter value is greater than the decision quantization parameter value, the following step 14 is performed.
[0124] Step 14: The electronic device reduces the resolution of the image data to the next lower resolution.
[0125] After executing the above step 14, the above step 11 is continued to be executed. At this time, the current resolution in step 11 and step 12 is the resolution selected in step 14 that is smaller than the first resolution.
[0126] If the quantization parameter value is less than or equal to the decision quantization parameter value, the following step 15 is performed.
[0127] Step 15: The electronic device transmits or stores the bit stream information encoded at the current resolution.
[0128] Step 16: The electronic device decodes the code stream information to obtain a pixel value matrix.
[0129] Step 17: The electronic device enhances the pixel value matrix to obtain an enhanced image frame.
[0130] Step 18: The electronic device plays the image frame.
[0131] It should be noted that the above steps 10 to 15 are the encoding process, that is, the execution steps of the encoding end, and the above steps 16 to 18 are the decoding process, that is, the execution steps of the decoding end.
[0132] In the embodiment of the present application, the resolution decision algorithm is used to select the appropriate resolution at the encoding end, retain more original information, reduce the encoding bit rate redundancy and thus reduce the bit rate. At the decoding end, the distortion of the image is obtained according to the quantization parameter QP, and the enhancement network is guided to enhance and achieve higher-definition image quality. Through effective strategies at both ends of the encoding and decoding, high-definition and low-code are better achieved.
[0133] Each of the above-mentioned method embodiments, or various possible implementation methods in each method embodiment can be executed separately, or any two or more of them can be executed in combination with each other. The specific implementation method can be determined according to actual usage requirements, and the embodiments of the present application do not limit this.
[0134] The image processing method provided in the embodiment of the present application can be executed by an image processing device. In the embodiment of the present application, an image processing device executing the image processing method is taken as an example to illustrate the image processing device provided in the embodiment of the present application.
[0135] Fig.10 A possible structural diagram of an image processing device involved in some embodiments of the present application is shown. Fig.10 As shown, the image processing device 70 may include: an encoding module 71 , a determination module 72 and an output module 73 .
[0136] The encoding module 71 is used to encode the image data with a first resolution to obtain first code stream information and a first quantization parameter.
[0137] The determination module 72 is used to determine a first decision quantization parameter value of the image data based on the first resolution and the second resolution, wherein the first decision quantization parameter value is a coding distortion critical value of the image data at the first resolution, and the second resolution is smaller than the first resolution.
[0138] The output module 73 is used to process the image data and output second code stream information based on the first quantization parameter obtained by the encoding module 71 and the first decision quantization parameter value determined by the determination module 72, where the second code stream information is the first code stream information, or is code stream information encoded at a resolution smaller than the first resolution.
[0139] In a possible implementation, the output module 72 is specifically configured to output the first code stream information when the first quantization parameter is less than or equal to the first decision quantization parameter value.
[0140] In a possible implementation, the output module 71 is specifically used to: when the first quantization parameter is greater than the first decision quantization parameter value, encode the image data using the second resolution to obtain third code stream information and the second quantization parameter; based on the second resolution and the third resolution, determine the second decision quantization parameter value of the image data, the second decision quantization parameter value is the coding distortion critical value of the image data at the second resolution, and the third resolution is less than the second resolution; specifically used to output the third code stream information when the second quantization parameter is less than or equal to the second decision quantization parameter value.
[0141] In a possible implementation, the image processing device 70 provided in the embodiment of the present application further includes: a decoding module and a processing module. The decoding module is used to decode the second code stream information after the output module outputs the second code stream information, and obtain a pixel value matrix and a third quantization parameter corresponding to the image data. The processing module is used to perform image quality enhancement processing on the pixel value matrix based on the third quantization parameter obtained by the decoding module, and output an image frame corresponding to the pixel value matrix.
[0142] In one possible implementation, the processing module is specifically used to: convert the third quantization parameter into an intensity coefficient, and use the intensity coefficient to adjust the intensity parameter in the preset enhancement network; and, based on the preset enhancement network after adjusting the intensity parameter, perform image quality enhancement processing on the pixel value matrix, and output an image frame corresponding to the pixel value matrix.
[0143] In one possible implementation, the processing module is specifically used to: perform feature extraction on the pixel value matrix based on a third quantization parameter to obtain feature information of the pixel value matrix; and, based on the feature information, perform image quality enhancement processing on the pixel value matrix to output an image frame corresponding to the pixel value matrix.
[0144] In a possible implementation, the image processing device 70 provided in the embodiment of the present application further includes an adjustment module. The adjustment module is used to adjust the decision quantization parameter value of the image data based on the capability information, wherein the capability information represents the image quality enhancement capability of the decoded image frame.
[0145] The embodiment of the present application provides an image processing device. Since the image processing device can encode image data with a first resolution to obtain a first quantization parameter, and the image processing device can determine a first decision quantization parameter value of the image data based on the first resolution and the second resolution, the image processing device can first encode the image data with any one resolution to obtain the first quantization parameter, and then determine the encoding distortion critical value of the image data at the current resolution based on a resolution smaller than the any one resolution, and then judge the output code stream information based on the encoding distortion critical value, that is, output the first code stream information, or the code stream information obtained by encoding with a resolution smaller than the first resolution, that is, based on the first quantization parameter and the encoding distortion critical value, it is possible to determine the code stream information whose encoding distortion degree does not exceed the encoding distortion critical value and whose encoding uses a lower resolution. Therefore, by selecting a suitable resolution to encode the image data, redundancy and insufficiency of the code rate are avoided, so that the image data can achieve a lower code rate while ensuring the image quality during the encoding process, so as to ensure the high-definition and low-code effect when the image processing device encodes the image.
[0146] The image processing device in the embodiment of the present application can be an electronic device, or a component in the electronic device, such as an integrated circuit or a chip. The electronic device can be a terminal, or other devices other than a terminal. Exemplarily, the electronic device can be a mobile phone, a tablet computer, a laptop computer, a PDA, a vehicle-mounted electronic device, a mobile Internet device (Mobile Internet Device, MID), an augmented reality (augmented reality, AR) / virtual reality (virtual reality, VR) device, a robot, a wearable device, an ultra-mobile personal computer (ultra-mobile personal computer, UMPC), a netbook or a personal digital assistant (personal digital assistant, PDA), etc., and can also be a server, a network attached storage (Network Attached Storage, NAS), a personal computer (personal computer, PC), a television (television, TV), a teller machine or a self-service machine, etc., which is not specifically limited in the embodiment of the present application.
[0147] The image processing device in the embodiment of the present application may be a device having an operating system. The operating system may be an Android operating system, an iOS operating system, or other possible operating systems, which are not specifically limited in the embodiment of the present application.
[0148] The image processing device provided in the embodiment of the present application can implement each process implemented in the above method embodiment, and will not be described again here to avoid repetition.
[0149] Alternatively, if Fig.11 As shown, an embodiment of the present application further provides an electronic device 1000, including a processor 1001 and a memory 1002, wherein the memory 1002 stores programs or instructions that can be executed on the processor 1001, and when the program or instructions are executed by the processor 1001, the various steps of the above-mentioned image processing method embodiment are implemented, and the same technical effect can be achieved. To avoid repetition, they are not described here.
[0150] It should be noted that the electronic devices in the embodiments of the present application include the mobile electronic devices and non-mobile electronic devices mentioned above.
[0151] Fig.12 A schematic diagram of the hardware structure of an electronic device to implement an embodiment of the present application.
[0152] The electronic device 100 includes but is not limited to components such as a radio frequency unit 101, a network module 102, an audio output unit 103, an input unit 104, a sensor 105, a display unit 106, a user input unit 107, an interface unit 108, a memory 109, and a processor 110.
[0153] Those skilled in the art will appreciate that the electronic device 100 may also include a power source (such as a battery) for supplying power to various components, and the power source may be logically connected to the processor 110 through a power management system, thereby implementing functions such as managing charging, discharging, and power consumption management through the power management system. Fig.12 The electronic device structure shown in the figure does not constitute a limitation on the electronic device. The electronic device may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently, which will not be described in detail here.
[0154] The processor 110 is used to encode the image data at a first resolution to obtain first bit stream information and a first quantization parameter.
[0155] The processor 110 is further configured to determine a first decision quantization parameter value of the image data based on the first resolution and the second resolution, wherein the first decision quantization parameter value is a coding distortion critical value of the image data at the first resolution, and the second resolution is smaller than the first resolution.
[0156] The processor 110 is further configured to process the image data based on the first quantization parameter and the first decision quantization parameter value and output second bitstream information, where the second bitstream information is the first bitstream information or bitstream information encoded at a resolution smaller than the first resolution.
[0157] An embodiment of the present application provides an electronic device. Since the electronic device can encode image data at a first resolution to obtain a first quantization parameter, and the electronic device can determine a first decision quantization parameter value of the image data based on the first resolution and the second resolution, the electronic device can first encode the image data at any one resolution to obtain the first quantization parameter, and then determine a coding distortion critical value of the image data at the current resolution based on a resolution smaller than the any one resolution, and then judge the output code stream information based on the coding distortion critical value, that is, output the first code stream information, or the code stream information obtained by encoding at a resolution smaller than the first resolution, that is, based on the first quantization parameter and the coding distortion critical value, it is possible to determine the code stream information whose coding distortion degree does not exceed the coding distortion critical value and whose encoding uses a lower resolution. Therefore, by selecting a suitable resolution to encode the image data, redundancy and insufficiency of the code rate are avoided, so that the image data can achieve a lower code rate while ensuring the image quality during the encoding process, thereby ensuring a high-definition and low-code effect when the electronic device encodes the image.
[0158] Optionally, the processor 110 is specifically configured to output the first code stream information when the first quantization parameter is less than or equal to the first decision quantization parameter value.
[0159] Optionally, the processor 110 is specifically configured to: when the first quantization parameter is greater than the first decision quantization parameter value, encode the image data using the second resolution to obtain third code stream information and the second quantization parameter; determine the second decision quantization parameter value of the image data based on the second resolution and the third resolution, the second decision quantization parameter value being a coding distortion critical value of the image data at the second resolution, and the third resolution being less than the second resolution; and specifically configured to output the third code stream information when the second quantization parameter is less than or equal to the second decision quantization parameter value.
[0160] Optionally, the processor 110 is further used to: after outputting the second code stream information, decode the second code stream information to obtain a pixel value matrix and a third quantization parameter corresponding to the image data; based on the third quantization parameter, perform image quality enhancement processing on the pixel value matrix, and output an image frame corresponding to the pixel value matrix.
[0161] Optionally, the processor 110 is specifically used to: convert the third quantization parameter into an intensity coefficient, and use the intensity coefficient to adjust the intensity parameter in the preset enhancement network; and, based on the preset enhancement network after adjusting the intensity parameter, perform image quality enhancement processing on the pixel value matrix, and output an image frame corresponding to the pixel value matrix.
[0162] Optionally, the processor 110 is specifically used to: extract features from the pixel value matrix based on a third quantization parameter to obtain feature information of the pixel value matrix; and, based on the feature information, perform image quality enhancement processing on the pixel value matrix to output an image frame corresponding to the pixel value matrix.
[0163] Optionally, the processor 110 is further used to adjust the decision quantization parameter value of the image data based on the capability information, and the capability information represents the image quality enhancement capability of the decoded image frame. The electronic device provided in the embodiment of the present application can implement the various processes implemented in the above method embodiment and can achieve the same technical effect. To avoid repetition, it will not be repeated here. The beneficial effects of various implementation methods in this embodiment can be specifically referred to the beneficial effects of the corresponding implementation methods in the above method embodiment. To avoid repetition, it will not be repeated here.
[0164] It should be understood that in the embodiment of the present application, the input unit 104 may include a graphics processor (Graphics Processing Unit, GPU) 1041 and a microphone 1042, and the graphics processor 1041 processes the image data of a static picture or video obtained by an image capture device (such as a camera) in a video capture mode or an image capture mode. The display unit 106 may include a display panel 1061, and the display panel 1061 may be configured in the form of a liquid crystal display, an organic light emitting diode, etc. The user input unit 107 includes a touch panel 1071 and at least one of other input devices 1072. The touch panel 1071 is also called a touch screen. The touch panel 1071 may include two parts: a touch detection device and a touch controller. Other input devices 1072 may include, but are not limited to, a physical keyboard, function keys (such as volume control keys, switch keys, etc.), a trackball, a mouse, and a joystick, which will not be repeated here.
[0165] The memory 109 can be used to store software programs and various data. The memory 109 may mainly include a first storage area for storing programs or instructions and a second storage area for storing data, wherein the first storage area may store an operating system, an application program or instructions required for at least one function (such as a sound playback function, an image playback function, etc.), etc. In addition, the memory 109 may include a volatile memory or a non-volatile memory, or the memory 109 may include both volatile and non-volatile memories. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), a static random access memory (SRAM), a dynamic random access memory (DRAM), a synchronous dynamic random access memory (SDRAM), a double data rate synchronous dynamic random access memory (DDRSDRAM), an enhanced synchronous dynamic random access memory (ESDRAM), a synchronous link dynamic random access memory (SLDRAM) and a direct memory bus random access memory (DRRAM). The memory 109 in the embodiment of the present application includes but is not limited to these and any other suitable types of memory.
[0166] The processor 110 may include one or more processing units; optionally, the processor 110 integrates an application processor and a modem processor, wherein the application processor mainly processes operations related to an operating system, a user interface, and application programs, and the modem processor mainly processes wireless communication signals, such as a baseband processor. It is understandable that the modem processor may not be integrated into the processor 110.
[0167] An embodiment of the present application also provides a readable storage medium, on which a program or instruction is stored. When the program or instruction is executed by a processor, each process of the above-mentioned image processing method embodiment is implemented, and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.
[0168] The processor is the processor in the electronic device described in the above embodiment. The readable storage medium includes a computer readable storage medium, such as a computer read-only memory ROM, a random access memory RAM, a magnetic disk or an optical disk.
[0169] An embodiment of the present application further provides a chip, which includes a processor and a communication interface, wherein the communication interface is coupled to the processor, and the processor is used to run programs or instructions to implement the various processes of the above-mentioned image processing method embodiment, and can achieve the same technical effect. To avoid repetition, it will not be repeated here.
[0170] It should be understood that the chip mentioned in the embodiments of the present application can also be called a system-level chip, a system chip, a chip system or a system-on-chip chip, etc.
[0171] An embodiment of the present application provides a computer program product, which is stored in a storage medium. The program product is executed by at least one processor to implement the various processes of the above-mentioned image processing method embodiment and can achieve the same technical effect. To avoid repetition, it will not be repeated here.
[0172] It should be noted that, in this article, the terms "comprise", "include" or any other variant thereof are intended to cover non-exclusive inclusion, so that the process, method, article or device including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise one..." do not exclude the presence of other identical elements in the process, method, article or device including the element. In addition, it should be pointed out that the scope of the methods and devices in the embodiments of the present application is not limited to performing functions in the order shown or discussed, and may also include performing functions in a substantially simultaneous manner or in reverse order according to the functions involved, for example, the described method may be performed in an order different from that described, and various steps may also be added, omitted, or combined. In addition, the features described with reference to certain examples may be combined in other examples.
[0173] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a computer software product, which is stored in a storage medium (such as ROM / RAM, a disk, or an optical disk), and includes a number of instructions for a terminal (which can be a mobile phone, a computer, a server, or a network device, etc.) to execute the methods described in each embodiment of the present application.
[0174] The embodiments of the present application are described above in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of the present application, ordinary technicians in this field can also make many forms without departing from the purpose of the present application and the scope of protection of the claims, all of which are within the protection of the present application.
Claims
1. An image processing method, characterized in that: include: Encoding the image data with a first resolution to obtain first bit stream information and a first quantization parameter; Determine a first decision quantization parameter value of the image data based on the first resolution and the second resolution, the first decision quantization parameter value being a coding distortion critical value of the image data at the first resolution, the second resolution being smaller than the first resolution; Based on the first quantization parameter and the first decision quantization parameter value, the image data is processed and second code stream information is output, where the second code stream information is the first code stream information, or is code stream information encoded using a resolution smaller than the first resolution.
2. The method according to claim 1, characterized in that The processing of the image data and outputting second code stream information based on the first quantization parameter and the first decision quantization parameter value includes: When the first quantization parameter is greater than the first decision quantization parameter value, encoding the image data using the second resolution to obtain third bitstream information and a second quantization parameter; Determine a second decision quantization parameter value of the image data based on the second resolution and the third resolution, the second decision quantization parameter value being a coding distortion threshold value of the image data at the second resolution, the third resolution being smaller than the second resolution; When the second quantization parameter is less than or equal to the second decision quantization parameter value, the third code stream information is output.
3. The method according to any one of claims 1 to 2, characterized in that After outputting the second code stream information, the method further includes: Decoding the second code stream information to obtain a pixel value matrix and a third quantization parameter corresponding to the image data; Based on the third quantization parameter, the pixel value matrix is subjected to image quality enhancement processing, and an image frame corresponding to the pixel value matrix is output.
4. The method according to claim 3, characterized in that The step of performing image quality enhancement processing on the pixel value matrix based on the third quantization parameter and outputting an image frame corresponding to the pixel value matrix includes: Converting the third quantization parameter into an intensity coefficient, and using the intensity coefficient to adjust an intensity parameter in a preset enhancement network; Based on the preset enhancement network after adjusting the intensity parameters, the pixel value matrix is subjected to image quality enhancement processing, and an image frame corresponding to the pixel value matrix is output.
5. The method according to claim 3, characterized in that: The step of performing image quality enhancement processing on the pixel value matrix based on the third quantization parameter and outputting an image frame corresponding to the pixel value matrix includes: Based on the third quantization parameter, feature extraction is performed on the pixel value matrix to obtain feature information of the pixel value matrix; Based on the feature information, the pixel value matrix is subjected to image quality enhancement processing, and an image frame corresponding to the pixel value matrix is output.
6. An image processing device, characterized in that: include: Encoding module, determination module and output module; The encoding module is used to encode the image data with a first resolution to obtain first code stream information and a first quantization parameter; The determining module is used to determine a first decision quantization parameter value of the image data based on the first resolution and the second resolution, the first decision quantization parameter value being a coding distortion threshold value of the image data at the first resolution, and the second resolution is smaller than the first resolution; The output module is used to process the image data and output second code stream information based on the first quantization parameter obtained by the encoding module and the first decision quantization parameter value determined by the determination module, where the second code stream information is the first code stream information, or is code stream information encoded using a resolution smaller than the first resolution.
7. The device according to claim 6, characterized in that The output module is specifically used for: When the first quantization parameter is greater than the first decision quantization parameter value, encoding the image data using the second resolution to obtain third bitstream information and a second quantization parameter; Determine a second decision quantization parameter value of the image data based on the second resolution and the third resolution, the second decision quantization parameter value being a coding distortion threshold value of the image data at the second resolution, the third resolution being smaller than the second resolution; When the second quantization parameter is less than or equal to the second decision quantization parameter value, the third code stream information is output.
8. The device according to any one of claims 6 to 7, characterized in that The device also includes: a decoding module and a processing module; The decoding module is used to decode the second code stream information after the output module outputs the second code stream information, so as to obtain a pixel value matrix and a third quantization parameter corresponding to the image data; The processing module is used to perform image quality enhancement processing on the pixel value matrix based on the third quantization parameter obtained by the decoding module, and output an image frame corresponding to the pixel value matrix.
9. The device according to claim 8, characterized in that The processing module is specifically used for: The third quantization parameter is converted into an intensity coefficient, and the intensity coefficient is used to adjust the intensity parameter in the preset enhancement network; and based on the preset enhancement network after adjusting the intensity parameter, the pixel value matrix is subjected to image quality enhancement processing, and an image frame corresponding to the pixel value matrix is output.
10. The device according to claim 8, characterized in that The processing module is specifically used for: Based on the third quantization parameter, feature extraction is performed on the pixel value matrix to obtain feature information of the pixel value matrix; and based on the feature information, image quality enhancement processing is performed on the pixel value matrix to output an image frame corresponding to the pixel value matrix.
Citation Information
Patent Citations
Real-time video encoder rate control using dynamic resolution switching
CN107040781A
Video encoding method and device, storage medium and equipment
CN108391127A
Coding resolution control method and terminal
CN109151470A
Video encoding method and device, video decoding method and device, storage medium and electronic device
CN110536134A
Multiple bit rate video encoding using variable bit rate and dynamic resolution for adaptive video streaming
US20100189183A1
