Image processing method, device, electronic device and storage medium

Abstract

The embodiment of the present application provides an image processing method, device, electronic device and storage medium, the method comprising: obtaining an image to be processed; dividing the image to be processed into an area of ​​interest and an area of ​​non-interest; filtering the area of ​​non-interest to obtain the filtered pixel value of each first pixel in the area of ​​non-interest; determining the distorted second pixel from each first pixel according to the filtered pixel value of each first pixel, correcting the filtered pixel value of the second pixel, and obtaining the processed area of ​​non-interest; obtaining the processed image according to the area of ​​interest and the processed area of ​​non-interest. The embodiment of the present application can reduce the loss of image details and help ensure image quality.

Description

Technical Field

[0001] The present application relates to the field of image processing, and in particular to an image processing method, device, electronic device and storage medium. Background Art

[0002] Video pre-processing refers to the processing before video encoding. The most common video pre-processing solution is to filter the video using traditional filtering methods to remove some high-frequency information. However, this solution will lose the details of the video image, and filtering the entire frame of the image will make the loss of details in the area of ​​interest in the image more noticeable, thus affecting the image quality. Summary of the invention

[0003] The present application provides an image processing method, device, electronic device and storage medium, which can reduce the loss of image details and help ensure image quality.

[0004] In a first aspect, an embodiment of the present application provides an image processing method, the method comprising:

[0005] Get the image to be processed;

[0006] Dividing the image to be processed into regions of interest and non-regions of interest;

[0007] Filtering the non-interested region to obtain a filtered pixel value of each first pixel point in the non-interested region;

[0008] Determine a distorted second pixel from each of the first pixel points according to the filtered pixel values ​​of each of the first pixel points, and correct the filtered pixel values ​​of the second pixel points to obtain a processed non-interested region;

[0009] A processed image is obtained according to the region of interest and the processed non-region of interest.

[0010] Through the above embodiment, the area to be processed is divided into an area of ​​interest and an area of ​​non-interest, the area of ​​interest is not processed, the area of ​​non-interest is filtered and the pixel values ​​​​generated with distortion after filtering are corrected, thereby the details of the area of ​​interest can be well preserved, and the loss of details in the area of ​​non-interest can be minimized, which helps to ensure image quality, and some high-frequency information in the area of ​​non-interest can be removed while ensuring image quality, which helps to reduce encoding pressure when used for video pre-processing.

[0011] In some possible implementations, determining the distorted second pixel from each of the first pixel points according to the filtered pixel value of each of the first pixel points includes:

[0012] Obtaining the original pixel value of each of the first pixel points in the image to be processed;

[0013] Determine a distorted second pixel point from each of the first pixel points according to the filtered pixel value and the original pixel value of each of the first pixel points.

[0014] Through the above implementation, the filtered pixel value and the original pixel value of the first pixel point can be used to reflect the change in the pixel value of the first pixel point before and after filtering, which is helpful to quickly and accurately determine the distorted second pixel point from each first pixel point.

[0015] In some possible implementations, determining the distorted second pixel point from each of the first pixel points according to the filtered pixel value and the original pixel value of each of the first pixel points includes:

[0016] Calculate the pixel difference between the filtered pixel value and the original pixel value of each of the first pixels;

[0017] Obtaining a just perceptible distortion value of each of the first pixels;

[0018] A distorted second pixel point is determined from each of the first pixel points according to the pixel difference value and the just perceptible distortion value of each of the first pixel points.

[0019] Through the above implementation, a just noticeable distortion value of the first pixel is introduced to determine whether the change in pixel value of the first pixel before and after filtering exceeds the tolerance of the human eye, that is, to determine whether the distortion of the first pixel can be perceived by the human eye, which is helpful to accurately determine the distorted second pixel from the first pixels.

[0020] In some possible implementations, obtaining the just perceptible distortion value of each of the first pixels includes:

[0021] Obtaining brightness sensitivity of each of the first pixels according to brightness information of each of the first pixels in the image to be processed;

[0022] Obtaining texture sensitivity of each of the first pixels according to gradient information of each of the first pixels in the image to be processed;

[0023] A just perceptible distortion value of each of the first pixels is determined according to the brightness sensitivity and the texture sensitivity of each of the first pixels.

[0024] Through the above implementation, the just noticeable distortion value of the first pixel is determined in combination with the brightness information and gradient information of the first pixel. The just noticeable distortion value determined in this way can more accurately reflect the degree of pixel value change that the human eye can tolerate for the first pixel.

[0025] In some possible implementations, determining a distorted second pixel from each of the first pixels according to the pixel difference value and the just perceptible distortion value of each of the first pixels includes:

[0026] A first pixel point whose absolute value of the pixel difference is greater than the just perceptible distortion value is determined as a distorted second pixel point.

[0027] Through the above implementation, by utilizing the size relationship between the pixel difference of the first pixel point before and after filtering and the just noticeable distortion value, it is possible to directly reflect whether the pixel value change of the first pixel point before and after filtering exceeds the tolerance of the human eye. This is helpful to accurately judge whether the first pixel point can be perceived by the human eye as distorted, thereby accurately determining the distorted second pixel point from the first pixel points.

[0028] In some possible implementations, the correcting the filtered pixel value of the second pixel includes:

[0029] When the filtered pixel value of the second pixel is greater than the original pixel value, correcting the filtered pixel value of the second pixel to the original pixel value of the second pixel plus the just perceptible distortion value;

[0030] When the filtered pixel value of the second pixel point is less than the original pixel value, the filtered pixel value of the second pixel point is corrected to the original pixel value of the second pixel point minus the just perceptible distortion value.

[0031] Through the above implementation, the filtered pixel value is corrected using the just noticeable distortion value of the second pixel point, so that the change amplitude of the corrected pixel value of the second pixel point relative to the original pixel value will not cause distortion perceived by the human eye, which helps to ensure image quality while retaining the filtering effect, helping to reduce encoding pressure.

[0032] In some possible implementations, obtaining the image to be processed includes:

[0033] A video to be processed is obtained, video frames are extracted from the video to be processed, and each extracted video frame is used as the image to be processed.

[0034] Through the above implementation, each video frame extracted from the video to be processed is used as the image to be processed, which is conducive to ensuring the video quality and reducing the pressure of video encoding without increasing the complexity of the encoding itself.

[0035] In a second aspect, an embodiment of the present application provides an image processing device, the device comprising:

[0036] An acquisition unit, used for acquiring an image to be processed;

[0037] A division unit, used for dividing the image to be processed into an area of ​​interest and an area of ​​no interest;

[0038] A filtering unit, configured to filter the non-interest region to obtain a filtered pixel value of each first pixel point in the non-interest region;

[0039] a correction unit, configured to determine a distorted second pixel point from each of the first pixel points according to the filtered pixel values ​​of each of the first pixel points, and correct the filtered pixel values ​​of the second pixel point to obtain a processed non-interested region;

[0040] A determining unit is used to determine a processed image according to the region of interest and the processed non-region of interest.

[0041] In some possible implementations, when the correction unit determines the distorted second pixel from each of the first pixel points according to the filtered pixel value of each of the first pixel points, it is specifically configured to:

[0042] Obtaining the original pixel value of each of the first pixel points in the image to be processed;

[0043] Determine a distorted second pixel point from each of the first pixel points according to the filtered pixel value and the original pixel value of each of the first pixel points.

[0044] In some possible implementations, when the correction unit determines the distorted second pixel point from each of the first pixel points according to the filtered pixel value and the original pixel value of each of the first pixel points, it is specifically configured to:

[0045] Calculate the pixel difference between the filtered pixel value and the original pixel value of each of the first pixels;

[0046] Obtaining a just perceptible distortion value of each of the first pixels;

[0047] A distorted second pixel point is determined from each of the first pixel points according to the pixel difference value and the just perceptible distortion value of each of the first pixel points.

[0048] In some possible implementations, when acquiring the just perceptible distortion value of each of the first pixels, the correction unit is specifically configured to:

[0049] Obtaining brightness sensitivity of each of the first pixels according to brightness information of each of the first pixels in the image to be processed;

[0050] Obtaining texture sensitivity of each of the first pixels according to gradient information of each of the first pixels in the image to be processed;

[0051] A just perceptible distortion value of each of the first pixels is determined according to the brightness sensitivity and the texture sensitivity of each of the first pixels.

[0052] In some possible implementations, when the correction unit determines the distorted second pixel from each of the first pixels according to the pixel difference value and the just perceptible distortion value of each of the first pixels, it is specifically configured to:

[0053] When the filtered pixel value of the second pixel is greater than the original pixel value, correcting the filtered pixel value of the second pixel to the original pixel value of the second pixel plus the just perceptible distortion value;

[0054] When the filtered pixel value of the second pixel point is less than the original pixel value, the filtered pixel value of the second pixel point is corrected to the original pixel value of the second pixel point minus the just perceptible distortion value.

[0055] In some possible implementations, when acquiring the image to be processed, the acquisition unit is specifically used to:

[0056] A video to be processed is obtained, video frames are extracted from the video to be processed, and each extracted video frame is used as the image to be processed.

[0057] In a third aspect, an embodiment of the present application provides an electronic device, including a memory and a processor, wherein the memory stores a computer program, and the processor implements the method in the above-mentioned first aspect and any possible implementation manner thereof when executing the computer program.

[0058] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, in which a computer program is stored. When the computer program is executed by a processor, the method in the above-mentioned first aspect and any possible implementation manner thereof is implemented.

[0059] In a fifth aspect, an embodiment of the present application provides a computer program product, which includes a computer program. When the computer program is executed by a processor, it implements the method in the above-mentioned first aspect and any possible implementation manner thereof.

[0060] It should be understood that the above general description and the following detailed description are merely exemplary and explanatory, and are not intended to limit the technical solutions of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0061] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the background technology, the drawings required for use in the embodiments of the present application or the background technology will be described below.

[0062] The drawings herein are incorporated into the specification and constitute a part of the specification. These drawings illustrate embodiments consistent with the present application and are used together with the specification to illustrate the technical solution of the present application.

[0063] Figure 1 It is a flowchart of an image processing method provided in an embodiment of the present application;

[0064] Figure 2 is a structural schematic diagram of an image processing device provided in an embodiment of the present application;

[0065] Figure 3 It is a structural schematic diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0066] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0067] The terms "first", "second", etc. in the embodiments of the present application are used to distinguish different objects, rather than to describe a specific order. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not limited to the listed steps or units, but optionally includes steps or units that are not listed, or optionally includes other steps or units inherent to these processes, methods, products or devices.

[0068] It should be understood that in the present application, "at least one (item)" means one or more, "more than one" means two or more, and "at least two (items)" means two or three and more than three. "And / or" is used to describe the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" can mean: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " can indicate that the previous and next associated objects are in an "or" relationship, which refers to any combination of these items, including any combination of single items (items) or plural items (items). For example, at least one item (item) of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.

[0069] Reference to "embodiments" in this application means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0070] Video pre-processing refers to the processing before video encoding. The most common video pre-processing solution is to filter the video using traditional filtering methods. However, this solution will lose the details of the video image, and filtering the entire frame of the image will make the loss of details in the area of ​​interest in the image more noticeable, thus affecting the image quality.

[0071] Based on this, an embodiment of the present application provides an image processing method that can well preserve the details of the area of ​​interest and minimize the loss of details in the area of ​​non-interest, which helps to ensure image quality, and can remove some high-frequency information in the area of ​​non-interest while ensuring image quality, which helps to reduce encoding pressure when used for video pre-processing.

[0072] The execution subject of the image processing method may be an image processing device, for example, the image processing method may be executed by a terminal device or a server or other processing device, wherein the terminal device may be, but is not limited to, various personal computers, laptops, tablet computers, smart phones, and portable wearable devices, and the server may be implemented as an independent server or a server cluster composed of multiple servers. In some possible implementations, the image processing method may be implemented by a processor calling computer-readable instructions stored in a memory.

[0073] See also Figure 1 , Figure 1 It is a flowchart of an image processing method provided in an embodiment of the present application. The image processing method includes the following steps S101 to S105.

[0074] S101, obtaining an image to be processed.

[0075] The image to be processed may be an image that needs to be processed before encoding. For example, the image to be processed may be a video frame extracted from a video to be encoded, and each video frame is processed before encoding.

[0076] S102, dividing the image to be processed into a region of interest and a region of no interest.

[0077] The region of interest (ROI) can be understood as the key area in the image, and the non-interest area refers to the other areas in the image except the region of interest. For example, assuming that the image to be processed is a portrait with a seaside background, the region of interest can be the portrait area, and the non-interest area can be the background area except the portrait area.

[0078] S103, filtering the non-interest region to obtain a filtered pixel value of each first pixel point in the non-interest region.

[0079] The first pixel point refers to a pixel point in the non-interested area. Filtering the non-interested area means adjusting the pixel value of each first pixel point in the non-interested area to the average value or weighted average value of the neighborhood pixel values ​​of the first pixel point, wherein the neighborhood pixel value includes the pixel values ​​of multiple pixel points around the first pixel point. Optionally, the filtering method adopts Gaussian filtering.

[0080] After filtering, the pixel value of the first pixel may change. For example, the pixel value of some first pixels after filtering is less than the pixel value before filtering, the pixel value of some first pixels after filtering is greater than the pixel value before filtering, and the pixel value of some first pixels after filtering is equal to the pixel value before filtering. The pixel value before filtering here refers to the pixel value of the first pixel in the image to be processed, that is, the original pixel value.

[0081] S104, determining a distorted second pixel from each first pixel according to the filtered pixel value of each first pixel, correcting the filtered pixel value of the second pixel, and obtaining a processed non-interest region.

[0082] After filtering, the pixel values ​​of different first pixel points may have different change amplitudes. For example, the pixel values ​​of some first pixel points may have a large change amplitude, while the pixel values ​​of some first pixel points may have a small change amplitude. Among them, the first pixel point with a large pixel value change amplitude is prone to lose more detail features, so it can be considered that it is distorted; the first pixel point with a small pixel value change amplitude may lose fewer detail features, so it can be considered that it is not distorted.

[0083] The first pixel may include both first pixel points that are distorted after filtering and first pixel points that are not distorted after filtering. To distinguish them, the first pixel points that are distorted after filtering are recorded as second pixel points, and the first pixel points that are not distorted after filtering are recorded as third pixel points. The filtered pixel value of the second pixel is corrected to reduce the detail loss of the second pixel, and the corrected pixel value of the second pixel is obtained.

[0084] In one example, the filtered pixel value of the third pixel point may not be processed, that is, the filtered pixel value of the third pixel point is kept unchanged. Thus, based on the corrected pixel values ​​of each second pixel point and the filtered pixel values ​​of each third pixel point, a processed non-interested region is obtained. The processed non-interested region retains the details of the non-interested region as much as possible, which helps to alleviate the distortion problem of the non-interested region, and removes some high-frequency information of the non-interested region, which helps to suppress the noise of the non-interested region and reduce the coding pressure.

[0085] In other examples, the filtered pixel value of the third pixel point can also be corrected under the condition that the third pixel point does not generate distortion, so as to remove the perceptual redundancy of the third pixel point and obtain the corrected pixel value of the third pixel point. Thus, based on the corrected pixel values ​​of each second pixel point and the corrected pixel values ​​of each third pixel point, the processed non-interested region is obtained. The processed non-interested region retains the details of the non-interested region as much as possible, which helps to alleviate the distortion problem of the non-interested region, and removes some high-frequency information of the non-interested region, which helps to suppress the noise of the non-interested region and reduce the subsequent encoding pressure, and also removes the redundant information of the non-interested region, which helps to improve the subsequent encoding efficiency.

[0086] S105, obtaining a processed image according to the region of interest and the processed non-region of interest.

[0087] The region of interest in the processed image is not processed to completely preserve the details of the region of interest. The processed image is obtained by combining the region of interest with the processed non-region of interest.

[0088] Through the above embodiment, the area to be processed is divided into an area of ​​interest and an area of ​​non-interest, the area of ​​interest is not processed, the area of ​​non-interest is filtered and the pixel values ​​​​generated with distortion after filtering are corrected, thereby the details of the area of ​​interest can be well preserved, and the loss of details in the area of ​​non-interest can be minimized, which helps to ensure image quality, and some high-frequency information in the area of ​​non-interest can be removed while ensuring image quality, which helps to reduce encoding pressure when used for video pre-processing.

[0089] In a possible implementation, determining a distorted second pixel point from each first pixel point based on the filtered pixel value of each first pixel point can specifically include: obtaining the original pixel value of each first pixel point in the image to be processed; and determining a distorted second pixel point from each first pixel point based on the filtered pixel value and the original pixel value of each first pixel point.

[0090] The original pixel value of the first pixel in the image to be processed is also the pixel value of the first pixel before filtering. For each first pixel, by comparing the pixel value after filtering and the original pixel value of the first pixel, the change of the pixel value of the first pixel before and after filtering can be obtained, and then it can be determined whether the first pixel is distorted according to the change of the pixel value.

[0091] Through the above implementation, the filtered pixel value and the original pixel value of the first pixel point can be used to reflect the change in the pixel value of the first pixel point before and after filtering, which is helpful to quickly and accurately determine the distorted second pixel point from each first pixel point.

[0092] In a possible implementation, determining a distorted second pixel from each first pixel based on the filtered pixel value and the original pixel value of each first pixel may specifically include: calculating the pixel difference between the filtered pixel value and the original pixel value of each first pixel; obtaining a just noticeable distortion value of each first pixel; and determining the distorted second pixel from each first pixel based on the pixel difference and the just noticeable distortion value of each first pixel.

[0093] For each first pixel point, a pixel difference between the filtered pixel value and the original pixel value of the first pixel point may be calculated to reflect the change in the pixel value of the first pixel point before and after filtering.

[0094] In one example, the pixel difference value may be the result of subtracting the original pixel value from the filtered pixel value. In this example, if the pixel difference value is a positive value, it means that the filtered pixel value is greater than the original pixel value, that is, the pixel value increases after filtering; if the pixel difference value is a negative value, it means that the filtered pixel value is less than the original pixel value, that is, the pixel value decreases after filtering; if the pixel difference value is zero, it means that the filtered pixel value is equal to the original pixel value, that is, the pixel value is not changed after filtering.

[0095] In another example, the pixel difference value may also be the result of subtracting the filtered pixel value from the original pixel value. In this example, if the pixel difference value is a positive value, it means that the filtered pixel value is less than the original pixel value, that is, the pixel value is reduced after filtering; if the pixel difference value is a negative value, it means that the filtered pixel value is greater than the original pixel value, that is, the pixel value is increased after filtering; if the pixel difference value is zero, it means that the filtered pixel value is equal to the original pixel value, that is, the pixel value is not changed after filtering.

[0096] Just Noticeable Distortion (JND) indicates the maximum distortion value that cannot be detected by the human eye. It can be used to reflect the human eye's tolerance to image changes. When the image change exceeds the JND value, it can be considered that the human eye can detect image distortion. It can be understood that the smaller the JND value, the easier it is for the human eye to detect distortion; the larger the JND value, the less likely it is for the human eye to detect distortion.

[0097] For each first pixel, a just noticeable distortion value of the first pixel can be obtained to reflect the human eye's tolerance to the pixel value change of the first pixel. Combining the pixel difference value and the just noticeable distortion value of the first pixel, it can be determined whether the first pixel generates distortion that can be perceived by the human eye.

[0098] Through the above implementation, a just noticeable distortion value of the first pixel is introduced to determine whether the change in pixel value of the first pixel before and after filtering exceeds the tolerance of the human eye, that is, to determine whether the distortion of the first pixel can be perceived by the human eye, which is helpful to accurately determine the distorted second pixel from the first pixels.

[0099] In a possible implementation, obtaining a just noticeable distortion value of each first pixel may specifically include: obtaining a brightness sensitivity of each first pixel based on brightness information of each first pixel in the image to be processed; obtaining a texture sensitivity of each first pixel based on gradient information of each first pixel in the image to be processed; and determining a just noticeable distortion value of each first pixel based on the brightness sensitivity and texture sensitivity of each first pixel.

[0100] Different first pixels may have different brightness and / or texture in the image to be processed, and the human eye has different sensitivities to different brightness and / or different textures. It can be understood that the degree of distortion that the human eye can tolerate is related to the sensitivity of the human eye. For areas where the human eye is more sensitive, the degree of distortion that the human eye can tolerate is lower; for areas where the human eye is less sensitive, the degree of distortion that the human eye can tolerate is higher. Therefore, the brightness and texture of the first pixel can be used to determine the just perceptible distortion value of the first pixel.

[0101] In an example, the calculation formula of the brightness sensitivity of the first pixel may be as follows:

[0102]

[0103] Wherein, L(x,y) represents the brightness sensitivity of the first pixel point (x,y); I(x,y) represents the brightness information of the first pixel point (x,y). Optionally, I(x,y) can be determined by the average brightness or weighted average brightness within a neighborhood pixel block centered on the first pixel point (x,y). Exemplarily, the neighborhood pixel block can be a 5×5 pixel area.

[0104] In an example, the calculation formula for the texture sensitivity of the first pixel may be as follows:

[0105] W(x,y)=β·G(x,y)·a

[0106] Wherein, W(x,y) represents the texture sensitivity of the first pixel point (x,y); G(x,y) represents the gradient information of the first pixel point (x,y). Optionally, G(x,y) may be the maximum value of the gradients obtained by performing edge detection on the first pixel point (x,y) in the horizontal and vertical directions; β is 0.117; a is a constant. The human eye is more sensitive to noise in flat and edge areas, but has a relatively greater tolerance for noise in texture areas. For example, if the first pixel point (x,y) belongs to the edge area, a=1; if the first pixel point (x,y) belongs to the texture area, a=3.

[0107] In an example, the calculation formula of the just perceptible distortion value of the first pixel point may be as follows:

[0108] JND(x,y)=L(x,y)+W(x,y)-b·min{L(x,y),W(x,y)}

[0109] Wherein, JND(x,y) represents the just noticeable distortion value of the first pixel point (x,y); min{} represents a function for obtaining the minimum value; b is a constant used to represent the degree of superposition of brightness sensitivity and texture sensitivity, and illustratively, b=0.3.

[0110] In other examples, JND(x,y) may also be the larger value of I(x,y) and JND(x,y), and JND(x,y) may also be the weighted sum of I(x,y) and JND(x,y), which is not limited in the embodiments of the present application.

[0111] Through the above implementation, the just noticeable distortion value of the first pixel is determined in combination with the brightness information and gradient information of the first pixel. The just noticeable distortion value determined in this way can more accurately reflect the degree of pixel value change that the human eye can tolerate for the first pixel.

[0112] In a possible implementation, determining a distorted second pixel from each first pixel based on the pixel difference and the just noticeable distortion value of each first pixel may specifically include: determining a first pixel whose absolute value of the pixel difference is greater than the just noticeable distortion value as the distorted second pixel.

[0113] For each first pixel, compare the absolute value of the pixel difference of the first pixel with the just noticeable distortion value of the first pixel. If the absolute value of the pixel difference of the first pixel is greater than the just noticeable distortion value of the first pixel, it means that the change in the pixel value of the first pixel before and after filtering exceeds the tolerance of the human eye, that is, the first pixel can be perceived as distorted by the human eye, and the first pixel is determined to be a distorted second pixel.

[0114] Through the above implementation, by utilizing the size relationship between the pixel difference of the first pixel point before and after filtering and the just noticeable distortion value, it is possible to directly reflect whether the pixel value change of the first pixel point before and after filtering exceeds the tolerance of the human eye. This is helpful to accurately judge whether the first pixel point can be perceived by the human eye as distorted, thereby accurately determining the distorted second pixel point from the first pixel points.

[0115] In a possible implementation, the filtered pixel value of the second pixel point is corrected, which may specifically include: when the filtered pixel value of the second pixel point is greater than the original pixel value, the filtered pixel value of the second pixel point is corrected to the original pixel value of the second pixel point plus a just noticeable distortion value; when the filtered pixel value of the second pixel point is less than the original pixel value, the filtered pixel value of the second pixel point is corrected to the original pixel value of the second pixel point minus the just noticeable distortion value.

[0116] In one example, the pixel difference value of the second pixel point is equal to the filtered pixel value minus the original pixel value, and the absolute value of the pixel difference value of the second pixel point is greater than the just noticeable distortion value, which includes the following two situations: one situation is that the pixel difference value of the second pixel point is a positive value, that is, the filtered pixel value of the second pixel point is greater than the original pixel value, that is, the filtered pixel value of the second pixel point is increased compared with the original pixel value, and the increase exceeds the just noticeable distortion value of the second pixel point; the other situation is that the pixel difference value of the second pixel point is a negative value, that is, the filtered pixel value of the second pixel point is less than the original pixel value, that is, the filtered pixel value of the second pixel point is reduced compared with the original pixel value, and the reduction exceeds the just noticeable distortion value of the second pixel point.

[0117] When the filtered pixel value of the second pixel is greater than the original pixel value, it means that the filtered pixel value of the second pixel has increased too much compared to the original pixel value, resulting in distortion that can be perceived by the human eye. The filtered pixel value of the second pixel can be corrected by reducing the increase amplitude of the filtered pixel value of the second pixel. Specifically, the filtered pixel value of the second pixel is corrected to the original pixel value of the second pixel plus the just noticeable distortion value of the second pixel, so that the increase amplitude of the corrected pixel value compared to the original pixel value will not cause distortion to be perceived by the human eye, while retaining the filtering effect. For example, if the just noticeable distortion value of a second pixel is 5, the original pixel value is 80, and the filtered pixel value is 87, then the corrected pixel value of the second pixel is 80+5, that is, 85.

[0118] When the filtered pixel value of the second pixel is less than the original pixel value, it means that the filtered pixel value of the second pixel is reduced too much compared to the original pixel value, resulting in distortion that can be perceived by the human eye. The filtered pixel value of the second pixel can be corrected by reducing the reduction amplitude of the filtered pixel value of the second pixel. Specifically, the filtered pixel value of the second pixel is corrected to the original pixel value of the second pixel minus the just noticeable distortion value of the second pixel, so that the reduction amplitude of the corrected pixel value compared to the original pixel value does not cause distortion to be perceived by the human eye, while retaining the filtering effect. For example, if the just noticeable distortion value of a second pixel is 5, the original pixel value is 80, and the filtered pixel value is 72, then the corrected pixel value of the second pixel is 80-5, that is, 75.

[0119] Through the above implementation, the filtered pixel value is corrected using the just noticeable distortion value of the second pixel point, so that the change amplitude of the corrected pixel value of the second pixel point relative to the original pixel value will not cause distortion perceived by the human eye, which helps to ensure image quality while retaining the filtering effect, helping to reduce encoding pressure.

[0120] Optionally, for the first pixel points whose pixel difference is not zero and whose absolute value is less than the just noticeable distortion value, for the purpose of distinction, these first pixel points are recorded as fourth pixel points. The just noticeable distortion value of the fourth pixel point can also be used to correct the filtered pixel value to remove the perceptual redundancy of the fourth pixel point, which helps to further improve the subsequent encoding efficiency while ensuring image quality.

[0121] In the case where the filtered pixel value of the fourth pixel is greater than the original pixel value, the filtered pixel value of the fourth pixel may be corrected to the original pixel value of the fourth pixel plus the just noticeable distortion value of the fourth pixel. For example, if the just noticeable distortion value of a fourth pixel is 5, the original pixel value is 80, and the filtered pixel value is 82, then the corrected pixel value of the fourth pixel is 80+5, i.e., 85.

[0122] In the case where the filtered pixel value of the second pixel is less than the original pixel value, the filtered pixel value of the fourth pixel may be corrected to the original pixel value of the fourth pixel minus the just noticeable distortion value of the fourth pixel. For example, if the just noticeable distortion value of a fourth pixel is 5, the original pixel value is 80, and the filtered pixel value is 78, then the corrected pixel value of the fourth pixel is 80-5, i.e., 75.

[0123] In a possible implementation, obtaining the image to be processed may specifically include: obtaining a video to be processed, extracting video frames from the video to be processed, and using each extracted video frame as the image to be processed.

[0124] The video to be processed may be a video that needs to be processed before encoding. The video to be processed is composed of a number of video frames, and video frames are extracted from the video to be processed, and each extracted video frame is used as an image to be processed. The extracted video frames are processed frame by frame, and the processed images corresponding to each video frame can be obtained through the embodiment of the present application. Then, the processed images corresponding to each video frame are combined according to the frame extraction order, and the processed video corresponding to the video to be processed can be obtained for subsequent encoding.

[0125] Through the above implementation, each video frame extracted from the video to be processed is used as the image to be processed, which is conducive to ensuring the video quality and reducing the pressure of video encoding without increasing the complexity of the encoding itself.

[0126] Those skilled in the art will appreciate that, in the above method of specific implementation, the order in which the steps are written does not imply a strict execution order and does not constitute any limitation on the implementation process. The specific execution order of the steps should be determined by their functions and possible internal logic.

[0127] The above description of various embodiments tends to emphasize the differences between the various embodiments. The same or similar aspects can be referenced to each other, and for the sake of brevity, they will not be repeated herein.

[0128] See also Figure 2 , Figure 2 200 is a schematic diagram of the structure of an image processing device provided in an embodiment of the present application. The image processing device 200 includes: an acquisition unit 201, a division unit 202, a filtering unit 203, a correction unit 204 and a determination unit 205, wherein:

[0129] An acquisition unit 201 is used to acquire an image to be processed;

[0130] A division unit 202, used for dividing the image to be processed into an area of ​​interest and an area of ​​no interest;

[0131] A filtering unit 203 is used to filter the non-interest region to obtain a filtered pixel value of each first pixel point in the non-interest region;

[0132] A correction unit 204 is used to determine a distorted second pixel point from each first pixel point according to the filtered pixel value of each first pixel point, and correct the filtered pixel value of the second pixel point to obtain a processed non-interested region;

[0133] The determination unit 205 is used to determine the processed image according to the region of interest and the processed non-region of interest.

[0134] In some possible implementations, when the correction unit 204 determines the distorted second pixel point from each first pixel point based on the filtered pixel value of each first pixel point, it is specifically used to: obtain the original pixel value of each first pixel point in the image to be processed; and determine the distorted second pixel point from each first pixel point based on the filtered pixel value and the original pixel value of each first pixel point.

[0135] In some possible implementations, when the correction unit 204 determines the distorted second pixel from each first pixel based on the filtered pixel value and the original pixel value of each first pixel, it is specifically used to: calculate the pixel difference between the filtered pixel value and the original pixel value of each first pixel; obtain the just noticeable distortion value of each first pixel; and determine the distorted second pixel from each first pixel based on the pixel difference and the just noticeable distortion value of each first pixel.

[0136] In some possible implementations, when obtaining the just noticeable distortion value of each first pixel point, the correction unit 204 is specifically used to: obtain the brightness sensitivity of each first pixel point based on the brightness information of each first pixel point in the image to be processed; obtain the texture sensitivity of each first pixel point based on the gradient information of each first pixel point in the image to be processed; and determine the just noticeable distortion value of each first pixel point based on the brightness sensitivity and texture sensitivity of each first pixel point.

[0137] In some possible implementations, when the correction unit 204 determines a distorted second pixel from each first pixel based on the pixel difference and the just noticeable distortion value of each first pixel, it is specifically used to: when the filtered pixel value of the second pixel is greater than the original pixel value, correct the filtered pixel value of the second pixel to the original pixel value of the second pixel plus the just noticeable distortion value; when the filtered pixel value of the second pixel is less than the original pixel value, correct the filtered pixel value of the second pixel to the original pixel value of the second pixel minus the just noticeable distortion value.

[0138] In some possible implementations, when acquiring the image to be processed, the acquisition unit 201 is specifically used to: acquire the video to be processed, extract video frames from the video to be processed, and use each extracted video frame as the image to be processed.

[0139] For the specific definition of the image processing device, please refer to the definition of the image processing method above, which will not be repeated here. Each unit in the above image processing device can be implemented in whole or in part by software, hardware and a combination thereof. Each of the above units can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above units.

[0140] See also Figure 3 , Figure 3 300 is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. The electronic device 300 includes a memory 301 and a processor 302. Optionally, the electronic device 300 also includes a communication interface 303 and a bus 304. The memory 301, the processor 302 and the communication interface 303 are connected to each other through the bus 304. The memory 301 stores a computer program, and the processor 302 is used to execute the computer program stored in the memory 301 to implement the methods in the above-mentioned method embodiments.

[0141] An embodiment of the present application further provides a computer-readable storage medium, in which a computer program is stored. When the computer program is executed by a processor, the methods in the above-mentioned method embodiments are implemented.

[0142] An embodiment of the present application further provides a computer program product, which includes a computer program. When the computer program is executed by a processor, the method in the above-mentioned method embodiments is implemented.

[0143] It should be understood that the memory / readable storage medium in the embodiments of the present application may be a volatile memory or a non-volatile memory, or may include both a volatile memory and a non-volatile memory. Among them, the non-volatile memory may be a hard disk drive (HDD), a solid-state drive (SSD), a read-only memory (ROM) or a flash memory, etc. The volatile memory may be a random access memory (RAM) or an external cache memory. As an illustration and not limitation, RAM may be in various forms, such as a static random access memory (SRAM) or a dynamic random access memory (DRAM), etc.

[0144] The processor in the embodiments of the present application may be a combination of one or more processing modules such as a central processing unit (CPU), a graphics processing unit (GPU) or a microprocessor (MPU), and may also be other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc.

[0145] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0146] The above-mentioned embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for ordinary technicians in this field, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application.

Claims

1. An image processing method, characterized in that: The method comprises: Acquire an image to be processed, where the image to be processed is an image that needs to be filtered before encoding; Dividing the image to be processed into regions of interest and non-regions of interest; Filtering the non-interested region to obtain a filtered pixel value of each first pixel point in the non-interested region; Determine a distorted second pixel from each of the first pixel points according to the filtered pixel values ​​of each of the first pixel points, and correct the filtered pixel values ​​of the second pixel points to obtain a processed non-interested region; A processed image is obtained according to the region of interest that has not been subjected to filtering processing and the processed non-region of interest.

2. The method according to claim 1, characterized in that The determining of the distorted second pixel point from each of the first pixel points according to the filtered pixel value of each of the first pixel points comprises: Obtaining the original pixel value of each of the first pixel points in the image to be processed; Determine a distorted second pixel point from each of the first pixel points according to the filtered pixel value and the original pixel value of each of the first pixel points.

3. The method according to claim 2, characterized in that The determining of the distorted second pixel point from each of the first pixel points according to the filtered pixel value and the original pixel value of each of the first pixel points comprises: Calculate the pixel difference between the filtered pixel value and the original pixel value of each of the first pixels; Obtaining a just perceptible distortion value of each of the first pixels; A distorted second pixel point is determined from each of the first pixel points according to the pixel difference value and the just perceptible distortion value of each of the first pixel points.

4. The method according to claim 3, characterized in that The obtaining of the just perceptible distortion value of each of the first pixels includes: Obtaining brightness sensitivity of each of the first pixels according to brightness information of each of the first pixels in the image to be processed; Obtaining texture sensitivity of each of the first pixels according to gradient information of each of the first pixels in the image to be processed; A just perceptible distortion value of each of the first pixels is determined according to the brightness sensitivity and the texture sensitivity of each of the first pixels.

5. The method according to claim 3, characterized in that: The determining of the distorted second pixel from each of the first pixel points according to the pixel difference value and the just perceptible distortion value of each of the first pixel points comprises: A first pixel point whose absolute value of the pixel difference is greater than the just perceptible distortion value is determined as a distorted second pixel point.

6. The method according to claim 5, characterized in that The correcting the filtered pixel value of the second pixel point includes: When the filtered pixel value of the second pixel is greater than the original pixel value, correcting the filtered pixel value of the second pixel to the original pixel value of the second pixel plus the just perceptible distortion value; When the filtered pixel value of the second pixel point is less than the original pixel value, the filtered pixel value of the second pixel point is corrected to the original pixel value of the second pixel point minus the just perceptible distortion value.

7. The method according to any one of claims 1 to 6, characterized in that The step of obtaining the image to be processed comprises: A video to be processed is obtained, video frames are extracted from the video to be processed, and each extracted video frame is used as the image to be processed.

8. An image processing device, characterized in that: The device comprises: An acquisition unit, used for acquiring an image to be processed, wherein the image to be processed is an image that needs to be filtered before encoding; A division unit, used for dividing the image to be processed into an area of ​​interest and an area of ​​no interest; A filtering unit, configured to filter the non-interest region to obtain a filtered pixel value of each first pixel point in the non-interest region; a correction unit, configured to determine a distorted second pixel point from each of the first pixel points according to the filtered pixel values ​​of each of the first pixel points, and correct the filtered pixel values ​​of the second pixel point to obtain a processed non-interested region; The determining unit is used to determine the processed image according to the region of interest that has not been subjected to filtering processing and the processed non-region of interest.

9. An electronic device, characterized in that: The method comprises a memory and a processor, wherein the memory stores a computer program, and the processor implements the method according to any one of claims 1 to 7 when executing the computer program.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images