Noise evaluation method, image processing method, device and storage medium thereof

Abstract

A noise assessment method and image processing method, apparatus, and storage medium thereof are disclosed. The noise assessment method comprises: obtaining an image to be assessed, the image to be assessed including pixels to be assessed; obtaining difference values of the pixels to be assessed along a preset noise assessment direction; obtaining a noise assessment value for the pixels to be assessed based on the difference values of the pixels to be assessed along different noise assessment directions; and obtaining a noise assessment result for the image to be assessed based on the noise assessment values of all pixels to be assessed. The noise assessment method is only relevant to the image to be assessed, effectively improving the accuracy of noise assessment, and can be used after any image processing step without affecting the accuracy of the assessment, thereby facilitating improved noise reduction results in subsequent steps.

Description

Technical Field

[0001] The present invention relates to the field of image processing, and in particular to a noise assessment method and an image processing method, and a device and a storage medium thereof. Background Art

[0002] Accurate image noise assessment is essential for many image processing modules to function properly. For example, the effectiveness of 2D and 3D noise reduction functions depends on accurate noise assessment.

[0003] Most commonly used image sensors are linear, meaning the digital signal increases linearly with the number of received photons, and the noise source can be approximated as wide-sense stationary white noise in both time and space. Current technology allows for relatively accurate noise assessment of unprocessed raw images (RAW images) captured by these image sensors, for example using Poisson and Gaussian noise models. Noise assessment here involves estimating the noise variance at the center of a small image window based on certain characteristic parameters of the image sensor and the average brightness within that window. Subsequent image processing modules can then perform operations such as noise reduction based on this noise variance at the current image location.

[0004] However, RAW images typically have a large data bit width. Performing noise reduction and other operations directly on RAW images requires significant hardware resources, impacting power consumption, real-time performance, circuit area, and increasing costs. Therefore, noise reduction and other operations are typically not performed directly on RAW images. Instead, they are applied elsewhere in the image processing pipeline, such as on images after tone mapping or gamma correction. In this case, the image noise no longer meets the characteristics of RAW image noise and cannot be accurately estimated using Poisson and Gaussian noise models based on image sensor parameters. Therefore, the noise variance obtained using these evaluation methods is ineffective for noise reduction. Summary of the Invention

[0005] The problem solved by the present invention is how to improve the accuracy of noise assessment.

[0006] To solve the above problems, the present invention provides a noise assessment method, comprising: obtaining an image to be assessed, the image to be assessed including pixels to be assessed; obtaining difference values of the pixels to be assessed along a preset noise assessment direction; obtaining noise assessment values of the pixels to be assessed based on the difference values of the pixels to be assessed along different noise assessment directions; and obtaining a noise assessment result of the image to be assessed based on the noise assessment values of all the pixels to be assessed.

[0007] Correspondingly, a noise assessment device is also provided, comprising: a memory and a processor, wherein the memory stores a computer program that can be run on the processor, and is characterized in that when the processor executes the program, the processor performs the steps of the noise assessment method of the present invention.

[0008] and a storage medium, wherein the storage medium is a non-volatile storage medium or a non-transient storage medium, on which computer instructions are stored, characterized in that when the computer instructions are executed, the steps of the noise evaluation method of the present invention are executed.

[0009] In addition, the present invention also provides an image processing method, which is characterized by comprising: performing noise evaluation and obtaining a noise evaluation result, wherein in the step of performing noise evaluation, the noise evaluation is performed by the noise evaluation method of the present invention.

[0010] An image processing device, characterized in that it includes: a memory and a processor, wherein the memory stores a computer program that can be run on the processor, and characterized in that when the processor executes the program, the processor performs the steps of the image processing method of the present invention.

[0011] A storage medium is provided, wherein the storage medium is a non-volatile storage medium or a non-transient storage medium, on which computer instructions are stored, and wherein the steps of the image processing method of the present invention are executed when the computer instructions are executed.

[0012] Compared with the prior art, the technical solution of the present invention has the following advantages:

[0013] In the technical solution of the present invention, a noise evaluation value of a pixel to be evaluated is obtained based on the difference values of the pixel to be evaluated along different noise evaluation directions in the image to be evaluated, thereby obtaining a noise evaluation result of the image to be evaluated. The difference values represent the fluctuation of the pixel values of the pixel to be evaluated along the corresponding noise evaluation direction, so the difference values include both noise and the contribution of image detail edges. The noise evaluation value obtained based on the difference values is only related to the image to be evaluated itself, which can effectively improve the accuracy of noise evaluation. Moreover, the noise evaluation method is based on the image to be evaluated and does not rely on the characteristic parameters of the image sensor. Therefore, it can be used after any step in the image processing pipeline without affecting the accuracy of the evaluation, which is conducive to improving the effect of subsequent noise reduction steps.

[0014] In an optional solution of the present invention, the difference image can also be obtained in combination with a preceding image; based on the difference values of the pixel to be evaluated along different noise evaluation directions, the noise evaluation value of the pixel to be evaluated is obtained in combination with the difference values of the pixel to be evaluated in the difference image. When the image to be evaluated is a static image, that is, the image does not change over time, the difference image contains pure noise information and does not contain image details and edge information. When the image to be evaluated is a frame of a moving image, the difference value of the difference image is large. Therefore, the noise evaluation value obtained by combining the difference values of the difference image can further effectively improve the accuracy of the noise evaluation. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 1 is a flow chart of an embodiment of a noise assessment method according to the present invention;

[0016] Figure 2 yes Figure 1 A schematic flow chart of the step of obtaining a difference value of the pixel to be evaluated along a preset noise evaluation direction in the embodiment of the noise evaluation method shown;

[0017] Figure 3 yes Figure 1 A schematic diagram of the pixels to be evaluated in the neighborhood to be evaluated along the row direction in the embodiment of the noise evaluation method shown;

[0018] Figure 4 yes Figure 1 A schematic diagram of the pixels to be evaluated in the neighborhood to be evaluated along the column direction in the embodiment of the noise evaluation method shown;

[0019] Figure 5 yes Figure 1 A schematic diagram of the pixels to be evaluated in the neighborhood to be evaluated along the first evaluation diagonal direction in the embodiment of the noise evaluation method shown;

[0020] Figure 6 yes Figure 1 A schematic diagram of the position of the pixels to be evaluated in the neighborhood to be evaluated along the second evaluation diagonal direction in the embodiment of the noise evaluation method shown;

[0021] Figure 7 yes Figure 1 A schematic flow chart of the steps of performing uniform filtering on the noise assessment result in the noise assessment method embodiment shown;

[0022] Figure 8 is a schematic diagram of an image to be evaluated obtained by the second embodiment of the noise evaluation method of the present invention;

[0023] Figure 9 When the neighborhood to be evaluated is a 7×7 square neighborhood, Figure 8Display results of the difference values of all pixels to be evaluated in the image to be evaluated along the row direction;

[0024] Figure 10 When the neighborhood to be evaluated is a 7×7 square neighborhood, Figure 8 Display results of difference values of all pixels to be evaluated in the image to be evaluated along the column direction;

[0025] Figure 11 When the neighborhood to be evaluated is a 7×7 square neighborhood, Figure 8 Display results of difference values of all pixels to be evaluated in the image to be evaluated along the first evaluation diagonal direction;

[0026] Figure 12 When the neighborhood to be evaluated is a 7×7 square neighborhood, Figure 8 Display results of difference values of all pixels to be evaluated in the image to be evaluated along the second evaluation diagonal direction;

[0027] Figure 13 When the neighborhood to be evaluated is a 7×7 square neighborhood, Figure 8 Display results of noise evaluation values of all pixels to be evaluated in the image to be evaluated;

[0028] Figure 14 is a flow chart of a third embodiment of the noise assessment method of the present invention;

[0029] Figure 15 is a schematic diagram of an image to be evaluated obtained by the fourth embodiment of the noise evaluation method of the present invention;

[0030] Figure 16 It is a direct calculation Figure 15 Schematic diagram of the noise variance results within the neighborhood to be evaluated in the image to be evaluated;

[0031] Figure 17 The "single frame method" of the present invention is used to Figure 15 Schematic diagram of evaluation results obtained after noise evaluation of the image to be evaluated;

[0032] Figure 18 The double frame method of the present invention is used to Figure 15 Schematic diagram of evaluation results obtained after noise evaluation of the image to be evaluated;

[0033] Figure 19 is a schematic diagram of an image to be evaluated obtained by the fifth embodiment of the noise evaluation method of the present invention;

[0034] Figure 20 yes Figure 19 The real noise image in the image to be evaluated is shown;

[0035] Figure 21 The "single frame method" of the present invention is used to Figure 19 Schematic diagram of evaluation results obtained after noise evaluation of the image to be evaluated;

[0036] Figure 22 The double frame method of the present invention is used to Figure 19 Schematic diagram of evaluation results obtained after noise evaluation of the image to be evaluated;

[0037] Figure 23 yes Figure 21 The "single frame method" shown is Figure 19 The schematic diagram of the evaluation results obtained after the noise evaluation of the image to be evaluated is shown in FIG. Figure 20 Schematic diagram of the result of the difference of the real noise image shown;

[0038] Figure 24 yes Figure 22 The "double frame method" shown is Figure 19 The schematic diagram of the evaluation results obtained after the noise evaluation of the image to be evaluated is shown in FIG. Figure 20 Schematic diagram of the difference results of the real noise image shown. DETAILED DESCRIPTION

[0039] As can be seen from the background technology, the noise assessment methods in the prior art have the problem of low assessment accuracy. Now, we analyze the reasons for the low accuracy of a noise assessment method:

[0040] To conserve hardware resources, the 2D and 3D noise reduction modules in cameras and camcorders are typically not used directly on RAW images. Instead, they are placed relatively late in the image signal processing pipeline (ISP pipeline), such as after tone mapping or gamma correction. Because these mapping operations are nonlinear, the noise profile of the RAW image changes significantly. Using the same noise estimation methods for RAW images at this stage can lead to inaccurate noise estimation, resulting in poor noise reduction effects for modules like 2D and 3D noise reduction. This can cause undesirable consequences, such as blurred images or inability to reduce noise.

[0041] To address these technical issues, the present invention provides a noise assessment method and image processing method, as well as a device and storage medium. The noise assessment result is only relevant to the image to be assessed, effectively improving the accuracy of the noise assessment. Furthermore, the noise assessment method is based on the image to be assessed and can be applied after any step in the image processing pipeline without affecting the accuracy of the assessment, thus improving the effectiveness of subsequent noise reduction steps.

[0042] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, specific embodiments of the present invention are described in detail below with reference to the accompanying drawings.

[0043] refer to Figure 1 , which shows a flow chart of an embodiment of the noise assessment method of the present invention.

[0044] The noise assessment method includes: executing step S110 to obtain an image to be assessed, wherein the image to be assessed includes pixels to be assessed; executing step S120 to obtain difference values of the pixels to be assessed along a preset noise assessment direction; executing step S130 to obtain a noise assessment value of the pixels to be assessed based on the difference values of the pixels to be assessed along different noise assessment directions; and executing step S140 to obtain a noise assessment result of the image to be assessed based on the noise assessment values of all pixels to be assessed.

[0045] The technical solution of the noise assessment method embodiment of the present invention will be described in detail below with reference to the accompanying drawings.

[0046] First, step S110 is performed to obtain an image to be evaluated, where the image to be evaluated includes pixels to be evaluated.

[0047] The image to be evaluated is an evaluation object of the noise evaluation method.

[0048] Specifically, the pixels of the image to be evaluated are arranged in an array of n rows and m columns, so any pixel position in the image to be evaluated can be represented by its row number and column number, that is, the pixel position of the j-th row and i-th column, where j is an integer in the range [1, n] and i is an integer in the range [1, m]. The image to be evaluated has pixel positions to be evaluated.

[0049] In some embodiments of the present invention, the image to be evaluated is an image in an image signal processing pipeline (ISP pipeline). In some embodiments, the image to be evaluated is an image that has undergone a nonlinear mapping operation. Specifically, the image to be evaluated is an image that has undergone tone mapping or gamma correction.

[0050] Then, step S120 is executed to obtain the difference value of the pixel to be evaluated along the preset noise evaluation direction.

[0051] The difference value is used to characterize the fluctuation of the pixel value along the noise evaluation direction corresponding to the pixel to be evaluated, which includes both noise information and image information (for example, object details and edge information).

[0052] The difference value represents the fluctuation of pixel values in a certain size neighborhood, so Figure 2 As shown, in some embodiments of the present invention, executing step S120, the step of obtaining the difference value of the pixel position to be evaluated along the preset noise evaluation direction includes: first executing step S121, obtaining the pixel value of each pixel position in a preset neighborhood to be evaluated based on the pixel position to be evaluated, and the neighborhood to be evaluated includes the pixel position to be evaluated.

[0053] The size and shape of the neighborhood to be evaluated and the position of the pixel to be evaluated in the neighborhood to be evaluated are preset and can be adjusted according to the actual situation of the image to be evaluated.

[0054] In some embodiments of the present invention, in the step S121 of obtaining the pixel value of each pixel in a preset neighborhood to be evaluated based on the pixel to be evaluated, the pixel to be evaluated is the center pixel of the neighborhood to be evaluated.

[0055] like Figure 1 In the embodiment shown, the neighborhood to be evaluated is a centrally symmetrical square, and the pixel to be evaluated is located at the symmetrical center of the neighborhood to be evaluated. Figure 3 As shown, the neighborhood to be evaluated is a neighborhood of 3×3 size, and the pixel position to be evaluated is the pixel position in the 2nd row and 2nd column of the neighborhood to be evaluated, that is, the pixel position P22.

[0056] After obtaining the pixel value of each pixel position in the neighborhood to be evaluated, step S122 is executed to obtain the difference value of the pixel position to be evaluated along the noise evaluation direction according to the pixel value of each pixel position in the neighborhood to be evaluated.

[0057] In some embodiments of the present invention, step S122 is executed to obtain the difference value of the pixel to be evaluated along the noise evaluation direction, and the difference value of the pixel to be evaluated along the noise evaluation direction is obtained based on the average difference in pixel values of any two adjacent pixels in the neighborhood to be evaluated along the noise evaluation direction.

[0058] It should be noted that, in some embodiments of the present invention, S120, in the step of obtaining the difference value of the pixel position to be evaluated along a preset noise evaluation direction, the noise evaluation direction includes: a row direction, a column direction, and at least two of a first evaluation diagonal direction and a second evaluation diagonal direction; wherein, the first evaluation diagonal direction is the direction of the line between the pixel position of the jth row and the i-th column and the pixel position of the (j-1)th row and the (i-1)th column; the second evaluation diagonal direction is the direction of the line between the pixel position of the jth row and the i-th column and the pixel position of the (j-1)th row and the (i+1)th column, wherein j is an integer in [1, n], and i is an integer in [1, m].

[0059] Figure 1 In the embodiment shown, the noise evaluation direction is 4 directions, namely the row direction hori (such as Figure 3 As shown), column direction vert (as shown Figure 4 As shown) and the first evaluation diagonal direction diag (as shown Figure 5 As shown) and the second evaluation diagonal direction anti (as Figure 6 As shown); therefore, the step of obtaining the difference value of the pixel position P22 to be evaluated along the noise evaluation direction includes: obtaining the difference value of the pixel position P22 to be evaluated along the row direction hori; obtaining the difference value of the pixel position P22 to be evaluated along the column direction vert; obtaining the difference value of the pixel position P22 to be evaluated along the first evaluation diagonal direction diag; obtaining the difference value of the pixel position P22 to be evaluated along the second evaluation diagonal direction anti.

[0060] Therefore, in the step of obtaining the difference value of the pixel to be evaluated P22 along the row direction hori, the difference value df of the pixel to be evaluated P22 along the row direction hori is hori for:

[0061]

[0062] In the step of obtaining the difference value of the pixel to be evaluated P22 along the column direction vert, the difference value df of the pixel to be evaluated P22 along the column direction vert vert for:

[0063]

[0064] In the step of obtaining the difference value of the pixel position to be evaluated P22 along the first evaluation diagonal direction diag, the difference value df of the pixel position to be evaluated P22 along the first evaluation diagonal direction diag is diag for:

[0065]

[0066] In the step of obtaining the difference value of the pixel position to be evaluated P22 along the second evaluation diagonal direction anti, the difference value df of the pixel position to be evaluated P22 along the second evaluation diagonal direction anti is anti for:

[0067]

[0068] Among them, df hori represents the difference value of the pixel to be evaluated P22 along the row direction hori; df vertrepresents the difference value of the pixel to be evaluated P22 along the column direction vert; df diag represents the difference value of the pixel to be evaluated P22 along the first evaluation diagonal direction diag; df anti represents the difference value of the pixel to be evaluated P22 along the second evaluation diagonal direction anti; 11 、p 12 、……、p 33 They respectively represent the pixel value of pixel position P11, the pixel value of pixel position P12, ..., the pixel value of pixel position P33; abs means taking the absolute value.

[0069] Continue to refer Figure 1 After obtaining the difference value, step S130 is executed to obtain the noise evaluation value of the pixel to be evaluated according to the difference values of the pixel to be evaluated along different noise evaluation directions.

[0070] In some embodiments of the present invention, in the step S130 of obtaining the noise estimation value for the pixel to be evaluated, the minimum difference value is used as the noise estimation value for the pixel to be evaluated. When the image information details in the neighborhood to be evaluated are directional, the fluctuation of the pixel values of each pixel is minimal along the direction of the details. Therefore, the fluctuation of the pixel values in the same direction is primarily composed of noise information. That is, the fluctuation of the pixel values in the direction with the minimum fluctuation can reflect the noise situation of the pixel to be evaluated.

[0071] Specifically, in some embodiments, in the step of obtaining the difference values of the pixel to be evaluated along preset noise evaluation directions, multiple noise evaluation directions are preset, namely, the noise evaluation directions include: a row direction, a column direction, and at least two of a first evaluation diagonal direction and a second evaluation diagonal direction. Therefore, in the step of obtaining the noise evaluation value of the pixel to be evaluated, the minimum value among the difference values of the pixel to be evaluated along the multiple noise evaluation directions is used as the noise evaluation value of the pixel to be evaluated.

[0072] Specific as Figures 1 to 6 In the embodiment shown, in the step of obtaining the noise evaluation value of the pixel to be evaluated, the difference value df of the pixel to be evaluated P22 along the row direction hori is used. hori , the difference value df of the pixel to be evaluated P22 along the column direction vert vert , the difference value df of the pixel to be evaluated P22 along the first evaluation diagonal direction diag diag and the difference value df between the pixel to be evaluated P22 and the pixel to be evaluated P22 along the second evaluation diagonal direction anti anti The minimum value among them is used as the noise evaluation value of the pixel position P22 to be evaluated.

[0073] Continue to refer Figure 1 After obtaining the noise evaluation value of the pixel to be evaluated, step S140 is executed to obtain the noise evaluation result of the image to be evaluated based on the noise evaluation values of all the pixel to be evaluated.

[0074] Specifically, the image to be evaluated has a plurality of pixels to be evaluated; a noise evaluation value of each pixel to be evaluated is obtained, thereby obtaining a noise evaluation result of the entire image to be evaluated.

[0075] It should be noted that, in some embodiments of the present invention, Figure 1 As shown, after obtaining the noise evaluation result of the image to be evaluated, the noise evaluation method further includes: executing step S180 to perform a uniform filtering on the noise evaluation result. Further uniform filtering on the noise evaluation result can effectively improve the noise reduction effect of the noise evaluation result.

[0076] like Figure 7 As shown, in some embodiments, executing step S180, the step of performing uniform filtering on the noise evaluation result includes: executing step S181, obtaining a uniform noise evaluation value of the pixel position to be evaluated based on the noise evaluation values of all pixel positions in the neighborhood to be evaluated; executing step S182, obtaining a noise evaluation result after uniform filtering based on the uniform noise evaluation values of all pixel positions to be evaluated.

[0077] Among them, in the step of executing step S181 to obtain the uniform noise evaluation value of the pixel position to be evaluated, the average value of the noise evaluation values of all pixel positions in the uniform filtering neighborhood is used as the uniform noise evaluation value of the pixel position to be evaluated, wherein the uniform filtering neighborhood is the same as the neighborhood to be evaluated, that is, the shape and position of the uniform filtering neighborhood are the same as the shape and position of the neighborhood to be evaluated.

[0078] Therefore, in the step of obtaining the uniform noise evaluation value of the pixel to be evaluated, the average of the noise evaluation values of all the pixels in the neighborhood to be evaluated is used as the uniform noise evaluation value of the pixel to be evaluated.

[0079] It should be noted that in the aforementioned embodiment, the neighborhood to be evaluated is a 3×3 neighborhood, and the pixel to be evaluated is the pixel in row 2 and column 2 of the neighborhood to be evaluated. However, this configuration is merely an example, and in other embodiments of the present invention, the pixel to be evaluated and the neighborhood to be evaluated may be configured in other ways.

[0080] refer to Figures 8 to 13 , which shows a schematic diagram of the results of each step in the second embodiment of the noise evaluation method of the present invention.

[0081] The present invention will not be repeated here for the similarities with the above embodiments. The difference from the above embodiments is that, in some embodiments of the present invention, the neighborhood to be evaluated is a square neighborhood of (2*hws+1)×(2*hws+1), where hws is an integer greater than or equal to 1, and the pixel to be evaluated is the pixel in the yth row and xth column.

[0082] In the step of obtaining the difference value of the pixel to be evaluated along the row direction, the difference value of the pixel to be evaluated along the row direction is:

[0083]

[0084] Among them, df hori (y, x) represents the difference value of the pixel to be evaluated along the row direction; Ω represents the neighborhood to be evaluated with the pixel to be evaluated as the symmetric center, cnt pair It is the logarithm of the adjacent pixel positions in the row direction in Ω, that is, the number of p(j, i) to p(j, i-1) in Ω; j∈[y-hws,y+hws], i∈[x-hws+1,x+hws].

[0085] In the step of obtaining the difference value of the pixel to be evaluated along the column direction, the difference value of the pixel to be evaluated along the column direction is:

[0086]

[0087] Among them, df vert (y, x) represents the difference value of the pixel to be evaluated along the column direction; Ω represents the neighborhood to be evaluated with the pixel to be evaluated as the symmetric center, cnt pair is the logarithm of adjacent pixel positions in the column direction in Ω, that is, the number of p(j, i) to p(j-1, i) in Ω; j∈[y-hws+1,y+hws], i∈[x-hws,x+hws].

[0088] In the step of obtaining the difference value of the pixel to be evaluated along the first evaluation diagonal direction, the difference value of the pixel to be evaluated along the first evaluation diagonal direction is:

[0089]

[0090] Among them, df diag (y, x) represents the difference value of the pixel to be evaluated along the first evaluation diagonal direction; Ω represents the neighborhood to be evaluated with the pixel to be evaluated as the symmetric center, cnt pairis the logarithm of adjacent pixel positions in the first evaluation diagonal direction in Ω, that is, the number of p(j, i) to p(j-1, i-1) in Ω; j∈[y-hws+1,y+hws], i∈[x-hws+1,x+hws].

[0091] In the step of obtaining the difference value of the pixel to be evaluated along the second evaluation diagonal direction, the difference value of the pixel to be evaluated along the second evaluation diagonal direction is:

[0092]

[0093] Among them, df anti (y, x) represents the difference value of the pixel to be evaluated along the second evaluation diagonal direction; Ω represents the neighborhood to be evaluated with the pixel to be evaluated as the symmetric center, cnt pair is the logarithm of adjacent pixel positions in the second evaluation diagonal direction in Ω, that is, the number of p(j, i) to p(j-1, i+1) in Ω; j∈[y-hws+1,y+hws], i∈[x-hws,x+hws-1].

[0094] Specifically, Figure 8 is the image to be evaluated obtained in the embodiment of the noise evaluation method; Figure 9 When hws=3, that is, the neighborhood to be evaluated is a 7×7 square neighborhood, Figure 8 Display results of difference values of all pixels to be evaluated in the image to be evaluated along the row direction; Figure 10 When hws=3, that is, the neighborhood to be evaluated is a 7×7 square neighborhood, Figure 8 Display results of difference values of all pixels to be evaluated in the image to be evaluated along the column direction; Figure 11 When hws=3, that is, the neighborhood to be evaluated is a 7×7 square neighborhood, Figure 8 Display results of difference values of all pixels to be evaluated in the image to be evaluated along the first evaluation diagonal direction; Figure 12 When hws=3, that is, the neighborhood to be evaluated is a 7×7 square neighborhood, Figure 8 The display result of the difference values of all the pixels to be evaluated in the image to be evaluated along the second evaluation diagonal direction is shown.

[0095] Therefore, in the step of obtaining the noise evaluation value of the pixel to be evaluated, the difference value df of the pixel to be evaluated along the row direction is used. hori (y, x), the difference value df of the pixel to be evaluated along the column direction vert (y, x), the difference value df of the pixel to be evaluated along the first evaluation diagonal direction diag(y, x) and the difference value df between the pixel to be evaluated and the pixel along the second evaluation diagonal direction anti The minimum value of (y, x) is used as the noise evaluation value of the pixel to be evaluated.

[0096] Specifically, Figure 13 When hws=3, that is, the neighborhood to be evaluated is a 7×7 square neighborhood, Figure 8 The display result of the minimum difference value of all pixels to be evaluated in the image to be evaluated is shown, that is, Figure 8 The display results of the noise evaluation values of all pixels to be evaluated in the image to be evaluated are shown.

[0097] It should be noted that, as shown in Table 1, the larger the hws value is, the larger the area of the neighborhood to be evaluated is, and the amount of calculation required to obtain the noise evaluation value of each pixel to be evaluated will also increase accordingly.

[0098] Table 1: When the neighborhood to be evaluated is a square neighborhood of different sizes, the statistical results of the amount of calculation required to obtain the noise evaluation value of each pixel to be evaluated are as follows: (The statistical results of the amount of calculation required to obtain the noise evaluation result after uniformization filtering are shown in brackets):

[0099] Neighborhood size 3×3 5×5 7×7 9×9 11×11 Number of additions 36(44) 140(164) 308(356) 540(620) 836(956) Number of comparisons 3 3 3 3 3 abs times 20 72 156 272 420 Counting times 20 72 156 272 420 Number of multiplications 4(5) 4(5) 4(5) 4(5) 4(5)

[0100] It should be noted that in the aforementioned embodiment, the noise assessment method embodiment obtains the noise assessment result based solely on the image to be assessed, and therefore the above embodiment can also be referred to as a "single-frame method." However, this approach is merely an example, and in other embodiments of the present invention, a "dual-frame method" can also be used for noise assessment.

[0101] refer to Figure 14 , which shows a flow chart of the third embodiment of the noise assessment method of the present invention.

[0102] like Figure 14 As shown, the noise evaluation method includes: executing step S310 to obtain an image to be evaluated, wherein the image to be evaluated includes pixels to be evaluated; executing step S320 to obtain difference values of the pixels to be evaluated along a preset noise evaluation direction.

[0103] It should be noted that, for the specific technical solutions of executing step S310 to obtain the image to be evaluated and executing step S320 to obtain the difference value of the pixel to be evaluated along the preset noise evaluation direction, refer to the above embodiment, and the present invention will not repeat them here.

[0104] The difference from the above embodiments is that, in some embodiments of the present invention, the noise evaluation of the image to be evaluated also needs to be combined with the previous image adjacent to the image to be evaluated in the time domain. Figure 14 As shown, before executing step S330 to obtain the noise evaluation value of the pixel to be evaluated, executing step S350 to obtain a previous image, wherein the previous image and the image to be evaluated are adjacent images in the time domain; executing step S360 to obtain a difference image based on the previous image and the image to be evaluated.

[0105] The preceding image and the image to be evaluated are adjacent images in the time domain. That is, the preceding image is one of the previous frame and the next frame of the image to be evaluated. In some embodiments, the preceding image is the previous frame of the image to be evaluated. Therefore, all pixel bits of the preceding image correspond one-to-one with all pixel bits of the image to be evaluated.

[0106] The difference image can remove image details and edge information as much as possible for the static area of the image to be evaluated, and only retain the noise, so as to improve the accuracy of noise evaluation for the static area in the video image.

[0107] In some embodiments of the present invention, in step S360, obtaining a difference image, the difference between the pixel value of the image to be evaluated and the pixel value of the previous image at each pixel position is obtained to obtain the difference image. It can be seen that the difference image and the image to be evaluated have exactly the same pixel position distribution, that is, all pixel positions of the difference image correspond one-to-one to all pixel positions of the image to be evaluated.

[0108] Continue to refer Figure 14 After obtaining the difference image, step S370 is executed to obtain the difference value of the pixel position to be evaluated in the difference image.

[0109] In some embodiments of the present invention, executing step S370, the step of obtaining the difference value of the pixel position to be evaluated in the difference image includes: obtaining the difference value of the pixel position to be evaluated in the difference image according to the pixel value of the difference image of each pixel position in the neighborhood to be evaluated.

[0110] It should be noted that all pixel bits of the difference image also correspond one-to-one to all pixel bits of the image to be evaluated, so the pixel bits to be evaluated and the neighborhood to be evaluated can both correspond one-to-one to the difference image. For example: executing step S370 to obtain the difference value of the pixel bit to be evaluated in the difference image means obtaining the difference value of the pixel bit corresponding to the pixel bit to be evaluated in the difference image; obtaining the difference value of the pixel bit to be evaluated in the difference image based on the pixel value of the difference image of each pixel bit in the neighborhood to be evaluated means obtaining the difference value of the pixel bit to be evaluated in the difference image based on the pixel value of the difference image of each pixel bit in the neighborhood corresponding to the neighborhood to be evaluated.

[0111] Because the difference image removes as much image detail and edge information as possible for the static region of the image to be evaluated, the fluctuations in the pixel values of the difference image can effectively reflect the noise level in the static region of the image to be evaluated. Specifically, in some embodiments, in step S370, the step of obtaining the difference value of the pixel position to be evaluated in the difference image is performed, and the difference value of the pixel position to be evaluated in the difference image is obtained based on the standard deviation of the pixel value of the difference image for each pixel position in the neighborhood to be evaluated.

[0112] It should be noted that, in the step of obtaining the difference value of the pixel to be evaluated in the difference image, in addition to the standard deviation of the pixel value of the difference image of each pixel in the neighborhood to be evaluated, it is also necessary to combine the variance coefficient caused by the standard deviation calculation process. The variance coefficient is That is, in the step of obtaining the difference value of the pixel to be evaluated in the difference image, the standard deviation of the pixel value of the difference image of each pixel in the neighborhood to be evaluated is divided by To obtain the difference value of the pixel to be evaluated in the difference image.

[0113] In some other embodiments of the present invention, step S370 is executed to obtain the difference value of the pixel position to be evaluated in the difference image, including: obtaining the average difference in pixel values between any pixel position and the adjacent pixel position in the difference neighborhood according to the pixel position to be evaluated, wherein the difference neighborhood includes the pixel position to be evaluated; and obtaining the difference value of the pixel position to be evaluated according to the average value of the average difference in pixel values between all pixel positions in the difference neighborhood and the adjacent pixel positions, combined with the difference variance coefficient.

[0114] It should be noted that in order to simplify the calculation and reduce the load, in some embodiments of the present invention, in the step of obtaining the average difference in pixel values between any pixel position and the adjacent pixel position in the difference neighborhood, the average difference in pixel values between any pixel position and the adjacent pixel positions along multiple preset difference directions is obtained.

[0115] The difference direction is a row direction, a column direction, a first difference diagonal direction, and a second difference diagonal direction; the first difference diagonal direction is the direction of the line between the pixel bit of the j-th row and the i-th column and the pixel bit of the (j+1)-th row and the (i+1)-th column; the second difference diagonal direction is the direction of the line between the pixel bit of the j-th row and the i-th column and the pixel bit of the (j+1)-th row and the (i-1)-th column, where j is an integer in [1, n] and i is an integer in [1, m].

[0116] It should be noted that after subtracting the two frames of image, the noise variance will increase to 2 times of the original value, so the difference variance coefficient can be

[0117] Therefore, the difference value of the pixel position to be evaluated in the difference image, that is, the difference value of the pixel position corresponding to the pixel position to be evaluated in the difference image is:

[0118]

[0119] Among them, df diff (y, x) is the difference value of the pixel position corresponding to the pixel position to be evaluated in the difference image; Ω represents the difference neighborhood with the pixel position corresponding to the pixel position to be evaluated as the symmetric center, and the difference neighborhood and the neighborhood to be evaluated have the same shape and size; cnt pair is the number of pixel bits traversed by j, i in Ω, and the difference variance coefficient is p(j, i) represents the pixel value of the pixel at the jth row and ith column in the difference image.

[0120] Continue to refer Figure 14 After obtaining the difference value of the pixel to be evaluated in the difference image, step S330 is executed. In the step of obtaining the noise evaluation value of the pixel to be evaluated, the noise evaluation value of the pixel to be evaluated is obtained according to the difference value of the pixel to be evaluated along different noise evaluation directions and in combination with the difference value of the pixel to be evaluated in the difference image.

[0121] Specifically, in the step of obtaining the noise evaluation value of the pixel to be evaluated, the difference value df of the pixel to be evaluated along the row direction is used. hori (y, x), the difference value df of the pixel to be evaluated along the column direction vert (y, x), the difference value df of the pixel to be evaluated along the first evaluation diagonal direction diag (y, x), the difference value df of the pixel to be evaluated along the second evaluation diagonal direction anti (y, x) and the difference value df of the pixel position corresponding to the pixel position to be evaluated in the difference image diff The minimum value of (y, x) is used as the noise evaluation value of the pixel to be evaluated.

[0122] For the static part of the image, the difference image obtained by subtracting the image to be evaluated from the previous image contains pure noise information and does not contain image details and edge information. The difference value of the difference image can more accurately reflect the noise situation. For the moving part of the image, the absolute value of the pixel value of each pixel bit of the difference image is usually large, and the difference value of the obtained difference image is also large. When the minimum value is taken as the noise evaluation value of the pixel bit to be evaluated, it does not affect the accuracy of the noise evaluation.

[0123] It should be noted that after the noise evaluation is performed using the "double-frame method", the noise reduction effect can also be improved through uniform filtering. Figure 14 As shown, after executing step S330 to obtain the noise evaluation value of the pixel position to be evaluated, executing step S340 to obtain the noise evaluation result of the image to be evaluated based on the noise evaluation values of all the pixel positions to be evaluated; then executing step S380 to perform uniform filtering on the noise evaluation result.

[0124] It should also be noted that, as shown in Table 2, the process of noise evaluation using the "dual-frame method" requires more computational effort than the "single-frame method" (as shown in Table 1) because it adds the steps of subtracting two frames and obtaining the difference value of the pixel to be evaluated in the difference image.

[0125] Table 2: When the neighborhood to be evaluated is a square neighborhood of different sizes, the statistical results of the amount of calculation required to obtain the noise evaluation value of each pixel to be evaluated are as follows: (The statistical results of the amount of calculation required to obtain the noise evaluation result after uniformization filtering are shown in brackets):

[0126] Neighborhood size 3x3 5x5 7x7 9x9 11x11 Number of additions 50(58) 224(248) 518(566) 932(1012) 1466(1586) Number of comparisons 4 4 4 4 4 abs times 28 120 276 496 780 Counting times 22 84 186 328 510 Number of multiplications 5(6) 5(6) 5(6) 5(6) 5(6)

[0127] Figure 15 Another image to be evaluated is shown, wherein the image to be evaluated is a noise-free image artificially mixed with Gaussian white noise of μ=0, σ=20; Figure 16 Shows a direct calculation Figure 15 A schematic diagram of the noise variance results within the neighborhood to be evaluated in the image to be evaluated; Figure 17 The "single frame method" of the present invention is used to Figure 15 Schematic diagram of evaluation results obtained after noise evaluation of the image to be evaluated; Figure 18 The double frame method of the present invention is used to Figure 15 Schematic diagram of the evaluation results obtained after noise evaluation of the image to be evaluated.

[0128] It should be noted that the "double frame method" of the present invention is used to Figure 15 In the process of performing noise evaluation on the image to be evaluated, the preceding image is an image of the same noise-free image randomly mixed with Gaussian white noise of μ=0, σ=20.

[0129] like Figure 16 As shown in the figure, since the image itself has textures and edges, the method of directly calculating the noise in the neighborhood to be evaluated will mix in a lot of image information itself and cannot extract accurate noise information; Figure 17 In the evaluation results shown, the mean of sig = 20.125, the standard deviation = 3.92; Figure 18In the evaluation results shown, the sig mean = 19.19, the standard deviation = 3.47; while the actual statistical results of the added Gaussian white noise are: sig mean = 19.34, the standard deviation = 2.8.

[0130] It should be noted that Figures 15 to 18 In the noise assessment method shown, the neighborhood to be assessed is a 5×5 square neighborhood; in addition, the sig mean refers to the average value of the noise assessment results calculated using a neighborhood of the same size as the neighborhood to be assessed in the entire image.

[0131] It can be seen that for Gaussian white noise, the evaluation results obtained by the "dual-frame method" for noise evaluation have a very high accuracy; and the "single-frame method" can also obtain noise evaluation results with good accuracy, but with a smaller amount of calculation.

[0132] Figure 19 Another image to be evaluated is shown, where the image to be evaluated is an image containing Poisson noise generated from a noise-free image. Figure 20 Shown Figure 19 The figure shows the result of calculating the standard deviation of the difference image after subtracting the image to be evaluated from the reference image, where the reference image and the image to be evaluated are two images containing Poisson noise that are randomly generated from the same noise-free image and satisfy the same Poisson distribution. Therefore, it can be considered that Figure 20 for Figure 19 The real noise image in the image to be evaluated is shown.

[0133] Figure 21 The "single frame method" of the present invention is used to Figure 19 Schematic diagram of evaluation results obtained after noise evaluation of the image to be evaluated; Figure 22 The double frame method of the present invention is used to Figure 19 Schematic diagram of evaluation results obtained after noise evaluation of the image to be evaluated; Figure 23 yes Figure 21 The "single frame method" shown is Figure 19 The schematic diagram of the evaluation results obtained after the noise evaluation of the image to be evaluated is shown in FIG. Figure 20 Schematic diagram of the result of the difference of the real noise image shown; Figure 24 yes Figure 22 The "double frame method" shown is Figure 19 The schematic diagram of the evaluation results obtained after the noise evaluation of the image to be evaluated is shown in FIG. Figure 20 The result diagram of the difference between the real noise image shown in Figure 1 is as follows. Figure 23 In the results shown, mean = 0.58, standard deviation = 4.37; Figure 24 In the results shown, mean = -0.35, standard deviation = 3.52.

[0134] When the neighborhood to be evaluated is also set to a 5×5 square neighborhood, the noise evaluation method based on the Poisson noise model is Figure 19 The noise evaluation results of the image to be evaluated are shown in Figure 20 In the difference result of the real noise image shown, the mean is 1.1 and the standard deviation is 4.65. The noise evaluation method based on the Poisson noise model is an existing technology and will not be described in detail.

[0135] It can be seen that for Poisson noise, the accuracy of the various evaluation results of the "single-frame method", "double-frame method" and noise evaluation methods based on the Poisson noise model are similar.

[0136] Accordingly, the present invention also provides a noise assessment device, comprising: a memory and a processor, wherein the memory stores a computer program that can be run on the processor, and when the processor executes the program, the processor performs the steps of the noise assessment method of the present invention.

[0137] and a storage medium, wherein the storage medium is a non-volatile storage medium or a non-transient storage medium, on which computer instructions are stored, and when the computer instructions are executed, the steps of the noise evaluation method of the present invention are executed.

[0138] Since the processor of the noise assessment device is used to execute the steps of the noise assessment method of the present invention; and the computer instructions stored in the storage medium execute the steps of the noise assessment method of the present invention when they are executed, the specific technical solutions of the noise assessment device and the storage medium refer to the embodiments of the noise assessment method described above, and the present invention will not be repeated here.

[0139] In addition, the present invention also provides an image processing method, comprising: performing noise evaluation and obtaining a noise evaluation result, wherein in the step of performing noise evaluation, the noise evaluation is performed by the noise evaluation method of the present invention.

[0140] In some embodiments of the present invention, noise assessment is performed after the nonlinear mapping operation. The results obtained by the noise assessment method of the present invention are only related to the image to be assessed, effectively improving the accuracy of the noise assessment. Furthermore, the noise assessment method is based on the image to be assessed, so even after the nonlinear mapping operation, the results obtained by the noise assessment method still have a high degree of accuracy. In other words, the nonlinear mapping operation does not affect the accuracy of the assessment, effectively saving hardware resources.

[0141] In some embodiments of the present invention, the image processing method further comprises: performing noise reduction processing according to the noise evaluation result. Specifically, the noise reduction processing may include: at least one of 2D noise reduction and 3D noise reduction.

[0142] Correspondingly, the present invention also provides an image processing device, characterized in that it includes: a memory and a processor, wherein the memory stores a computer program that can be run on the processor, and when the processor executes the program, the processor executes the steps of the image processing method of the present invention.

[0143] A storage medium is provided, which is a non-volatile storage medium or a non-transient storage medium, on which computer instructions are stored. When the computer instructions are executed, the steps of the image processing method of the present invention are executed.

[0144] Since the processor of the image processing device is used to execute the steps of the image processing method of the present invention; the computer instructions stored in the storage medium execute the steps of the image processing method of the present invention when running, the specific technical solutions of the image processing device and the storage medium refer to the embodiments of the aforementioned image processing method, and the present invention will not be repeated here.

[0145] In summary, based on the difference values of the pixel positions to be evaluated in the image to be evaluated along different noise evaluation directions, the noise evaluation value of the pixel positions to be evaluated is obtained, and then the noise evaluation result of the image to be evaluated is obtained. The difference values represent the fluctuation of the pixel values of the pixel positions to be evaluated along the corresponding noise evaluation directions, so the difference values include both noise and the contribution of image detail edges. The noise evaluation value obtained based on the difference values is only related to the image to be evaluated itself, which can effectively improve the accuracy of noise evaluation. Moreover, the noise evaluation method is based on the image to be evaluated and can therefore be used after any step of image processing without affecting the accuracy of the evaluation, which is conducive to improving the effect of subsequent noise reduction steps.

[0146] Although the present invention is disclosed as above, the present invention is not limited thereto. Any person skilled in the art can make various changes and modifications without departing from the spirit and scope of the present invention. Therefore, the scope of protection of the present invention should be based on the scope defined by the claims.

Claims

1. A noise assessment method, characterized in that: include: Obtaining an image to be evaluated, wherein the image to be evaluated includes pixels to be evaluated; Obtaining a difference value of the pixel to be evaluated along a preset noise evaluation direction, wherein in the step of obtaining the difference value of the pixel to be evaluated along the noise evaluation direction, the difference value of the pixel to be evaluated along the noise evaluation direction is obtained based on the pixel values of all adjacent pixels in the neighborhood to be evaluated along the noise evaluation direction; Obtaining a noise evaluation value of the pixel to be evaluated according to difference values of the pixel to be evaluated along different noise evaluation directions, wherein in the step of obtaining the noise evaluation value of the pixel to be evaluated, the minimum difference value is used as the noise evaluation value of the pixel to be evaluated; A noise evaluation result of the image to be evaluated is obtained according to the noise evaluation values of all the pixel positions to be evaluated.

2. The noise evaluation method according to claim 1, wherein: The pixels of the image are arranged into an array of n rows and m columns; In the step of obtaining the difference value of the pixel to be evaluated along a preset noise evaluation direction, the noise evaluation direction includes: a row direction, a column direction, and at least two of a first evaluation diagonal direction and a second evaluation diagonal direction; The first evaluation diagonal direction is the direction of the line between the pixel bit in the j-th row and the i-th column and the pixel bit in the (j-1)-th row and the (i-1)-th column; the second evaluation diagonal direction is the direction of the line between the pixel bit in the j-th row and the i-th column and the pixel bit in the (j-1)-th row and the (i+1)-th column, where j is an integer in [1, n] and i is an integer in [1, m].

3. The noise evaluation method according to claim 1, wherein: The step of obtaining the difference value of the pixel to be evaluated along a preset noise evaluation direction includes: Obtaining a pixel value of each pixel in a preset neighborhood to be evaluated according to the pixel to be evaluated, wherein the neighborhood to be evaluated includes the pixel to be evaluated; According to the pixel value of each pixel in the neighborhood to be evaluated, a difference value of the pixel to be evaluated along the noise evaluation direction is obtained.

4. The noise evaluation method according to claim 3, wherein: In the step of obtaining the pixel value of each pixel in a preset neighborhood to be evaluated according to the pixel to be evaluated, the pixel to be evaluated is the central pixel of the neighborhood to be evaluated.

5. The noise evaluation method according to claim 3, wherein: In the step of obtaining the difference value of the pixel to be evaluated along the noise evaluation direction, the difference value of the pixel to be evaluated along the noise evaluation direction is obtained according to the average difference in pixel values of any two adjacent pixels in the neighborhood to be evaluated along the noise evaluation direction.

6. The noise evaluation method according to claim 1, wherein: Also includes: Obtaining a previous image, where the previous image and the image to be evaluated are adjacent images in the time domain; Obtaining a difference image according to the previous image and the image to be evaluated; Obtaining a difference value of the pixel to be evaluated in the difference image; In the step of obtaining the noise evaluation value of the pixel to be evaluated, the noise evaluation value of the pixel to be evaluated is obtained according to the difference values of the pixel to be evaluated along different noise evaluation directions and in combination with the difference values of the pixel to be evaluated in the difference image.

7. The noise evaluation method according to claim 6, wherein: In the step of obtaining the difference image, the difference between the pixel value of the image to be evaluated and the pixel value of the previous image at each pixel position is obtained to obtain the difference image; The step of obtaining the difference value of the pixel position to be evaluated in the difference image includes: obtaining the difference value of the pixel position to be evaluated in the difference image according to the pixel value of the difference image of each pixel position in the neighborhood to be evaluated.

8. The noise evaluation method according to claim 7, wherein: In the step of obtaining the difference value of the pixel position to be evaluated in the difference image, the difference value of the pixel position to be evaluated in the difference image is obtained according to the standard deviation of the pixel value of the difference image of each pixel position in the neighborhood to be evaluated.

9. The noise evaluation method according to claim 7, wherein: The step of obtaining the difference value of the pixel to be evaluated in the difference image includes: According to the pixel position to be evaluated, an average difference in pixel values between any pixel position and an adjacent pixel position in a difference neighborhood is obtained, wherein the difference neighborhood includes the pixel position to be evaluated; and according to the average value of the average differences in pixel values between all pixel positions in the difference neighborhood and the adjacent pixel positions, combined with the difference variance coefficient, a difference value of the pixel position to be evaluated is obtained.

10. The noise evaluation method according to claim 9, wherein: In the step of obtaining the average difference in pixel values between any pixel position and adjacent pixel positions in the difference neighborhood, the average difference in pixel values between any pixel position and adjacent pixel positions along multiple preset difference directions is obtained.

11. The noise evaluation method according to claim 10, wherein: The pixels of the image are arranged into an array of n rows and m columns; The difference direction is a row direction, a column direction, a first difference diagonal direction, and a second difference diagonal direction; The first difference diagonal direction is the direction of the line between the pixel bit in the j-th row and the i-th column and the pixel bit in the (j+1)-th row and the (i+1)-th column; the second difference diagonal direction is the direction of the line between the pixel bit in the j-th row and the i-th column and the pixel bit in the (j+1)-th row and the (i-1)-th column, where j is an integer in [1, n] and i is an integer in [1, m].

12. The noise evaluation method according to claim 1, wherein: Also includes: Performing homogenization filtering on the noise evaluation result.

13. The noise evaluation method according to claim 12, wherein: The step of performing homogenization filtering on the noise evaluation result includes: Obtaining a uniform noise evaluation value of the pixel to be evaluated based on the noise evaluation values of all pixel positions in the neighborhood to be evaluated; The noise evaluation result after uniform filtering is obtained according to the uniform noise evaluation values of all pixel positions to be evaluated.

14. The noise evaluation method according to claim 13, wherein: In the step of obtaining the uniform noise evaluation value of the pixel to be evaluated, the average of the noise evaluation values of all the pixels in the neighborhood to be evaluated is used as the uniform noise evaluation value of the pixel to be evaluated.

15. A noise assessment device, characterized in that: include: A memory and a processor, wherein the memory stores a computer program that can be run on the processor, and wherein when the processor executes the program, the processor performs the steps of the noise assessment method according to any one of claims 1 to 14.

16. A storage medium, wherein the storage medium is a non-volatile storage medium or a non-transient storage medium, and a computer instruction is stored thereon, characterized in that: When the computer instructions are executed, the steps of the noise evaluation method according to any one of claims 1 to 14 are executed.

17. An image processing method, characterized in that: include: Noise evaluation is performed to obtain a noise evaluation result. In the step of performing noise evaluation, the noise evaluation is performed by the noise evaluation method according to any one of claims 1 to 14.

18. The image processing method according to claim 17, wherein: After the nonlinear mapping operation, noise estimation is performed.

19. The image processing method according to claim 17, wherein: Also includes: Noise reduction processing is performed according to the noise evaluation result.

20. An image processing device, characterized in that: include: A memory and a processor, wherein the memory stores a computer program that can be run on the processor, and wherein when the processor executes the program, the processor executes the steps of the image processing method according to any one of claims 17 to 19.

21. A storage medium, wherein the storage medium is a non-volatile storage medium or a non-transient storage medium, and a computer instruction is stored thereon, characterized in that: When the computer instructions are executed, the steps of the image processing method according to any one of claims 17 to 19 are executed.

Images