An image processing method, apparatus, electronic device, and storage medium

Abstract

This application provides an image processing method, apparatus, electronic device, and storage medium, relating to the field of image processing technology. The method includes: converting an RGB image to be processed into a YUV image; obtaining the first brightness of each pixel in the YUV image via the Y channel, with each pixel corresponding to a first brightness; using each pixel as a center pixel; obtaining pixels within a preset range to obtain pixel blocks; performing brightness dilation processing on the center pixel of each pixel block based on the first brightness of each pixel in each pixel block to obtain a second brightness of each pixel; determining the gradient of each pixel based on the second brightness of each pixel; performing boundary sharpening processing on the center pixel of each pixel block according to the gradient of each pixel in each pixel block to obtain a target brightness of each pixel; and calculating a target RGB image by combining the target brightness of the Y channel with the UV channel of the RGB image to be processed, which can improve the image clarity in real time.

Description

Technical Field

[0001] This application relates to the field of image processing, and more specifically, to an image processing method, apparatus, electronic device, and storage medium. Background Technology

[0002] In live video streaming scenarios, video smoothness and clarity are the most important aspects of the customer experience. Image super-resolution technology can transform low-resolution images into high-resolution images. Traditional image super-resolution uses pixel interpolation to achieve this, which, while providing acceptable video playback smoothness, results in poor image clarity. Summary of the Invention

[0003] The purpose of this application is to provide an image processing method, apparatus, electronic device, and storage medium that can improve the clarity of images in real time.

[0004] To achieve the above objectives, the technical solutions adopted in the embodiments of this application are as follows:

[0005] In a first aspect, embodiments of this application provide an image processing method, the method comprising:

[0006] Convert the RGB image to be processed into a YUV image;

[0007] Obtain the first brightness of each pixel in the Y channel of the YUV image, wherein the YUV image is composed of multiple pixels, and each pixel corresponds to a first brightness;

[0008] Each pixel is used as the center pixel, and pixels within a preset range are obtained to obtain each pixel block;

[0009] Based on the first brightness of each pixel in each pixel block, a bright area dilation process is performed on the central pixel in each pixel block to obtain the second brightness of each pixel.

[0010] The gradient of each pixel is determined based on the second brightness of each pixel;

[0011] Based on the gradient of each pixel in each pixel block, the center pixel in each pixel block is subjected to boundary sharpening processing to obtain the target brightness of each pixel;

[0012] The target RGB image is obtained by calculating the target brightness of the Y channel and the UV channels of the RGB image to be processed.

[0013] In an optional implementation, the step of performing a bright area dilation process on the center pixel of each pixel block based on the first brightness of each pixel in each pixel block to obtain the second brightness of each pixel includes:

[0014] For each pixel block, all first target pixels that satisfy the first set condition are determined from the pixels corresponding to the first detection direction obtained from that pixel block;

[0015] Based on the first brightness of each of the first target pixels, the center pixel is subjected to brightness dilation processing to obtain the center pixel with updated brightness.

[0016] From the pixels corresponding to the second detection direction obtained from the pixel block, determine all second target pixels that satisfy the first set condition;

[0017] Based on the first brightness of each second target pixel, the brightness-updated center pixel is subjected to bright area dilation processing to obtain the brightness-updated center pixel;

[0018] From the pixels corresponding to the Nth detection direction obtained from the pixel block, determine all Nth target pixels that satisfy the first set condition;

[0019] Based on the first brightness of each of the Nth target pixels, the center pixel after the N-1 brightness updates is subjected to bright area dilation processing to obtain the center pixel after the Nth update;

[0020] The process continues until the center pixel is dilated based on all detection directions, resulting in the second brightness of each pixel.

[0021] In an optional implementation, the step of determining all first target pixels satisfying the first preset condition from the pixels corresponding to the first detection direction obtained from each pixel block includes:

[0022] For each pixel block, when the first detection direction is upward, obtain the first set of pixels above the center pixel and the second set of pixels below the center pixel;

[0023] Determine the first minimum and first maximum values ​​of the first brightness for each pixel in the first pixel set;

[0024] Determine the second minimum and second maximum values ​​of the first brightness for each pixel in the second pixel set;

[0025] When the first minimum value is greater than the second maximum value, each pixel in the first pixel set is determined as all first target pixels that satisfy the first set condition;

[0026] When the second minimum value is greater than the first maximum value, each pixel in the second pixel set is determined as all first target pixels that satisfy the first set condition.

[0027] In an optional implementation, the step of determining the gradient of each pixel based on the second brightness of each pixel includes:

[0028] Determine the first convolution kernel in the horizontal direction and the second convolution kernel in the vertical direction of the YUV image;

[0029] Calculate the first square of the second brightness of each pixel and the first product of the first convolution kernel;

[0030] Calculate the second square of the second product of the second brightness of each pixel and the second product of the second convolution kernel;

[0031] The square root of the sum of the first square and the second square is determined as the gradient of each pixel.

[0032] In an optional implementation, the step of performing boundary sharpening processing on the center pixel of each pixel block based on the gradient of each pixel in each pixel block to obtain the target brightness of each pixel includes:

[0033] For each pixel block, obtain the pixels corresponding to different detection directions from that pixel block;

[0034] Sequentially determine whether there are gradient pixels that satisfy the second set condition in each pixel corresponding to each detection direction, wherein the different detection directions correspond to a certain order;

[0035] When there is a gradient pixel that satisfies the second set condition in each pixel corresponding to any detection direction, determine the gradient pixel that satisfies the second set condition in each pixel corresponding to the target detection direction.

[0036] Based on the gradient of the gradient pixel, the center pixel is subjected to boundary sharpening processing to obtain the target brightness of each pixel.

[0037] In an optional implementation, the step of sequentially determining whether there are gradient pixels satisfying the second preset condition among the pixels corresponding to each detection direction includes:

[0038] For each detection direction, determine the set of third pixels and the set of fourth pixels in each pixel corresponding to that detection direction;

[0039] Determine the third minimum and third maximum values ​​of the gradient of each pixel in the third pixel set;

[0040] Determine the fourth minimum and fourth maximum values ​​of the gradient of each pixel in the fourth pixel set;

[0041] When the third minimum value is greater than the fourth maximum value, it is determined that there are gradient pixels that satisfy the second set condition among the pixels corresponding to the detection direction;

[0042] When the fourth minimum value is greater than the third maximum value, it is determined that there are gradient pixels that satisfy the second set condition among the pixels corresponding to the detection direction.

[0043] Secondly, embodiments of this application provide an image processing apparatus, the apparatus comprising:

[0044] The conversion module is used to convert the RGB image to be processed into a YUV image;

[0045] The first acquisition module is used to acquire the first brightness of each pixel in the Y channel of the YUV image, wherein the YUV image is composed of multiple pixels, and each pixel corresponds to a first brightness.

[0046] The second acquisition module is used to acquire pixels within a preset range as center pixels to obtain pixel blocks;

[0047] The first processing module is used to perform bright area dilation processing on the central pixel in each pixel block based on the first brightness of each pixel in each pixel block to obtain the second brightness of each pixel.

[0048] The first determining module is used to determine the gradient of each pixel based on the second brightness of each pixel;

[0049] The second processing module is used to perform boundary sharpening processing on the center pixel in each pixel block according to the gradient of each pixel in each pixel block, so as to obtain the target brightness of each pixel.

[0050] The second determining module calculates the target RGB image by combining the target brightness of the Y channel with the UV channels of the RGB image to be processed.

[0051] Thirdly, embodiments of this application provide an electronic device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps of the image processing method.

[0052] Fourthly, embodiments of this application provide a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the image processing method.

[0053] This application has the following beneficial effects:

[0054] This application improves the clarity of the processed target image by converting the RGB image to a YUV image and performing brightness dilation and edge sharpening operations on the first brightness of the Y channel. Specifically, the brightness dilation operation based on the first brightness of each pixel's Y channel is performed as follows: each pixel is used as the center pixel, and pixels within a preset range are obtained to form pixel blocks. Based on the first brightness of each pixel in each pixel block, brightness dilation processing is performed on the center pixel of each pixel block to obtain the second brightness of each pixel, making the brightness of the center pixel in each pixel block higher. The edge sharpening operation on the pixels after brightness adjustment is performed as follows: based on the second brightness of each pixel, the gradient of each pixel is determined. According to the gradient of each pixel in each pixel block, edge sharpening processing is performed on the center pixel in each pixel block to obtain the target brightness of each pixel. This can enhance the edge contrast of the image, thereby improving the image sharpness. The target brightness of the Y channel is calculated with the UV channels of the RGB image to be processed to obtain the target RGB image output. The final target RGB image is clearer and can meet the requirements of continuous processing of dynamic video images. Attached Figure Description

[0055] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0056] Figure 1 A block diagram illustrating an electronic device provided in an embodiment of this application;

[0057] Figure 2 This is one of the flowcharts of an image processing method provided in an embodiment of this application;

[0058] Figure 3 This is a second flowchart illustrating the steps of an image processing method provided in an embodiment of this application.

[0059] Figure 4 This is a schematic diagram of different detection directions for pixel blocks provided in an embodiment of this application;

[0060] Figure 5 This is the third step in a flowchart of an image processing method provided in an embodiment of this application.

[0061] Figure 6 The fourth step of an image processing method provided in this application embodiment;

[0062] Figure 7 This is a structural block diagram of an image processing apparatus provided in an embodiment of this application. Detailed Implementation

[0063] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. The components of the embodiments of this application described and shown in the accompanying drawings can generally be arranged and designed in various different configurations.

[0064] Therefore, the following detailed description of the embodiments of this application provided in the accompanying drawings is not intended to limit the scope of the claimed application, but merely to illustrate selected embodiments of the application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without inventive effort are within the scope of protection of this application.

[0065] It should be noted that similar labels and letters in the following figures indicate similar items. Therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures.

[0066] In the description of this application, it should be noted that if terms such as "upper," "lower," "inner," or "outer" are used to indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings, or the orientation or positional relationship in which the product of the invention is usually placed during use, they are only for the convenience of describing this application and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of this application.

[0067] Furthermore, the terms "first" and "second" are used only to distinguish descriptions and should not be interpreted as indicating or implying relative importance.

[0068] Through extensive research, the inventors discovered that in live video streaming scenarios, video smoothness and clarity are the most important aspects of the user experience. Image super-resolution technology can transform low-resolution images into high-resolution images. Traditional image super-resolution uses pixel interpolation to achieve this, which, while providing acceptable video playback smoothness, results in poor image clarity.

[0069] In view of the above-mentioned problems, this embodiment provides an image processing method, apparatus, electronic device and storage medium that can perform bright area dilation and boundary sharpening operations on the Y channel of the converted YUV image to make the processed target RGB image have higher clarity. The solution provided in this embodiment will be described in detail below.

[0070] This embodiment provides an electronic device capable of processing images. In one possible implementation, the electronic device can be a user terminal, such as, but not limited to, a server, smartphone, personal computer (PC), tablet computer, personal digital assistant (PDA), mobile internet device (MID), and image acquisition device.

[0071] Please refer to Figure 1 , Figure 1 This is a schematic diagram of the structure of the electronic device 100 provided in the embodiments of this application. The electronic device 100 may further include... Figure 1 The more or fewer components shown, or having the same Figure 1 The different configurations shown. Figure 1 The components shown can be implemented using hardware, software, or a combination thereof.

[0072] The electronic device 100 includes an image processing device 110, a memory 120, and a processor 130.

[0073] The components of the memory 120 and processor 130 are electrically connected directly or indirectly to achieve data transmission or interaction. For example, these components can be electrically connected to each other through one or more communication buses or signal lines. The image processing device 110 includes at least one software function module that can be stored in the memory 120 in the form of software or firmware or embedded in the operating system (OS) of the electronic device 100. The processor 130 is used to execute the executable modules stored in the memory 120, such as the software function modules and computer programs included in the image processing device 110.

[0074] The memory 120 may be, but is not limited to, Random Access Memory (RAM), Read Only Memory (ROM), Programmable Read-Only Memory (PROM), Erasable Programmable Read-Only Memory (EPROM), Electrically Erasable Programmable Read-Only Memory (EEPROM), etc. The memory 120 is used to store programs, and the processor 130 executes the programs after receiving execution instructions.

[0075] Please refer to Figure 2 , Figure 2 For application Figure 1 The flowchart below shows an image processing method for an electronic device 100, and the method includes each step in detail.

[0076] Step 201: Convert the RGB image to be processed into a YUV image.

[0077] Step 202: Obtain the first brightness of each pixel in the Y channel of the YUV image.

[0078] The YUV image is composed of multiple pixels, and each pixel corresponds to a first brightness.

[0079] Step 203: Using each pixel as the center pixel, obtain pixels within a preset range to get each pixel block.

[0080] Step 204: Based on the first brightness of each pixel in each pixel block, perform bright area dilation processing on the center pixel in each pixel block to obtain the second brightness of each pixel.

[0081] Step 205: Determine the gradient of each pixel based on the second brightness of each pixel.

[0082] Step 206: Based on the gradient of each pixel in each pixel block, perform boundary sharpening processing on the center pixel in each pixel block to obtain the target brightness of each pixel.

[0083] Step 207: Calculate the target RGB image by combining the target brightness of the Y channel with the UV channels of the RGB image to be processed.

[0084] It should be noted that the RGB images to be processed include various types, such as real-world video scenes, like game interfaces or anime videos, as well as live stream videos or videos or images to be processed that are retrieved and stored from electronic devices.

[0085] To facilitate brightness processing of the RGB image, it is necessary to convert the RGB image to a YUV image. This conversion can be performed using the following methods:

[0086] Y=0.299*R+0.587*G+0.114*B, U=-0.169*R-0.331*G+0.500*B, V=0.500*R-0.439*G-0.081*B.

[0087] The first brightness of each pixel in the Y channel of the converted YUV image is obtained, and the bright area dilation and boundary sharpening processing are performed on the Y channel. The U and V channels are not processed and are retained.

[0088] To significantly reduce the search range while increasing the search speed, each pixel can be used as the center pixel, and pixels within a preset range can be acquired to form a pixel block.

[0089] It should be noted that the preset range can be two units of pixels in each direction according to the center point, that is, the obtained pixel block is a 3*3 pixel block. The specific number of pixels contained in the pixel block can be set by those skilled in the art according to the actual situation.

[0090] For the center pixel in each pixel block, a bright area dilation process is performed based on the first brightness of each pixel in the pixel block, that is, the center pixel is brightened. After the bright area dilation process is performed on each pixel based on the first brightness of each pixel in the corresponding pixel block, each pixel in the YUV image is brightened, and the pixel after the bright area dilation process is updated from the first brightness to the second brightness.

[0091] After bright area dilation, each pixel is sharpened at its edge to enhance image edge contrast and improve image clarity.

[0092] For example, image edge sharpening is performed based on the gradient of each pixel: the gradient of each pixel is calculated, and the center pixel in each pixel block is sharpened based on the gradient in each pixel block, so as to obtain the final brightness of each pixel of the YUV image, i.e., the target brightness.

[0093] The final, sharper target RGB image is obtained by combining the calculated target luminance of the new Y channel with the original UV channels of the RGB image to be processed. The calculation method for obtaining the target RGB image can be as follows:

[0094] R=Y+1.402*V, G=Y-0.344*U-0.792*V, B=Y+1.772*U.

[0095] The above method can be executed on a user terminal, for example, when the user terminal is running an application related to image processing, such as when the user is watching a live broadcast, playing a game, or performing image processing through the application. The above method can be used to process the live broadcast screen, the game interface, and the image to be processed.

[0096] This application improves the clarity of the processed target image by converting the RGB image to a YUV image and performing brightness dilation and edge sharpening operations on the first brightness of the Y channel. Specifically, the brightness dilation operation based on the first brightness of each pixel's Y channel is performed as follows: each pixel is used as the center pixel, and pixels within a preset range are obtained to form pixel blocks. Based on the first brightness of each pixel in each pixel block, brightness dilation processing is performed on the center pixel of each pixel block to obtain the second brightness of each pixel, making the brightness of the center pixel in each pixel block higher. The edge sharpening operation on the pixels after brightness adjustment is performed as follows: based on the second brightness of each pixel, the gradient of each pixel is determined. According to the gradient of each pixel in each pixel block, edge sharpening processing is performed on the center pixel in each pixel block to obtain the target brightness of each pixel. This can enhance the edge contrast of the image, thereby improving the image sharpness. The target brightness of the Y channel is calculated with the UV channels of the RGB image to be processed to obtain the target RGB image output. The final target RGB image is clearer and can meet the requirements of continuous processing of dynamic video images.

[0097] There are multiple ways to perform bright area dilation on the center pixel of each pixel block. In one implementation, such as... Figure 3 As shown, it may include the following steps:

[0098] Step 204-1: For each pixel block, determine all first target pixels that satisfy the first set conditions from the pixels corresponding to the first detection direction obtained in the pixel block.

[0099] Step 204-2: Perform bright area dilation processing on the center pixel based on the first brightness of each first target pixel to obtain the center pixel with updated brightness.

[0100] Step 204-3: Determine all second target pixels that satisfy the first set condition from the pixels corresponding to the second detection direction obtained from the pixel block.

[0101] Step 204-4: Perform bright area dilation processing on the center pixel after brightness update based on the first brightness of each second target pixel to obtain the center pixel after brightness update.

[0102] Step 204-5: Determine all Nth target pixels that satisfy the first set condition from the pixels corresponding to the Nth detection direction obtained from the pixel block.

[0103] Step 204-6: Based on the first brightness of each Nth target pixel, perform bright area dilation processing on the center pixel after the N-1 brightness updates to obtain the center pixel after the Nth update.

[0104] Step 204-7: Continue until the dilation process of the center pixel based on all detection directions is completed, and the second brightness of each pixel is obtained.

[0105] For each pixel block, there are N detection directions. This embodiment illustrates the case when N equals 8.

[0106] like Figure 4 As shown, there are 8 detection directions for any pixel block.

[0107] Figure 4 In the diagram, (a) represents the first detection direction, and the pixels corresponding to the first detection direction are: the three B pixels above the center pixel and the three A pixels below the center pixel.

[0108] The brightness of pixel blocks is detected based on different detection directions, for example:

[0109] The method for obtaining the first target pixel that satisfies the first set condition from the pixels corresponding to the first detection direction of the pixel block is as follows: For each pixel block, when the first detection direction is upward, obtain the first pixel set above the center pixel and the second pixel set below the center pixel, determine the first minimum and first maximum values ​​of the first brightness of each pixel in the first pixel set, determine the second minimum and second maximum values ​​of the first brightness of each pixel in the second pixel set, when the first minimum value is greater than the second maximum value, determine each pixel in the first pixel set as all the first target pixels that satisfy the first set condition, and when the second minimum value is greater than the first maximum value, determine each pixel in the second pixel set as all the first target pixels that satisfy the first set condition.

[0110] When the first minimum value is greater than the second maximum value, in the first detection direction, the brightness of the first pixel set is higher than the brightness of the second pixel set, that is, the brightness above the center pixel is higher than the brightness below it. When the second minimum value is greater than the first maximum value, the brightness of the second pixel set is higher than the brightness of the first pixel set, that is, the brightness below the center pixel is higher than the brightness above it.

[0111] For example: when Figure 4In (a), when the first brightness of the three pixels B above the center pixel C is 3, 4, and 5 from left to right, and the first brightness of the three pixels A below the center pixel C is 1.5, 2, and 2.5 from left to right, the three pixels B are determined as the first pixel set, and the pixels A are determined as the second pixel set. The first minimum value min(B) = 3 and the first maximum value max(B) = 5 of the first brightness of each pixel B in the first pixel set are determined. The second minimum value min(A) = 1.5 and the second maximum value max(A) = 2.5 of the first brightness of each pixel A in the second pixel set are determined. Since min(B) > max(A), each pixel B in the first pixel set is then designated as the first target pixel.

[0112] Based on the first brightness of each first target pixel, a brightness dilation process is performed on the center pixel to obtain the center pixel with updated brightness, specifically including:

[0113] C' = α*C + (1-α)*(X1+X2+X3) / 3, where α∈[0,1], X is the brightness of the first target pixel, C' is the updated brightness, and C is the first brightness of the center pixel.

[0114] It should be noted that α controls the degree of diffusion of the bright area. The larger α is, the greater the degree of expansion of the bright area. Those skilled in the art can set the value of α according to the actual situation.

[0115] The center pixel with updated brightness is obtained based on the first detection direction, and the center pixel with updated brightness is used as the center pixel of the pixel block in the second detection direction. At this time, the brightness of the center pixel in the second detection direction has been updated.

[0116] Figure 4 In the diagram, (b) represents the second detection direction, and the pixels corresponding to the second detection direction are: three A pixels above the center pixel and three B pixels below the center pixel. The first pixel set in the second detection direction consists of the three B pixels below the center pixel, and the second pixel set consists of the three A pixels above the center pixel.

[0117] Figure 4 In the middle (c), the third detection direction is defined. The pixels corresponding to the third detection direction are: three A pixels and three B pixels, including the center pixel. Figure 4 In (d), the fourth detection direction is represented by the following pixels: three B pixels and three A pixels, including the center pixel.

[0118] Figure 4 In the diagram, (e), (f), (g), and (h) are schematic diagrams of each pixel in the fifth, sixth, seventh, and eighth detection directions, respectively.

[0119] The first set of pixels corresponding to any detection direction is a set consisting of B pixels, and the second set of pixels is a set consisting of A pixels.

[0120] All target pixels that meet the first set condition and are consistent with the first detection direction are determined from the pixels corresponding to each detection direction, which will not be elaborated here.

[0121] The above operation of updating the brightness of the center pixel is performed in all eight detection directions. The brightness of the center pixel is updated after each operation until the dilation process of the center pixel based on all detection directions is completed, and the second brightness of each pixel is obtained.

[0122] For example, when the order of the detection directions is from the first detection direction to the eighth detection direction, when performing bright area dilation, if there is a first target pixel in the first detection direction, the bright area of ​​the center pixel is dilated based on the brightness of the first target pixel to obtain the brightness C1 of the center pixel. If there is a second target pixel in the second detection direction, the bright area of ​​the center pixel is dilated based on the brightness C1 of the second target pixel to obtain the brightness C2 of the center pixel. ... If there is an eighth target pixel in the eighth detection direction, the bright area is dilated based on the brightness C7 of the eighth target pixel to obtain the brightness C8 of the center pixel. C8 is the second brightness of the center pixel.

[0123] For determining the gradient of each pixel, such as Figure 5 As shown, it may include the following steps:

[0124] Step 205-1: Determine the first convolution kernel in the horizontal direction and the second convolution kernel in the vertical direction of the YUV image.

[0125] Step 205-2: Calculate the first square of the second brightness of each pixel and the first product of the first convolution kernel.

[0126] Step 205-3: Calculate the second square of the second product of the second brightness of each pixel and the second product of the second convolution kernel.

[0127] Step 205-4: Determine the square root of the sum of the first square and the second square, and use it as the gradient of each pixel.

[0128] For example: the first convolution kernel in the horizontal direction of the YUV image can be [-1,0,1; -2,0,2; -1,0,1], and the second convolution kernel in the vertical direction can be [-1,0,1; -2,0,2; -1,0,1].

[0129] Multiplying the first convolution kernel by the second brightness of each pixel yields the gradient in the horizontal direction of each pixel, and multiplying the second convolution kernel by the second brightness of each pixel yields the gradient in the vertical direction of each pixel.

[0130] The final gradient of each pixel is determined based on its horizontal and vertical gradients as follows:

[0131] Grad = sqrt(gradX*gradX+gradY*gradY), where gradX is the gradient in the horizontal direction of the pixel, gradY is the gradient in the vertical direction of the pixel, and sqrt is the square root.

[0132] There are multiple ways to obtain the target brightness of each pixel by sharpening the boundary of the center pixel in each pixel block. In one implementation, such as... Figure 6 As shown, it may include the following steps:

[0133] Step 206-1: For each pixel block, obtain the pixels corresponding to different detection directions from that pixel block.

[0134] Step 206-2: Sequentially determine whether there are gradient pixels that satisfy the second set condition in each pixel corresponding to each detection direction.

[0135] The different detection directions have a specific order.

[0136] Step 206-3: When there are gradient pixels that satisfy the second set condition in each pixel corresponding to any detection direction, determine the gradient pixels that satisfy the second set condition in each pixel corresponding to the target detection direction.

[0137] Step 206-4: Sharpen the boundaries of the center pixel based on the gradient of the gradient pixel to obtain the target brightness of each pixel.

[0138] Still refer to Figure 4 Obtain the pixels corresponding to different detection directions. Figure 4 It contains eight detection directions, which have a specific order. It should be noted that the order of the different detection directions can be from the first detection direction to the eighth detection direction, i.e., from image (a) to image (h). Alternatively, it can be from the eighth detection direction to the first detection direction, i.e., from image (h) to image (a). The order of the first to eighth detection directions can also be completely shuffled, for example: the order could be the first detection direction, the fourth detection direction, the second detection direction, the sixth detection direction, the fifth detection direction, the third detection direction, the eighth detection direction, and the seventh detection direction. This application does not impose specific restrictions on the order of the different detection directions.

[0139] In one example, the method of sequentially determining whether there are gradient pixels satisfying a second set condition in each pixel block corresponding to each detection direction is as follows:

[0140] for Figure 4For any detection direction, for each detection direction, determine the third set of pixels (i.e., all B pixels) and the fourth set of pixels (i.e., all A pixels) in each pixel corresponding to that detection direction. Determine the third minimum value min(B) and the third maximum value max(B) of the gradient of each pixel in the third set of pixels. Determine the fourth minimum value min(A) and the fourth maximum value max(A) of the gradient of each pixel in the fourth set of pixels. When the third minimum value min(B) is greater than the fourth maximum value max(A), it is determined that there are gradient pixels in each pixel corresponding to that detection direction that satisfy the second set condition. When the fourth minimum value min(A) is greater than the third maximum value max(B), it is determined that there are gradient pixels in each pixel corresponding to that detection direction that satisfy the second set condition.

[0141] The gradient of the gradient pixel is used to sharpen the boundary of the center pixel based on the following formula:

[0142] C”=β*C+(1-β)*(X1+X2+X3) / 3, where β∈[0,1], X is the gradient of the gradient pixel, C” is the target brightness, and C is the gradient of the center pixel.

[0143] It should be noted that β controls the sharpness of the image boundaries. The larger β is, the sharper the image boundaries. Those skilled in the art can set the value of β according to the actual situation.

[0144] For example, if there are gradient pixels that satisfy the second set condition among the pixels corresponding to the first detection direction, then the gradient pixels that satisfy the second set condition among the pixels corresponding to the first detection direction are determined, the boundary sharpening processing of the center pixel is performed based on the gradient pixels, and the step of determining whether there are gradient pixels that satisfy the second set condition among the pixels corresponding to the detection direction ends, thus completing the boundary sharpening operation of the center pixel.

[0145] Please refer to Figure 7 This application embodiment also provides an application for Figure 1 The image processing device 110 of the electronic device 100 includes:

[0146] Conversion module 111 is used to convert the RGB image to be processed into a YUV image;

[0147] The first acquisition module 112 is used to acquire the first brightness of the Y channel of each pixel in the YUV image, wherein the YUV image is composed of multiple pixels, and each pixel corresponds to a first brightness;

[0148] The second acquisition module 113 is used to acquire pixels within a preset range to obtain each pixel block, with each pixel as the center pixel.

[0149] The first processing module 114 is used to perform bright area dilation processing on the center pixel in each pixel block based on the first brightness of each pixel in each pixel block to obtain the second brightness of each pixel.

[0150] The first determining module 115 is used to determine the gradient of each pixel based on the second brightness of each pixel;

[0151] The second processing module 116 is used to perform boundary sharpening processing on the center pixel in each pixel block according to the gradient of each pixel in each pixel block, so as to obtain the target brightness of each pixel.

[0152] The second determining module 117 is used to calculate the target RGB image by combining the target brightness of the Y channel with the UV channels of the RGB image to be processed.

[0153] It should be noted that the image processing apparatus provided in this embodiment has the same basic principle and technical effect as the method embodiment described above. For the sake of brevity, any parts not mentioned in this embodiment can be referred to the corresponding content in the method embodiment described above.

[0154] This application also provides an electronic device 100, which includes a processor 130 and a memory 120. The memory 120 stores computer-executable instructions, which, when executed by the processor 130, implement the image processing method.

[0155] This application embodiment also provides a computer-readable storage medium storing a computer program, which, when executed by a processor 130, implements the image processing method.

[0156] In summary, this application improves the clarity of the processed target image by converting the RGB image to be processed into a YUV image and performing bright area dilation and boundary sharpening operations on the first brightness of the Y channel, and can meet the requirements of continuous processing of dynamic video images.

[0157] In the embodiments provided in this application, it should be understood that the disclosed apparatus and methods can also be implemented in other ways. The apparatus embodiments described above are merely illustrative. For example, the flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of apparatus, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than those marked in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram and / or flowchart, and combinations of blocks in block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0158] Furthermore, the functional modules in the various embodiments of this application can be integrated together to form an independent part, or each module can exist independently, or two or more modules can be integrated to form an independent part. If the function is implemented as a software functional module and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes: USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, optical disks, and other media capable of storing program code.

[0159] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0160] The above descriptions are merely various embodiments of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. An image processing method, characterized by, The method includes: Convert the RGB image to be processed into a YUV image; Obtain the first brightness of each pixel in the Y channel of the YUV image, wherein the YUV image is composed of multiple pixels, and each pixel corresponds to a first brightness; Each pixel is used as the center pixel, and pixels within a preset range are obtained to obtain each pixel block; Based on the first brightness of each pixel in each pixel block, a bright area dilation process is performed on the central pixel in each pixel block to obtain the second brightness of each pixel. The gradient of each pixel is determined based on the second brightness of each pixel; For each pixel block, obtain the pixels corresponding to different detection directions from that pixel block; Sequentially determine whether there are gradient pixels that satisfy the second set condition in each pixel corresponding to each detection direction, wherein the different detection directions correspond to a certain order; When there is a gradient pixel that satisfies the second set condition in each pixel corresponding to any detection direction, determine the gradient pixel that satisfies the second set condition in each pixel corresponding to the target detection direction. Based on the gradient of the gradient pixel, the center pixel is subjected to boundary sharpening processing to obtain the target brightness of each pixel; The target RGB image is obtained by calculating the target brightness of the Y channel and the UV channels of the RGB image to be processed.

2. The method according to claim 1, characterized in that, The step of performing brightness dilation processing on the center pixel of each pixel block based on the first brightness of each pixel in each pixel block to obtain the second brightness of each pixel includes: For each pixel block, all first target pixels that satisfy the first set condition are determined from the pixels corresponding to the first detection direction obtained from that pixel block; Based on the first brightness of each of the first target pixels, the center pixel is subjected to brightness dilation processing to obtain the center pixel with updated brightness. From the pixels corresponding to the second detection direction obtained from the pixel block, determine all second target pixels that satisfy the first set condition; Based on the first brightness of each second target pixel, the brightness-updated center pixel is subjected to bright area dilation processing to obtain the brightness-updated center pixel; From the pixels corresponding to the Nth detection direction obtained from the pixel block, determine all Nth target pixels that satisfy the first set condition; Based on the first brightness of each of the Nth target pixels, the center pixel after the N-1 brightness updates is subjected to bright area dilation processing to obtain the center pixel after the Nth update; The process continues until the center pixel is dilated based on all detection directions, resulting in the second brightness of each pixel.

3. The method according to claim 2, characterized in that, The step of determining all first target pixels that satisfy the first preset condition from the pixels corresponding to the first detection direction obtained for each pixel block includes: For each pixel block, when the first detection direction is upward, obtain the first set of pixels above the center pixel and the second set of pixels below the center pixel; Determine the first minimum and first maximum values ​​of the first brightness for each pixel in the first pixel set; Determine the second minimum and second maximum values ​​of the first brightness for each pixel in the second pixel set; When the first minimum value is greater than the second maximum value, each pixel in the first pixel set is determined as all first target pixels that satisfy the first set condition; When the second minimum value is greater than the first maximum value, each pixel in the second pixel set is determined as all first target pixels that satisfy the first set condition.

4. The method according to claim 1, characterized in that, The step of determining the gradient of each pixel based on the second brightness of each pixel includes: Determine the first convolution kernel in the horizontal direction and the second convolution kernel in the vertical direction of the YUV image; Calculate the first square of the second brightness of each pixel and the first product of the first convolution kernel; Calculate the second square of the second product of the second brightness of each pixel and the second product of the second convolution kernel; The square root of the sum of the first square and the second square is determined as the gradient of each pixel.

5. The method according to claim 1, characterized in that, The step of sequentially determining whether there are gradient pixels satisfying the second preset condition among the pixels corresponding to each detection direction includes: For each detection direction, determine the set of third pixels and the set of fourth pixels in each pixel corresponding to that detection direction; Determine the third minimum and third maximum values ​​of the gradient of each pixel in the third pixel set; Determine the fourth minimum and fourth maximum values ​​of the gradient of each pixel in the fourth pixel set; When the third minimum value is greater than the fourth maximum value, it is determined that there are gradient pixels that satisfy the second set condition among the pixels corresponding to the detection direction; When the fourth minimum value is greater than the third maximum value, it is determined that there are gradient pixels that satisfy the second set condition among the pixels corresponding to the detection direction.

6. An image processing apparatus, characterized in that, The device includes: The conversion module is used to convert the RGB image to be processed into a YUV image; The first acquisition module is used to acquire the first brightness of each pixel in the Y channel of the YUV image, wherein the YUV image is composed of multiple pixels, and each pixel corresponds to a first brightness. The second acquisition module is used to acquire pixels within a preset range as center pixels to obtain pixel blocks; The first processing module is used to perform bright area dilation processing on the central pixel in each pixel block based on the first brightness of each pixel in each pixel block to obtain the second brightness of each pixel. The first determining module is used to determine the gradient of each pixel based on the second brightness of each pixel; The second processing module is used to, for each pixel block, obtain each pixel corresponding to different detection directions from the pixel block; sequentially determine whether there are gradient pixels that satisfy a second set condition among the pixels corresponding to each detection direction, wherein the different detection directions correspond in an order; when there are gradient pixels that satisfy the second set condition among the pixels corresponding to any detection direction, determine the gradient pixels that satisfy the second set condition among the pixels corresponding to the target detection direction; and perform boundary sharpening processing on the center pixel based on the gradient of the gradient pixels to obtain the target brightness of each pixel; The second determining module is used to calculate the target RGB image by combining the target brightness of the Y channel with the UV channels of the RGB image to be processed.

7. An electronic device, characterized in that, It includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the steps of the method according to any one of claims 1-5.

8. A storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the computer program implements the steps of the method described in any one of claims 1-5.

Images