Image processing method and device, storage medium and electronic equipment
By acquiring the brightness information and high-frequency characteristics of the image and performing sharpening processing, the problem of low image enhancement accuracy in the prior art is solved, and higher image processing accuracy and quality are achieved.
Patent Information
- Application Number
- CN202310713999.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2023-06-15
- Publication Date
- 2024-12-17
AI Technical Summary
The image enhancement technology in the prior art has low processing accuracy, resulting in poor quality of the obtained image.
By obtaining the brightness information of the image to be processed, the airspace characteristics and initial high-frequency characteristics of the target pixel point are determined, and the sharpening process is performed based on these characteristics to obtain the target high-frequency characteristics, thereby improving the accuracy and quality of the image processing.
The accuracy and efficiency of image processing are improved, and the target image quality is better, more details are more, the calculation amount is reduced, and the processor burden is reduced.
Smart Images
Figure CN119151812A_ABST
Abstract
Description
Background Art
[0002] With the rapid development of display technology, users' requirements for image quality are becoming higher and higher, and the application of image enhancement technology is becoming more and more extensive.
[0003] However, the image enhancement technology in the related art has low processing accuracy and the quality of the obtained image is poor. Summary of the invention
[0004] The purpose of the present disclosure is to provide an image processing method, an image processing device, a computer-readable medium and an electronic device, thereby improving the accuracy of image processing at least to a certain extent.
[0005] According to a first aspect of the present disclosure, there is provided an image processing method, comprising: obtaining brightness information of all pixel points of an image to be processed, and determining a plurality of target pixel points among the pixel points according to the brightness information; obtaining spatial domain features of the target pixel points and initial high-frequency features of the target pixel points in at least one dimension based on the brightness information; determining intermediate high-frequency features according to the spatial domain features and the initial high-frequency features; sharpening the intermediate high-frequency features based on the intermediate high-frequency features and preset sharpening rules to obtain target high-frequency features; and obtaining a target image based on the target high-frequency features and the image to be processed.
[0006] According to a second aspect of the present disclosure, there is provided an image processing device, comprising: a brightness acquisition module, used to acquire brightness information of all pixels of an image to be processed, and determine a plurality of target pixels among the pixels according to the brightness information; a feature extraction module, used to acquire spatial features of the target pixels and initial high-frequency features of the target pixels in at least one dimension based on the brightness information; a feature determination module, used to determine intermediate high-frequency features according to the spatial features and the initial high-frequency features; an image sharpening module, used to sharpen the intermediate high-frequency features based on the intermediate high-frequency features and preset sharpening rules to obtain target high-frequency features; and an image processing module, used to obtain a target image based on the target high-frequency features and the image to be processed.
[0007] According to a third aspect of the present disclosure, a computer-readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the above method is implemented.
[0008] According to a fourth aspect of the present disclosure, an electronic device is provided, characterized in that it includes: one or more processors; and a memory for storing one or more programs, which enables the one or more processors to implement the above method when the one or more programs are executed by the one or more processors.
[0009] An image processing method provided by an embodiment of the present disclosure, on the one hand, obtains the spatial domain features of the target pixel points and the initial high-frequency features of the target pixel points in at least one dimension based on brightness information, can obtain high-frequency features in multiple dimensions to obtain the target image, can improve the details of the target image, and has higher processing accuracy when performing image enhancement on the image to be processed. On the other hand, the intermediate high-frequency features are determined based on the spatial domain features and the initial high-frequency features; the intermediate high-frequency features are sharpened based on the intermediate high-frequency features and preset sharpening rules to obtain the target high-frequency features, and the image processing is performed by fusing the spatial domain features and the high-frequency features, which further improves the processing accuracy and makes the quality of the obtained target image better. On the other hand, obtaining the target pixel points among the pixel points for image processing can reduce the overall amount of calculation, and can filter out the pixel points that do not need to be processed, thereby reducing the amount of calculation of the processor and improving the efficiency of image processing.
[0010] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS
[0011] The accompanying drawings herein are incorporated into the specification and constitute a part of the specification, illustrate embodiments consistent with the present disclosure, and together with the specification are used to explain the principles of the present disclosure. Obviously, the accompanying drawings described below are only some embodiments of the present disclosure, and for ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without creative work.
[0012] Figure 1 A structural framework diagram of an image processing in an exemplary embodiment of the present disclosure is schematically shown.
[0013] Figure 2 A schematic diagram showing an exemplary system architecture to which the embodiments of the present disclosure can be applied is shown.
[0014] Figure 3 The flowchart schematically shows an image processing method in an exemplary embodiment of the present disclosure.
[0015] Figure 4 A comparison diagram schematically illustrates a gamma conversion in an exemplary embodiment of the present disclosure.
[0016] Figure 5 A schematic diagram showing an edge detection in an exemplary embodiment of the present disclosure.
[0017] Figure 6 A schematic diagram showing a method of obtaining initial high-frequency features in an exemplary embodiment of the present disclosure.
[0018] Figure 7A schematic diagram of obtaining intermediate high-frequency features in an exemplary embodiment of the present disclosure is shown.
[0019] Figure 8 A schematic diagram showing a sharpening process in an exemplary embodiment of the present disclosure.
[0020] Fig. 9 A schematic diagram schematically illustrates an updating target high-frequency feature in an exemplary embodiment of the present disclosure.
[0021] Fig.10 A data flow diagram of an image processing method in an exemplary embodiment of the present disclosure is schematically shown.
[0022] Fig.11 The following is a schematic diagram schematically showing the composition of an image processing device in an exemplary embodiment of the present disclosure.
[0023] Fig.12 A schematic diagram of an electronic device to which the embodiments of the present disclosure can be applied is shown. DETAILED DESCRIPTION
[0024] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be implemented in a variety of forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that the disclosure will be more comprehensive and complete and to fully convey the concepts of the example embodiments to those skilled in the art. The described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments.
[0025] In addition, the accompanying drawings are only schematic illustrations of the present disclosure and are not necessarily drawn to scale. The same reference numerals in the figures represent the same or similar parts, and their repeated description will be omitted. Some of the block diagrams shown in the accompanying drawings are functional entities and do not necessarily correspond to physically or logically independent entities. These functional entities can be implemented in software form, or implemented in one or more hardware modules or integrated circuits, or implemented in different networks and / or processor devices and / or microcontroller devices.
[0026] In an example implementation, after the image signal is converted from analog to digital, the photosensitive chip outputs a Bayer signal, which is converted into an RGB signal through demosaicing, and finally converted into a YUV signal through RGB. Because the signal is lost and noise is introduced during the transmission process, it is necessary to perform denoising and sharpening in the Y domain to enhance the image signal. Figure 1As shown, specifically, the signal can be divided into three paths first. The first path is sent to 5x5 sharp_noise_filter for limiting processing; the second path is to limit the maximum and minimum values of the signal change after all processing is completed, that is, to improve the common black and white edges; the third path signal is subjected to normal denoising and sharpening signal processing.
[0027] When processing the third signal, it can first be denoised through a 5X5 spatial filter. Before sharpening, denoising can be performed to avoid further amplification of noise during sharpening. Then, it can be filtered through a 5X5 high-pass FIR filter to obtain the high-frequency part. After sharpening, it is added to the processed signal to obtain the third target signal, and then the signal size limitation module and 5x5sharp_noise_filter are used to limit the processing to obtain the final target signal.
[0028] In the above technical solution, the denoising module is placed in the front position. If the denoising is too strong at this time, it will be not conducive to the extraction of the high-frequency part of the subsequent sharpening module. The high-frequency signals of different frequencies are not divided, resulting in uneven extraction of the high-frequency part. The brightness of different scenes and the brightness of each pixel in a single frame cannot be adaptively processed. If the same sharpening processing is performed on dark and bright pixels, there will be different noises.
[0029] Based on the above shortcomings, the present invention provides an image processing method. Figure 2 A schematic diagram of a system architecture that can implement the above-mentioned image processing method is shown, and the system architecture 200 may include a terminal 210 and a server 220. The terminal 210 may be a terminal device such as a smart phone, a tablet computer, a desktop computer, a laptop computer, etc. The server 220 generally refers to a background system that provides image processing related services in this exemplary embodiment, and may be a server or a cluster formed by multiple servers. The terminal 210 and the server 220 may be connected via a wired or wireless communication link to perform data exchange.
[0030] In one embodiment, the above-mentioned image processing method can be performed by the terminal 210. For example, a user uses the terminal 210 to obtain an image to be processed, and the terminal 210 first obtains the brightness information of all pixels of the image to be processed, and determines multiple target pixels in the pixels according to the brightness information; then obtains the spatial domain features of the target pixels and the initial high-frequency features of the target pixels in at least one dimension based on the brightness information; then determines the intermediate high-frequency features according to the spatial domain features and the initial high-frequency features; then, sharpens the intermediate high-frequency features based on the intermediate high-frequency features and preset sharpening rules to obtain the target high-frequency features; finally, obtains the target image based on the target high-frequency features and the image to be processed.
[0031] In one embodiment, the image processing method described above may be performed by the server 220. For example, after the user uses the terminal 210 to obtain the image to be processed, the terminal 110 uploads the image to be processed to the server 220, and the server 220 processes the image to be processed. Specifically, the server may first obtain the brightness information of all pixels of the image to be processed, and determine multiple target pixels in the pixels according to the brightness information; then obtain the spatial domain features of the target pixels and the initial high-frequency features of the target pixels in at least one dimension based on the brightness information; then determine the intermediate high-frequency features based on the spatial domain features and the initial high-frequency features; then sharpen the intermediate high-frequency features based on the intermediate high-frequency features and the preset sharpening rules to obtain the target high-frequency features; finally obtain the target image based on the target high-frequency features and the image to be processed, and return the target image to the terminal 210.
[0032] As can be seen from the above, the execution subject of the image processing method in this exemplary embodiment may be the terminal 210 or the server 220, and the present disclosure does not limit this.
[0033] Combine the following Figure 3 The image processing method in this exemplary embodiment is described. Figure 3 An exemplary process of the image processing method is shown, which may include steps S310 to S350.
[0034] In step S310, the brightness information of all pixels of the image to be processed is obtained, and a plurality of target pixels are determined among the pixels according to the brightness information.
[0035] In an example implementation of the present disclosure, an image to be processed is obtained. The image to be processed may be an image in YUV format. In this case, obtaining the brightness information of all pixels of the image to be processed may be obtaining the value of the Y domain of each pixel.
[0036] It should be noted that the above-mentioned image to be processed can also be in RGB format. In this case, when obtaining the brightness information of all pixels, the RGB image can be first converted into a YUV image. At this time, obtaining the brightness information of all pixels of the image to be processed can be obtaining the value of the Y domain of each pixel.
[0037] In an example implementation of the present disclosure, after obtaining the above-mentioned brightness information, the above-mentioned image to be processed can be gamma converted to update the brightness information. Since the sensitivity of the human eye to the external light source is not linearly related to the input light intensity, but exponentially related, and the image sensor output is a linear transformation, therefore, performing gamma conversion can make the obtained image more suitable for user viewing.
[0038] Specifically, refer to Figure 4As shown, the gamma conversion can be performed by the following formula:
[0039] B=255A X
[0040] Among them, B is the brightness information after conversion, A is the brightness information before conversion, and X is the gamma value.
[0041] For example, Figure 4 S1 is the brightness curve when the gamma value is 1 / 2.2, and S2 is the brightness curve when the gamma value is 2.2. S1 corresponds to the overall comparison of the picture, but the image contrast is poor, and S2 corresponds to a higher picture contrast. Users can choose the appropriate gamma value according to their needs, and no specific limitation is made in this example implementation.
[0042] In this exemplary embodiment, after the brightness information of all the above pixels is obtained, multiple target pixels can be determined from the above pixels according to the brightness information. Figure 5 As shown, edge detection can be performed on each of the above pixel points to obtain the edge value of each pixel point.
[0043] When performing edge detection on a pixel point, you can first obtain an N*N pixel matrix with the pixel point as the center, where N is an odd number. The specific value can be 5, 7, etc., and can also be customized according to user needs. There is no specific limitation in this example implementation.
[0044] After obtaining the N*N pixel matrix, the brightness value of each pixel in the pixel matrix can be obtained, and then the edge value of the above pixel is calculated based on the brightness value of the pixel. Specifically, Figure 5 As shown, taking N equal to 5 as an example, the average of the difference between the brightness value of the first row of pixels and the brightness value of the fourth row of pixels is calculated, and the average of the difference between the brightness value of the second row of pixels and the brightness value of the fifth row of pixels is calculated, and then the average of the difference between the brightness value of the first column of pixels and the brightness value of the fourth column of pixels is calculated, and the average of the difference between the brightness value of the second column of pixels and the brightness value of the fifth column of pixels is calculated. Then the maximum value of the above averages is taken as the edge value of the pixel.
[0045] In this example implementation, an edge threshold may be set first, which may be customized according to user needs and is not specifically limited in this example implementation. Then, pixels with edge values greater than the edge threshold may be used as the target pixels.
[0046] In an exemplary embodiment of the present disclosure, if the pixel point is at the edge of the image to be processed, such as the upper left corner, the upper right corner, etc., when obtaining the pixel matrix, the pixel matrix can be obtained by symmetrical padding. Pixel points that cannot directly obtain the pixel matrix in the image to be processed can also be ignored, and no specific limitation is made in this exemplary embodiment.
[0047] Obtaining target pixels from the pixels for image processing can reduce the overall amount of calculation and filter out pixels that do not need to be processed, thereby reducing the amount of calculation for the processor and improving the efficiency of image processing.
[0048] In step S320, the spatial domain features of the target pixel point and the initial high-frequency features of the target pixel point in at least one dimension are acquired based on the brightness information.
[0049] In this example implementation, after obtaining the target pixel point, the spatial domain features of the target pixel point and the initial high-frequency features of the target pixel point in at least one dimension may be acquired based on the brightness information.
[0050] Specifically, when obtaining spatial domain features, the first area of a first preset size centered on the target pixel point can be first determined; then the average brightness of the brightness information of the pixels other than the target pixel point in the first area is obtained; finally, the spatial domain features of the target pixel point are determined based on the brightness information and the average brightness of the target pixel point.
[0051] When obtaining the initial high-frequency features of at least one dimension, at least one target area may be first obtained with the target pixel point as the center. The target area may be an M*M pixel matrix, where M may be an odd number greater than or equal to 3. For example, 3, 7, 13, etc. The value of M may also be customized according to user needs. When there are multiple target areas, the dimensions of the multiple target areas are different.
[0052] When acquiring at least one target area, at least one convolution kernel for high-level feature extraction can be first determined, and the size of the convolution kernel can be determined. The value of M of the target area can be determined based on the size of the convolution kernel. For example, if the target area is a 3*3 pixel matrix, the size of the corresponding convolution kernel can also be 3*3.
[0053] Specifically, the processor can be configured with multiple convolution kernels for user high-frequency feature extraction, and one or more can be selected according to user needs to complete the high-frequency feature extraction of the target pixel. That is to say, the value of M in the target area can depend on the size of the convolution sum of the high-frequency feature extraction configured in the processor.
[0054] When extracting high-frequency features of the above target pixels, you can refer to Figure 6As shown, the above initial high-frequency features are multiplied by the above M*M matrix and M*M convolution kernel.
[0055] In step S330, intermediate high-frequency features are determined according to the spatial features and the initial high-frequency features.
[0056] After obtaining the above-mentioned spatial domain features and the above-mentioned initial high-frequency features, the intermediate high-frequency features can be calculated according to the above-mentioned initial high-frequency features and the spatial domain features.
[0057] Specifically, if the number of the above-mentioned initial high-frequency features is 1, the first weighted average of the above-mentioned initial high-frequency features and spatial domain features is directly calculated, and the weighted average of the initial high-frequency features and the spatial domain features is used as the intermediate high-frequency feature. The weight information corresponding to the initial high-frequency features and the spatial domain features can be customized according to user needs, and is not specifically limited in this example implementation.
[0058] If the number of the above initial high-frequency features is multiple, refer to Figure 7 As shown, the second weighted average of the multiple initial high-frequency features can be determined first, and then the second weighted average and the third weighted average of the spatial domain features can be determined, and the third weighted average can be used as the intermediate high-frequency feature.
[0059] For example, the above initial high-frequency information includes three, namely, the first initial high-frequency information, the second initial high-frequency information and the third initial high-frequency information, and the corresponding weights are ω 1 ,ω 2 and ω 3 , where ω 1 ,ω 2 and ω 3 The sum of is 1, and the specific value can be customized according to user needs.
[0060] After obtaining the second weighted average value, the second weighted average value and the third weighted average value of the spatial feature can be obtained. For example, the weight information corresponding to the second weighted average value and the spatial feature is ω 4 and ω 5 , where ω 4 and ω 5 The sum of is 1, and the specific value can be customized according to user needs.
[0061] In step S340, the intermediate high-frequency features are sharpened based on the intermediate high-frequency features and a preset sharpening rule to obtain target high-frequency features.
[0062] After obtaining the above intermediate high-frequency features, the intermediate high-frequency features can be sharpened using a preset sharpening rule to obtain a target high-frequency feature. Specifically, the intermediate high-frequency features can be preprocessed first so that the value of the intermediate high-frequency features is within a target range, the target range including a first sub-range, a second sub-range, a third sub-range, and a fourth sub-range connected in the first place, wherein the value in the fourth sub-range is greater than the value in the first sub-range.
[0063] For example, the value of the above intermediate high-frequency feature can be between -128 and 128. After preprocessing, the value of the above intermediate high-frequency feature can be between 0 and 255, that is, 128 corresponds to flat area pixels. The farther away from 128, that is, the higher the frequency, the stronger the sharpening intensity needs to be.
[0064] In an exemplary implementation, the maximum value of the second sub-range is 128, the minimum value of the third sub-range is greater than 128, and the first sub-range, the second sub-range, the third sub-range and the fourth sub-range are connected end to end.
[0065] For example, refer to Figure 8 As shown, the first sub-range may be 0-96, the second sub-range may be 96-128, the third sub-range may be 128-160, and the fourth sub-range may be 160-255. The specific range division may be customized according to user needs and is not specifically limited in this example implementation.
[0066] In which, in response to the intermediate high frequency feature in the first sub-range, the intermediate high frequency feature is sharpened using the first sharpening value to obtain the target high frequency feature; in response to the intermediate high frequency feature in the second sub-range, the second sharpening value is determined using the first mapping relationship and the intermediate high frequency feature, and the intermediate high frequency feature is sharpened using the second sharpening value to obtain the target high frequency feature, wherein in the first mapping relationship, the intermediate high frequency feature and the second sharpening value are negatively correlated; in response to the intermediate high frequency feature in the third sub-range, the third sharpening value is determined using the second mapping relationship and the intermediate high frequency feature, and the intermediate high frequency feature is sharpened using the third sharpening value to obtain the target high frequency feature; wherein in the second mapping relationship, the intermediate high frequency feature and the third sharpening value are positively correlated; in response to the intermediate high frequency feature in the fourth sub-range, the intermediate high frequency feature is sharpened using the first sharpening value to obtain the target high frequency feature. In which, the first sharpening value and the fourth sharpening value can be 25, that is, the maximum value of the sharpening value, the above-mentioned first mapping relationship and the second mapping relationship are both linear relationships, and the specific slope can be determined according to the division of the sub-range and the maximum value of the sharpening value, which is not specifically limited in this example implementation.
[0067] In an exemplary embodiment of the present disclosure, referring to Fig. 9As shown, the above-mentioned image processing method may also include determining the sensitivity of the image to be processed, and the distance information between the target pixel and the center point of the image to be processed, and adaptively gaining the target high-frequency features according to the sensitivity of the image to be processed, the brightness value of the target pixel and the distance information to update the target high-frequency features.
[0068] Specifically, the first gain coefficient is determined according to the sensitivity of the image to be processed, and the first gain coefficient is negatively correlated with the sensitivity; for example, if the maximum sensitivity is 6400 and the minimum sensitivity is 100, the corresponding first gain coefficient may be 0-128, that is, the first gain coefficient corresponding to 6400 is 0, and the first gain coefficient corresponding to the sensitivity of 100 is 128, wherein other data may be calculated based on the linear relationship determined by the correspondence between the above two sensitivities and the first gain coefficient.
[0069] The second gain coefficient is determined according to the brightness value of the target pixel point, and the second gain coefficient is positively correlated with the brightness value of the target pixel point; wherein the positive correlation may be a linear correlation, that is, the larger the brightness value, the larger the second gain coefficient.
[0070] The third gain coefficient is determined according to the distance information, and the third gain coefficient is negatively correlated with the distance information; wherein the negative correlation may be a linear correlation, that is, the closer the distance to the center point is, the greater the third gain coefficient is.
[0071] In this example implementation, after obtaining the first gain coefficient, the second gain coefficient and the third gain coefficient, the first gain coefficient, the second gain coefficient and the third gain coefficient may be used to perform adaptive gain on the target high frequency feature to update the target high frequency feature.
[0072] In step S350, a target image is obtained based on the target high-frequency features and the image to be processed.
[0073] In this example implementation, after obtaining the target high-frequency features, the target image may be determined using the image to be processed and the target high-frequency features.
[0074] Specifically, the target high-frequency features of all the target pixels are superimposed on the image to be processed to obtain an upper intermediate image. The specific process of obtaining the above intermediate image can refer to the following formula:
[0075] Y new =Y cur +FIR_OUTPUT*coring*A*B*C
[0076] Among them, Y new , is the brightness value of the target pixel in the target image, Y curis the brightness value of the target pixel in the image to be processed, FIR_OUTPUT is the intermediate high-frequency feature, coring is the sharpening value, A is the first gain coefficient, B is the second gain coefficient, and C is the third gain coefficient.
[0077] After obtaining the intermediate image, the target image can be obtained by limiting the maximum value of the pixels of the intermediate image. Specifically, the brightness value of the intermediate image is limited according to the set maximum and minimum values to prevent the appearance of black and white edges and enhance the quality of the obtained image. For example, the brightness value of the final target image is limited to less than 255.
[0078] In an exemplary embodiment of the present disclosure, referring to Fig.10 As shown, the above method may also include performing low-pass filtering on the obtained target image to update the target image, which can reduce the noise generated by sharpening, improve the progress of the target image, and enhance the accuracy of image processing.
[0079] In summary, in this exemplary embodiment, on the one hand, the spatial domain features of the target pixel points and the initial high-frequency features of the target pixel points in at least one dimension are obtained based on the brightness information, and the high-frequency features can be obtained in multiple dimensions to obtain the target image, which can improve the details of the target image. When the image is enhanced for the processed image, the processing accuracy is higher. On the other hand, the intermediate high-frequency features are determined according to the spatial domain features and the initial high-frequency features; the intermediate high-frequency features are sharpened based on the intermediate high-frequency features and the preset sharpening rules to obtain the target high-frequency features, and the spatial domain features and the high-frequency features are fused to perform image processing, which further improves the processing accuracy and makes the quality of the obtained target image better. On the other hand, the target pixel points are obtained from the pixel points for image processing, which can reduce the overall calculation amount, and the pixels that do not need to be processed can be screened out, which reduces the calculation amount of the processor and improves the efficiency of image processing. On the other hand, the initial high-frequency features of multiple dimensions are used to enhance the image to be processed, which can make the obtained target image have more details. On the other hand, the obtained target image is low-pass filtered to update the target image, which can reduce the noise generated by sharpening, improve the progress of the target image, and enhance the accuracy of image processing.
[0080] It should be noted that the above figures are only schematic illustrations of the processes included in the method according to the exemplary embodiments of the present disclosure, and are not intended to be limiting. It is easy to understand that the processes shown in the above figures do not indicate or limit the time sequence of these processes. In addition, it is also easy to understand that these processes can be performed synchronously or asynchronously, for example, in multiple modules.
[0081] For further reference, Fig.11As shown, in the embodiment of this example, an image processing device 1100 is also provided, including a brightness acquisition module 1110, a feature extraction module 1120, a feature determination module 1130, an image sharpening module 1140 and an image processing module 1150. Among them:
[0082] The brightness acquisition module 1110 can be used to obtain the brightness information of all pixels of the image to be processed, and determine multiple target pixels among the pixels based on the brightness information; the feature extraction module 1120 can be used to obtain the spatial characteristics of the target pixels and the initial high-frequency characteristics of the target pixels in at least one dimension based on the brightness information; the feature determination module 1130 can be used to determine the intermediate high-frequency characteristics based on the spatial characteristics and the initial high-frequency characteristics; the image sharpening module 1140 can be used to sharpen the intermediate high-frequency characteristics based on the intermediate high-frequency characteristics and preset sharpening rules to obtain the target high-frequency characteristics; the image processing module 1150 can be used to obtain the target image based on the target high-frequency characteristics and the image to be processed.
[0083] In an example implementation, the brightness acquisition module 1110 may be configured to perform edge detection on each pixel based on brightness information to obtain an edge value of each pixel; and use a pixel whose edge value is greater than an edge threshold as a target pixel.
[0084] In an example implementation, the brightness acquisition module 1110 may be configured to perform a gamma conversion on the image to be processed to update the brightness information.
[0085] In an example implementation, the feature extraction module 1120 may be configured to determine a first region of a first preset size with the target pixel as the center; obtain the average brightness of the brightness information of the pixels other than the target pixel in the first region; determine the spatial features of the target pixel based on the brightness information and the average brightness of the target pixel. Obtain at least one target region with the target pixel as the center; extract high-frequency features from the target region to obtain at least one target region; wherein, when there are multiple target regions, the sizes of the multiple target regions are different.
[0086] In this example implementation, when acquiring at least one target area with the target pixel point as the center, at least one convolution kernel for high-frequency feature extraction can be first determined, and the size of the convolution kernel can be determined; based on the convolution kernel size, at least one target area is acquired with the target pixel point as the center.
[0087] In an example implementation, the feature determination module 1130 may be configured to, in response to an initial high-frequency feature being one, determine a first weighted average of the initial high-frequency feature and the spatial domain feature, and use the weighted average of the initial high-frequency feature and the spatial domain feature as an intermediate high-frequency feature; in response to multiple initial high-frequency features being multiple, determine a second weighted average of the multiple initial high-frequency features; determine a third weighted average of the second weighted average and the spatial domain feature, and use the third weighted average as the intermediate high-frequency feature.
[0088] In an example implementation, the image sharpening module 1140 may be configured to preprocess the intermediate high-frequency feature so that the value of the intermediate high-frequency feature is within a target range, the target range including a first sub-range, a second sub-range, a third sub-range, and a fourth sub-range connected in the first place, wherein the value within the fourth sub-range is greater than the value within the first sub-range; in response to the intermediate high-frequency feature being in the first sub-range, the intermediate high-frequency feature is sharpened using a first sharpening value to obtain a target high-frequency feature; in response to the intermediate high-frequency feature being in the second sub-range, a second sharpening value is determined using the first mapping relationship and the intermediate high-frequency feature, and the intermediate high-frequency feature is sharpened using the second sharpening value The target high-frequency feature is obtained by sharpening, wherein, in the first mapping relationship, the intermediate high-frequency feature and the second sharpening value are negatively correlated; in response to the intermediate high-frequency feature being in the third sub-range, the third sharpening value is determined using the second mapping relationship and the intermediate high-frequency feature, and the intermediate high-frequency feature is sharpened using the third sharpening value to obtain the target high-frequency feature; wherein, in the second mapping relationship, the intermediate high-frequency feature and the third sharpening value are positively correlated; in response to the intermediate high-frequency feature being in the fourth sub-range, the intermediate high-frequency feature is sharpened using the first sharpening value to obtain the target high-frequency feature; wherein the first sharpening value is the maximum value of the sharpening value, and the maximum value of the second sub-range is the median of the target range.
[0089] In an example implementation, the image processing device may further include an image gain module, which may be configured to determine the sensitivity of the image to be processed, and the distance information between the target pixel and the center point of the image to be processed; adaptively gain the target high-frequency features according to the sensitivity of the image to be processed, the brightness value of the target pixel, and the distance information to update the target high-frequency features. Specifically, a first gain coefficient is determined according to the sensitivity of the image to be processed, and the first gain coefficient is negatively correlated with the sensitivity; a second gain coefficient is determined according to the brightness value of the target pixel, and the second gain coefficient is positively correlated with the brightness value of the target pixel; a third gain coefficient is determined according to the distance information, and the third gain coefficient is negatively correlated with the distance information; and the first gain coefficient, the second gain coefficient, and the third gain coefficient are used to adaptively gain the target high-frequency features to update the target high-frequency features.
[0090] In an example implementation, the image processing module 1150 may be configured to superimpose the target high-frequency features and the image to be processed to obtain an intermediate image; and to perform pixel maximum value limitation on the intermediate image to obtain the target image.
[0091] In an exemplary implementation, the image processing apparatus may further include an image filtering module, and the image filtering module may be configured to perform low-pass filtering on the target image to update the target image.
[0092] The specific details of each module in the above device have been described in detail in the implementation method of the method part. The undisclosed details can be found in the implementation method of the method part, so they will not be repeated here.
[0093] The exemplary embodiment of the present disclosure also provides an electronic device for executing the above image processing method, and the electronic device may be the above terminal 210 or the server 220. Generally, the electronic device may include a processor and a memory, the memory is used to store executable instructions of the processor, and the processor is configured to execute the above image processing method by executing the executable instructions.
[0094] Below Figure 2 The mobile terminal 1200 in FIG. 1 is taken as an example to exemplify the structure of the electronic device. It should be understood by those skilled in the art that, in addition to the components specifically used for mobile purposes, Fig.12 The construction in can also be applied to fixed type equipment.
[0095] like Fig.12 As shown, the mobile terminal 1200 may specifically include: a processor 1201, a memory 1202, a bus 1203, a mobile communication module 1204, an antenna 1, a wireless communication module 1205, an antenna 2, a display screen 1206, a camera module 1207, an audio module 1208, a power module 1209 and a sensor module 1210.
[0096] The processor 1201 may include one or more processing units, for example, the processor 1201 may include an AP (Application Processor), a modem processor, a GPU (Graphics Processing Unit), an ISP (Image Signal Processor), a controller, an encoder, a decoder, a DSP (Digital Signal Processor), a baseband processor and / or an NPU (Neural-Network Processing Unit), etc. The image processing method in this exemplary embodiment may be executed by an AP, a GPU or a DSP, and when the method involves processing related to a neural network, it may be executed by an NPU.
[0097] The processor 1201 may be connected to the memory 1202 or other components via a bus 1203 .
[0098] The memory 1202 may be used to store computer executable program codes, which include instructions. The processor 1201 executes various functional applications and data processing of the mobile terminal 1200 by running the instructions stored in the memory 1202. The memory 1202 may also store application data, such as images, videos and other files.
[0099] The communication function of the mobile terminal 1200 can be implemented by the mobile communication module 1204, antenna 1, wireless communication module 1205, antenna 2, a modulation and demodulation processor, and a baseband processor. Antenna 1 and antenna 2 are used to transmit and receive electromagnetic wave signals. The mobile communication module 1204 can provide 2G, 3G, 4G, 5G and other mobile communication solutions applied to the mobile terminal 1200. The wireless communication module 1205 can provide wireless communication solutions such as wireless LAN, Bluetooth, and near field communication applied to the mobile terminal 1200.
[0100] The display screen 1206 is used to implement display functions, such as displaying user interfaces, images, videos, etc. The camera module 1207 is used to implement shooting functions, such as shooting images, videos, etc. The audio module 1208 is used to implement audio functions, such as playing audio, collecting voice, etc. The power module 1209 is used to implement power management functions, such as charging the battery, powering the device, monitoring the battery status, etc. The sensor module 1210 may include a depth sensor 12101, a pressure sensor 12102, a gyroscope sensor 12103, an air pressure sensor 12104, etc., to implement corresponding sensing detection functions.
[0101] Those skilled in the art will appreciate that various aspects of the present disclosure may be implemented as systems, methods or program products. Therefore, various aspects of the present disclosure may be specifically implemented in the following forms, namely: complete hardware implementation, complete software implementation (including firmware, microcode, etc.), or a combination of hardware and software, which may be collectively referred to herein as "circuits", "modules" or "systems".
[0102] The exemplary embodiments of the present disclosure also provide a computer-readable storage medium on which a program product capable of implementing the above-mentioned method of the present specification is stored. In some possible implementations, various aspects of the present disclosure may also be implemented in the form of a program product, which includes a program code, and when the program product is run on a terminal device, the program code is used to enable the terminal device to execute the steps according to various exemplary embodiments of the present disclosure described in the above-mentioned "Exemplary Method" section of the present specification.
[0103] It should be noted that the computer-readable medium shown in the present disclosure may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.
[0104] In the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that may be used by or in combination with an instruction execution system, apparatus, or device. In the present disclosure, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries a computer-readable program code. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium, which may send, propagate, or transmit a program for use by or in combination with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium may be transmitted using any suitable medium, including but not limited to: wireless, wire, optical cable, RF, etc., or any suitable combination of the above.
[0105] In addition, program code for performing the operations of the present disclosure may be written in any combination of one or more programming languages, including object-oriented programming languages such as Java, C++, etc., and conventional procedural programming languages such as "C" or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user device, as a separate software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server. In cases involving a remote computing device, the remote computing device may be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computing device (e.g., through the Internet using an Internet service provider).
[0106] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary technical means in the art that are not disclosed in the present disclosure. The specification and examples are to be considered as exemplary only, and the true scope and spirit of the present disclosure are indicated by the claims.
[0107] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.
Claims
1. An image processing method, It is characterized in that include: Acquire brightness information of all pixels of the image to be processed, and determine a plurality of target pixels among the pixels according to the brightness information; Acquire the spatial domain features of the target pixel point and the initial high-frequency features of the target pixel point in at least one dimension based on the brightness information; Determine an intermediate high-frequency feature according to the spatial domain feature and the initial high-frequency feature; Based on the intermediate high-frequency features and a preset sharpening rule, the intermediate high-frequency features are sharpened to obtain target high-frequency features; A target image is obtained based on the target high-frequency features and the image to be processed.
2. The method according to claim 1, It is characterized in that Before determining a plurality of target pixel points among the pixel points according to the brightness information, the method further includes: Gamma conversion is performed on the image to be processed to update the brightness information.
3. The method according to claim 1, It is characterized in that Determining a plurality of target pixel points among the pixel points according to the brightness information comprises: Performing edge detection on each of the pixels based on the brightness information to obtain an edge value of each of the pixels; The pixel point whose edge value is greater than the edge threshold is taken as the target pixel point.
4. The method according to claim 1, It is characterized in that The acquiring the spatial domain feature of the target pixel point based on the brightness information comprises: Determine a first area of a first preset size with the target pixel as the center; Obtaining an average brightness of brightness information of pixel points other than the target pixel point in the first area; The spatial domain feature of the target pixel is determined based on the brightness information of the target pixel and the average brightness.
5. The method according to claim 1, It is characterized in that The step of obtaining the initial high-frequency feature of at least one dimension of the target pixel point based on the brightness information includes: Acquire at least one target area with the target pixel point as the center; Extracting high-frequency features from the target area to obtain at least one target area; When there are multiple target areas, the sizes of the multiple target areas are different.
6. The method according to claim 5, It is characterized in that Acquiring at least one target area with the target pixel point as the center comprises: Determine at least one convolution kernel for high-frequency feature extraction, and determine the size of the convolution kernel; According to the convolution kernel size, at least one target area is acquired with the target pixel point as the center.
7. The method according to claim 1, It is characterized in that The determining of the intermediate high-frequency feature according to the spatial domain feature and the initial high-frequency feature comprises: In response to the initial high-frequency feature being one, determining a first weighted average of the initial high-frequency feature and the spatial domain feature, and taking the weighted average of the initial high-frequency feature and the spatial domain feature as the intermediate high-frequency feature; In response to the initial high-frequency feature being multiple, determining a second weighted average of the multiple initial high-frequency features; A third weighted average value of the second weighted average value and the spatial domain feature is determined, and the third weighted average value is used as the intermediate high-frequency feature.
8. The method according to claim 1, It is characterized in that Sharpening the intermediate high-frequency features based on the intermediate high-frequency features and the preset sharpening rules to obtain the target high-frequency features includes: Preprocessing the intermediate high-frequency feature so that the value of the intermediate high-frequency feature is within a target range, the target range includes a first sub-range, a second sub-range, a third sub-range, and a fourth sub-range connected in the first place, wherein the value in the fourth sub-range is greater than the value in the first sub-range; In response to the intermediate high-frequency feature being in the first sub-range, sharpening the intermediate high-frequency feature using a first sharpening value to obtain a target high-frequency feature; In response to the intermediate high-frequency feature being in the second sub-range, determining a second sharpening value using the first mapping relationship and the intermediate high-frequency feature, and performing sharpening processing on the intermediate high-frequency feature using the second sharpening value to obtain a target high-frequency feature, wherein in the first mapping relationship, the intermediate high-frequency feature and the second sharpening value are negatively correlated; In response to the intermediate high-frequency feature being in a third sub-range, a third sharpening value is determined using the second mapping relationship and the intermediate high-frequency feature, and the intermediate high-frequency feature is sharpened using the third sharpening value to obtain a target high-frequency feature; wherein in the second mapping relationship, the intermediate high-frequency feature and the third sharpening value are positively correlated; In response to the intermediate high-frequency feature being in the fourth sub-range, sharpening the intermediate high-frequency feature using a first sharpening value to obtain a target high-frequency feature; The first sharpening value is the maximum value of the sharpening value, and the maximum value of the second sub-range is the median of the target range.
9. The method according to claim 1, It is characterized in that The method further comprises: Determine the sensitivity of the image to be processed, and the distance information between the target pixel point and the center point of the image to be processed; The target high-frequency feature is adaptively gained according to the sensitivity of the image to be processed, the brightness value of the target pixel point and the distance information to update the target high-frequency feature.
10. The method according to claim 9, It is characterized in that Performing adaptive gain on the target high-frequency feature according to the sensitivity of the image to be processed, the brightness value of the target pixel point, and the distance information to update the target high-frequency feature includes: Determining a first gain coefficient according to the sensitivity of the image to be processed, wherein the first gain coefficient is negatively correlated with the sensitivity; Determine a second gain coefficient according to the brightness value of the target pixel point, wherein the second gain coefficient is positively correlated with the brightness value of the target pixel point; Determine a third gain coefficient according to the distance information, wherein the third gain coefficient is negatively correlated with the distance information; The target high-frequency feature is adaptively gained by using the first gain coefficient, the second gain coefficient, and the third gain coefficient to update the target high-frequency feature.
11. The method according to claim 1, It is characterized in that The obtaining of the target image based on the target high-frequency features and the image to be processed comprises: Superimposing the target high-frequency features and the image to be processed to obtain an intermediate image; The target image is obtained by limiting the pixel maximum value of the intermediate image.
12. The method according to claim 1, It is characterized in that The method further comprises: A low-pass filtering process is performed on the target image to update the target image.
13. An image processing device, It is characterized in that include: A brightness acquisition module, used to acquire brightness information of all pixels of the image to be processed, and determine a plurality of target pixels among the pixels according to the brightness information; A feature extraction module, used to obtain the spatial domain features of the target pixel point and the initial high-frequency features of the target pixel point in at least one dimension based on the brightness information; A feature determination module, used to determine an intermediate high-frequency feature according to the spatial domain feature and the initial high-frequency feature; An image sharpening module, used for sharpening the intermediate high-frequency features based on the intermediate high-frequency features and preset sharpening rules to obtain target high-frequency features; An image processing module is used to obtain a target image based on the target high-frequency features and the image to be processed.
14. A computer-readable storage medium having a computer program stored thereon, It is characterized in that When the program is executed by a processor, the image processing method according to any one of claims 1 to 12 is implemented.
15. An electronic device, It is characterized in that include: one or more processors; as well as A memory for storing one or more programs, which, when executed by the one or more processors, enables the one or more processors to implement the image processing method according to any one of claims 1 to 12.