Sharpening Image Generation Method, Apparatus, Device, and Storage Medium

Abstract

The present application discloses a method, apparatus, device and storage medium for generating a sharpened image, which can be applied to various scenarios such as cloud technology, artificial intelligence, intelligent transportation, vehicle networking, etc. The method includes: determining a high-frequency image based on an original image; determining target pixel values corresponding to at least two target coordinate points in the high-frequency image; performing non-linear remapping processing on the target pixel values corresponding to at least two target coordinate points respectively to obtain updated pixel values for each target coordinate point; generating a sharpened image according to the original pixel values of original coordinate points in the original image and the updated pixel values corresponding to at least two target coordinate points respectively. By performing non-linear remapping processing on the pixel values corresponding to each coordinate point in the high-frequency image, the sharpened image obtained in the present application is clearer and more natural, improving the visual experience of users.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular, to a method, apparatus, device, and storage medium for generating a sharpened image. Background Art

[0002] In related technical solutions, simple linear mapping is often used in high-frequency information processing. Such a mapping method can only uniformly control the strength of high-frequency information and lacks the adaptive ability to process high-frequency information with different amplitudes. To achieve a better enhancement effect, if the high-frequency information intensity is too high, the naturalness of the sharp area is damaged, resulting in image distortion; or to make the sharp area more natural, the high-frequency information is adjusted weakly, but this will sacrifice the enhancement effect, resulting in a low image clarity. Summary of the Invention

[0003] This application provides a method, apparatus, device, and storage medium for generating a sharpened image, which can accurately calculate the similarity between the first feedback text fed back by the target object and the second feedback text in the preset text vector library, thereby improving the accuracy of generating the sharpened image.

[0004] On the one hand, this application provides a method for generating a sharpened image, and the method includes:

[0005] Based on the original image, determine the high-frequency image;

[0006] Determine the target pixel values corresponding to at least two target coordinate points in the high-frequency image;

[0007] Perform non-linear remapping processing on the target pixel values corresponding to the at least two target coordinate points to obtain the updated pixel value of each target coordinate point; wherein, the at least two target coordinate points include a first target coordinate point and a second target coordinate point, the first target coordinate point includes target coordinate points with target pixel values less than a first threshold and target coordinate points with target pixel values greater than a second threshold, the first threshold is less than the second threshold; the updated pixel value corresponding to the first target coordinate point is less than a first preset pixel value; the updated pixel value corresponding to the second target coordinate point is greater than a second preset pixel value; the first preset pixel value is obtained by linearly processing the target pixel value corresponding to the first target coordinate point, and the second preset pixel value is obtained by linearly processing the target pixel value corresponding to the second target coordinate point;

[0008] Generate a sharpened image according to the original pixel value of the original coordinate point in the original image and the updated pixel values corresponding to the at least two target coordinate points.

[0009] On the other hand, a device for generating a sharpened image is provided, and the device includes:

[0010] A high-frequency image determination module, configured to determine a high-frequency image based on an original image;

[0011] A target pixel value determination module, configured to determine target pixel values corresponding to at least two target coordinate points in the high-frequency image respectively;

[0012] An updated pixel value determination module, configured to perform non-linear remapping processing on the target pixel values corresponding to the at least two target coordinate points respectively to obtain updated pixel values for each target coordinate point; wherein, the at least two target coordinate points include a first target coordinate point and a second target coordinate point, the first target coordinate point includes target coordinate points with target pixel values less than a first threshold and target coordinate points with target pixel values greater than a second threshold, the first threshold is less than the second threshold; the updated pixel value corresponding to the first target coordinate point is less than a first preset pixel value; the updated pixel value corresponding to the second target coordinate point is greater than a second preset pixel value; the first preset pixel value is obtained by performing linear processing on the target pixel value corresponding to the first target coordinate point, and the second preset pixel value is obtained by performing linear processing on the target pixel value corresponding to the second target coordinate point;

[0013] A sharpened image generation module, configured to generate a sharpened image according to the original pixel values of the original coordinate points in the original image and the updated pixel values corresponding to the at least two target coordinate points respectively.

[0014] On the other hand, a sharpened image generation device is provided, the device includes a processor and a memory, and at least one instruction or at least one program segment is stored in the memory, and the at least one instruction or the at least one program segment is loaded and executed by the processor to implement the sharpened image generation method as described above.

[0015] On the other hand, a computer storage medium is provided, the computer storage medium stores at least one instruction or at least one program segment, and the at least one instruction or at least one program segment is loaded and executed by a processor to implement the sharpened image generation method as described above.

[0016] On the other hand, a computer program product or a computer program is provided, the computer program product or the computer program includes computer instructions, and the computer instructions are stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes to implement the sharpened image generation method as described above.

[0017] The sharpened image generation method, device, equipment and storage medium provided by this application have the following technical effects:

[0018] Based on the original image, this application determines a high-frequency image; determines the target pixel values corresponding to at least two target coordinate points in the high-frequency image; performs a non-linear remapping process on the target pixel values corresponding to the at least two target coordinate points to obtain the updated pixel value of each target coordinate point; wherein, the at least two target coordinate points include a first target coordinate point and a second target coordinate point, the first target coordinate point includes target coordinate points with target pixel values less than a first threshold and target coordinate points with target pixel values greater than a second threshold, the first threshold is less than the second threshold; the updated pixel value corresponding to the first target coordinate point is less than a first preset pixel value; the updated pixel value corresponding to the second target coordinate point is greater than a second preset pixel value; the first preset pixel value is obtained by performing a linear process based on the target pixel value corresponding to the first target coordinate point, and the second preset pixel value is obtained by performing a linear process based on the target pixel value corresponding to the second target coordinate point; generates a sharpened image according to the original pixel value of the original coordinate point in the original image and the updated pixel values corresponding to the at least two target coordinate points respectively. By performing a non-linear remapping process on the pixel values corresponding to each coordinate point in the high-frequency image, this application makes the updated pixel values of high-frequency information with too low or too high pixel values smaller, and the updated pixel values of high-frequency information with medium pixel values larger, so that the obtained sharpened image is clearer and more natural, improving the user's visual experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] To more clearly illustrate the technical solutions and advantages in the embodiments of the present application or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the following drawings are only some embodiments of the present application. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0020] Figure 1 is a schematic diagram of a sharpened image generation system provided by an embodiment of the present application;

[0021] Figure 2 is a flowchart of a sharpened image generation method provided by an embodiment of the present application;

[0022] Figure 3 is a flowchart of a method for performing a non-linear remapping process on the target pixel values corresponding to the at least two target coordinate points respectively to obtain the updated pixel value of each target coordinate point;

[0023] Figure 4 is a flowchart of a method for constructing a non-linear remapping function provided by an embodiment of the present application;

[0024] Figure 5is a flowchart of a method for constructing a fractional function and an exponential function corresponding to the pixel provided in an embodiment of the present application;

[0025] Figure 6 is a flowchart of a method for constructing an exponential function corresponding to the pixel according to the absolute value function, the second parameter, the third parameter and the fourth parameter provided in an embodiment of the present application;

[0026] Figure 7 It is a flowchart of a method for generating a sharpened image according to the original pixel value of the original coordinate point in the original image and the updated pixel values corresponding to the at least two target coordinate points, provided by an embodiment of the present application;

[0027] Figure 8 It is a flowchart of a method for generating a sharpened image according to original pixel values corresponding to remaining coordinate points in the original image and sharpened pixel values corresponding to at least two target coordinate points, provided by an embodiment of the present application;

[0028] Figure 9 It is a schematic diagram of intensity comparison of a high-frequency image after linear remapping and S-shaped remapping provided by an embodiment of the present application;

[0029] Figure 10 is a structural schematic diagram of a sharpened image generating device provided in an embodiment of the present application;

[0030] Figure 11 It is a structural diagram of a server provided in an embodiment of the present application. DETAILED DESCRIPTION

[0031] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application.

[0032] First, some nouns or terms that appear in the description of the embodiments of the present application are explained as follows:

[0033] Video sharpening: Video sharpening is a technology that extracts and enhances the high-frequency information of a video, making the texture content of the video sharper and allowing users to perceive the video as clearer.

[0034] High-frequency information: High-frequency information refers to the information in the video signal with complex texture content and drastic changes, such as the texture of grass and trees.

[0035] Low-frequency information: Low-frequency information refers to the information with simple texture content and gentle changes in the video, such as the sky, etc.

[0036] Low-pass filter: Low-pass filtering is a filtering method. The rule is that low-frequency signals can pass through normally, while high-frequency signals exceeding the set critical value are blocked and weakened. However, the degree of blocking and weakening will change according to different frequencies and different filtering procedures (purposes). Sometimes it is also called high-cut filter or treble-cut filter. Low-pass filtering is the opposite of high-pass filtering. By means such as averaging, low-pass filtering can suppress high-frequency information in the video, making the video look more blurred.

[0037] Remapping: Placing the pixel points in one image to the specified positions in another image is called remapping; a new image is obtained by modifying the positions of the pixel points.

[0038] Intelligent transportation makes full use of new-generation information technologies such as the Internet of Things, spatial perception, cloud computing, and mobile Internet in the entire transportation field. It comprehensively applies theories and tools such as traffic science, systematic methods, artificial intelligence, and knowledge mining. Aiming at comprehensive perception, deep integration, active service, and scientific decision-making, by building a real-time dynamic information service system, deeply mining transportation-related data, forming a problem analysis model, realizing the improvement of the industry's resource allocation ability, public decision-making ability, industry management ability, and public service ability, promoting the safer, more efficient, more convenient, more economical, more environmentally friendly, and more comfortable operation and development of transportation, and driving the transformation and upgrading of transportation-related industries.

[0039] It should be noted that the terms "first", "second", etc. in the description, claims, and above-mentioned drawings of this application are used to distinguish similar objects and do not necessarily describe a specific order or sequence. It should be understood that such used data can be interchanged under appropriate circumstances so that the embodiments of this application described here can be implemented in an order different from those illustrated or described here. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or server that includes a series of steps or units does not necessarily have to be limited to those steps or units clearly listed, but may include other steps or units not clearly listed or inherent to these processes, methods, products, or devices.

[0040] Please refer to Figure 1 , Figure 1 which is a schematic diagram of a sharpened image generation system provided by an embodiment of this application, as Figure 1As shown in the figure, the sharpened image generation system may at least include a server 01 and a client 02.

[0041] Specifically, in the embodiments of the present application, the server 01 may include an independently operating server, or a distributed server, or a server cluster composed of multiple servers, and may also be a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN (Content Delivery Network), and big data and artificial intelligence platforms. The server 01 may include a network communication unit, a processor, a memory, and so on. Specifically, the server 01 may be used to generate a sharpened image based on the original image.

[0042] Specifically, in the embodiments of the present application, the client 02 may include physical devices such as smartphones, desktop computers, tablet computers, laptop computers, digital assistants, smart wearable devices, smart speakers, vehicle terminals, smart TVs, etc., and may also include software running on the physical devices, such as web pages provided by some service providers to users, or applications provided by these service providers to users. Specifically, the client 02 may be used to send the original image to the server 01 and display the sharpened image.

[0043] The following introduces a method for generating a sharpened image according to the present application. Figure 2 It is a schematic flowchart of a method for generating a sharpened image provided by the embodiments of the present application. This specification provides the method operation steps as described in the embodiments or flowcharts, but based on routine or non-creative labor, there may be more or fewer operation steps. The step order listed in the embodiments is only one way among the execution orders of numerous steps and does not represent the only execution order. When the actual system or server product executes, it may be executed in the order of the embodiments or the method shown in the drawings, or executed in parallel (for example, in an environment of parallel processors or multi-threaded processing). Specifically, as Figure 2 shown, the method may include:

[0044] S201: Determine a high-frequency image based on the original image.

[0045] In the embodiments of the present application, the high-frequency image is an image composed of high-frequency information in the original image.

[0046] In some embodiments, the determining a high-frequency image based on the original image includes:

[0047] Extract the low-frequency image in the original image based on a low-pass filtering algorithm;

[0048] In the embodiments of the present application, the low-frequency image is an image composed of low-frequency information; after the low-frequency image is determined, all coordinate points in the low-frequency image are defined as low-frequency coordinate points, and the pixel value of each low-frequency coordinate point in the low-frequency image can be calculated. Specifically, it can be determined according to the adjacent coordinate points of each low-frequency coordinate point in the original image.

[0049] In some embodiments, the method for determining the pixel value of each low-frequency coordinate point in the low-frequency image includes:

[0050] Obtain the first pixel value of any low-frequency coordinate point in the original image;

[0051] Obtain the second pixel values of a preset number of adjacent coordinate points of the any low-frequency coordinate point in the original image;

[0052] Calculate the pixel average value of the first pixel value and the preset number of second pixel values;

[0053] Take the pixel average value as the pixel value of the any low-frequency coordinate point in the low-frequency image.

[0054] In a specific embodiment, the corresponding low-frequency image is obtained based on low-pass filtering Specifically, for each pixel I at position (x, y) in the original image x,y , calculate the average value of the pixel of any coordinate point in the low-frequency image in the original image and the pixels of the adjacent coordinate points around the coordinate point to obtain the pixel value in the low-frequency image. The calculation formula is as follows:

[0055]

[0056] Among them, this formula is the pixel average value of 25 coordinate points.

[0057] Determine the high-frequency image according to the original image and the low-frequency image.

[0058] In the embodiments of the present application, the high-frequency image can be determined according to the original image and the low-frequency image.

[0059] In the embodiments of the present application, the original image, the low-frequency image, and the high-frequency image may include the same number of coordinate points. In the low-frequency image, the high-frequency information is blurred, and the pixel value of the corresponding coordinate point is 0. Therefore, the pixel value of each coordinate point in the high-frequency image can be determined by the method of taking the difference.

[0060] In the embodiments of the present application, the determining the high-frequency image according to the original image and the low-frequency image includes:

[0061] Obtain the original pixel values of each coordinate point in the original image and the low-frequency pixel values of each coordinate point in the low-frequency image;

[0062] Calculate the difference between the original pixel value and the low-frequency pixel value corresponding to each coordinate point to obtain the target pixel value of each coordinate point;

[0063] Generate the high-frequency image according to the target pixel values of each coordinate point.

[0064] In a specific embodiment, based on the original image I and the low-frequency image Extract the corresponding high-frequency image For each pixel at position (x, y), the calculation formula for the pixel value in the corresponding high-frequency image is:

[0065]

[0066] In the embodiment of the present application, before determining the high-frequency image based on the original image, the method further includes:

[0067] Obtain the target video;

[0068] Decompose the target video into at least two original images;

[0069] Correspondingly, after updating the original pixel value of each target coordinate point in the original image to the sharpened pixel value and generating the sharpened image, the method further includes:

[0070] Generate a sharpened video according to the sharpened images corresponding to each of the at least two original images.

[0071] In the embodiment of the present application, the original image can be a video frame image extracted from a video, and each frame image in the video can be sharpened to generate a sharpened video with high clarity, naturalness and no distortion. This embodiment can be used in the fields of video calls, video account live broadcasts, video accounts, and short video playbacks, and can effectively improve the clarity of the video and enhance the subjective experience of users.

[0072] S203: Determine the target pixel values corresponding to at least two target coordinate points in the high-frequency image.

[0073] In the embodiment of the present application, the coordinate points in the high-frequency image are defined as target coordinate points, and the high-frequency image may include at least two target coordinate points, and the target pixel values corresponding to each target coordinate point can be obtained.

[0074] S205: Perform a non-linear remapping process on the target pixel values corresponding to the at least two target coordinate points to obtain the updated pixel value of each target coordinate point; wherein, the at least two target coordinate points include a first target coordinate point and a second target coordinate point, the first target coordinate point includes target coordinate points with target pixel values less than a first threshold and target coordinate points with target pixel values greater than a second threshold, the first threshold is less than the second threshold; the updated pixel value corresponding to the first target coordinate point is less than a first preset pixel value; the updated pixel value corresponding to the second target coordinate point is greater than a second preset pixel value; the first preset pixel value is obtained by performing a linear process on the target pixel value corresponding to the first target coordinate point, and the second preset pixel value is obtained by performing a linear process on the target pixel value corresponding to the second target coordinate point;

[0075] In an embodiment of the present application, the non-linear remapping processing line corresponding to the at least two target coordinate points intersects with the linear processing line;

[0076] In a specific embodiment, as Figure 9 shown, Figure 9 FIG. is a schematic diagram of the intensity comparison after linear remapping and S-shaped remapping processing of a high-frequency image, wherein the straight line is the result of linear remapping processing, and the corresponding linear processing formula is: The S curve is the result of non-linear remapping processing, and the corresponding formula is:

[0077]

[0078] It can be seen that the first target coordinate point includes target coordinate points with target pixel values less than a first threshold and target coordinate points with target pixel values greater than a second threshold, the first threshold is less than the second threshold; the updated pixel value corresponding to the first target coordinate point is less than a first preset pixel value; the updated pixel value corresponding to the second target coordinate point is greater than a second preset pixel value; the first preset pixel value is obtained by performing a linear process on the target pixel value corresponding to the first target coordinate point, and the second preset pixel value is obtained by performing a linear process on the target pixel value corresponding to the second target coordinate point.

[0079] For a video frame, it is desired to enhance its sharpness by adding high-frequency information. The stronger the high-frequency information, the better the enhancement effect. However, for some areas that are originally sharp, such as, it is not desired to over-enhance. If the enhancement is too excessive, it is very easy to make the enhanced result unnatural. The simple linear mapping of traditional methods can only uniformly adjust the strength of high-frequency information. Reflected in the enhancement effect, in order to achieve a better enhancement effect, the intensity of high-frequency information is too large, which destroys the naturalness of sharp areas. Or, in order to make the sharp areas more natural, the high-frequency information is adjusted weakly, but this will sacrifice the enhancement effect. When the intensity of high-frequency information is low, the mapping value is weak because the area with low high-frequency information intensity itself does not have much high-frequency information and may be a defocused area in the original image and does not require excessive enhancement. When the intensity of high-frequency information is high, it is also suppressed because the area with high high-frequency information intensity itself has relatively drastic texture changes, and further enhancement will make the target area look less natural visually. In this embodiment, when the intensity of high-frequency information is low, the mapping value is weak because the area with low high-frequency information intensity itself does not have much high-frequency information and may be a defocused area in the original image and does not require excessive enhancement. When the intensity of high-frequency information is high, it is also suppressed.

[0080] In the embodiment of the present application, as Figure 3 shown, the non-linear remapping process of the target pixel values corresponding to the at least two target coordinate points respectively to obtain the updated pixel value of each target coordinate point includes:

[0081] S2051: Construct a non-linear remapping function;

[0082] In the embodiment of the present application, as Figure 4 shown, the construction of the non-linear remapping function includes:

[0083] S20511: Construct an assignment function corresponding to the pixel, and the assignment function represents the corresponding relationship between the pixel value and a preset value;

[0084] In the embodiment of the present application, the construction of the assignment function corresponding to the pixel includes:

[0085] Construct a preset mapping relationship between the pixel value of the pixel and a preset value;

[0086] In the embodiment of the present application, the construction of the preset mapping relationship between the pixel value of the pixel and a preset value includes:

[0087] If the pixel value is greater than zero, determine that the preset value is a positive number;

[0088] In the embodiment of the present application, the positive number can be 1.

[0089] If the pixel value is less than zero, determine that the preset value is negative;

[0090] In an embodiment of the present application, the negative number can be -1.

[0091] If the pixel value is equal to zero, determine that the preset value is zero.

[0092] Construct the assignment function according to the preset mapping relationship.

[0093] In an embodiment of the present application, the assignment function can be When Greater than 0 Is 1, when Less than 0 Is -1, when Equal to 0 Is 0; where sign is also called sgn, which means sign. The assignment function (usually represented by sign(x)) is a very useful type of function that can help us achieve some constructions that are difficult to directly implement in the Geometer's Sketchpad. The assignment function can separate the sign of the function. In mathematics and computer operations, its function is to take the sign (positive or negative) of a certain number.

[0094] S20513: Construct the absolute value function of the pixel;

[0095] In an embodiment of the present application, the absolute value function can be

[0096] S20515: Construct the non-linear remapping function according to the assignment function and the absolute value function.

[0097] In an embodiment of the present application, as Figure 5 Shown, constructing the non-linear remapping function according to the assignment function and the absolute value function includes:

[0098] S205151: Construct the first function corresponding to the pixel according to the absolute value function and the first parameter; the first function is a fraction with the absolute value function as the numerator and the sum of the absolute value function and the first parameter as the denominator;

[0099] In an embodiment of the present application, the first parameter is d, and the first function can be

[0100] S205153: Construct the second function corresponding to the pixel according to the absolute value function, the second parameter, the third parameter, and the fourth parameter;

[0101] In an embodiment of the present application, as Figure 6As shown, constructing the second function corresponding to the pixel according to the absolute value function, the second parameter, the third parameter, and the fourth parameter includes:

[0102] S2051351: Construct a third function with the difference between the absolute value function and the second parameter as the numerator and the third parameter as the denominator;

[0103] In the embodiment of the present application, the second parameter is b, the third parameter is c, and the third function can be

[0104] S2051353: Construct a fourth function with the natural constant as the base and the third function as the exponent;

[0105] In the embodiment of the present application, the fourth function can be

[0106] S2051355: Determine the sum of the fourth function and a preset constant as the fifth function;

[0107] In the embodiment of the present application, the preset constant can be 1, and the fifth function can be

[0108] S2051357: Construct the second function corresponding to the pixel with the fifth function as the denominator and the fourth parameter as the numerator.

[0109] In the embodiment of the present application, the fourth parameter is a, and the second function can be

[0110] S205155: Determine the product of the assignment function, the first function, and the second function as the non - linear remapping function.

[0111] In the embodiment of the present application, the non - linear remapping function can be:

[0112]

[0113] In a specific embodiment, the parameters a, b, c, and d in the function can take values of 20, 8, 4, and 4 respectively.

[0114] S2053: Based on the non - linear remapping function, perform non - linear remapping processing on the target pixel values corresponding to the at least two target coordinate points to obtain the updated pixel values of each target coordinate point.

[0115] In the embodiment of the present application, the updated pixel value of each target coordinate point can be calculated through the corresponding formula of the non - linear remapping function.

[0116] S207: Generate a sharpened image based on the original pixel values of the original coordinate points in the original image and the updated pixel values corresponding to each of the at least two target coordinate points.

[0117] In the embodiments of the present application, as Figure 7 shown, the generating a sharpened image based on the original pixel values of the original coordinate points in the original image and the updated pixel values corresponding to each of the at least two target coordinate points includes:

[0118] S2071: Use the sum of the target pixel value and the updated pixel value corresponding to each of the at least two target coordinate points as the sharpened pixel value of each target coordinate point.

[0119] S2073: Generate a sharpened image based on the original pixel values corresponding to the remaining coordinate points in the original image and the sharpened pixel values corresponding to each of the at least two target coordinate points; the remaining coordinate points are other coordinate points in the original image except the at least two target coordinate points.

[0120] In the embodiments of the present application, by performing sharpening processing on the high-frequency image in the original image, a sharpened image with high clarity, naturalness, and no distortion can be generated, improving the user's visual experience.

[0121] In the embodiments of the present application, as Figure 8 shown, the generating a sharpened image based on the original pixel values corresponding to the remaining coordinate points in the original image and the sharpened pixel values corresponding to each of the at least two target coordinate points includes:

[0122] S20731: Update the original pixel values corresponding to each of the at least two target coordinate points in the original image to the corresponding sharpened pixel values.

[0123] In the embodiments of the present application, the pixel values of the high-frequency information in the original image can be updated to obtain a sharpened high-frequency image.

[0124] S20733: Generate a sharpened image based on the sharpened pixel values corresponding to each of the at least two target coordinate points and the original pixel values corresponding to the remaining coordinate points in the original image.

[0125] In the embodiments of the present application, based on the non-linear remapping processing of the high-frequency information in the image, while ensuring the sharpening enhancement effect, it can less enhance the areas that are prone to excessive enhancement, thus adaptively completing the enhancement of the sharpness clarity of the image. As a result, for the high-frequency information with too low or too high pixel values, the updated pixel values are relatively small, and for the high-frequency information with pixel values in the middle, the updated pixel values are relatively large, making the obtained sharpened image clearer and more natural, and improving the user's visual experience.

[0126] As can be seen from the technical solutions provided by the embodiments of the present application above, the embodiments of the present application determine a high-frequency image based on an original image; determine target pixel values corresponding to at least two target coordinate points in the high-frequency image; perform a non-linear remapping process on the target pixel values corresponding to the at least two target coordinate points to obtain updated pixel values for each target coordinate point; wherein the at least two target coordinate points include a first target coordinate point and a second target coordinate point, the first target coordinate point includes target coordinate points with target pixel values less than a first threshold and target coordinate points with target pixel values greater than a second threshold, the first threshold is less than the second threshold; the updated pixel value corresponding to the first target coordinate point is less than a first preset pixel value; the updated pixel value corresponding to the second target coordinate point is greater than a second preset pixel value; the first preset pixel value is obtained by performing a linear process on the target pixel value corresponding to the first target coordinate point, and the second preset pixel value is obtained by performing a linear process on the target pixel value corresponding to the second target coordinate point; generate a sharpened image according to the original pixel value of the original coordinate point in the original image and the updated pixel values corresponding to the at least two target coordinate points respectively. By performing a non-linear remapping process on the pixel values corresponding to each coordinate point in the high-frequency image, the present application makes the updated pixel values of high-frequency information with too low or too high pixel values smaller, and the updated pixel values of high-frequency information with pixel values in the middle larger, so that the obtained sharpened image is clearer and more natural, improving the user's visual experience.

[0127] The embodiments of the present application also provide a sharpened image generation device, as Figure 10 shown, the device includes:

[0128] A high-frequency image determination module 1010, configured to determine a high-frequency image based on an original image;

[0129] A target pixel value determination module 1020, configured to determine target pixel values corresponding to at least two target coordinate points in the high-frequency image;

[0130] The updated pixel value determination module 1030 is configured to perform a non-linear remapping process on the target pixel values corresponding to the at least two target coordinate points respectively, to obtain the updated pixel value of each target coordinate point; wherein, the at least two target coordinate points include a first target coordinate point and a second target coordinate point, the first target coordinate point includes target coordinate points with target pixel values less than a first threshold and target coordinate points with target pixel values greater than a second threshold, the first threshold is less than the second threshold; the updated pixel value corresponding to the first target coordinate point is less than a first preset pixel value; the updated pixel value corresponding to the second target coordinate point is greater than a second preset pixel value; the first preset pixel value is obtained by linearly processing the target pixel value corresponding to the first target coordinate point, and the second preset pixel value is obtained by linearly processing the target pixel value corresponding to the second target coordinate point;

[0131] The sharpened image generation module 1040 is configured to generate a sharpened image according to the original pixel values of the original coordinate points in the original image and the updated pixel values corresponding to the at least two target coordinate points respectively.

[0132] In some embodiments, the updated pixel value determination module may include:

[0133] A function construction sub-module, configured to construct a non-linear remapping function;

[0134] An updated pixel value determination sub-module, configured to perform a non-linear remapping process on the target pixel values corresponding to the at least two target coordinate points respectively based on the non-linear remapping function, to obtain the updated pixel value of each target coordinate point.

[0135] In some embodiments, the function construction sub-module may include:

[0136] An assignment function construction unit, configured to construct an assignment function corresponding to the pixel, and the assignment function represents the corresponding relationship between the pixel value and a preset value;

[0137] An absolute value function construction unit, configured to construct an absolute value function of the pixel;

[0138] A remapping function construction unit, configured to construct the non-linear remapping function according to the assignment function and the absolute value function.

[0139] In some embodiments, the remapping function construction unit includes:

[0140] A first function construction sub-unit, configured to construct a first function corresponding to the pixel according to the absolute value function and a first parameter; the first function is a fraction with the absolute value function as the numerator and the sum of the absolute value function and the first parameter as the denominator;

[0141] A second function construction subunit, configured to construct a second function corresponding to the pixel according to the absolute value function, a second parameter, a third parameter, and a fourth parameter;

[0142] A remapping function construction subunit, configured to determine the product of the assignment function, the first function, and the second function as the non-linear remapping function.

[0143] In some embodiments, the second function construction subunit may include:

[0144] A third function construction subunit, configured to construct a third function with the difference between the absolute value function and the second parameter as the numerator and the third parameter as the denominator;

[0145] A fourth function construction subunit, configured to construct a fourth function with the natural constant as the base and the third function as the exponent;

[0146] A fifth function construction subunit, configured to determine the sum of the fourth function and a preset constant as the fifth function;

[0147] A function subunit, configured to construct the second function corresponding to the pixel with the fifth function as the denominator and the fourth parameter as the numerator.

[0148] In some embodiments, the assignment function construction unit may include:

[0149] A preset mapping relationship construction subunit, configured to construct a preset mapping relationship between the pixel value of the pixel and a preset value;

[0150] An assignment function construction subunit, configured to construct the assignment function according to the preset mapping relationship.

[0151] In some embodiments, the preset mapping relationship construction subunit may include:

[0152] A first assignment subunit, configured to determine that the preset value is a positive number if the pixel value is greater than zero;

[0153] A second assignment subunit, configured to determine that the preset value is a negative number if the pixel value is less than zero;

[0154] A third assignment subunit, configured to determine that the preset value is zero if the pixel value is equal to zero.

[0155] In some embodiments, the sharpened image generation module may include:

[0156] A sharpened pixel value determination sub-module, configured to use the sum of the target pixel values and the updated pixel values corresponding to the at least two target coordinate points as the sharpened pixel value of each target coordinate point;

[0157] A sharpened image generation sub-module, configured to generate a sharpened image according to the original pixel values corresponding to the remaining coordinate points in the original image and the sharpened pixel values corresponding to each of the at least two target coordinate points; the remaining coordinate points are other coordinate points in the original image except the at least two target coordinate points.

[0158] In some embodiments, the apparatus may further include:

[0159] A video acquisition module, configured to acquire a target video;

[0160] An original image decomposition module, configured to decompose the target video into at least two original images;

[0161] In some embodiments, the apparatus may further include:

[0162] A video generation module, configured to generate a sharpened video according to the sharpened images corresponding to each of the at least two original images.

[0163] The apparatus in the apparatus embodiment and the method embodiment are based on the same inventive concept.

[0164] An embodiment of the present application provides a sharpened image generation device, which includes a processor and a memory. At least one instruction or at least one segment of program is stored in the memory, and the at least one instruction or at least one segment of program is loaded and executed by the processor to implement the sharpened image generation method provided in the above method embodiment.

[0165] An embodiment of the present application further provides a computer storage medium, which can be set in a terminal to store at least one instruction or at least one segment of program related to implementing a sharpened image generation method in a method embodiment. The at least one instruction or at least one segment of program is loaded and executed by the processor to implement the sharpened image generation method provided in the above method embodiment.

[0166] An embodiment of the present application further provides a computer program product or a computer program, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes to implement the sharpened image generation method provided in the above method embodiment.

[0167] Optionally, in the embodiments of the present application, the storage medium may be located in at least one of multiple network servers of a computer network. Optionally, in this embodiment, the above storage medium may include, but is not limited to, various media that can store program codes, such as USB flash drives, read-only memories (ROMs), random access memories (RAMs), mobile hard disks, magnetic disks, or optical discs.

[0168] The memory in the embodiments of the present application can be used to store software programs and modules. The processor executes various functional applications and data processing by running the software programs and modules stored in the memory. The memory mainly includes a program storage area and a data storage area. Among them, the program storage area can store an operating system, application programs required for functions, etc.; the data storage area can store data created according to the use of the device, etc. In addition, the memory may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one magnetic disk storage device, a flash memory device, or other volatile solid-state storage devices. Correspondingly, the memory may further include a memory controller to provide the processor with access to the memory.

[0169] The embodiment of the sharpened image generation method provided in the embodiments of the present application can be executed on a mobile terminal, a computer terminal, a server, or a similar computing device. Taking running on a server as an example, Figure 11 is the hardware structure block diagram of a server for the sharpened image generation method provided in the embodiments of the present application. As Figure 11As shown, the server 1100 can vary significantly depending on configuration or performance, and may include one or more central processing units (CPUs) 1110 (the central processing unit 1110 may include, but is not limited to, processing devices such as microprocessor MCUs or programmable logic devices FPGAs), a memory 1130 for storing data, and one or more storage media 1120 for storing application programs 1123 or data 1122 (such as one or more mass storage devices). Among them, the memory 1130 and the storage media 1120 can be transient storage or persistent storage. The programs stored in the storage media 1120 may include one or more modules, and each module may include a series of instruction operations on the server. Further, the central processing unit 1110 can be set to communicate with the storage media 1120 and execute a series of instruction operations in the storage media 1120 on the server 1100. The server 1100 may also include one or more power supplies 1160, one or more wired or wireless network interfaces 1150, one or more input / output interfaces 1140, and / or one or more operating systems 1121, such as Windows ServerTM, Mac OS XTM, UnixTM, LinuxTM, FreeBSDTM, and so on.

[0170] The input / output interface 1140 can be used to receive or send data via a network. Specific examples of the above-mentioned network may include a wireless network provided by the communication provider of the server 1100. In one example, the input / output interface 1140 includes a network interface controller (NIC), which can be connected to other network devices through a base station and thus communicate with the Internet. In one example, the input / output interface 1140 can be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0171] Those of ordinary skill in the art can understand that Figure 11 the structure shown is only schematic and does not limit the structure of the above-mentioned electronic device. For example, the server 1100 may also include more or fewer components than Figure 11 shown, or have a different configuration from Figure 11 shown.

[0172] As can be seen from the embodiments of the sharpening image generation method, apparatus, device or storage medium provided by the present application above, the present application determines a high-frequency image based on an original image; determines target pixel values corresponding to at least two target coordinate points in the high-frequency image; performs a non-linear remapping process on the target pixel values corresponding to the at least two target coordinate points to obtain updated pixel values for each target coordinate point; wherein the at least two target coordinate points include a first target coordinate point and a second target coordinate point, the first target coordinate point includes target coordinate points with target pixel values less than a first threshold and target coordinate points with target pixel values greater than a second threshold, the first threshold is less than the second threshold; the updated pixel value corresponding to the first target coordinate point is less than a first preset pixel value; the updated pixel value corresponding to the second target coordinate point is greater than a second preset pixel value; the first preset pixel value is obtained by performing a linear process on the target pixel value corresponding to the first target coordinate point, and the second preset pixel value is obtained by performing a linear process on the target pixel value corresponding to the second target coordinate point; and generates a sharpened image according to the original pixel value of the original coordinate point in the original image and the updated pixel values corresponding to the at least two target coordinate points respectively. By performing a non-linear remapping process on the pixel values corresponding to each coordinate point in the high-frequency image, the present application makes the updated pixel values of the high-frequency information with too low or too high pixel values smaller, and the updated pixel values of the high-frequency information with pixel values in the middle larger, so that the obtained sharpened image is clearer and more natural, improving the user's visual experience.

[0173] It should be noted that: the above sequence of the embodiments of the present application is only for description and does not represent the superiority or inferiority of the embodiments. And the above describes specific embodiments of this specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims can be performed in a different order than in the embodiments and still achieve the desired result. Additionally, the processes depicted in the figures do not necessarily require the particular order or sequential order shown to achieve the desired result. In certain embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0174] Each embodiment in this specification is described in a progressive manner. The same or similar parts among the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the embodiments of the apparatus, device, and storage medium, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can refer to the partial description of the method embodiments.

[0175] Those of ordinary skill in the art can understand that all or part of the steps to implement the above embodiments can be completed by hardware, or can be completed by instructing relevant hardware through a program. The program can be stored in a computer storage medium. The above-mentioned storage medium can be a read-only memory, a disk, an optical disc, etc.

[0176] The above are only the preferred embodiments of the present application and are not intended to limit the present application. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present application shall be included within the protection scope of the present application.

Claims

1. A method for generating a sharpened image, characterized in that, The method includes: Determining a high-frequency image based on an original image; Determining target pixel values corresponding to at least two target coordinate points in the high-frequency image; Performing non-linear remapping processing on the target pixel values corresponding to the at least two target coordinate points to obtain updated pixel values for each target coordinate point; wherein, the at least two target coordinate points include a first target coordinate point and a second target coordinate point, the first target coordinate point includes target coordinate points with target pixel values less than a first threshold and target coordinate points with target pixel values greater than a second threshold, the first threshold is less than the second threshold; the updated pixel value corresponding to the first target coordinate point is less than a first preset pixel value; the updated pixel value corresponding to the second target coordinate point is greater than a second preset pixel value; the first preset pixel value is obtained by performing linear processing on the target pixel value corresponding to the first target coordinate point, and the second preset pixel value is obtained by performing linear processing on the target pixel value corresponding to the second target coordinate point; Generating a sharpened image according to the original pixel values of the original coordinate points in the original image and the updated pixel values corresponding to the at least two target coordinate points.

2. The method according to claim 1, wherein The performing non-linear remapping processing on the target pixel values corresponding to the at least two target coordinate points to obtain updated pixel values for each target coordinate point includes: Constructing a non-linear remapping function; Based on the non-linear remapping function, performing non-linear remapping processing on the target pixel values corresponding to the at least two target coordinate points to obtain updated pixel values for each target coordinate point.

3. The method according to claim 2, wherein The constructing the non-linear remapping function includes: Constructing an assignment function corresponding to the pixel, the assignment function characterizing the correspondence between the pixel value and a preset value; Constructing the absolute value function of the pixel; Constructing the non-linear remapping function according to the assignment function and the absolute value function.

4. The method according to claim 3, wherein The constructing the non-linear remapping function according to the assignment function and the absolute value function includes: Constructing a first function corresponding to the pixel according to the absolute value function and a first parameter; the first function is a fraction with the absolute value function as the numerator and the sum of the absolute value function and the first parameter as the denominator; Constructing a second function corresponding to the pixel according to the absolute value function, a second parameter, a third parameter, and a fourth parameter; Determining the product of the assignment function, the first function, and the second function as the non-linear remapping function.

5. The method according to claim 4, wherein The constructing the second function corresponding to the pixel according to the absolute value function, a second parameter, a third parameter, and a fourth parameter includes: Constructing a third function with the difference between the absolute value function and the second parameter as the numerator and the third parameter as the denominator; Constructing a fourth function with the natural constant as the base and the third function as the exponent; Determining the sum of the fourth function and a preset constant as a fifth function; Constructing the second function corresponding to the pixel with the fifth function as the denominator and the fourth parameter as the numerator.

6. The method according to claim 3, wherein The constructing the assignment function corresponding to the pixel includes: Constructing a preset mapping relationship between the pixel value of the pixel and a preset value; Construct the assignment function according to the preset mapping relationship.

7. The method according to claim 6, wherein The construction of the preset mapping relationship between the pixel value of the pixel and the preset value includes: If the pixel value is greater than zero, determine that the preset value is a positive number; If the pixel value is less than zero, determine that the preset value is a negative number; If the pixel value is equal to zero, determine that the preset value is zero.

8. The method according to any one of claims 1 to 7, characterized in that, The generation of the sharpened image according to the original pixel value of the original coordinate point in the original image and the updated pixel values corresponding to the at least two target coordinate points respectively includes: Use the sum of the target pixel value and the updated pixel value corresponding to each of the at least two target coordinate points as the sharpened pixel value of each target coordinate point; Generate a sharpened image according to the original pixel value corresponding to the remaining coordinate points in the original image and the sharpened pixel values corresponding to the at least two target coordinate points respectively; the remaining coordinate points are other coordinate points in the original image except the at least two target coordinate points.

9. The method according to any one of claims 1-7, characterized in that, Before determining the high-frequency image based on the original image, the method further includes: Obtain a target video; Decompose the target video into at least two original images; Correspondingly, after updating the original pixel value of each target coordinate point in the original image to the sharpened pixel value to generate a sharpened image, the method further includes: Generate a sharpened video according to the sharpened images corresponding to the at least two original images respectively.

10. An apparatus for generating a sharpened image, characterized in that, The device includes: A high-frequency image determination module, configured to determine a high-frequency image based on an original image; A target pixel value determination module, configured to determine the target pixel values corresponding to at least two target coordinate points in the high-frequency image respectively; An updated pixel value determination module, configured to perform a non-linear remapping process on the target pixel values corresponding to the at least two target coordinate points respectively to obtain the updated pixel value of each target coordinate point; wherein, the at least two target coordinate points include a first target coordinate point and a second target coordinate point, the first target coordinate point includes target coordinate points with target pixel values less than a first threshold and target coordinate points with target pixel values greater than a second threshold, the first threshold is less than the second threshold; the updated pixel value corresponding to the first target coordinate point is less than a first preset pixel value; the updated pixel value corresponding to the second target coordinate point is greater than a second preset pixel value; the first preset pixel value is obtained by performing a linear process on the target pixel value corresponding to the first target coordinate point, and the second preset pixel value is obtained by performing a linear process on the target pixel value corresponding to the second target coordinate point; A sharpened image generation module, configured to generate a sharpened image according to the original pixel value of the original coordinate point in the original image and the updated pixel values corresponding to the at least two target coordinate points respectively.

11. The device according to claim 10, characterized in that, The updated pixel value determination module includes: A function construction sub-module, configured to construct a non-linear remapping function; An updated pixel value determination sub-module, configured to perform a non-linear remapping process on the target pixel values corresponding to the at least two target coordinate points respectively based on the non-linear remapping function to obtain the updated pixel value of each target coordinate point.

12. The device according to claim 11, characterized in that, The function construction sub-module includes: An assignment function construction unit for constructing an assignment function corresponding to a pixel, where the assignment function represents the correspondence between pixel values and preset values; An absolute value function construction unit for constructing the absolute value function of the pixel; A remapping function construction unit for constructing the non-linear remapping function according to the assignment function and the absolute value function.

13. The device according to claim 12, characterized in that, The remapping function construction unit includes: A first function construction subunit for constructing a first function corresponding to the pixel according to the absolute value function and a first parameter; the first function is a fraction with the absolute value function as the numerator and the sum of the absolute value function and the first parameter as the denominator; A second function construction subunit for constructing a second function corresponding to the pixel according to the absolute value function, a second parameter, a third parameter, and a fourth parameter; A remapping function construction subunit for determining the product of the assignment function, the first function, and the second function as the non-linear remapping function.

14. The device according to claim 13, wherein The second function construction subunit includes: A third function construction subunit for constructing a third function with the difference between the absolute value function and the second parameter as the numerator and the third parameter as the denominator; A fourth function construction subunit for constructing a fourth function with the natural constant as the base and the third function as the exponent; A fifth function construction subunit for determining the sum of the fourth function and a preset constant as the fifth function; A function subunit for constructing a second function corresponding to the pixel with the fifth function as the denominator and the fourth parameter as the numerator.

15. The device according to claim 12, characterized in that, The assignment function construction unit includes: A preset mapping relationship construction subunit for constructing a preset mapping relationship between the pixel value of the pixel and a preset value; An assignment function construction subunit for constructing the assignment function according to the preset mapping relationship.

16. The device according to claim 15, characterized in that The preset mapping relationship construction subunit includes: A first assignment subunit for determining the preset value as a positive number if the pixel value is greater than zero; A second assignment subunit for determining the preset value as a negative number if the pixel value is less than zero; A third assignment subunit for determining the preset value as zero if the pixel value is equal to zero.

17. The device according to any one of claims 10 - 16, characterized in that, The sharpened image generation module includes: A sharpened pixel value determination sub-module for using the sum of the target pixel values and the updated pixel values corresponding to the at least two target coordinate points as the sharpened pixel value of each target coordinate point; A sharpened image generation sub-module for generating a sharpened image according to the original pixel values corresponding to the remaining coordinate points in the original image and the sharpened pixel values corresponding to the at least two target coordinate points respectively; the remaining coordinate points are other coordinate points in the original image except the at least two target coordinate points.

18. The device according to any one of claims 10-16, characterized in that, The device further includes: A video acquisition module for acquiring a target video; An original image decomposition module for decomposing the target video into at least two original images; The device further includes: A video generation module for generating a sharpened video according to the sharpened images corresponding to the at least two original images respectively.

19. A sharpened image generation device, characterized in that, The device includes a processor and a memory. At least one instruction or at least one program segment is stored in the memory, and the at least one instruction or the at least one program segment is loaded and executed by the processor to implement the sharpened image generation method according to any one of claims 1-9.

20. A computer storage medium, characterized in that, At least one instruction or at least one program segment is stored in the computer storage medium, and the at least one instruction or the at least one program segment is loaded and executed by a processor to implement the sharpened image generation method according to any one of claims 1-9.

21. A computer program product comprising computer instructions, characterized in that, When the computer instruction is executed by a processor, it implements the sharpened image generation method according to any one of claims 1-9.

Images