Image processing method and device, electronic equipment, storage medium and program product

Abstract

The invention provides an image processing method and device, electronic equipment, a storage medium and a program product. The method comprises the steps that sharpening processing is carried out on an obtained to-be-processed image, a sharpened image of the to-be-processed image is obtained, and the sharpened image comprises a plurality of sharpened pixel points and a plurality of reference pixel points; wherein the definition of the reference pixel points is smaller than that of the sharpening pixel points, and the sharpening pixel points form a sharpening boundary of the sharpening image; boundary expansion is carried out on the sharpening boundary in the sharpening image, a reference sharpening image of the to-be-processed image is obtained, and the number of sharpening pixel points in the sharpening boundary of the reference sharpening image is larger than the number of sharpening pixel points in the sharpening boundary of the sharpening image; and carrying out image fusion on the to-be-processed image, the sharpened image and the reference sharpened image to obtain a target image of the to-be-processed image. According to the invention, the image quality presented by the image can be effectively improved.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to an image processing method, device, electronic device, storage medium and program product. Background Art

[0002] With the development of image rendering technology, in order to simulate more realistic virtual scenes, virtual scene images are usually sharpened to improve the expressiveness and authenticity of the image. Sharpening is an effective image enhancement method to improve the subjective quality of video images. Sharpening in the preprocessing stage before encoding game videos can effectively highlight video textures and edges and significantly improve video quality.

[0003] In the related art, image sharpening is performed by sharpening the entire image to the same degree, which results in low distinction between the subject and the background, fails to achieve the effect of highlighting the subject, and results in poor picture quality of the sharpened image. Summary of the invention

[0004] The embodiments of the present application provide an image processing method, device, electronic device, computer-readable storage medium, and computer program product, which can effectively improve the picture quality presented by an image.

[0005] The technical solution of the embodiment of the present application is implemented as follows:

[0006] The present application provides an image processing method, including:

[0007] Performing sharpening processing on the acquired image to be processed to obtain a sharpened image of the image to be processed, wherein the sharpened image includes a plurality of sharpened pixel points and a plurality of reference pixel points;

[0008] The clarity of the reference pixel is smaller than the clarity of the sharpened pixel, and the plurality of sharpened pixel constitutes a sharpened boundary of the sharpened image;

[0009] Performing boundary extension on the sharpened boundary in the sharpened image to obtain a reference sharpened image of the image to be processed, wherein the number of sharpened pixel points in the sharpened boundary of the reference sharpened image is greater than the number of sharpened pixel points in the sharpened boundary of the sharpened image;

[0010] The image to be processed, the sharpened image and the reference sharpened image are fused to obtain a target image of the image to be processed.

[0011] The present application provides an image processing device, including:

[0012] A sharpening module, used for performing sharpening processing on the acquired image to be processed to obtain a sharpened image of the image to be processed, wherein the sharpened image includes a plurality of sharpened pixels and a plurality of reference pixels; wherein the clarity of the reference pixels is less than the clarity of the sharpened pixels, and the plurality of sharpened pixels constitute a sharpening boundary of the sharpened image;

[0013] A boundary extension module, used for performing boundary extension on the sharp boundary in the sharpened image to obtain a reference sharpened image of the image to be processed, wherein the number of sharpened pixel points in the sharp boundary of the reference sharpened image is greater than the number of sharpened pixel points in the sharp boundary of the sharpened image;

[0014] The fusion module is used to fuse the image to be processed, the sharpened image and the reference sharpened image to obtain a target image of the image to be processed.

[0015] In the above scheme, the above-mentioned sharpening module is also used to perform sharpening processing on the acquired image to be processed to obtain a sharpened image of the image to be processed, wherein the sharpened image includes multiple sharpened pixels and multiple reference pixels; wherein the clarity of the reference pixels is less than the clarity of the sharpened pixels, and the multiple sharpened pixels constitute the sharpened boundary of the sharpened image; the sharpened boundary in the sharpened image is extended to obtain a reference sharpened image of the image to be processed, and the number of sharpened pixels in the sharpened boundary of the reference sharpened image is greater than the number of sharpened pixels in the sharpened boundary of the sharpened image; the image to be processed, the sharpened image and the reference sharpened image are fused to obtain a target image of the image to be processed.

[0016] In the above scheme, the pixels in the image to be processed correspond one-to-one to the pixels in the blurred image. The above sharpening module is also used to subtract the pixel value of the corresponding pixel in the blurred image from the pixel value of each pixel in the image to be processed to obtain the target pixel value of the pixel; and update the pixel value of each pixel in the image to be processed to the corresponding target pixel value to obtain the difference image.

[0017] In the above scheme, the pixels in the above difference image correspond one-to-one to the pixels in the image to be processed, and the above sharpening module is also used to obtain a target sharpening coefficient, which is associated with the image type of the image to be processed; for each pixel in the difference image, the product of the pixel value of the pixel and the sharpening coefficient is determined as a reference pixel value, and the reference pixel value is summed with the pixel value of the corresponding pixel in the image to be processed to obtain a summed pixel value; the pixel value of each pixel in the image to be processed is updated to the corresponding summed pixel value to obtain the sharpened image.

[0018] In the above scheme, the above sharpening module is also used to obtain the mapping relationship between multiple image types and corresponding sharpening coefficients, and the image type of the image to be processed; from the mapping relationship, determine the target mapping relationship including the image type of the image to be processed, and determine the sharpening coefficient in the target mapping relationship as the target sharpening coefficient.

[0019] In the above scheme, the above-mentioned image processing device also includes: a boundary determination module, which is used to obtain a clarity threshold and the clarity of each pixel in the sharpened image; and perform the following processing on each pixel in the sharpened image: when the clarity of the pixel is greater than the clarity threshold, the pixel is determined as the sharpened pixel; when the clarity of the pixel is less than or equal to the clarity threshold, the pixel is determined as the reference pixel; and the area formed by each of the sharpened pixels in the sharpened image is determined as the sharpened boundary.

[0020] In the above scheme, the above-mentioned boundary extension module is also used to blur the reference pixel points in the sharpened image to obtain a first reference image; determine the associated pixel points associated with the sharpened boundary of the first reference image from the first reference pixel points of the first reference image, and the first reference pixel points of the first reference image correspond one-to-one to the reference pixel points in the sharpened image; determine the area formed by the associated pixel points and the sharpened pixel points in the first reference image as the extended sharpened boundary of the first reference image; enhance the clarity of the extended sharpened boundary in the first reference image to obtain the reference sharpened image.

[0021] In the above scheme, the above-mentioned boundary extension module is also used to blur each pixel point in the sharpened image to obtain a sharpened blurred image of the sharpened image, wherein the sharpened blurred image includes sharpened blurred pixel points corresponding one by one to the sharpened pixel points; and to enhance the clarity of each of the sharpened blurred pixel points in the sharpened blurred image to obtain the first reference image.

[0022] In the above scheme, the above-mentioned boundary extension module is also used to obtain a brightness threshold, and perform the following processing for each of the first reference pixel points in the first reference image: obtain the brightness value of the first reference pixel point, and when the brightness value is greater than the brightness threshold, determine the shortest distance between the first reference pixel point and the sharpening boundary; when the shortest distance is less than the distance threshold, determine the first reference pixel point as the associated pixel point.

[0023] In the above scheme, the above image processing device also includes: a size adjustment module, used to obtain the target size of the set target image and determine the image size of the image to be processed; when the target size is different from the image size, the image size of the image to be processed is adjusted to the target size to obtain the image to be processed of the target size; the above fusion module is also used to fuse the image to be processed of the target size, the sharpened image and the reference sharpened image to obtain the target image of the image to be processed.

[0024] In the above scheme, the above fusion module is also used to determine, for each pixel point in the image to be processed of the target size, a first associated pixel point corresponding to the pixel point from the sharpened image, determine a second associated pixel point corresponding to the pixel point from the reference sharpened image, and fuse the pixel values ​​corresponding to the pixel point, the first associated pixel point and the second associated pixel point respectively to obtain an updated pixel value of the pixel point; update the pixel value of each pixel point in the image to be processed of the target size to the updated pixel value to obtain the target image.

[0025] An embodiment of the present application provides an electronic device, including:

[0026] A memory for storing computer executable instructions or computer programs;

[0027] The processor is used to implement the image processing method provided in the embodiment of the present application when executing the computer executable instructions or computer program stored in the memory.

[0028] An embodiment of the present application provides a computer-readable storage medium storing computer-executable instructions for causing a processor to execute the image processing method provided in the embodiment of the present application.

[0029] The embodiment of the present application provides a computer program product, which includes a computer program or a computer executable instruction, and the computer program or the computer executable instruction is stored in a computer-readable storage medium. The processor of the electronic device reads the computer executable instruction from the computer-readable storage medium, and the processor executes the computer executable instruction, so that the electronic device performs the above-mentioned image processing method of the embodiment of the present application.

[0030] The embodiments of the present application have the following beneficial effects:

[0031] The sharpened image of the image to be processed is obtained by sharpening the acquired image to be processed, the sharpened boundary in the sharpened image is extended to obtain a reference sharpened image of the image to be processed, and the image to be processed, the sharpened image and the reference sharpened image are fused to obtain a target image of the image to be processed. In this way, the target image obtained by image fusion retains the original image information of the image to be processed on the one hand, and retains the enhanced sharpening effect of the sharpened image and the reference sharpened image on the other hand, thereby effectively improving the picture quality presented by the obtained target image. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Figure 1 is a schematic diagram of the architecture of an image processing system provided in an embodiment of the present application;

[0033] Figure 2 is a schematic diagram of the structure of an electronic device for image processing provided by an embodiment of the present application;

[0034] Figures 3 to 6 is a flowchart of an image processing method provided in an embodiment of the present application;

[0035] Figure 7 It is a schematic diagram of the effect of the image processing method provided in the embodiment of the present application;

[0036] Figure 8 is a flowchart of an image processing method provided in an embodiment of the present application;

[0037] Fig. 9 It is a schematic diagram of the effect of the image processing method provided in the embodiment of the present application. DETAILED DESCRIPTION

[0038] In order to make the purpose, technical solutions and advantages of the present application clearer, the present application will be further described in detail below in conjunction with the accompanying drawings. The described embodiments should not be regarded as limiting the present application. All other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of this application.

[0039] In the following description, reference is made to “some embodiments”, which describe a subset of all possible embodiments, but it will be understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0040] In the following description, the terms "first\second\third" involved are merely used to distinguish similar objects and do not represent a specific ordering of the objects. It can be understood that "first\second\third" can be interchanged with a specific order or sequence where permitted, so that the embodiments of the present application described here can be implemented in an order other than that illustrated or described here.

[0041] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein are only for the purpose of describing the embodiments of this application and are not intended to limit this application.

[0042] Before further describing the embodiments of the present application in detail, the nouns and terms involved in the embodiments of the present application are explained. The nouns and terms involved in the embodiments of the present application are subject to the following interpretations.

[0043] 1) Sharpening: Also known as image sharpening, it is to compensate for the image contour, enhance the image edge and grayscale jump part, and make the image clear. It is divided into two categories: spatial domain processing and frequency domain processing. Image sharpening is to highlight the edge, contour, or characteristics of some linear target elements on the image. It increases the contrast between the edge of the object and the surrounding pixels, so it is also called edge enhancement.

[0044] 2) Image clarity: refers to the clarity of the detailed patterns and their boundaries on the image.

[0045] 3) In response to: used to indicate the conditions or states on which the executed operations depend. When the dependent conditions or states are met, one or more operations executed may be in real time or have a set delay. Unless otherwise specified, there is no restriction on the order in which the multiple operations executed are executed.

[0046] 4) Virtual scene: It is a virtual scene displayed (or provided) when the application is running on the terminal. The virtual scene can be a simulation environment of the real world, a semi-simulation and semi-fictitious virtual environment, or a purely fictitious virtual environment. The virtual scene can be any one of a two-dimensional virtual scene, a 2.5-dimensional virtual scene, or a three-dimensional virtual scene. The embodiment of the present application does not limit the dimension of the virtual scene. For example, the virtual scene may include the sky, land, ocean, etc. The land may include environmental elements such as deserts and cities, and users can control virtual objects to move in the virtual scene.

[0047] 5) Virtual objects: images of various people and objects that can interact in a virtual scene, or movable objects in a virtual scene. The movable objects can be virtual characters, virtual animals, cartoon characters, etc., such as characters, animals, plants, oil drums, walls, stones, etc. displayed in a virtual scene. The virtual object can be a virtual image in the virtual scene that is used to represent the user. The virtual scene can include multiple virtual objects, each of which has its own shape and volume in the virtual scene and occupies a part of the space in the virtual scene. Optionally, the virtual object can be a user character controlled by operations on the client, or an artificial intelligence (AI) set in a virtual scene battle through training, or a non-user character (NPC) set in a virtual scene interaction. Optionally, the virtual object can be a virtual character that performs confrontational interaction in a virtual scene. Optionally, the number of virtual objects participating in the interaction in the virtual scene can be pre-set or dynamically determined according to the number of clients joining the interaction.

[0048] 6) Game Program: The game program may be any of a massively multiplayer online role-playing game (MMORPG), a first-person shooter game (FPS), a third-person shooter game, a multiplayer online tactical competitive game (MOBA), a virtual reality application, a three-dimensional map program, a simulation program, or a multiplayer gunfight survival game.

[0049] 7) Augmented Reality (AR): It is a technology that cleverly integrates virtual information with the real world. It widely uses multimedia, three-dimensional modeling, real-time registration, intelligent interaction, sensing and other technical means to simulate computer-generated text, images, three-dimensional models, music, video and other virtual information, and apply them to the real world. The two types of information complement each other, thereby achieving "enhancement" of the real world. Augmented reality technology is also called augmented reality. AR augmented reality technology is a relatively new technical content that promotes the integration of real-world information and virtual-world information content. It simulates and processes physical information that is difficult to experience in the spatial scope of the real world on the basis of computers and other scientific technologies, and superimposes virtual information content to effectively apply it in the real world. In this process, it can be perceived by human senses, thereby achieving a sensory experience beyond reality. After the real environment and virtual objects overlap, they can exist simultaneously in the same picture and space.

[0050] 8) Virtual engine: A virtual engine refers to some already-written editable computer virtual systems or core components of some interactive real-time graphics applications. These systems provide virtual scene designers with various tools needed to write virtual scenes. The purpose is to enable designers to write programs easily and quickly. Virtual engines include rendering engines (rendering engines include two-dimensional rendering engines and three-dimensional rendering engines), physics engines, collision detection engines, sound engines, script engines, animation engines, artificial intelligence engines, network engines, and scene management engines, etc.

[0051] 9) Shader: It is an editable program used to implement image rendering and replace the fixed rendering pipeline. The vertex shader is mainly responsible for the calculation of the geometric relationship of vertices, and the pixel shader is mainly responsible for the calculation of the color of the fragment. The shader replaces the traditional fixed rendering pipeline and can realize the relevant calculations in 3D graphics. Due to its editability, it can achieve a variety of image effects without being restricted by the fixed rendering pipeline of the graphics card.

[0052] 10) Image Processing: A technique that uses computers to analyze images to achieve the desired results. Also known as image processing. Image processing generally refers to digital image processing. A digital image is a large two-dimensional array obtained by shooting with industrial cameras, video cameras, scanners and other equipment. The elements of the array are called pixels, and their values ​​are called grayscale values. Image processing technology generally includes three parts: image compression, enhancement and restoration, matching, description and recognition.

[0053] During the implementation of the embodiments of the present application, the applicant discovered that the related technology has the following problems:

[0054] In the related art, image sharpening is performed by sharpening the entire image to the same degree, which results in low distinction between the subject and the background, fails to achieve the effect of highlighting the subject, and results in poor picture quality of the sharpened image.

[0055] The embodiments of the present application provide an image processing method, device, electronic device, computer-readable storage medium and computer program product, which can effectively improve the picture quality presented by the image. The following describes an exemplary application of the image processing system provided by the embodiments of the present application.

[0056] See also Figure 1 , Figure 1 It is a schematic diagram of the architecture of the image processing system 100 provided in an embodiment of the present application. The terminal (terminal 400 is shown as an example) is connected to the server 200 via the network 300. The network 300 can be a wide area network or a local area network, or a combination of the two.

[0057] The terminal 400 is used for the user to use the client 410 to display a target image on a graphical interface 410-1 (graphic interface 410-1 is shown as an example). The terminal 400 and the server 200 are connected to each other via a wired or wireless network.

[0058] In some embodiments, the server 200 may be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (CDN), and big data and artificial intelligence platforms. The terminal 400 may be a smart phone, a tablet computer, a laptop computer, a desktop computer, a smart speaker, a smart TV, a smart watch, a car terminal, etc., but is not limited thereto. The electronic device provided in the embodiment of the present application may be implemented as a terminal or as a server. The terminal and the server may be directly or indirectly connected via wired or wireless communication, which is not limited in the embodiment of the present application.

[0059] In some embodiments, the server 200 performs sharpening processing on the acquired image to be processed to obtain a sharpened image of the image to be processed, performs boundary extension on the sharpened boundary in the image to be processed to obtain a reference sharpened image of the image to be processed, and sends the image to be processed, the sharpened image and the reference sharpened image to the terminal 400. The terminal 400 performs image fusion on the image to be processed, the sharpened image and the reference sharpened image to obtain a target image of the image to be processed.

[0060] In other embodiments, the terminal 400 obtains the image to be processed and sends the image to be processed to the server 200. The server 200 sharpens the obtained image to be processed to obtain a sharpened image of the image to be processed, extends the sharpened boundaries in the image to be processed to obtain a reference sharpened image of the image to be processed, fuses the image to be processed, the sharpened image and the reference sharpened image to obtain a target image of the image to be processed, and sends the target image to the server 200.

[0061] In other embodiments, the embodiments of the present application can be implemented with the aid of cloud technology. Cloud technology refers to a hosting technology that unifies a series of resources such as hardware, software, and networks within a wide area network or a local area network to achieve data computing, storage, processing, and sharing.

[0062] Cloud technology is a general term for network technology, information technology, integration technology, management platform technology, and application technology based on the cloud computing business model. It can form a resource pool that can be used on demand and is flexible and convenient. Cloud computing technology will become an important support. The background services of the technical network system require a large amount of computing and storage resources.

[0063] See also Figure 2 , Figure 2 is a schematic diagram of the structure of an electronic device 500 for image processing provided in an embodiment of the present application, wherein: Figure 2 The electronic device 500 shown may be Figure 1 The server 200 or the terminal 400 in Figure 2 The electronic device 500 shown includes: at least one processor 410, a memory 450, and at least one network interface 420. The various components in the electronic device 500 are coupled together via a bus system 440. It is understood that the bus system 440 is used to achieve connection and communication between these components. In addition to the data bus, the bus system 440 also includes a power bus, a control bus, and a status signal bus. However, for the sake of clarity, the bus system 440 is not described in detail. Figure 2 Various buses are labeled as bus system 440 .

[0064] Processor 410 can be an integrated circuit chip with signal processing capabilities, such as a general-purpose processor, a digital signal processor (DSP), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc., where the general-purpose processor can be a microprocessor or any conventional processor, etc.

[0065] The memory 450 may be removable, non-removable, or a combination thereof. Exemplary hardware devices include solid-state memory, hard disk drives, optical disk drives, etc. The memory 450 may optionally include one or more storage devices that are physically remote from the processor 410.

[0066] The memory 450 includes a volatile memory or a non-volatile memory, and may also include both volatile and non-volatile memories. The non-volatile memory may be a read-only memory (ROM), and the volatile memory may be a random access memory (RAM). The memory 450 described in the embodiments of the present application is intended to include any suitable type of memory.

[0067] In some embodiments, memory 450 can store data to support various operations, examples of which include programs, modules, and data structures, or a subset or superset thereof, as exemplarily described below.

[0068] Operating system 451, including system programs for processing various basic system services and performing hardware-related tasks, such as a framework layer, a core library layer, a driver layer, etc., for implementing various basic services and processing hardware-based tasks;

[0069] The network communication module 452 is used to reach other electronic devices via one or more (wired or wireless) network interfaces 420. Exemplary network interfaces 420 include: Bluetooth, Wireless Fidelity (WiFi), and Universal Serial Bus (USB).

[0070] In some embodiments, the image processing device provided in the embodiments of the present application can be implemented in a software manner. Figure 2 The image processing device 455 stored in the memory 450 is shown, which can be software in the form of a program and a plug-in, etc., including the following software modules: a sharpening module 4551, a boundary extension module 4552, and a fusion module 4553. These modules are logical, so they can be arbitrarily combined or further divided according to the functions implemented. The functions of each module will be described below.

[0071] In other embodiments, the image processing device provided in the embodiments of the present application can be implemented in hardware. As an example, the image processing device provided in the embodiments of the present application can be a processor in the form of a hardware decoding processor, which is programmed to execute the image processing method provided in the embodiments of the present application. For example, the processor in the form of a hardware decoding processor can adopt one or more application-specific integrated circuits (ASICs), DSPs, programmable logic devices (PLDs), complex programmable logic devices (CPLDs), field-programmable gate arrays (FPGAs) or other electronic components.

[0072] In some embodiments, the terminal or server can implement the image processing method provided in the embodiment of the present application by running a computer program or a computer executable instruction. For example, the computer program can be a native program (e.g., a dedicated image processing program) or a software module in the operating system, for example, an image processing module that can be embedded in any program (such as an instant messaging client, an album program, an electronic map client, a navigation client); for example, it can be a native application (APP, Application), that is, a program that needs to be installed in the operating system to run. In short, the above-mentioned computer program can be any form of application, module or plug-in.

[0073] The image processing method provided in the embodiment of the present application will be described in conjunction with the exemplary application and implementation of the server or terminal provided in the embodiment of the present application.

[0074] See also Figure 3 , Figure 3 is a flowchart of the image processing method provided in the embodiment of the present application, which will be combined with Figure 3 Steps 101 to 103 are shown for illustration. The image processing method provided in the embodiment of the present application can be implemented by a server or a terminal alone, or by a server and a terminal in collaboration. The following description will be made using the server alone as an example.

[0075] In step 101, a sharpening process is performed on the acquired image to be processed to obtain a sharpened image of the image to be processed.

[0076] In some embodiments, the sharpened image includes a plurality of sharpened pixels and a plurality of reference pixels, the clarity of the reference pixels is less than the clarity of the sharpened pixels, and the plurality of sharpened pixels constitute a sharpened boundary of the sharpened image.

[0077] In some embodiments, sharpening is also called image sharpening, which is to compensate for the contour of the image, enhance the edge and grayscale jump of the image, and make the image clear. It is divided into two categories: spatial domain processing and frequency domain processing. Image sharpening is to highlight the edge, contour, or characteristics of certain linear target elements of the object on the image. This filtering method increases the contrast between the edge of the object and the surrounding pixels, so it is also called edge enhancement.

[0078] In some embodiments, see Figure 4 , Figure 4 is a flowchart of an image processing method provided in an embodiment of the present application, Figure 3 Step 101 shown may be performed by Figure 4 Steps 1011 to 1013 are implemented as shown.

[0079] In step 1011, the image to be processed is blurred to obtain a blurred image of the image to be processed.

[0080] In some embodiments, the blurring process refers to the process of performing low-frequency processing on the image to be processed. The blurred image obtained by blurring the image to be processed is a low-frequency image of the image to be processed.

[0081] In some embodiments, the blurring of the image to be processed may be achieved by obtaining a low-pass image filter, performing low-frequency filtering on the image to be processed, and obtaining a blurred image corresponding to the image to be processed.

[0082] As an example, the expression of the above blurred image can be:

[0083] S=F(s) (1)

[0084] Among them, S is used to indicate a blurred image, s is used to indicate an image to be processed, and F() is used to indicate a low-pass image filter.

[0085] In step 1012, the image to be processed and the blurred image are subjected to difference processing to obtain a difference image of the image to be processed.

[0086] In some embodiments, the pixels in the image to be processed correspond one-to-one to the pixels in the blurred image.

[0087] In some embodiments, the difference image is a high-frequency image of the image to be processed, and the blurred image is a low-frequency image of the image to be processed.

[0088] In some embodiments, the difference processing refers to a process of subtracting the pixel value of each pixel point of the image to be processed from the pixel value of each pixel point in the blurred image.

[0089] As an example, the expression of the difference image of the above-mentioned image to be processed can be:

[0090] S2=S1-S (2)

[0091] Among them, S2 is used to indicate the difference image of the image to be processed, S1 is used to indicate the difference image, and S is used to indicate the blurred image.

[0092] In some embodiments, the above step 1012 can be implemented as follows: for each pixel point in the image to be processed, subtract the pixel value of the corresponding pixel point in the blurred image from the pixel value of the pixel point to obtain the target pixel value of the pixel point; update the pixel value of each pixel point in the image to be processed to the corresponding target pixel value to obtain a difference image.

[0093] As an example, the expression of the above target pixel value can be:

[0094] X2=X1-X (3)

[0095] Wherein, X2 is used to indicate the target pixel value, X1 is used to indicate the pixel value of the pixel point in the image to be processed, and X is used to indicate the pixel value of the corresponding pixel point in the blurred image.

[0096] In step 1013, the difference image and the image to be processed are combined to perform sharpening processing on the image to be processed to obtain a sharpened image.

[0097] In some embodiments, the pixels in the difference image correspond one-to-one to the pixels in the image to be processed.

[0098] In some embodiments, the above step 1013 can also be implemented in the following manner: obtain the target sharpening coefficient; for each pixel in the difference image, determine the product of the pixel value of the pixel and the sharpening coefficient as the reference pixel value, and sum the reference pixel value with the pixel value of the corresponding pixel in the image to be processed to obtain the summed pixel value; update the pixel value of each pixel in the image to be processed to the corresponding summed pixel value to obtain a sharpened image.

[0099] In some embodiments, the target sharpening coefficient is associated with the image type of the image to be processed. Different image types correspond to different target sharpening coefficients. The value range of the target sharpening coefficient is between 0 and 1. The numerical value of the target sharpening coefficient is related to the sharpening degree of the sharpening processing.

[0100] As an example, the expression of the reference pixel value may be:

[0101] X3=αX2 (4)

[0102] Among them, X3 is used to indicate the reference pixel value, α is used to indicate the target sharpening coefficient, and X2 is used to indicate the pixel value of the pixel point in the difference image.

[0103] As an example, the above expression for summing pixel values ​​can be:

[0104] X4=X3+X1 (5)

[0105] Among them, X4 is used to indicate the summed pixel value, X3 is used to indicate the reference pixel value, and X1 is used to indicate the pixel value of the pixel point in the image to be processed.

[0106] In some embodiments, the above-mentioned acquisition of the target sharpening coefficient can be achieved in the following manner: obtaining the mapping relationship between multiple image types and corresponding sharpening coefficients, and the image type of the image to be processed; from the mapping relationship, determining the target mapping relationship including the image type of the image to be processed, and determining the sharpening coefficient in the target mapping relationship as the target sharpening coefficient.

[0107] In some embodiments, the above-mentioned sharpening coefficient is associated with the image type of the image to be processed. Different image types correspond to different sharpening coefficients. The value range of the sharpening coefficient is greater than 0, and the numerical value of the sharpening coefficient is related to the sharpening degree of the sharpening processing.

[0108] In some embodiments, after the above step 101, the sharpening boundary can also be determined in the following manner: obtain a clarity threshold and the clarity of each pixel in the sharpened image; perform the following processing on each pixel in the sharpened image: when the clarity of the pixel is greater than the clarity threshold, determine the pixel as a sharpened pixel; when the clarity of the pixel is less than or equal to the clarity threshold, determine the pixel as a reference pixel; determine the area formed by the sharpened pixels in the sharpened image as a sharpening boundary.

[0109] In some embodiments, the above-mentioned sharpened boundary refers to an image area composed of sharpened pixels in the sharpened image, the above-mentioned sharpened pixels refer to pixels in the sharpened image whose clarity is greater than a clarity threshold, and the above-mentioned reference pixels refer to pixels in the sharpened image whose clarity is less than a clarity threshold.

[0110] In some embodiments, the area formed by the reference pixels in the sharpened image is determined as a reference boundary, and the union of the reference boundary and the sharpened boundary constitutes the sharpened image.

[0111] In this way, by performing sharpening processing on the image to be processed, a sharpened image corresponding to the image to be processed is obtained, thereby effectively enhancing the clarity of the image to be processed and achieving effective compensation of the contour edge of the image to be processed.

[0112] In step 102, the sharpened boundary in the sharpened image is extended to obtain a reference sharpened image of the image to be processed.

[0113] In some embodiments, the number of sharpened pixels in the sharpened boundary of the reference sharpened image is greater than the number of sharpened pixels in the sharpened boundary of the sharpened image.

[0114] In some embodiments, the above-mentioned boundary extension is used to expand the sharp boundary in the sharpened image so that the number of sharpened pixel points in the sharp boundary of the reference sharpened image is greater than the number of sharpened pixel points in the sharp boundary of the sharpened image.

[0115] In some embodiments, see Figure 5 , Figure 5 is a flowchart of an image processing method provided in an embodiment of the present application, Figure 3 The illustrated step 102 may be performed by Figure 5 Steps 1021 to 1024 are implemented as shown.

[0116] In step 1021, blur processing is performed on the reference pixels in the sharpened image to obtain a first reference image.

[0117] In some embodiments, the blurring process refers to the process of performing low-frequency processing on the reference pixels in the sharpened image.

[0118] In some embodiments, the above-mentioned blurring of the reference pixels in the sharpened image may be implemented in the following manner: obtaining a low-pass image filter, performing low-frequency filtering on the reference pixels in the sharpened image, and obtaining a first reference image.

[0119] As an example, the expression of the first reference image may be:

[0120] C=F(c) (6)

[0121] Wherein, C is used to indicate the first reference image, c is used to indicate the reference pixel point in the sharpened image, and F() is used to indicate the low-pass image filter.

[0122] In some embodiments, the above-mentioned step 1021 can be implemented by the following method: blurring each pixel point in the sharpened image to obtain a sharpened blurred image of the sharpened image, wherein the sharpened blurred image includes sharpened blurred pixel points corresponding one to one to the sharpened pixel points; and enhancing the clarity of each sharpened blurred pixel point in the sharpened blurred image to obtain a first reference image.

[0123] In some embodiments, the above-mentioned clarity enhancement is the inverse process of the blurring process. After blurring all pixels in the sharpened image, the clarity of the sharpened blurred image corresponding to the sharpened pixels is enhanced, thereby blurring the sharpened blurred pixels in the sharpened image except those corresponding to the sharpened pixels. This makes the pixels in the obtained first reference image except the sharpened blurred pixels more blurred than those in the sharpened image, effectively enhancing the contrast between the sharpened boundary and the reference boundary in the sharpened image.

[0124] In step 1022, associated pixel points associated with the sharpened boundary of the first reference image are determined from the first reference pixel points of the first reference image.

[0125] In some embodiments, the first reference pixel points of the first reference image correspond one-to-one to the reference pixel points in the sharpened image.

[0126] In some embodiments, the above-mentioned step 1022 can be implemented as follows: obtain a brightness threshold, and perform the following processing on each first reference pixel in the first reference image: obtain the brightness value of the first reference pixel, and when the brightness value is greater than the brightness threshold, determine the shortest distance between the first reference pixel and the sharpening boundary; when the shortest distance is less than the distance threshold, determine the first reference pixel as an associated pixel.

[0127] In some embodiments, the first reference pixel is screened based on its brightness value. When the brightness value is greater than a brightness threshold, the shortest distance between the first reference pixel and the sharpening boundary is determined. When the shortest distance is less than the distance threshold, the first reference pixel is determined as an associated pixel.

[0128] In some embodiments, determining the shortest distance between the first reference pixel and the sharp boundary can be achieved as follows: for each pixel in the sharp boundary, determining the distance between the pixel and the first reference pixel; determining the shortest distance from the distances of each pixel, and determining the shortest distance as the shortest distance between the first reference pixel and the sharp boundary.

[0129] In this way, the first reference pixel points are screened by the brightness value of the first reference pixel point and the shortest distance between the first reference pixel point and the sharp boundary, and the first reference pixel point whose brightness value is greater than the brightness threshold and the shortest distance between the first reference pixel point and the sharp boundary is less than the distance threshold is determined as the associated pixel point, thereby achieving effective screening of the associated pixel points.

[0130] In step 1023, the area formed by the associated pixels and the sharpened pixels in the first reference image is determined as the extended sharpened boundary of the first reference image.

[0131] In some embodiments, since the area formed by the sharpened pixels in the first reference image is a sharpened boundary, the area formed by the sharpened pixels and the associated pixels in the first reference image is an extended sharpened boundary. Then, it is obvious that the number of pixels in the extended sharpened boundary is greater than the number of pixels in the sharpened boundary, thereby achieving effective extension of the sharpened boundary, so that the obtained reference sharpened image has a stronger sharpening effect.

[0132] In step 1024, the definition of the extended sharpened boundary in the first reference image is enhanced to obtain a reference sharpened image.

[0133] In some embodiments, the clarity enhancement is an inverse process of blurring, so that the clarity of the extended sharp boundary in the obtained reference sharpened image is greater than the clarity of the extended sharp boundary in the first reference image.

[0134] In this way, by extending the sharp boundary in the sharpened image, a reference sharpened image of the image to be processed is obtained, thereby achieving the extension of the sharp boundary, so that the obtained reference image has a more obvious sharpening effect than the sharpened image.

[0135] In step 103, the image to be processed, the sharpened image and the reference sharpened image are fused to obtain a target image of the image to be processed.

[0136] In some embodiments, before the above step 103, the image to be processed can be resized in the following manner: obtain the target size of the set target image and determine the image size of the image to be processed; when the target size is different from the image size, adjust the image size of the image to be processed to the target size to obtain the image to be processed of the target size.

[0137] In some embodiments, since the image to be processed can be displayed on a mobile terminal or a computer terminal, the target size of the target image set above can be the screen size of the mobile terminal or the screen size of the computer terminal where the image to be processed is displayed.

[0138] In some embodiments, when the target size is different from the image size, in order to make the image to be processed fit the target size, the size of the image to be processed may be adjusted to the target size to obtain the image to be processed of the target size.

[0139] In some embodiments, the above step 103 can be implemented in the following manner: performing image fusion on the image to be processed of the target size, the sharpened image and the reference sharpened image to obtain the target image of the image to be processed.

[0140] In some embodiments, see Figure 6 , Figure 6is a flowchart of an image processing method provided in an embodiment of the present application, Figure 3 The illustrated step 103 may be performed by Figure 6 Steps 1031 to 1033 are shown to be implemented.

[0141] In step 1031, for each pixel point in the to-be-processed image of the target size, a first associated pixel point corresponding to the pixel point is determined from the sharpened image, and a second associated pixel point corresponding to the pixel point is determined from the reference sharpened image.

[0142] In some embodiments, the image size of the target size of the image to be processed is inconsistent with the image size of the sharpened image. When the image size of the target size of the image to be processed is consistent with the image size of the sharpened image, the number of pixels in the target size of the image to be processed and the sharpened image is the same. When the image size of the target size of the image to be processed and the sharpened image is inconsistent, the number of pixels of each is different, that is, the pixel points of the target size of the image to be processed and the sharpened image are not one-to-one corresponding. Then, the number of first associated pixel points determined in the sharpened image to correspond to the pixel points is at least one, that is, the pixel points in the target size of the image to be processed correspond to at least one first associated pixel point.

[0143] In some embodiments, the image size of the target size of the image to be processed is inconsistent with that of the reference sharpened image. When the image size of the target size of the image to be processed is consistent with that of the reference sharpened image, the number of pixels in the target size of the image to be processed and the reference sharpened image is the same. When the image size of the target size of the image to be processed and the reference sharpened image is inconsistent, the number of pixels of each is different, that is, the pixel points of the target size of the image to be processed and the reference sharpened image are not one-to-one corresponding. Then, the number of second associated pixel points determined in the reference sharpened image to correspond to the pixel point is at least one, that is, the pixel point in the target size of the image to be processed corresponds to at least one second associated pixel point.

[0144] In step 1032, the pixel values ​​corresponding to the pixel point, the first associated pixel point and the second associated pixel point are merged to obtain an updated pixel value of the pixel point.

[0145] In some embodiments, the above step 1032 may be implemented in the following manner: pixel values ​​corresponding to the pixel point, the first associated pixel point, and the second associated pixel point are added together to obtain an updated pixel value of the pixel point.

[0146] As an example, the expression for updating the pixel value of a pixel point can be:

[0147] Z=Z1+Z2+Z3 (7)

[0148] Among them, Z is used to indicate the updated pixel value of the pixel point, Z1 is used to indicate the pixel value corresponding to the pixel point, Z2 is used to indicate the pixel value corresponding to the first associated pixel point, and Z3 is used to indicate the pixel value corresponding to the second associated pixel point.

[0149] In step 1033, the pixel value of each pixel in the to-be-processed image of the target size is updated to the updated pixel value to obtain the target image.

[0150] As an example, the image to be processed of the target size includes pixel A, pixel B, pixel C and pixel D. The pixel value of pixel A in the image to be processed of the target size is updated to the corresponding updated pixel value, the pixel value of pixel B in the image to be processed of the target size is updated to the corresponding updated pixel value, the pixel value of pixel C in the image to be processed of the target size is updated to the corresponding updated pixel value, and the pixel value of pixel D in the image to be processed of the target size is updated to the corresponding updated pixel value to obtain the target image.

[0151] In this way, by performing sharpening processing on the acquired image to be processed, a sharpened image of the image to be processed is obtained, the sharpened boundary in the sharpened image is extended to obtain a reference sharpened image of the image to be processed, and the image to be processed, the sharpened image and the reference sharpened image are fused to obtain a target image of the image to be processed. The target image obtained by image fusion retains the original image information of the image to be processed on the one hand, and retains the enhanced sharpening effect of the sharpened image and the reference sharpened image on the other hand, thereby effectively improving the picture quality presented by the obtained target image.

[0152] The following is an explanation of an exemplary application of the embodiments of the present application in an actual online game application scenario.

[0153] With the development of image rendering technology, in order to simulate more realistic virtual scenes, virtual scene images are usually sharpened to improve the expressiveness and authenticity of the image. Sharpening is an effective image enhancement method to improve the subjective quality of video images. Sharpening in the preprocessing stage before encoding game videos can effectively highlight video textures and edges and significantly improve video quality.

[0154] See also Figure 7 , Figure 7This is a schematic diagram of the effect of the image processing method provided by the embodiment of the present application. After the game screen image is processed by the image processing method provided by the embodiment of the present application, the visual art information received by the game player during the game will be more delicate and rich, because the range of high-frequency color changes is sharpened and the contrast is increased, such as the gaps between door panels and wooden boards, and the concave and convex positions that express the structure of objects. At the same time, in order to avoid a large amount of sharpening that causes noise in the picture or more obvious jagged edges, the processing of the low-frequency color change area is filtered out, and the fuzzy transition is still smooth.

[0155] In some embodiments, see Figure 8 , Figure 8 is a flow chart of the image processing method provided in the embodiment of the present application. The image processing method provided in the embodiment of the present application can be Figure 8 Steps 201 to 208 are shown to be implemented.

[0156] In step 201, an image to be processed is obtained.

[0157] In step 202, the image to be processed is resized to obtain a first image.

[0158] In some embodiments, the original texture, the original resolution texture, comes out with a 2048 version. The goal is to use a 512x512 texture in the mobile package, so here is a batch script in Unity to blit it into a 512 texture (temporarily referred to as Figure A).

[0159] In step 203, the image to be processed is sharpened to obtain a sharpened area.

[0160] In step 204, the image to be processed is globally sharpened to obtain a sharpened original image.

[0161] In step 205, the black and white area map is sharpened.

[0162] In step 206, the sharpened black and white area map is blurred twice in the vertical and horizontal directions to obtain a blurred black and white area map.

[0163] In step 207, the black-white contrast is increased to obtain a second image.

[0164] In step 208, the first image, the second image and the sharpened original image are fused to obtain a target image.

[0165] In some embodiments, a sharpened large image is saved for backup (temporarily referred to as Figure B), and then the output of the sharpening shader is slightly modified to process the original large image to obtain a black and white sharpened area, which is then processed with a blur shader to obtain a blurred map, and then input into a special shader to continue processing to discard most of the low-frequency fragmented sharpened areas, retain and strengthen the sharpened areas. What this special shader does is to discard some dark grays, and uniformly brighten the light gray, white and other areas into white areas. Among them, the blur processing in this application embodiment uses ordinary BoxBlur, and then uses the following formula to highlight the mask result, and weakens the fragmented areas by squaring the black and white results: Mask = smoothstep(0, 0.5, pow(blurMask, 2)).

[0166] In some embodiments, see Fig. 9 , Fig. 9 It is a schematic diagram of the effect of the image processing method provided in an embodiment of the present application, in which the image to be processed is sharpened to obtain a filtered image, the filtered image is blurred to obtain a blurred image, the low-frequency area of ​​the blurred image is weakened, and the high-frequency area is strengthened to obtain a target image.

[0167] In some embodiments, see Fig. 9 , Fig. 9 It is a schematic diagram of the effect of the image processing method provided by the embodiment of the present application, and the target image is finally obtained. The details are retained and the look and feel is sharp in places with large color changes, and no processing is done in places with small color changes. The most direct screen problem of low-end mobile phones is severe jaggedness and blurred textures. Under the condition that the package, anti-aliasing restrictions, and rendering pipeline remain unchanged, try to improve the performance that can be presented by low-end mobile phones. In addition, compared with directly sharpening the texture, the target image obtained by the embodiment of the present application will be more in line with expectations, and most areas of the texture can be sharpened, there is no fragmentation, and the picture quality is effectively improved.

[0168] In some embodiments, the sharpening area can be modified, and different parameters can be input according to different types of maps to change the sharpening area. It is also possible to consider adding brightness judgment. In addition to the above sharpening area, brightness judgment is added, and the bright area has weaker sharpening and the dark area has more obvious sharpening stripes. The relevant judgment parameters are input according to the art style.

[0169] Thus, by performing sharpening processing on the acquired image to be processed, a sharpened image of the image to be processed is obtained, the sharpened boundary in the sharpened image is extended to obtain a reference sharpened image of the image to be processed, and the image to be processed, the sharpened image and the reference sharpened image are fused to obtain a target image of the image to be processed. Thus, the target image obtained by image fusion retains the original image information of the image to be processed on the one hand, and retains the enhanced sharpening effect of the sharpened image and the reference sharpened image on the other hand, thereby effectively improving the picture quality presented by the obtained target image.

[0170] It is understandable that in the embodiments of the present application, when data related to the image to be processed is involved, when the embodiments of the present application are applied to specific products or technologies, user permission or consent is required, and the collection, use and processing of the relevant data need to comply with relevant laws, regulations and standards of relevant countries and regions.

[0171] The following is a description of an exemplary structure of the image processing device 455 provided in the embodiment of the present application implemented as a software module. In some embodiments, Figure 2 As shown, the software modules stored in the image processing device 455 of the memory 450 may include: a sharpening module 4551, used to perform sharpening processing on the acquired image to be processed to obtain a sharpened image of the image to be processed, wherein the sharpened image includes multiple sharpened pixels and multiple reference pixels; wherein the clarity of the reference pixels is less than the clarity of the sharpened pixels, and the multiple sharpened pixels constitute the sharpened boundary of the sharpened image; a boundary extension module 4552, used to perform boundary extension on the sharpened boundary in the sharpened image to obtain a reference sharpened image of the image to be processed, wherein the number of sharpened pixels in the sharpened boundary of the reference sharpened image is greater than the number of sharpened pixels in the sharpened boundary of the sharpened image; a fusion module 4553, used to perform image fusion on the image to be processed, the sharpened image and the reference sharpened image to obtain a target image of the image to be processed.

[0172] In some embodiments, the above-mentioned sharpening module is also used to perform sharpening processing on the acquired image to be processed to obtain a sharpened image of the image to be processed, wherein the sharpened image includes multiple sharpened pixels and multiple reference pixels; wherein the clarity of the reference pixels is less than the clarity of the sharpened pixels, and the multiple sharpened pixels constitute the sharpened boundary of the sharpened image; the sharpened boundary in the sharpened image is extended to obtain a reference sharpened image of the image to be processed, and the number of sharpened pixels in the sharpened boundary of the reference sharpened image is greater than the number of sharpened pixels in the sharpened boundary of the sharpened image; the image to be processed, the sharpened image and the reference sharpened image are image-fused to obtain a target image of the image to be processed.

[0173] In some embodiments, the pixels in the image to be processed correspond one-to-one to the pixels in the blurred image, and the sharpening module is further used to, for each pixel in the image to be processed, subtract the pixel value of the corresponding pixel in the blurred image from the pixel value of the pixel to obtain the target pixel value of the pixel; and update the pixel value of each pixel in the image to be processed to the corresponding target pixel value to obtain the difference image.

[0174] In some embodiments, the pixels in the difference image correspond one-to-one to the pixels in the image to be processed, and the sharpening module is further used to obtain a target sharpening coefficient, which is associated with the image type of the image to be processed; for each pixel in the difference image, the product of the pixel value of the pixel and the sharpening coefficient is determined as a reference pixel value, and the reference pixel value is summed with the pixel value of the corresponding pixel in the image to be processed to obtain a summed pixel value; the pixel value of each pixel in the image to be processed is updated to the corresponding summed pixel value to obtain the sharpened image.

[0175] In some embodiments, the above-mentioned sharpening module is also used to obtain the mapping relationship between multiple image types and corresponding sharpening coefficients, and the image type of the image to be processed; from the mapping relationship, determine the target mapping relationship including the image type of the image to be processed, and determine the sharpening coefficient in the target mapping relationship as the target sharpening coefficient.

[0176] In some embodiments, the above-mentioned image processing device also includes: a boundary determination module, which is used to obtain a clarity threshold and the clarity of each pixel in the sharpened image; and perform the following processing on each pixel in the sharpened image: when the clarity of the pixel is greater than the clarity threshold, the pixel is determined as the sharpened pixel; when the clarity of the pixel is less than or equal to the clarity threshold, the pixel is determined as the reference pixel; and the area formed by each of the sharpened pixels in the sharpened image is determined as the sharpened boundary.

[0177] In some embodiments, the above-mentioned boundary extension module is also used to blur the reference pixel points in the sharpened image to obtain a first reference image; determine the associated pixel points associated with the sharpened boundary of the first reference image from the first reference pixel points of the first reference image, and the first reference pixel points of the first reference image correspond one-to-one to the reference pixel points in the sharpened image; determine the area formed by the associated pixel points and the sharpened pixel points in the first reference image as the extended sharpened boundary of the first reference image; enhance the clarity of the extended sharpened boundary in the first reference image to obtain the reference sharpened image.

[0178] In some embodiments, the above-mentioned boundary extension module is also used to blur each pixel point in the sharpened image to obtain a sharpened blurred image of the sharpened image, wherein the sharpened blurred image includes sharpened blurred pixel points corresponding one to one to the sharpened pixel points; and enhance the clarity of each of the sharpened blurred pixel points in the sharpened blurred image to obtain the first reference image.

[0179] In some embodiments, the above-mentioned boundary extension module is also used to obtain a brightness threshold, and perform the following processing on each of the first reference pixel points in the first reference image: obtain the brightness value of the first reference pixel point, and when the brightness value is greater than the brightness threshold, determine the shortest distance between the first reference pixel point and the sharpening boundary; when the shortest distance is less than the distance threshold, determine the first reference pixel point as the associated pixel point.

[0180] In some embodiments, the above-mentioned image processing device also includes: a size adjustment module, which is used to obtain the target size of the set target image and determine the image size of the image to be processed; when the target size is different from the image size, the image size of the image to be processed is adjusted to the target size to obtain the image to be processed of the target size; the above-mentioned fusion module is also used to fuse the image to be processed of the target size, the sharpened image and the reference sharpened image to obtain the target image of the image to be processed.

[0181] In some embodiments, the above-mentioned fusion module is also used to determine, for each pixel point in the image to be processed of the target size, a first associated pixel point corresponding to the pixel point from the sharpened image, determine a second associated pixel point corresponding to the pixel point from the reference sharpened image, and fuse the pixel values ​​corresponding to the pixel point, the first associated pixel point and the second associated pixel point respectively to obtain an updated pixel value of the pixel point; update the pixel value of each pixel point in the image to be processed of the target size to the updated pixel value to obtain the target image.

[0182] The embodiment of the present application provides a computer program product, which includes a computer program or a computer executable instruction, and the computer program or the computer executable instruction is stored in a computer-readable storage medium. The processor of the electronic device reads the computer executable instruction from the computer-readable storage medium, and the processor executes the computer executable instruction, so that the electronic device performs the above-mentioned image processing method of the embodiment of the present application.

[0183] The present application embodiment provides a computer-readable storage medium storing computer-executable instructions, wherein the computer-executable instructions are stored. When the computer-executable instructions are executed by a processor, the processor will be caused to execute the image processing method provided by the present application embodiment, for example, Figure 3 The image processing method is shown.

[0184] In some embodiments, the computer-readable storage medium may be a memory such as FRAM, ROM, PROM, EPROM, EEP ROM, flash memory, magnetic surface storage, optical disk, or CD-ROM; or it may be various electronic devices including one or any combination of the above memories.

[0185] In some embodiments, computer executable instructions may be in the form of a program, software, software module, script or code, written in any form of programming language (including compiled or interpreted languages, or declarative or procedural languages), and may be deployed in any form, including as a stand-alone program or as a module, component, subroutine or other unit suitable for use in a computing environment.

[0186] As an example, computer-executable instructions may, but need not, correspond to a file in a file system, may be stored as part of a file storing other programs or data, such as in one or more scripts in a HyperText Markup Language (HTML) document, in a single file dedicated to the program in question, or in multiple coordinated files (e.g., files storing one or more modules, subroutines, or code portions).

[0187] As an example, computer executable instructions may be deployed to be executed on one electronic device, or on multiple electronic devices located at one site, or on multiple electronic devices distributed at multiple sites and interconnected by a communication network.

[0188] In summary, the embodiments of the present application have the following beneficial effects:

[0189] (1) The acquired image to be processed is sharpened to obtain a sharpened image of the image to be processed, the sharpened boundary in the sharpened image is extended to obtain a reference sharpened image of the image to be processed, and the image to be processed, the sharpened image and the reference sharpened image are fused to obtain a target image of the image to be processed. In this way, the target image obtained by image fusion retains the original image information of the image to be processed on the one hand, and retains the enhanced sharpening effect of the sharpened image and the reference sharpened image on the other hand, thereby effectively improving the picture quality presented by the obtained target image.

[0190] (2) By extending the sharp boundary in the sharpened image, a reference sharpened image of the image to be processed is obtained, thereby achieving the extension of the sharp boundary, so that the sharpening effect of the obtained reference image is more obvious than that of the sharpened image.

[0191] (3) The first reference pixel points are screened by the brightness value of the first reference pixel point and the shortest distance between the first reference pixel point and the sharp boundary, and the first reference pixel point whose brightness value is greater than the brightness threshold and whose shortest distance between the first reference pixel point and the sharp boundary is less than the distance threshold is determined as the associated pixel point, thereby achieving effective screening of the associated pixel points.

[0192] (4) Clarity enhancement is the inverse process of blurring. After blurring all pixels in the sharpened image, the sharpened blurred image corresponding to the sharpened pixels is enhanced to achieve blurring of the sharpened blurred pixels except those corresponding to the sharpened pixels in the sharpened image. The pixels in the obtained first reference image except for the sharpened blurred pixels are more blurred than those in the sharpened image, thereby effectively enhancing the contrast between the sharpened boundary and the reference boundary in the sharpened image.

[0193] (5) By performing sharpening processing on the image to be processed, a sharpened image corresponding to the image to be processed is obtained, thereby effectively enhancing the clarity of the image to be processed and achieving effective compensation of the contour edge of the image to be processed.

[0194] (6) In order to simulate a more realistic virtual scene, the virtual scene image is usually sharpened to improve the image's expressiveness and authenticity. Sharpening is an effective image enhancement method to improve the subjective quality of video images. Sharpening in the preprocessing stage before encoding game videos can effectively highlight video textures and edges, significantly improving video quality.

[0195] (7) Details are preserved and the look is sharp in places with large color changes, and no processing is done in places with small color changes. The most direct screen problem of low-end mobile phones is severe aliasing and blurred textures. Under the condition that the package, anti-aliasing restrictions, and rendering pipeline remain unchanged, the performance that can be presented by low-end mobile phones is improved to the greatest extent possible. In addition, compared with directly sharpening the texture, the target image obtained by the embodiment of the present application will be more in line with expectations, and most areas of the texture can be sharpened without fragmentation, effectively improving the picture quality.

[0196] The above is only an embodiment of the present application and is not intended to limit the protection scope of the present application. Any modifications, equivalent substitutions and improvements made within the spirit and scope of the present application are included in the protection scope of the present application.

Claims

1. An image processing method, characterized in that: The method comprises: Performing sharpening processing on the acquired image to be processed to obtain a sharpened image of the image to be processed, wherein the sharpened image includes a plurality of sharpened pixel points and a plurality of reference pixel points; The clarity of the reference pixel is smaller than the clarity of the sharpened pixel, and the plurality of sharpened pixel constitutes a sharpened boundary of the sharpened image; Performing boundary extension on the sharpened boundary in the sharpened image to obtain a reference sharpened image of the image to be processed, wherein the number of sharpened pixel points in the sharpened boundary of the reference sharpened image is greater than the number of sharpened pixel points in the sharpened boundary of the sharpened image; The image to be processed, the sharpened image and the reference sharpened image are fused to obtain a target image of the image to be processed.

2. The method according to claim 1, characterized in that The step of performing sharpening processing on the acquired image to be processed to obtain a sharpened image of the image to be processed includes: Performing blur processing on the image to be processed to obtain a blurred image of the image to be processed; Performing difference processing on the image to be processed and the blurred image to obtain a difference image of the image to be processed; The difference image and the image to be processed are combined to perform sharpening processing on the image to be processed to obtain the sharpened image.

3. The method according to claim 2, characterized in that The pixel points in the image to be processed correspond to the pixel points in the blurred image one by one, and performing difference processing on the image to be processed and the blurred image to obtain the difference image of the image to be processed includes: For each pixel in the image to be processed, subtract the pixel value of the corresponding pixel in the blurred image from the pixel value of the pixel to obtain a target pixel value of the pixel; The pixel value of each pixel point in the image to be processed is updated to the corresponding target pixel value to obtain the difference image.

4. The method according to claim 2, characterized in that: The pixel points in the difference image correspond to the pixel points in the image to be processed one by one, and the sharpening process is performed on the image to be processed by combining the difference image and the image to be processed to obtain the sharpened image, including: Acquire a target sharpening coefficient, where the target sharpening coefficient is associated with the image type of the image to be processed; For each pixel in the difference image, multiplying the pixel value of the pixel by the sharpening coefficient is determined as a reference pixel value, and summing the reference pixel value with the pixel value of the corresponding pixel in the image to be processed to obtain a summed pixel value; The pixel value of each pixel point in the image to be processed is updated to the corresponding summed pixel value to obtain the sharpened image.

5. The method according to claim 4, characterized in that The step of obtaining a target sharpening coefficient comprises: Acquire mapping relationships between a plurality of image types and corresponding sharpening coefficients, and the image type of the image to be processed; A target mapping relationship including the image type of the image to be processed is determined from the mapping relationship, and the sharpening coefficient in the target mapping relationship is determined as the target sharpening coefficient.

6. The method according to claim 1, characterized in that After performing sharpening processing on the acquired image to be processed to obtain a sharpened image of the image to be processed, the method further includes: Obtaining a clarity threshold and the clarity of each pixel in the sharpened image; The following processing is performed for each pixel in the sharpened image: when the clarity of the pixel is greater than the clarity threshold, the pixel is determined as the sharpening pixel; when the clarity of the pixel is less than or equal to the clarity threshold, the pixel is determined as the reference pixel; The area formed by the sharpened pixel points in the sharpened image is determined as the sharpening boundary.

7. The method according to claim 1, characterized in that The step of extending the sharpened boundary in the sharpened image to obtain a reference sharpened image of the image to be processed includes: Performing blur processing on reference pixels in the sharpened image to obtain a first reference image; Determining, from first reference pixels of the first reference image, associated pixels associated with a sharpened boundary of the first reference image, wherein the first reference pixels of the first reference image correspond one-to-one to the reference pixels in the sharpened image; Determine the area formed by the associated pixel points and the sharpened pixel points in the first reference image as an extended sharpening boundary of the first reference image; The extended sharpened boundary in the first reference image is enhanced in clarity to obtain the reference sharpened image.

8. The method according to claim 7, characterized in that The blurring of the reference pixel points in the sharpened image to obtain the first reference image includes: Performing blur processing on each pixel point in the sharpened image to obtain a sharpened blurred image of the sharpened image, wherein the sharpened blurred image includes sharpened blurred pixel points corresponding to the sharpened pixel points one by one; The clarity of each of the sharpened blurred pixels in the sharpened blurred image is enhanced to obtain the first reference image.

9. The method according to claim 7, characterized in that: The step of determining, from the first reference pixel points of the first reference image, the associated pixel points associated with the sharpened boundary of the first reference image comprises: Obtain a brightness threshold, and perform the following processing on each of the first reference pixels in the first reference image: Acquire a brightness value of the first reference pixel, and when the brightness value is greater than the brightness threshold, determine the shortest distance between the first reference pixel and the sharpening boundary; When the shortest distance is less than a distance threshold, the first reference pixel is determined as the associated pixel.

10. The method according to claim 1, characterized in that Before fusing the image to be processed, the sharpened image and the reference sharpened image to obtain the target image of the image to be processed, the method further includes: Acquire the set target size of the target image, and determine the image size of the image to be processed; When the target size is different from the image size, adjusting the image size of the image to be processed to the target size to obtain the image to be processed of the target size; The step of fusing the image to be processed, the sharpened image and the reference sharpened image to obtain a target image of the image to be processed includes: The image to be processed of the target size, the sharpened image and the reference sharpened image are fused to obtain a target image of the image to be processed.

11. The method according to claim 10, characterized in that The step of fusing the image to be processed of the target size, the sharpened image and the reference sharpened image to obtain the target image of the image to be processed comprises: For each pixel point in the to-be-processed image of the target size, determine a first associated pixel point corresponding to the pixel point from the sharpened image, determine a second associated pixel point corresponding to the pixel point from the reference sharpened image, and fuse pixel values ​​corresponding to the pixel point, the first associated pixel point, and the second associated pixel point, respectively, to obtain an updated pixel value of the pixel point; The pixel value of each pixel in the to-be-processed image of the target size is updated to the updated pixel value to obtain the target image.

12. An image processing device, characterized in that: The device comprises: A sharpening module, used for performing sharpening processing on the acquired image to be processed to obtain a sharpened image of the image to be processed, wherein the sharpened image includes a plurality of sharpened pixels and a plurality of reference pixels; wherein the clarity of the reference pixels is less than the clarity of the sharpened pixels, and the plurality of sharpened pixels constitute a sharpening boundary of the sharpened image; A boundary extension module, used for performing boundary extension on the sharp boundary in the sharpened image to obtain a reference sharpened image of the image to be processed, wherein the number of sharpened pixel points in the sharp boundary of the reference sharpened image is greater than the number of sharpened pixel points in the sharp boundary of the sharpened image; The fusion module is used to fuse the image to be processed, the sharpened image and the reference sharpened image to obtain a target image of the image to be processed.

13. An electronic device, characterized in that: The electronic device comprises: A memory for storing computer executable instructions or computer programs; A processor, configured to implement the image processing method according to any one of claims 1 to 11 when executing the computer executable instructions or computer programs stored in the memory.

14. A computer-readable storage medium storing computer-executable instructions, characterized in that: When the computer executable instructions are executed by a processor, the image processing method according to any one of claims 1 to 11 is implemented.

15. A computer program product comprising a computer program or computer executable instructions, characterized in that: When the computer program or computer executable instructions are executed by a processor, the image processing method according to any one of claims 1 to 11 is implemented.

Images