Image processing method and device, computer equipment and storage medium

Abstract

The invention provides an image processing method and device, computer equipment and a storage medium, and belongs to the technical field of image processing. The method comprises the steps that a second texture image is generated based on a first texture image, and the resolution ratio of the second texture image is lower than that of the first texture image; the second texture image is subjected to up-sampling, a third texture image is obtained, the resolution ratio of the third texture image is higher than that of the second texture image, and the resolution ratio of the third texture image is equal to that of the first texture image; determining an image gap based on the first texture image and the third texture image; and adjusting the pixel values of the pixel points in the second texture image by taking minimization of the image gap as a target. According to the technical scheme, the generated second texture image with the lower resolution can better reserve detail information in the first texture image, and the image processing quality when the image resolution is reduced is improved.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to an image processing method, apparatus, computer equipment, and storage medium. Background Art

[0002] On low-end devices or with limited resources, it's often necessary to reduce the resolution of texture images to accommodate different scenarios or device resolution requirements, or to use lower-resolution texture images to save memory. Reducing the resolution of texture images is a key area of ​​research in this field.

[0003] Currently, a bilinear interpolation algorithm is commonly used, that is, a new pixel value is generated by linearly weighting multiple adjacent pixel values, and the new pixel value replaces the original pixel values, thereby achieving the purpose of reducing the resolution.

[0004] However, the new pixel values ​​in the above technical solution are limited by the range of surrounding pixel values, which may cause blurring, distortion or jagged edges when processing the edges of objects in the image, resulting in loss of image details and affecting the visual effect, that is, poor image processing quality. Summary of the Invention

[0005] The embodiments of the present application provide an image processing method, apparatus, computer device, and storage medium, so that a second texture image with a lower resolution generated can better preserve the detail information in the first texture image, thereby improving the image processing quality when reducing the image resolution. The technical solution is as follows:

[0006] In one aspect, an image processing method is provided, the method comprising:

[0007] generating a second texture image based on the first texture image, wherein the resolution of the second texture image is lower than the resolution of the first texture image;

[0008] Upsampling the second texture image to obtain a third texture image, wherein a resolution of the third texture image is higher than a resolution of the second texture image and a resolution of the third texture image is equal to a resolution of the first texture image;

[0009] determining an image gap based on the first texture image and the third texture image, wherein the image gap is used to represent a difference between the first texture image and the third texture image;

[0010] With the goal of minimizing the image gap, the pixel values ​​of the pixels in the second texture image are adjusted.

[0011] In another aspect, an image processing apparatus is provided, the apparatus comprising:

[0012] a generating module, configured to generate a second texture image based on the first texture image, wherein the resolution of the second texture image is lower than that of the first texture image;

[0013] a processing module, configured to upsample the second texture image to obtain a third texture image, wherein the resolution of the third texture image is higher than that of the second texture image and the resolution of the third texture image is equal to that of the first texture image;

[0014] a determining module, configured to determine an image gap based on the first texture image and the third texture image, wherein the image gap is used to represent a difference between the first texture image and the third texture image;

[0015] The first adjustment module is configured to adjust the pixel values ​​of the pixels in the second texture image with the goal of minimizing the image gap.

[0016] In some embodiments, the processing module includes:

[0017] a first determining unit configured to determine, in a process of calculating a pixel value of any pixel point in the third texture image, a plurality of first pixel points from the second texture image based on the pixel point, where positional relationships between the plurality of first pixel points and the pixel satisfy a condition;

[0018] The second determining unit is configured to determine the pixel value of the pixel point in the third texture image based on the pixel values ​​of the plurality of first pixel points.

[0019] In some embodiments, the second determination unit is used to map the pixel point in the third texture image to the second texture image to obtain a second pixel point in the process of calculating the pixel value of any pixel point in the third texture image; for any first pixel point among the multiple first pixel points, determine the weight of the first pixel point based on the position between the first pixel point and the second pixel point; and based on the weights of the multiple first pixel points, perform weighted summation on the pixel values ​​of the multiple first pixel points to obtain the pixel value of the pixel point in the third texture image.

[0020] In some embodiments, the pixels in the first texture image correspond to the pixels in the third texture image in a one-to-one manner;

[0021] The determination module is used to determine, for any pixel point in the third texture image, a third pixel point corresponding to the pixel point from the first texture image based on the position of the pixel point; determine a pixel gap between the pixel point and the third pixel point based on the pixel value of the pixel point and the pixel value of the third pixel point, where the pixel value of the pixel point includes three pixel components of red, green and blue, and the pixel gap is used to represent the difference in pixel values ​​between the pixel points in the three color channels of red, green and blue; and determine the image gap based on the pixel gaps corresponding to multiple pixel points in the third texture image.

[0022] In some embodiments, the first adjustment module includes:

[0023] a third determining unit configured to determine, for any pixel point in the second texture image, a gradient of the pixel point based on the image difference and the pixel value of the pixel point when the image difference is not less than a difference threshold, wherein the gradient is used to represent an adjustment direction of the pixel value in the second texture image when reducing the image difference;

[0024] An adjustment unit is used to adjust the pixel value of the pixel point based on the gradient of the pixel point.

[0025] In some embodiments, the third determination unit is used to determine, for any pixel point in the second texture image, multiple fourth pixel points from the third texture image based on the pixel point when the image gap is not less than the gap threshold, and the multiple fourth pixel points are used to calculate the pixel value of the pixel point when generating the second texture image based on the third texture image; determine a first gradient based on the image gap and the pixel values ​​of the multiple fourth pixel points, the first gradient being used to represent the adjustment direction of the pixel value in the third texture image when reducing the image gap; determine a second gradient based on the pixel value of the pixel point and the pixel values ​​of the multiple fourth pixel points; the second gradient is used to represent the adjustment direction of the pixel value in the second texture image when generating the third texture image based on the second texture image; determine the gradient of the pixel point in the second texture image based on the first gradient and the second gradient.

[0026] In some embodiments, the adjustment unit is used to weight the first step size based on the gradient of the pixel point to obtain a second step size, where the first step size is used to represent the basic increment of each adjustment of the pixel value; and based on the second step size, the pixel value of the pixel point is adjusted.

[0027] In some embodiments, the determining module is further configured to determine a new image gap based on the adjusted second texture image;

[0028] The device further comprises:

[0029] The second adjustment module is configured to adjust the first step length when the image gap becomes larger.

[0030] On the other hand, a computer device is provided, which includes a processor and a memory, wherein the memory is used to store at least one computer program, and the at least one computer program is loaded and executed by the processor to implement the image processing method in the embodiment of the present application.

[0031] On the other hand, a computer-readable storage medium is provided, in which at least one computer program is stored. The at least one computer program is loaded and executed by a processor to implement the image processing method in the embodiment of the present application.

[0032] On the other hand, a computer program product is provided, including a computer program, which is stored in a computer-readable storage medium. A processor of a computer device reads the computer program from the computer-readable storage medium, and the processor executes the computer program, so that the computer device performs the image processing method provided in the above-mentioned various aspects or various optional implementations of various aspects.

[0033] An embodiment of the present application provides an image processing method. In the process of reducing the resolution of a first texture image, a second texture image with a lower resolution is generated and upsampled to restore the second texture image with the lower resolution to a third texture image with the same resolution as the first texture image. Since some detail information (such as the edges of objects in the texture image) is lost when the resolution of the original texture image is reduced, even if the generated texture image with the lower resolution is restored to the resolution of the original texture image, the lost detail information is not restored. Therefore, by calculating the difference between the first texture image and the third texture image, the detail information lost when the resolution of the first texture image is reduced can be accurately determined. Then, by adjusting the pixel values ​​of pixels in the second texture image to minimize the image difference, the detail information lost when the resolution of the first texture image is reduced can be minimized. In other words, the generated second texture image with the lower resolution can better retain the detail information in the first texture image, thereby ensuring the visual effect of the second texture image and improving the image processing quality when the image resolution is reduced. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0035] Figure 1 is a schematic diagram of an implementation environment of an image processing method provided according to an embodiment of the present application;

[0036] Figure 2 is a flowchart of an image processing method provided according to an embodiment of the present application;

[0037] Figure 3 is a flowchart of another image processing method provided according to an embodiment of the present application;

[0038] Figure 4 is a schematic diagram of upsampling provided according to an embodiment of the present application;

[0039] Figure 5 A schematic diagram of determining a fourth pixel point is provided according to an embodiment of the present application;

[0040] Figure 6 is a flowchart of another image processing method provided according to an embodiment of the present application;

[0041] Figure 7 is a comparison chart of image processing effects provided according to an embodiment of the present application;

[0042] Figure 8 is a block diagram of an image processing device provided according to an embodiment of the present application;

[0043] Figure 9 is a block diagram of another image processing device provided according to an embodiment of the present application;

[0044] Figure 10 This is a structural block diagram of a terminal provided according to an embodiment of the present application;

[0045] Figure 11 It is a structural diagram of a server provided according to an embodiment of the present application. DETAILED DESCRIPTION

[0046] In order to make the objectives, technical solutions and advantages of this application clearer, the implementation methods of this application will be further described in detail below with reference to the accompanying drawings.

[0047] In this application, the terms "first", "second", etc. are used to distinguish identical or similar items with substantially the same effects and functions. It should be understood that there is no logical or temporal dependency between "first", "second", and "nth", nor is there any limitation on the quantity and execution order.

[0048] In the present application, the term "at least one" means one or more, and the term "plurality" means two or more.

[0049] It should be noted that the information (including but not limited to user device information, user personal information, etc.), data (including but not limited to data used for analysis, storage, display, etc.), and signals involved in this application are all authorized by the user or fully authorized by all parties, and the collection, use, and processing of relevant data must comply with the relevant laws, regulations, and standards of the relevant countries and regions. For example, the texture images involved in this application were obtained with full authorization.

[0050] For ease of understanding, the terms involved in this application are explained below.

[0051] Artificial Intelligence (AI) refers to the theories, methods, techniques, and application systems that use digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, to perceive the environment, acquire knowledge, and use that knowledge to achieve optimal results. In other words, AI is a comprehensive technology within computer science that seeks to understand the essence of intelligence and produce new intelligent machines that can respond in a manner similar to human intelligence. AI also studies the design principles and implementation methods of various intelligent machines, enabling them to possess the capabilities of perception, reasoning, and decision-making.

[0052] Artificial intelligence (AI) technology is a comprehensive discipline encompassing a wide range of fields, encompassing both hardware and software technologies. Foundational AI technologies generally include sensors, specialized AI chips, cloud computing, distributed storage, big data processing, pre-trained models, operating / interaction systems, and mechatronics. Pre-trained models, also known as large models or basic models, can be fine-tuned and widely applied to downstream tasks across various AI disciplines. AI software technologies primarily encompass computer vision, speech processing, natural language processing, and machine learning / deep learning.

[0053] Computer Vision (CV): Computer vision is the science of making machines "see." Specifically, it refers to using cameras and computers to replace the human eye in identifying and measuring objects, and then further processing the images to make them more suitable for human observation or transmission to instruments. As a scientific discipline, computer vision studies related theories and technologies, aiming to build artificial intelligence systems that can extract information from images or multidimensional data. Large model technology has brought significant changes to the development of computer vision technology. Pre-trained models in the field of vision, such as the swin-transformer, ViT (Vision Transformer), V-MOE (Vision-Mixture of Experts), and MAE (Masked Autoencoders), can be quickly and widely applied to specific downstream tasks through fine-tuning. Computer vision technology generally includes image processing, image recognition, image semantic understanding, image retrieval, OCR (Optical Character Recognition), video processing, video semantic understanding, video content / behavior recognition, three-dimensional object reconstruction, 3D (Three Dimensions) technology, virtual reality, augmented reality and map construction, and also includes common biometric recognition technology. The image processing method provided in the embodiment of the present application can be applied to the field of computer vision technology. For example, the image processing method provided in the embodiment of the present application can generate a more accurate low-resolution image when reducing the image resolution.

[0054] Texture resampling is the process of adjusting a texture to a different size or resolution. In computer graphics and image processing, texture resampling is an important technique used to adjust the size or level of detail of a texture during rendering or in image processing. Texture resampling typically involves both texture enlargement (upsampling) and texture reduction (downsampling).

[0055] The image processing method provided in the embodiment of the present application can be executed by a computer device. In some embodiments, the computer device is a terminal or a server. The following first takes the computer device as an example to introduce the implementation environment of the image processing method provided in the embodiment of the present application. Figure 1 Schematic diagram of an implementation environment of an image processing method provided according to an embodiment of the present application. Figure 1 The implementation environment includes a terminal 101 and a server 102. The terminal 101 and the server 102 can be directly or indirectly connected via wired or wireless communication, which is not limited in this application.

[0056] In some embodiments, the terminal 101 is a smart phone, a tablet computer, a laptop computer, a desktop computer, a smart speaker, a smart watch, an intelligent voice interaction device, a smart home appliance, a car terminal, etc., but is not limited thereto. The terminal 101 installs and runs an application that supports image processing. The application can be a game application, a traffic application, a clipping application, an editing application, or a communication application, etc., and the embodiments of the present application are not limited to this. Schematically, the terminal 101 is a terminal used by a user. The terminal 101 is configured with a GPU (Graphics Processing Unit). The terminal processes the texture image through the GPU to retain the detail information of the texture image as much as possible while reducing the resolution of the texture image.

[0057] Terminal 101 generally refers to one of multiple terminals. This embodiment uses terminal 101 as an example. Those skilled in the art will appreciate that the number of terminals may be greater or lesser. For example, there may be a few terminals, or dozens, hundreds, or even more. This embodiment does not limit the number or device type of terminals.

[0058] In some embodiments, server 102 is an independent physical server, or a server cluster or distributed system composed of multiple physical servers. It can also be a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN (Content Delivery Network), big data and artificial intelligence platforms. Server 102 is used to provide background services for applications that support image processing. In some embodiments, server 102 undertakes the main computing work and terminal 101 undertakes the secondary computing work; or, server 102 undertakes the secondary computing work and terminal 101 undertakes the main computing work; or, server 102 and terminal 101 adopt a distributed computing architecture for collaborative computing.

[0059] Figure 2 This is a flowchart of an image processing method provided according to an embodiment of the present application. Figure 2 In the embodiment of the present application, the image processing method is described as being executed by a terminal. The image processing method includes the following steps:

[0060] 201. The terminal generates a second texture image based on the first texture image, where the resolution of the second texture image is lower than that of the first texture image.

[0061] In an embodiment of the present application, the first texture image can be a texture image of any three-dimensional model in a virtual space, or it can be a texture image of any object. The embodiment of the present application does not limit the source of the first texture image. Taking the example that the first texture image is a texture image of any three-dimensional model in a virtual space, the terminal can two-dimensionally unfold the surface of the three-dimensional model, and then map the coordinates of the three-dimensional model to a two-dimensional plane to obtain texture coordinates. Then, the terminal maps the texture coordinates to a texture map to obtain a first texture image. The above process can be completed by the GPU on the terminal. Then, the terminal reduces the resolution of the first texture image to obtain a second texture image. Among them, the method of reducing the resolution can be a linear filtering method such as nearest neighbor interpolation, bilinear interpolation or bicubic interpolation, or it can be a super-resolution technology based on a neural network, etc., and the embodiment of the present application does not limit this.

[0062] 202. The terminal upsamples the second texture image to obtain a third texture image. The resolution of the third texture image is higher than that of the second texture image, and the resolution of the third texture image is equal to that of the first texture image.

[0063] In an embodiment of the present application, during upsampling of the second texture image, the terminal uses the resolution of the first texture image as a reference and calculates the pixel values ​​of the pixels in the third texture image based on the pixel values ​​of the pixels in the second texture image, thereby obtaining the third texture image. The resolution of the third texture image is equal to the resolution of the first texture image. The number of pixels in the third texture image is equal to the number of pixels in the first texture image. The embodiment of the present application does not limit the upsampling method. Upsampling is a method of texture resampling. Accordingly, this step is to resample the second texture image to restore the lower-resolution second texture image to a third texture image with the same resolution as the first texture image.

[0064] 203. The terminal determines an image gap based on the first texture image and the third texture image, where the image gap is used to represent a difference between the first texture image and the third texture image.

[0065] In an embodiment of the present application, since the resolution of the first texture image is equal to the resolution of the third texture image, the pixel points in the first texture image and the pixel points in the third texture image can correspond one to one according to the position of the pixel points in the texture image. The terminal calculates the image gap between the first texture image and the third texture image based on the pixel values ​​of the pixel points in the first texture image and the pixel values ​​of the pixel points in the third texture image. Since some detail information (such as the edges of objects in the texture image) will be lost when the resolution of the original texture image is reduced, even if the generated lower resolution texture image is restored to the resolution size of the original texture image, the lost detail information will not be restored. Therefore, by calculating the image gap between the first texture image and the third texture image, it is possible to accurately determine the detail information lost when the resolution of the first texture image is reduced.

[0066] 204. The terminal adjusts the pixel values ​​of the pixels in the second texture image with the goal of minimizing the image difference.

[0067] In an embodiment of the present application, the terminal adjusts the pixel values ​​of the pixels in the second texture image to minimize the image gap. That is, by adjusting the pixel values ​​of the pixels in the second texture image, the second texture image can better retain the detail information in the first texture image. When the image gap meets the gap condition, the terminal stops adjusting the pixel values ​​of the pixels in the second texture image. The gap condition can be that the image gap is lower than a certain gap threshold, or it can be that the image gap is within a certain gap interval, which is not limited in the embodiment of the present application.

[0068] An embodiment of the present application provides an image processing method. In the process of reducing the resolution of a first texture image, a second texture image with a lower resolution is generated and upsampled to restore the second texture image with the lower resolution to a third texture image with the same resolution as the first texture image. Since some detail information (such as the edges of objects in the texture image) is lost when the resolution of the original texture image is reduced, even if the generated texture image with the lower resolution is restored to the resolution of the original texture image, the lost detail information is not restored. Therefore, by calculating the difference between the first texture image and the third texture image, the detail information lost when the resolution of the first texture image is reduced can be accurately determined. Then, by adjusting the pixel values ​​of pixels in the second texture image to minimize the image difference, the detail information lost when the resolution of the first texture image is reduced can be minimized. In other words, the generated second texture image with the lower resolution can better retain the detail information in the first texture image, thereby ensuring the visual effect of the second texture image and improving the image processing quality when the image resolution is reduced.

[0069] Figure 3is a flowchart of another image processing method provided in accordance with an embodiment of the present application, see Figure 3 In the embodiment of the present application, the image processing method is described as being executed by a terminal. The image processing method includes the following steps:

[0070] 301. The terminal generates a second texture image based on the first texture image, where the resolution of the second texture image is lower than that of the first texture image.

[0071] In an embodiment of the present application, the first texture image can be an image input into the terminal by the user, or it can be a texture image obtained by the terminal from other computer devices. The embodiment of the present application does not limit the method of obtaining the first texture image. The first texture image includes multiple pixels. The pixel value of each pixel includes three pixel components: red, green, and blue. The terminal reduces the resolution of the first texture image to obtain a second texture image. The more pixels in the texture image, the higher the resolution of the texture image. That is, the number of pixels in the second texture image is less than the number of pixels in the first texture image. Among them, the terminal can calculate the pixel values ​​of the pixels of the second texture image based on the pixel values ​​of the pixels in the first texture image. The embodiment of the present application does not limit the method of reducing the resolution of the first texture image.

[0072] 302. The terminal upsamples the second texture image to obtain a third texture image. The resolution of the third texture image is higher than that of the second texture image, and the resolution of the third texture image is equal to that of the first texture image.

[0073] In an embodiment of the present application, the terminal determines the resolution in the third texture image with reference to the resolution of the first texture image. That is, the terminal determines the position of each pixel in the third texture image based on the distribution of pixels in the first texture image. Then, in the process of calculating the pixel value of any pixel in the third texture image, the terminal determines multiple first pixels from the second texture image based on the pixel. Then, the terminal determines the pixel value of the pixel in the third texture image based on the pixel values ​​of the multiple first pixels. Among them, the positional relationship between the multiple first pixels and the pixel meets the condition. The condition can be that the pixel in the second texture image is adjacent to the mapping position of the pixel in the third texture image; or, the distance between the mapping position of the pixel in the second texture image and the pixel in the third texture image is less than the distance threshold, etc. The embodiment of the present application does not limit this. The solution provided by the embodiment of the present application determines multiple first pixel points that satisfy a positional relationship with pixel points in the third texture image from the second texture image, and calculates the pixel value of the pixel point in the third texture image using the multiple first pixel values, so that the pixel point can reflect the information of the corresponding position in the second texture image, that is, retains the information in the second texture image as much as possible, thereby improving the image processing quality when increasing the image resolution.

[0074] In some embodiments, the terminal may weight the pixel values ​​of multiple first pixels to calculate the pixel value of a pixel in the third texture image. Accordingly, the process of determining the pixel value of a pixel in the third texture image based on the pixel values ​​of the multiple first pixels includes: in calculating the pixel value of any pixel in the third texture image, the terminal maps the pixel in the third texture image to the second texture image to obtain a second pixel. Then, for any first pixel among the multiple first pixels, the terminal determines a weight for the first pixel based on the position between the first pixel and the second pixel. Then, based on the weights of the multiple first pixels, the terminal performs a weighted summation of the pixel values ​​of the multiple first pixels to obtain the pixel value of the pixel in the third texture image. The weight of a first pixel is positively correlated with the distance between the first pixel and the second pixel. That is, the closer the distance between the first pixel and the second pixel, the higher the weight of the first pixel; the farther the distance between the first pixel and the second pixel, the lower the weight of the first pixel. The solution provided in the embodiment of the present application determines the weight of the first pixel point through the position between the first pixel point and the second pixel point, so that the weight can accurately reflect the influence of the pixel value of the first pixel point on the pixel point in the third texture image, and then performs weighted summation on the pixel values ​​of multiple first pixels to calculate the pixel value of the pixel point in the third texture image, so that the pixel point can reflect the information of the corresponding position in the second texture image, that is, retain the information in the second texture image as much as possible, thereby improving the image processing quality when increasing the image resolution.

[0075] For example, Figure 4 Schematic diagram of upsampling according to an embodiment of the present application. Figure 4 , the number of pixels in the second texture image is 4*4=16, and the number of pixels in the third texture image is 8*8=64. That is, the resolution of the second texture image is 4*4, and the resolution of the third texture image is 8*8. In the process of calculating the pixel value of pixel 401 in the third texture image, the terminal maps pixel 401 to the second texture image to obtain second pixel 402. Then, the terminal uses pixel 403, pixel 404, pixel 405, and pixel 406 adjacent to second pixel 402 as multiple first pixel points. Then, for any first pixel point, the terminal determines the weight of the first pixel point based on the position between the first pixel point and the second pixel point. Then, based on the weights of the multiple first pixel points, the terminal performs a weighted summation on the pixel values ​​of the multiple first pixel points to obtain the pixel value of pixel 401 in the third texture image.

[0076] by Figure 4 Taking the up-sampling process as an example, the terminal can calculate the pixel value of the pixel point in the third texture image by using the following formula 1.

[0077] Formula 1:

[0078]

[0079] Among them, U i used to represent the pixel value of pixel point 401 in the third texture image; used to represent the pixel value of pixel point 403 in the second texture image; Used to represent the weight of pixel 403, 0.25; used to represent the pixel value of pixel point 404 in the second texture image; Used to represent the weight of pixel 404, used to represent the pixel value of pixel point 405 in the second texture image; Used to represent the weight of pixel 405, used to represent the pixel value of pixel point 406 in the second texture image; Used to represent the weight of pixel 406, Here, 0.25 and 0.75 refer to the ratio of the distance to the size of a pixel in the second texture image. The above upsampling process is a linear interpolation calculation method.

[0080] 303. The terminal determines an image gap based on the first texture image and the third texture image, where the image gap is used to represent a difference between the first texture image and the third texture image.

[0081] In an embodiment of the present application, the pixels in the first texture image and the pixels in the third texture image correspond one to one. The pixel values ​​of the two corresponding pixels may be the same or different, and the embodiment of the present application does not limit this. The terminal compares the pixel values ​​of the pixels in the first texture image with the pixel values ​​of the pixels in the third texture image one by one, determines the difference between the pixel values ​​of each corresponding pixel, and thus determines the image difference between the first texture image and the third texture image. Since some detail information is lost when the resolution of the first texture image is reduced, even if the generated second texture image with a lower resolution is restored to the resolution size of the first texture image (the third texture image), the lost detail information will not be restored. The less detail information is lost when the resolution is reduced, the closer it may be to the original texture image when it is restored to the original resolution. Therefore, the image difference can reflect the detail information lost when the resolution of the first texture image is reduced.

[0082] In some embodiments, the pixel value of each pixel includes three pixel components: red, green, and blue. The terminal can calculate the image disparity based on these three pixel components. Accordingly, the terminal determines the image disparity based on the first and third texture images, including: for any pixel in the third texture image, the terminal determines a third pixel corresponding to the pixel in the first texture image based on the pixel's position. The terminal then determines the pixel disparity between the pixel and the third pixel based on the pixel value of the pixel and the pixel value of the third pixel. The pixel value of the pixel includes three pixel components: red, green, and blue. The pixel disparity represents the difference in pixel values ​​between the pixels in the three color channels of red, green, and blue. The terminal then determines the image disparity based on the pixel disparities corresponding to multiple pixels in the third texture image. The solution provided by the embodiments of the present application calculates the pixel disparity between corresponding pixels based on these three pixel components, enabling accurate determination of the information lost at each pixel when the resolution of the first texture image is reduced. This allows for the calculation of the disparity between the first and third texture images, enabling accurate determination of the detail information lost when the resolution of the first texture image is reduced, thereby facilitating subsequent preservation of the detail information in the first texture image as much as possible.

[0083] In some embodiments, the terminal may calculate the image difference between the first texture image and the third texture image using the following formula 2.

[0084] Formula 2:

[0085]

[0086] Wherein, MSE is used to represent the image error; pixelCount is used to represent the number of pixels in the first texture image (third texture image); Used to represent the i-th pixel U in the third texture image i The pixel component of the red channel; Used to represent the i-th pixel T in the first texture image i The pixel component of the red channel; used to represent; Used to represent the i-th pixel U in the third texture image i The pixel component of the green channel; Used to represent the i-th pixel T in the first texture image i The pixel component of the green channel; Used to represent the i-th pixel U in the third texture image i The pixel component of the blue channel; Used to represent the i-th pixel T in the first texture image iThe pixel component of the blue channel; 3 is used to represent the total number of color channels, including red channel, green channel and blue channel.

[0087] 304. When the image gap is not less than the gap threshold, for any pixel point in the second texture image, the terminal determines a gradient of the pixel point based on the image gap and the pixel value of the pixel point, where the gradient is used to represent an adjustment direction of the pixel value in the second texture image when reducing the image gap.

[0088] In the embodiment of the present application, the embodiment of the present application does not limit the difference threshold. For any pixel in the second texture image, the terminal calculates the partial derivative of the relationship between the image difference and the pixel value of the pixel to obtain the gradient of the pixel. Among them, for the three color components (pixel components) in the pixel value of the pixel, the terminal can calculate the gradient corresponding to each color component separately, so that each color component can be adjusted based on the gradient corresponding to each color component.

[0089] In some embodiments, because the image disparity is determined based on the first and third texture images and has no direct relationship with the second texture image, the difference between the image disparity and the second texture image cannot be directly determined. In this case, the terminal can use the third texture image as a medium to determine the difference between the image disparity and the second texture image. That is, the terminal can determine the gradient of a pixel in the second texture image based on the relationship between the image disparity and the third texture image, as well as the relationship between the second texture image and the third texture image. Accordingly, the terminal determines the gradient of a pixel in the second texture image based on the image disparity and the pixel value, including: if the image disparity is not less than a disparity threshold, for any pixel in the second texture image, the terminal determines, based on the pixel, a plurality of fourth pixels from the third texture image. The plurality of fourth pixels are used to calculate the pixel values ​​when generating the second texture image based on the third texture image. The terminal determines a first gradient based on the image disparity and the pixel values ​​of the plurality of fourth pixels. The first gradient represents the direction in which the pixel values ​​in the third texture image should be adjusted to reduce the image disparity. The terminal then determines a second gradient based on the pixel value and the pixel values ​​of the plurality of fourth pixels. The second gradient is used to indicate the direction in which pixel values ​​in the second texture image are adjusted when generating the third texture image based on the second texture image. Then, the terminal determines the gradient of the pixel points in the second texture image based on the first gradient and the second gradient.

[0090] In some embodiments, the terminal may calculate the gradient of a pixel point in the second texture image using the following formula three.

[0091] Formula 3:

[0092]

[0093] Among them, MSE is used to represent the image gap; C k Used to represent the pixel value of the k-th pixel in the second texture image; Used to represent the gradient of the k-th pixel in the second texture image; U i Used to represent the pixel value of the i-th pixel in the third texture image; Used to represent the first gradient; Used to represent the second gradient; pixelCount is used to represent the number of pixels in the third texture image.

[0094] For the first gradient in Formula 3, the terminal can determine the first gradient by taking the partial derivative of the pixel value of each pixel in the third texture image using Formula 2. Accordingly, the terminal can calculate the first gradient using the following Formula 4.

[0095] Formula 4:

[0096]

[0097] Among them, MSE is used to represent the image gap; U i Used to represent the pixel value of the i-th pixel in the third texture image; Used to represent the first gradient; pixelCount is used to represent the number of pixels in the third texture image; T i Used to represent the pixel value of the i-th pixel in the first texture image.

[0098] In some embodiments, the embodiments of the present application do not limit the number of multiple fourth pixel points. For any pixel point in the second texture image, in the process of determining the multiple fourth pixel points corresponding to the pixel point, the terminal can map the pixel point to the third texture image and determine the mapping position of the pixel point; then, the terminal determines the pixel point that meets the conditions from the mapping position from the third texture image as the fourth pixel point. The condition can be that the pixel point in the third texture image is adjacent to the mapping position of the pixel point in the second texture image; or, the distance between the pixel point in the third texture image and the pixel point in the second texture image is less than the distance threshold, etc. The embodiments of the present application do not limit this. The multiple fourth pixel points can be regarded as the pixel points used when calculating the pixel values ​​of the pixel points in the second texture image in the process of generating the second texture image based on the third texture image.

[0099] For example, Figure 5 FIG. 1 is a schematic diagram of determining a fourth pixel point according to an embodiment of the present application. Figure 5If the pixel value of pixel 501 in the second texture image is calculated based on GPU bilinear filtering sampling in the pixel points of the third texture image, the pixel points used are the 16 gray pixels in the third texture image.

[0100] by Figure 5 Taking the method of determining multiple fourth pixel points in as an example, the terminal can calculate the gradient of the pixel points in the second texture image through the following formula 5.

[0101] Formula 5:

[0102]

[0103] Among them, MSE is used to represent the image error; C k Used to represent the pixel value of the k-th pixel in the second texture image; Used to represent the gradient of the k-th pixel in the second texture image; U i Used to represent the pixel value of the i-th pixel in the third texture image; Used to represent the first gradient; Used to represent the second gradient; pixelCount is used to represent the number of pixels in the third texture image; T i It is used to represent the pixel value of the i-th pixel in the first texture image; 16 is used to represent the number of the fourth pixel; U j Used to represent the pixel value of the j-th fourth pixel in the third texture image; T j Used to represent the first texture image with U j The corresponding pixel value; The terminal can calculate the weight of each fourth pixel using the weight calculation method in step 302. The code for calculating the weight of each fourth pixel is as follows:

[0104] xIndex=i.uv.x+_UpSamplingTex_TexelSize*0.5;

[0105] yIndex=i.uv.y-_UpSamplingTex_TexelSize*1.5;

[0106] xWeight=(xIndex<_UpSamplingTex_TexelSize||xIndex+_UpSamplingTex_TexelSize>1)? 1:0.75;

[0107] yWeight=(yIndex<_UpSamplingTex_TexelSize||yIndex+_UpSamplingTex_TexelSize>1)? 1:0.25;

[0108] upSampleIndex[2]=half3(xIndex,yIndex,xWeight*yWeight).

[0109] In the embodiment of the present application, the image difference MSE and the pixel value C of the pixel point in the second texture image can be obtained according to Formula 5. k The second-order derivative of is greater than zero, as shown in the following formula 6.

[0110] Formula 6:

[0111]

[0112] Therefore, the calculation method of the image difference MSE (Formula 2) is a convex function, which can be optimized using the gradient descent method. Accordingly, the terminal continues to execute step 305.

[0113] 305. The terminal adjusts the pixel value of the pixel based on the gradient of the pixel.

[0114] In an embodiment of the present application, a terminal adjusts the pixel value of a pixel in the second texture image based on the pixel's gradient. For any pixel in the second texture image, the terminal weights the first step size based on the pixel's gradient to obtain a second step size. The first step size represents the base increment for each pixel value adjustment. The terminal then adjusts the pixel value based on the second step size.

[0115] In some embodiments, the terminal may adjust the pixel value of the pixel point in the second texture image using the following formula 7.

[0116] Formula 7:

[0117]

[0118] Among them, C k It is used to represent the pixel value of the kth pixel in the second texture image; step is used to represent the first step length; MSE is used to represent the image gap; Used to represent the gradient of the k-th pixel in the second texture image; Used to indicate the second step length.

[0119] In some embodiments, the first step length is dynamic. The terminal determines a new image gap based on the adjusted second texture image. Then, if the image gap increases, the terminal adjusts the first step length.

[0120] During the process of calculating the gradient of each pixel in the second texture image and updating the pixel value of each pixel, the terminal can use the GPU to accelerate the calculation in parallel. That is, the terminal uses the GPU to calculate the gradient of each pixel in the second texture image in parallel and update the pixel value of each pixel in parallel. This method can speed up image processing.

[0121] In the process of updating the pixels of each pixel point in the second texture image, the terminal can use the following code to implement it. As shown below:

[0122]

[0123]

[0124] The terminal may repeat steps 302 to 305 until the image gap satisfies the gap condition. When the image gap satisfies the gap condition, the terminal stops adjusting the pixel values ​​of the pixels in the second texture image. The gap condition may be that the image gap is lower than a certain gap threshold, or it may be that the image gap is in a certain gap interval, which is not limited in the embodiment of the present application. In addition to being MSE (Mean Squared Error), the image gap may also be PSNR (Peak Signal-to-Noise Ratio), which is not limited in the embodiment of the present application. The terminal may display each calculated image gap to reflect the progress of image processing to the user. When the image gap converges, the image processing process is stopped.

[0125] In order to more clearly describe the image processing method provided in the embodiment of the present application, the image processing method is further described below with reference to the accompanying drawings. Figure 6 This is a flowchart of another image processing method provided according to an embodiment of the present application. Figure 6, the terminal obtains a first texture image (an image with a higher resolution). Then, the terminal reduces the resolution of the first texture image to obtain a second texture image. Then, the terminal upsamples the second texture image to obtain a third texture image. Then, the terminal calculates the image gap between the first texture image and the third texture image. Then, when the image gap is not less than the gap threshold, the terminal calculates the gradient of the pixel components of each color channel of each pixel point in the second texture image. Then, for any pixel point in the second texture image, the terminal adjusts the three pixel components of the pixel point based on the gradient of the three pixel components of the pixel point. Then, the terminal re-upsamples based on the adjusted second texture image. When the image gap is less than the gap threshold, the terminal stops adjusting the second texture image and outputs the final second texture image (an image with a lower resolution).

[0126] The solution provided by the embodiment of the present application does not depend on a specific engine, programming language, or graphics API. Taking the implementation in Unity as an example, the present invention will provide the following files: SampleShader.shader file, GPUDeltaShader.shader file, GPUGatherShader.shader file, GradientShader.shader file, and TextureOptimization.cs file. The SampleShader.shader file is used to perform upsampling using the GPU. The GPUDeltaShader.shader file is used to calculate the total amount of the difference in the process of calculating the image difference. The GPUGatherShader.shader file is used to average the image gap weights during the image gap calculation process to obtain the image gap. The GradientShader.shader file is used to use the GPU to calculate gradients and update pixel values. The TextureOptimization.cs file is used to implement the script for the image processing method provided in the embodiments of this application.

[0127] In contrast, traditional bilinear interpolation methods calculate pixel values ​​within a specific range and cannot effectively preserve important texture features. The method provided in the embodiments of the present application can obtain a wider range of information beyond the surrounding pixel value range, which is expected to improve the quality and fidelity of resampling. This technical solution is applicable to multiple fields, including game development, computer graphics, virtual reality, etc., and can bring higher levels of image processing performance and result quality to these fields.

[0128] For example, Figure 7 is a comparison chart of image processing effects provided according to an embodiment of the present application. Figure 7, Figure 7 (a) in the figure is a 2048*2048 texture image. The PSNR calculated by using the traditional bilinear interpolation method after upsampling is 41.10 compared with the original image. However, after using the method provided by the present invention, the PSNR can reach 44.29. Figure 7 (a) in the figure is a 512*512 texture image. The PSNR calculated by using the traditional bilinear interpolation method after upsampling is 31.15 compared with the original image. However, after using the method provided by the present invention, the PSNR can reach 33.19.

[0129] An embodiment of the present application provides an image processing method. In the process of reducing the resolution of a first texture image, a second texture image with a lower resolution is generated and upsampled to restore the second texture image with the lower resolution to a third texture image with the same resolution as the first texture image. Since some detail information (such as the edges of objects in the texture image) is lost when the resolution of the original texture image is reduced, even if the generated texture image with the lower resolution is restored to the resolution of the original texture image, the lost detail information is not restored. Therefore, by calculating the difference between the first texture image and the third texture image, the detail information lost when the resolution of the first texture image is reduced can be accurately determined. Then, by adjusting the pixel values ​​of pixels in the second texture image to minimize the image difference, the detail information lost when the resolution of the first texture image is reduced can be minimized. In other words, the generated second texture image with the lower resolution can better retain the detail information in the first texture image, thereby ensuring the visual effect of the second texture image and improving the image processing quality when the image resolution is reduced.

[0130] Figure 8 This is a block diagram of an image processing device provided according to an embodiment of the present application. The image processing device is used to execute the steps of the above-mentioned image processing method. Figure 8 , the image processing device comprises:

[0131] A generating module 801 is configured to generate a second texture image based on the first texture image, wherein the resolution of the second texture image is lower than that of the first texture image;

[0132] A processing module 802 is configured to upsample the second texture image to obtain a third texture image, wherein the resolution of the third texture image is higher than that of the second texture image and the resolution of the third texture image is equal to that of the first texture image;

[0133] A determination module 803 is configured to determine an image gap based on the first texture image and the third texture image, where the image gap is used to represent a difference between the first texture image and the third texture image;

[0134] The first adjustment module 804 is configured to adjust the pixel values ​​of the pixels in the second texture image with the goal of minimizing the image gap.

[0135] In some embodiments, Figure 9 is a block diagram of another image processing device provided according to an embodiment of the present application. Figure 9 , the processing module 802 includes:

[0136] The first determining unit 8021 is configured to determine, based on the pixel point, a plurality of first pixel points from the second texture image in a process of calculating a pixel value of any pixel point in the third texture image, where a positional relationship between the plurality of first pixel points and the pixel satisfies a condition;

[0137] The second determining unit 8022 is configured to determine the pixel value of a pixel point in the third texture image based on the pixel values ​​of the plurality of first pixel points.

[0138] In some embodiments, see Figure 9 The second determining unit 8022 is configured to, in the process of calculating the pixel value of any pixel point in the third texture image, map the pixel point in the third texture image to the second texture image to obtain the second pixel point; for any first pixel point among the multiple first pixel points, determine the weight of the first pixel point based on the position between the first pixel point and the second pixel point; and perform weighted summation of the pixel values ​​of the multiple first pixel points based on the weights of the multiple first pixel points to obtain the pixel value of the pixel point in the third texture image.

[0139] In some embodiments, pixels in the first texture image correspond one-to-one to pixels in the third texture image;

[0140] Continue to see Figure 9 The determination module 803 is used to determine, for any pixel point in the third texture image, a third pixel point corresponding to the pixel point from the first texture image based on the position of the pixel point; determine a pixel difference between the pixel point and the third pixel point based on the pixel value of the pixel point and the pixel value of the third pixel point, where the pixel value of the pixel point includes three pixel components of red, green, and blue, and the pixel difference is used to represent the difference in pixel values ​​between the pixels in the three color channels of red, green, and blue; and determine an image difference based on the pixel differences corresponding to multiple pixel points in the third texture image.

[0141] In some embodiments, see Figure 9 The first adjustment module 804 includes:

[0142] A third determining unit 8041 is configured to determine, for any pixel in the second texture image, a gradient of the pixel based on the image difference and the pixel value when the image difference is not less than the difference threshold, wherein the gradient represents a direction in which the pixel value in the second texture image is adjusted when reducing the image difference.

[0143] The adjustment unit 8042 is configured to adjust the pixel value of the pixel based on the gradient of the pixel.

[0144] In some embodiments, see Figure 9 The third determining unit 8041 is configured to, when the image gap is not less than the gap threshold, determine, for any pixel point in the second texture image, a plurality of fourth pixels from the third texture image based on the pixel point, the plurality of fourth pixels being used to calculate the pixel value of the pixel point when generating the second texture image based on the third texture image; determine, based on the image gap and the pixel values ​​of the plurality of fourth pixels, a first gradient, the first gradient being used to indicate an adjustment direction of the pixel value in the third texture image when reducing the image gap; determine, based on the pixel value of the pixel point and the pixel values ​​of the plurality of fourth pixels, a second gradient being used to indicate an adjustment direction of the pixel value in the second texture image when generating the third texture image based on the second texture image; and determine, based on the first gradient and the second gradient, a gradient of the pixel point in the second texture image.

[0145] In some embodiments, see Figure 9 The adjustment unit 8042 is used to weight the first step length based on the gradient of the pixel point to obtain a second step length, where the first step length is used to represent the basic increment of each adjustment of the pixel value; and adjust the pixel value of the pixel point based on the second step length.

[0146] In some embodiments, see Figure 9 , the determination module 803 is further used to determine a new image gap based on the adjusted second texture image;

[0147] The device also includes:

[0148] The second adjustment module 805 is configured to adjust the first step length when the image gap becomes larger.

[0149] An embodiment of the present application provides an image processing device that, in the process of reducing the resolution of a first texture image, restores the lower-resolution second texture image to a third texture image with the same resolution as the first texture image by upsampling the generated lower-resolution second texture image. Since some detail information (such as the edges of objects in the texture image) is lost when the resolution of the original texture image is reduced, even if the generated lower-resolution texture image is restored to the resolution of the original texture image, the lost detail information is not restored. Therefore, by calculating the difference between the first texture image and the third texture image, the detail information lost when the resolution of the first texture image is reduced can be accurately determined. Then, by adjusting the pixel values ​​of pixels in the second texture image to minimize the image difference, the detail information lost when the resolution of the first texture image is reduced can be minimized. In other words, the generated lower-resolution second texture image can better retain the detail information in the first texture image, thereby ensuring the visual effect of the second texture image and improving the image processing quality when the image resolution is reduced.

[0150] It should be noted that the image processing device provided in the above embodiment, when reducing the resolution of a texture image, is illustrated only by the division of the above functional modules. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. In addition, the image processing device provided in the above embodiment and the image processing method embodiment are based on the same concept. The specific implementation process is detailed in the method embodiment and will not be repeated here.

[0151] In the embodiments of the present application, the computer device can be configured as a terminal or a server. When the computer device is configured as a terminal, the terminal can be used as the execution subject to implement the technical solution provided in the embodiments of the present application. When the computer device is configured as a server, the server can be used as the execution subject to implement the technical solution provided in the embodiments of the present application. The technical solution provided in the present application can also be implemented through interaction between the terminal and the server. The embodiments of the present application do not limit this.

[0152] Figure 10The following is a block diagram of a terminal 1000 according to an embodiment of the present application. Terminal 1000 may be a portable mobile terminal, such as a smartphone, tablet computer, MP3 player (Moving Picture Experts Group Audio Layer III), MP4 player (Moving Picture Experts Group Audio Layer IV), laptop computer, or desktop computer. Terminal 1000 may also be referred to as user equipment, portable terminal, laptop terminal, desktop terminal, or other similar names.

[0153] Typically, the terminal 1000 includes a processor 1001 and a memory 1002 .

[0154] The processor 1001 may include one or more processing cores, such as a 4-core processor, an 8-core processor, etc. The processor 1001 may be implemented in at least one hardware form of DSP (Digital Signal Processing), FPGA (Field-Programmable Gate Array), or PLA (Programmable Logic Array). The processor 1001 may also include a main processor and a coprocessor. The main processor is a processor for processing data in the awake state, also known as a CPU (Central Processing Unit); the coprocessor is a low-power processor for processing data in the standby state. In some embodiments, the processor 1001 may be integrated with a GPU (Graphics Processing Unit), which is responsible for rendering and drawing the content to be displayed on the display screen. In some embodiments, the processor 1001 may also include an AI (Artificial Intelligence) processor, which is used to process computing operations related to machine learning.

[0155] The memory 1002 may include one or more computer-readable storage media, which may be non-transitory. The memory 1002 may also include high-speed random access memory and non-volatile memory, such as one or more disk storage devices and flash memory storage devices. In some embodiments, the non-transitory computer-readable storage medium in the memory 1002 is used to store at least one computer program, which is executed by the processor 1001 to implement the image processing method provided in the method embodiment of the present application.

[0156] In some embodiments, terminal 1000 may optionally include a peripheral device interface 1003 and at least one peripheral device. Processor 1001, memory 1002, and peripheral device interface 1003 may be connected via a bus or signal lines. Each peripheral device may be connected to peripheral device interface 1003 via a bus, signal lines, or circuit boards. Specifically, the peripheral device may include at least one of a radio frequency circuit 1004, a display screen 1005, a camera assembly 1006, an audio circuit 1007, and a power supply 1008.

[0157] The peripheral device interface 1003 can be used to connect at least one I / O (Input / Output)-related peripheral device to the processor 1001 and the memory 1002. In some embodiments, the processor 1001, the memory 1002, and the peripheral device interface 1003 are integrated on the same chip or circuit board; in some other embodiments, any one or two of the processor 1001, the memory 1002, and the peripheral device interface 1003 can be implemented on separate chips or circuit boards, which is not limited in this embodiment.

[0158] The RF circuit 1004 is used to receive and transmit RF (Radio Frequency) signals, also known as electromagnetic signals. The RF circuit 1004 communicates with communication networks and other communication devices via electromagnetic signals. The RF circuit 1004 converts electrical signals into electromagnetic signals for transmission, or converts received electromagnetic signals into electrical signals. In some embodiments, the RF circuit 1004 includes an antenna system, an RF transceiver, one or more amplifiers, a tuner, an oscillator, a digital signal processor, a codec chipset, a user identity module card, and the like. The RF circuit 1004 can communicate with other terminals via at least one wireless communication protocol. Such wireless communication protocols include, but are not limited to, the World Wide Web, metropolitan area networks, intranets, various generations of mobile communication networks (2G, 3G, 4G, and 5G), wireless local area networks, and / or WiFi (Wireless Fidelity) networks. In some embodiments, the RF circuit 1004 may also include circuitry related to Near Field Communication (NFC), although this application does not limit this.

[0159] The display screen 1005 is used to display a UI (User Interface). The UI may include graphics, text, icons, videos, and any combination thereof. When the display screen 1005 is a touch screen display, the display screen 1005 also has the ability to collect touch signals on the surface or above the surface of the display screen 1005. The touch signal can be input as a control signal to the processor 1001 for processing. At this time, the display screen 1005 can also be used to provide virtual buttons and / or virtual keyboards, also known as soft buttons and / or soft keyboards. In some embodiments, there can be one display screen 1005, which is set on the front panel of the terminal 1000; in other embodiments, there can be at least two display screens 1005, which are respectively set on different surfaces of the terminal 1000 or in a folding design; in other embodiments, the display screen 1005 can be a flexible display screen, which is set on the curved surface or folding surface of the terminal 1000. Even more, the display screen 1005 can be set to a non-rectangular irregular shape, that is, a special-shaped screen. The display screen 1005 can be made of materials such as LCD (Liquid Crystal Display) and OLED (Organic Light-Emitting Diode).

[0160] The camera assembly 1006 is used to capture images or videos. In some embodiments, the camera assembly 1006 includes a front camera and a rear camera. Typically, the front camera is arranged on the front panel of the terminal, and the rear camera is arranged on the back of the terminal. In some embodiments, there are at least two rear cameras, which are any one of a main camera, a depth of field camera, a wide-angle camera, and a telephoto camera, so as to realize the fusion of the main camera and the depth of field camera to realize the background blur function, the fusion of the main camera and the wide-angle camera to realize panoramic shooting and VR (Virtual Reality) shooting function or other fusion shooting functions. In some embodiments, the camera assembly 1006 may also include a flash. The flash can be a single-color temperature flash or a dual-color temperature flash. A dual-color temperature flash refers to a combination of a warm light flash and a cold light flash, which can be used for light compensation at different color temperatures.

[0161] The audio circuit 1007 may include a microphone and a speaker. The microphone is used to collect sound waves from the user and the environment, and convert the sound waves into electrical signals that are input into the processor 1001 for processing, or input into the RF circuit 1004 to achieve voice communication. For the purpose of stereo sound collection or noise reduction, there may be multiple microphones, each located in different parts of the terminal 1000. The microphone may also be an array microphone or an omnidirectional collection microphone. The speaker is used to convert electrical signals from the processor 1001 or the RF circuit 1004 into sound waves. The speaker may be a traditional thin film speaker or a piezoelectric ceramic speaker. When the speaker is a piezoelectric ceramic speaker, it can not only convert electrical signals into sound waves audible to humans, but also convert electrical signals into sound waves inaudible to humans for purposes such as ranging. In some embodiments, the audio circuit 1007 may also include a headphone jack.

[0162] Power supply 1008 is used to power various components in terminal 1000. Power supply 1008 can be AC ​​power, DC power, a disposable battery, or a rechargeable battery. When power supply 1008 includes a rechargeable battery, the rechargeable battery can be a wired rechargeable battery or a wireless rechargeable battery. A wired rechargeable battery is charged via a wired line, while a wireless rechargeable battery is charged via a wireless coil. The rechargeable battery can also support fast charging technology.

[0163] In some embodiments, the terminal 1000 further includes one or more sensors 1009 . The one or more sensors 1009 include, but are not limited to, an acceleration sensor 1010 , a gyroscope sensor 1011 , a pressure sensor 1012 , an optical sensor 1013 , and a proximity sensor 1014 .

[0164] The accelerometer 1010 can detect the magnitude of acceleration along the three coordinate axes of the coordinate system established by the terminal 1000. For example, the accelerometer 1010 can be used to detect the components of gravity acceleration along the three coordinate axes. The processor 1001 can control the display screen 1005 to display the user interface in a landscape or portrait view based on the gravity acceleration signal collected by the accelerometer 1010. The accelerometer 1010 can also be used to collect game or user motion data.

[0165] The gyroscope sensor 1011 can detect the orientation and rotation angle of the terminal 1000. The gyroscope sensor 1011 can work with the acceleration sensor 1010 to collect the user's 3D movements on the terminal 1000. Based on the data collected by the gyroscope sensor 1011, the processor 1001 can implement the following functions: motion sensing (such as changing the UI based on the user's tilt operation), image stabilization during shooting, game control, and inertial navigation.

[0166] The pressure sensor 1012 can be provided on the side frame of the terminal 1000 and / or below the display screen 1005. When the pressure sensor 1012 is provided on the side frame of the terminal 1000, it can detect the user's gripping signal of the terminal 1000, and the processor 1001 can perform left-hand or right-hand recognition or shortcut operations based on the gripping signal collected by the pressure sensor 1012. When the pressure sensor 1012 is provided below the display screen 1005, the processor 1001 controls the operable controls on the UI interface based on the user's pressure operation on the display screen 1005. Operable controls include at least one of button controls, scroll bar controls, icon controls, and menu controls.

[0167] Optical sensor 1013 is used to detect ambient light intensity. In one embodiment, processor 1001 can control the display brightness of display screen 1005 based on the ambient light intensity detected by optical sensor 1013. Specifically, when the ambient light intensity is high, the display brightness of display screen 1005 is increased; when the ambient light intensity is low, the display brightness of display screen 1005 is decreased. In another embodiment, processor 1001 can also dynamically adjust the shooting parameters of camera assembly 1006 based on the ambient light intensity detected by optical sensor 1013.

[0168] Proximity sensor 1014, also known as a distance sensor, is typically located on the front panel of terminal 1000. Proximity sensor 1014 is used to detect the distance between the user and the front of terminal 1000. In one embodiment, when proximity sensor 1014 detects that the distance between the user and the front of terminal 1000 is gradually decreasing, processor 1001 controls display screen 1005 to switch from the screen-on state to the screen-off state. When proximity sensor 1014 detects that the distance between the user and the front of terminal 1000 is gradually increasing, processor 1001 controls display screen 1005 to switch from the screen-off state to the screen-on state.

[0169] Those skilled in the art will understand that Figure 10 The structure shown in the figure does not constitute a limitation on the terminal 1000, and the terminal 1000 may include more or fewer components than shown in the figure, or combine certain components, or adopt a different component arrangement.

[0170] Figure 111 is a schematic diagram of the structure of a server provided in accordance with an embodiment of the present application. The server 1100 may vary greatly due to different configurations or performances, and may include one or more processors (Central Processing Units, CPU) 1101 and one or more memories 1102, wherein the memories 1102 store at least one computer program, which is loaded and executed by the processor 1101 to implement the image processing methods provided in the above-mentioned various method embodiments. Of course, the server 1100 may also have components such as a wired or wireless network interface, a keyboard, and an input / output interface for input and output. The server 1100 may also include other components for implementing device functions, which will not be described in detail here.

[0171] The present application also provides a computer-readable storage medium that stores at least one computer program. The at least one computer program is loaded and executed by a processor of a computer device to implement the operations performed by the computer device in the image processing method of the above embodiment. For example, the computer-readable storage medium can be a read-only memory (ROM), a random access memory (RAM), a compact disc read-only memory (CD-ROM), a magnetic tape, a floppy disk, an optical data storage device, etc.

[0172] The present application also provides a computer program product, including a computer program stored in a computer-readable storage medium. A processor of a computer device reads the computer program from the computer-readable storage medium and executes the computer program, causing the computer device to perform the image processing methods provided in the various optional implementations described above.

[0173] Those skilled in the art will understand that all or part of the steps to implement the above embodiments may be accomplished by hardware, or by a program to instruct the relevant hardware, and the program may be stored in a computer-readable storage medium, which may be a read-only memory, a disk, or an optical disk, etc.

[0174] The above description is merely an optional embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present application shall be included in the scope of protection of the present application.

Claims

1. An image processing method, characterized in that: The method comprises: generating a second texture image based on the first texture image, wherein the resolution of the second texture image is lower than the resolution of the first texture image; Upsampling the second texture image to obtain a third texture image, wherein a resolution of the third texture image is higher than a resolution of the second texture image and a resolution of the third texture image is equal to a resolution of the first texture image; determining an image gap based on the first texture image and the third texture image, wherein the image gap is used to represent a difference between the first texture image and the third texture image; With the goal of minimizing the image gap, the pixel values ​​of the pixels in the second texture image are adjusted.

2. The method according to claim 1, wherein The upsampling of the second texture image to obtain a third texture image includes: In the process of calculating the pixel value of any pixel point in the third texture image, based on the pixel point, a plurality of first pixel points are determined from the second texture image, and a positional relationship between the plurality of first pixel points and the pixel point satisfies a condition; Based on the pixel values ​​of the plurality of first pixel points, the pixel value of the pixel point in the third texture image is determined.

3. The method according to claim 2, characterized in that The determining, based on the pixel values ​​of the plurality of first pixel points, the pixel values ​​of the pixel points in the third texture image includes: In the process of calculating the pixel value of any pixel point in the third texture image, mapping the pixel point in the third texture image to the second texture image to obtain a second pixel point; For any first pixel point among the plurality of first pixel points, determining a weight of the first pixel point based on a position between the first pixel point and the second pixel point; Based on the weights of the multiple first pixel points, pixel values ​​of the multiple first pixel points are weighted summed to obtain the pixel value of the pixel point in the third texture image.

4. The method according to claim 1, wherein The pixels in the first texture image correspond to the pixels in the third texture image in one-to-one correspondence; The determining of the image gap based on the first texture image and the third texture image includes: For any pixel point in the third texture image, determining a third pixel point corresponding to the pixel point from the first texture image based on the position of the pixel point; determining a pixel difference between the pixel point and the third pixel point based on a pixel value of the pixel point and a pixel value of the third pixel point, wherein the pixel value of the pixel point includes three pixel components of red, green, and blue, and the pixel difference is used to represent a difference in pixel values ​​between the pixels in the three color channels of red, green, and blue; The image distance is determined based on pixel distances corresponding to a plurality of pixel points in the third texture image.

5. The method according to claim 1, wherein: The step of adjusting the pixel values ​​of the pixels in the second texture image with the goal of minimizing the image gap includes: When the image gap is not less than a gap threshold, determining, for any pixel point in the second texture image, a gradient of the pixel point based on the image gap and a pixel value of the pixel point, the gradient being used to represent an adjustment direction of the pixel value in the second texture image when reducing the image gap; Based on the gradient of the pixel point, the pixel value of the pixel point is adjusted.

6. The method according to claim 5, characterized in that The step of determining, for any pixel point in the second texture image based on the image difference and the pixel value of the pixel point when the image difference is not less than the difference threshold, comprising: When the image gap is not less than a gap threshold, for any pixel point in the second texture image, determining a plurality of fourth pixel points from the third texture image based on the pixel point, the plurality of fourth pixel points being used to calculate a pixel value of the pixel point when generating the second texture image based on the third texture image; determining a first gradient based on the image gap and the pixel values ​​of the plurality of fourth pixels, wherein the first gradient is used to represent an adjustment direction of pixel values ​​in the third texture image when reducing the image gap; determining a second gradient based on the pixel value of the pixel point and the pixel values ​​of the plurality of fourth pixels; wherein the second gradient is used to indicate an adjustment direction of pixel values ​​in the second texture image when generating the third texture image based on the second texture image; Based on the first gradient and the second gradient, a gradient of the pixel point in the second texture image is determined.

7. The method according to claim 5, characterized in that The adjusting the pixel value of the pixel point based on the gradient of the pixel point includes: Based on the gradient of the pixel point, weighting the first step length to obtain a second step length, wherein the first step length is used to represent a basic increment for each adjustment of the pixel value; Based on the second step size, the pixel value of the pixel point is adjusted.

8. The method according to claim 7, characterized in that The method further comprises: determining a new image gap based on the adjusted second texture image; When the image gap becomes larger, the first step length is adjusted.

9. An image processing device, characterized in that: The device comprises: a generating module, configured to generate a second texture image based on the first texture image, wherein the resolution of the second texture image is lower than that of the first texture image; a processing module, configured to upsample the second texture image to obtain a third texture image, wherein the resolution of the third texture image is higher than that of the second texture image and the resolution of the third texture image is equal to that of the first texture image; a determining module, configured to determine an image gap based on the first texture image and the third texture image, wherein the image gap is used to represent a difference between the first texture image and the third texture image; The first adjustment module is configured to adjust the pixel values ​​of the pixels in the second texture image with the goal of minimizing the image gap.

10. A computer device, characterized in that: The computer device includes a processor and a memory, the memory is used to store at least one computer program, and the at least one computer program is loaded by the processor to execute the image processing method according to any one of claims 1 to 8.

11. A computer-readable storage medium, characterized in that The computer-readable storage medium is used to store at least one computer program, and the at least one computer program is used to execute the image processing method according to any one of claims 1 to 8.

12. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the image processing method according to any one of claims 1 to 8 is implemented.

Images