Image processing method, scene updating method, device, equipment and storage medium

Abstract

The application discloses an image processing method, a scene updating method, a device, equipment and a storage medium, and belongs to the computer technical field. The method comprises the following steps: generating a first target image, the resolution of the first target image being a first resolution; acquiring a first reference image and a second reference image, the content of the first reference image being the same as that of the second reference image, the resolution of the first reference image being the first resolution, the resolution of the second reference image being a second resolution, and the first resolution being lower than the second resolution; determining a first difference image between the first target image and the first reference image; adjusting the resolution of the first difference image to obtain a second difference image with the second resolution; and superimposing the second reference image and the second difference image to obtain a second target image. The method can improve the resolution of the image, is high in efficiency, low in power consumption of the equipment, and can improve the performance of the equipment when processing the image.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to an image processing method, a scene update method, an apparatus, a device, and a storage medium. Background Technology

[0002] Image resolution refers to the number of pixels per unit size in an image. Higher resolution images offer more detail and produce better display results. In recent years, with the rapid development of computer technology, the image resolution supported by computer devices has continuously improved, leading to increasingly higher user demands for higher image resolution. Therefore, to meet these growing needs, a method to improve image resolution is urgently required. Summary of the Invention

[0003] This application provides an image processing method, scene update method, apparatus, device, and storage medium that can improve image resolution. The technical solution is as follows:

[0004] According to one aspect of the embodiments of this application, an image processing method is provided, the method comprising:

[0005] Generate a first target image, wherein the resolution of the first target image is a first resolution;

[0006] Acquire a first reference image and a second reference image, wherein the content of the first reference image and the second reference image are the same, and the resolution of the first reference image is the first resolution, the resolution of the second reference image is the second resolution, and the first resolution is lower than the second resolution;

[0007] Determine a first difference image between the first target image and the first reference image;

[0008] Adjust the resolution of the first difference image to obtain a second difference image with the second resolution;

[0009] The second reference image is superimposed on the second difference image to obtain the second target image.

[0010] According to another aspect of the embodiments of this application, a scene update method is provided, the method comprising:

[0011] In response to a first scene update command, a first scene image of a first resolution is rendered based on the first scene update command and the currently displayed virtual scene image;

[0012] A first reference image at the first resolution and a second reference image at the second resolution are obtained. The first reference image and the second reference image are scene images with the same content rendered in response to a second scene update instruction. The first resolution is lower than the second resolution, and the second scene update instruction is generated earlier than the first scene update instruction.

[0013] A first difference image is determined between the first scene image and the first reference image, and the resolution of the first difference image is adjusted to obtain a second difference image with the second resolution.

[0014] The second reference image is superimposed on the second difference image to obtain the second scene image. Based on the second scene image, the currently displayed virtual scene image is updated.

[0015] According to another aspect of the embodiments of this application, an image processing apparatus is provided, the apparatus comprising:

[0016] An image generation module is used to generate a first target image, wherein the resolution of the first target image is a first resolution;

[0017] An image acquisition module is used to acquire a first reference image and a second reference image, wherein the content of the first reference image and the second reference image are the same, and the resolution of the first reference image is the first resolution, the resolution of the second reference image is the second resolution, and the first resolution is lower than the second resolution;

[0018] The image determination module is used to determine a first difference image between the first target image and the first reference image;

[0019] A resolution adjustment module is used to adjust the resolution of the first difference image to obtain a second difference image with the second resolution;

[0020] An image overlay module is used to overlay the second reference image and the second difference image to obtain a second target image.

[0021] According to another aspect of the embodiments of this application, a scene update apparatus is provided, the apparatus comprising:

[0022] An image rendering module is used to render a first scene image of a first resolution based on the first scene update command and the currently displayed virtual scene image in response to the first scene update command.

[0023] The image acquisition module is used to acquire a first reference image at the first resolution and a second reference image at the second resolution. The first reference image and the second reference image are scene images with the same content rendered in response to a second scene update instruction. The first resolution is lower than the second resolution, and the second scene update instruction is generated earlier than the first scene update instruction.

[0024] A resolution adjustment module is used to determine a first difference image between the first scene image and the first reference image, and adjust the resolution of the first difference image to obtain a second difference image with the second resolution.

[0025] The scene update module is used to overlay the second reference image and the second difference image to obtain a second scene image, and update the currently displayed virtual scene image based on the second scene image.

[0026] According to another aspect of the embodiments of this application, a computer device is provided, the computer device including a processor and a memory; the memory stores at least one piece of program code, the at least one piece of program code being executed by the processor to implement the method as described above.

[0027] According to another aspect of the embodiments of this application, a chip is provided, the chip including programmable logic circuitry and / or program instructions, which, when the chip is run on a computer device, are used to implement the methods described above.

[0028] According to another aspect of the embodiments of this application, a computer-readable storage medium is provided, the storage medium storing at least one piece of program code, the at least one piece of program code being executed by a processor to implement the method as described above.

[0029] According to another aspect of the embodiments of this application, a computer program product is provided, which stores at least one piece of program code, the at least one piece of program code being executed by a processor to implement the method described above.

[0030] The solution provided in this application takes into account the relationship between a low-resolution target image and a low-resolution reference image, which is similar to the relationship between a high-resolution target image and a high-resolution reference image. Therefore, the difference image between the low-resolution target image and the low-resolution reference image is the same as the difference image between the high-resolution target image and the high-resolution reference image, only differing in resolution. Thus, the second difference image obtained by adjusting the resolution of the first difference image is the difference image between the second target image and the second reference image. The image obtained by superimposing the second reference image and the second difference image is the second target image with the same content as the first target image but at the second resolution. Therefore, this solution can improve image resolution. Furthermore, due to the simplicity of the process, this solution is not only highly efficient in improving image resolution but also consumes less power, thereby improving the performance of the device when processing images. Attached Figure Description

[0031] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0032] Figure 1 This application shows a structural block diagram of a computer device provided in an exemplary embodiment;

[0033] Figure 2 A flowchart illustrating an image processing method provided in an exemplary embodiment of this application is shown;

[0034] Figure 3 A flowchart illustrating another image processing method provided by an exemplary embodiment of this application is shown;

[0035] Figure 4 This illustration shows a schematic diagram of a process for acquiring a second target image patch according to an exemplary embodiment of this application;

[0036] Figure 5 This illustration shows a schematic diagram of an image processing procedure based on a hardware accelerator, provided by an exemplary embodiment of this application.

[0037] Figure 6 A flowchart illustrating yet another image processing method provided by an exemplary embodiment of this application is shown;

[0038] Figure 7 A flowchart illustrating another image processing method provided by an exemplary embodiment of this application is shown;

[0039] Figure 8 A flowchart illustrating another image processing method provided by an exemplary embodiment of this application is shown;

[0040] Figure 9 A block diagram of an image processing apparatus provided in an exemplary embodiment of this application is shown;

[0041] Figure 10 A block diagram of a scene update apparatus provided in an exemplary embodiment of this application is shown;

[0042] Figure 11 This invention provides a structural block diagram of a terminal according to an exemplary embodiment of the present application.

[0043] Figure 12 A structural block diagram of a server provided in an exemplary embodiment of this application is shown. Detailed Implementation

[0044] To make the objectives, technical solutions, and advantages of this application clearer, the embodiments of this application will be described in further detail below with reference to the accompanying drawings.

[0045] In this article, "at least one" refers to one or more, and "multiple" refers to two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone. The character " / " generally indicates that the preceding and following related objects have an "or" relationship.

[0046] It should be noted that all information (including but not limited to user device information, user personal information, etc.), data (including but not limited to data used for analysis, stored data, displayed data, etc.), and signals involved in this application have been authorized by the user or fully authorized by all parties, and the collection, use, and processing of related data must comply with the relevant laws, regulations, and standards of the relevant countries and regions. For example, the image data and operational data involved in this application were obtained with full authorization.

[0047] This application provides an image processing method, executed by a computer device 100. The computer device 100 can increase the resolution of an image using the method provided in this application. In some embodiments, the computer device 100 is a terminal, such as a mobile phone, camera, desktop computer, laptop computer, tablet computer, or other types of terminals.

[0048] Figure 1 This application shows a structural block diagram of a computer device provided in an exemplary embodiment, with reference to... Figure 1The computer device 100 includes a graphics processing unit (GPU) 101 and a hardware accelerator (HWA) 102. The GPU 101 renders images, and the hardware accelerator 102 adjusts the resolution of the images rendered by the GPU 101. For example, the GPU 101 renders an image at a second resolution, uses this image as a second reference image, and acquires an image with the same content as the second reference image but at a first resolution, uses this image as a first reference image, and sends the first reference image and the second reference image to the hardware accelerator 102. The hardware accelerator 102 receives and stores the first and second reference images. Afterwards, the GPU 101 renders a first target image at the first resolution and sends it to the hardware accelerator 102. The hardware accelerator 102, based on the first reference image, the second reference image, and the first target image, reconstructs a second target image with the same content as the first target image but at a second resolution. The first resolution is lower than the second resolution.

[0049] The image processing method provided in this application can be applied to game scenarios. For example, when a user performs an interactive operation on the game interface of a terminal, or during the game, when the terminal determines that an interface update is needed based on the game mechanics, the terminal generates a scene update command. Based on the scene update command and the currently displayed virtual scene image, a virtual scene image of a second resolution is rendered, and the game interface is updated based on the rendered virtual scene image of the second resolution. Furthermore, the terminal also acquires a virtual scene image with the same content as the second resolution virtual scene image but at a first resolution. This second resolution virtual scene image and the first resolution virtual scene image are used as reference images, and the resolution of the subsequently rendered first resolution virtual scene image is increased based on these reference images. In other words, the terminal uses the second resolution virtual scene image as a second reference image and the first resolution virtual scene image as a first reference image. When the terminal generates a scene update command again, it renders a virtual scene image at a first resolution based on the scene update command and the currently displayed virtual scene image. Then, using the method provided in this application, it adjusts the resolution of the virtual scene image based on a first reference image and a second reference image to obtain a virtual scene image with the same content as the virtual scene but at a second resolution. The game interface is then updated based on this virtual scene image. This method not only improves game graphics but also reduces game power consumption.

[0050] The image processing method provided in this application can be applied to extended reality (XR) scenarios. For example, when a user performs an interactive operation in a virtual scene, or when the terminal determines that a scene update is imminent based on the virtual scene's update mechanism, the terminal generates a scene update command. Based on the scene update command and the currently displayed virtual scene image, a virtual scene image of a second resolution is rendered, and the virtual scene is updated based on the rendered second-resolution virtual scene image. Furthermore, the terminal also acquires a virtual scene image with the same content as the second-resolution virtual scene image but at a first resolution. This second-resolution virtual scene image and the first-resolution virtual scene image are used as reference images, and the resolution of the subsequently rendered first-resolution virtual scene image is increased based on these reference images. In other words, the terminal uses the second-resolution virtual scene image as a second reference image and the first-resolution virtual scene image as a first reference image. When the terminal generates a scene update command again, it renders a virtual scene image of the first resolution based on the scene update command and the currently displayed virtual scene image. Then, using the method provided in this application, it adjusts the resolution of the virtual scene image based on the first reference image and the second reference image to obtain a virtual scene image with the same content as the virtual scene but at the second resolution. Then, it updates the virtual scene based on the virtual scene image.

[0051] Of course, the image processing method provided in this application embodiment can also be applied to other scenarios, and this application embodiment does not limit this.

[0052] Figure 2 A flowchart illustrating an image processing method provided in an exemplary embodiment of this application is shown. See also: Figure 2 The method includes:

[0053] 201. The terminal generates a first target image, and the resolution of the first target image is a first resolution.

[0054] Optionally, the first target image is one of multiple frames rendered continuously by the terminal. Optionally, in response to an image rendering instruction, the terminal renders the first target image using the GPU based on the image rendering instruction.

[0055] The first resolution can be any value, and this application does not impose any restrictions on it.

[0056] 202. The terminal acquires a first reference image and a second reference image. The content of the first reference image and the second reference image are the same, and the resolution of the first reference image is the first resolution, and the resolution of the second reference image is the second resolution. The first resolution is lower than the second resolution.

[0057] The value of the second resolution can be any value, and this application embodiment does not impose any restrictions on it.

[0058] Optionally, the first reference image and the second reference image are generated and stored in the terminal before the first target image is generated. Accordingly, the terminal directly obtains the stored first reference image and the second reference image.

[0059] Optionally, the first reference image and the second reference image are rendered by the GPU in response to the same image rendering instruction. The GPU renders the first reference image and the second reference image according to different rendering parameters. The rendered first reference image and the second reference image have the same content, but different resolutions. Optionally, the second reference image is generated by the GPU, and the first reference image is obtained by downsampling the second reference image. Therefore, the first reference image and the second reference image have the same content, but the resolution of the first reference image is lower than that of the second reference image.

[0060] 203. The terminal determines the first difference image between the first target image and the first reference image.

[0061] In this first difference image, the pixel value of each pixel is the difference between the pixel value of a pixel in the first target image and the pixel value of a pixel in the first reference image. For example, the pixel value of the first pixel in the first difference image is the difference between the pixel value of the first pixel in the first target image and the pixel value of the first pixel in the first reference image. The pixel value of the second pixel in the first difference image is the difference between the pixel value of the second pixel in the first target image and the pixel value of the second pixel in the first reference image, and so on.

[0062] 204. The terminal adjusts the resolution of the first difference image to obtain a second difference image with a second resolution.

[0063] In one possible implementation, the step includes: the terminal determining the ratio of the second resolution to the first resolution as a resolution adjustment parameter; and adjusting the resolution of the first difference image according to the resolution adjustment parameter to obtain a second difference image with a second resolution. Since the first resolution is lower than the second resolution, and the resolution adjustment parameter is the ratio of the second resolution to the first resolution, adjusting the resolution of the first difference image according to the resolution adjustment parameter can increase the resolution of the first difference image, thus obtaining a second difference image with a second resolution.

[0064] Optionally, the terminal determines the ratio of the second resolution to the first resolution as the interpolation coefficient of the image interpolation algorithm, and then performs image interpolation on the first difference image according to the interpolation coefficient using the image difference algorithm to obtain the second difference image. The image difference algorithm can be any type of image difference algorithm; for example, the image interpolation algorithm includes nearest neighbor interpolation, bilinear interpolation, bicubic interpolation, etc., and this application embodiment does not impose any limitations on this.

[0065] 205. The terminal overlays the second reference image and the second difference image to obtain the second target image.

[0066] In this context, the pixel value of each pixel in the second target image is the sum of the pixel values ​​of the pixels in the second reference image and the pixels in the second difference image. For example, the pixel value of the first pixel in the second target image is the sum of the pixel values ​​of the first pixel in the second reference image and the first pixel in the second difference image. The pixel value of the second pixel in the second target image is the sum of the pixel values ​​of the second pixel in the second reference image and the second pixel in the second difference image, and so on.

[0067] The solution provided in this application takes into account the relationship between a low-resolution target image and a low-resolution reference image, which is similar to the relationship between a high-resolution target image and a high-resolution reference image. Therefore, the difference image between the low-resolution target image and the low-resolution reference image is the same as the difference image between the high-resolution target image and the high-resolution reference image, only differing in resolution. Thus, the second difference image obtained by adjusting the resolution of the first difference image is the difference image between the second target image and the second reference image. The image obtained by superimposing the second reference image and the second difference image is the second target image with the same content as the first target image but at the second resolution. Therefore, this solution can improve image resolution. Furthermore, due to the simplicity of the process, this solution is not only highly efficient in improving image resolution but also consumes less power, thereby improving the performance of the device when processing images.

[0068] The above Figure 2 The illustrated embodiment simply describes the process of reconstructing a high-resolution image from a low-resolution image. The following... Figure 3 The illustrated embodiment describes the process of reconstructing a high-resolution image on a patch-by-pattern basis.

[0069] See Figure 3 , Figure 3 A flowchart illustrating an exemplary embodiment of this application provides an image processing method, the method comprising:

[0070] 301. The terminal generates a first target image, and the resolution of the first target image is a first resolution.

[0071] 302. The terminal acquires a first reference image and a second reference image. The content of the first reference image and the second reference image are the same, and the resolution of the first reference image is the first resolution, and the resolution of the second reference image is the second resolution. The first resolution is lower than the second resolution.

[0072] For the implementation of steps 301-302, please refer to steps 201-202 above, which will not be repeated here.

[0073] 303. The terminal determines multiple first target image blocks in the first target image.

[0074] The size of the first target image block can be arbitrary. For example, a 5x5 size means that the first target image block includes 5 rows and 5 columns of pixels, or that the first target image includes 25 pixels. Optionally, the sizes of multiple first target image blocks in the first target image may be the same or different, and this embodiment does not impose any restrictions on this.

[0075] Optionally, these multiple target image blocks do not overlap, and the pixels in these multiple target image blocks cover all the pixels in the first target image. For example, after the terminal determines a first target image block in the first target image, it determines the next first target image block that is adjacent to and does not overlap with the first target image block, and then determines the next first target image block that is adjacent to and does not overlap with the newly determined first target image block, until the pixels in the determined multiple target image blocks cover all the pixels in the first target image.

[0076] Optionally, these multiple target image blocks do not overlap, and the pixels in these multiple target image blocks cover all the pixels in the first target image. For example, after the terminal determines a first target image block in the first target image, it determines the next first target image block that is adjacent to and does not overlap with the first target image block, and then determines the next first target image block that is adjacent to and does not overlap with the newly determined first target image block, until the pixels in the determined multiple target image blocks cover all the pixels in the first target image.

[0077] Optionally, adjacent image blocks in these multiple target image blocks may overlap, and the pixels in these multiple target image blocks may cover all pixels in the first target image. For example, after the terminal determines a first target image block in the first target image, it determines the next first target image block that does not completely overlap with the first target image block, and then determines the next first target image block that does not completely overlap with the newly determined first target image block, until the pixels in the determined multiple target image blocks cover all pixels in the first target image.

[0078] 304. In the first reference image, the terminal determines the first reference image block that matches each first target image block.

[0079] Optionally, for any first target image block, the terminal determines the image block in the first reference image that best matches the first target image block, and the image block is the first reference image block that matches the first target image.

[0080] Optionally, the terminal uses an image patch matching algorithm to determine the first reference image patch that matches each first target image patch in the first reference image. This image patch matching algorithm can be of any type. For example, image patch matching algorithms include feature-point-based patch matching algorithms, patch matching optical flow algorithms, exhaustive search (ES), etc., and this application embodiment does not impose any limitations on this.

[0081] 305. The terminal determines the first difference image block between each first target image block and the matching first reference image block.

[0082] Wherein, the pixel value of the pixel in the first difference image block between the first target image block and the matching first reference image block is the difference between the pixel value of the pixel in the first target image block and the pixel value of the pixel in the matching first reference image block.

[0083] After step 305, the terminal obtains a plurality of first difference image blocks. Accordingly, the first difference image between the first target image and the first reference image includes these plurality of first difference image blocks.

[0084] In this embodiment, based on the first target image block and the matching first reference image block, the pixel value of the pixel in the first difference image block is determined to be the difference between the pixel value of the pixel in the first target image block and the pixel value of the pixel in the matching first reference image block. Then, the second difference image block obtained from the first difference image block can represent the difference between the pixel value of the pixel in the second target image block and the pixel value of the pixel in the matching second reference image block. In this way, the pixel value of the pixel in the second target image block can be determined based on the pixel value of the pixel in the second reference image and the pixel values ​​of the pixel in the multiple second difference image blocks, thereby generating a high-resolution second target image.

[0085] 306. The terminal adjusts the resolution of each first difference image block to obtain the corresponding second difference image block with the second resolution.

[0086] Optionally, the terminal determines the ratio of the second resolution to the first resolution as a resolution adjustment parameter; according to this resolution adjustment parameter, the resolution of the first difference image block is adjusted to obtain a second difference image block with the corresponding second resolution. Since the first resolution is lower than the second resolution, and the resolution adjustment parameter is the ratio of the second resolution to the first resolution, the resolution of the first difference image block can be increased according to this resolution adjustment parameter to obtain a second difference image block with the second resolution.

[0087] Optionally, the terminal determines the ratio of the second resolution to the first resolution as the interpolation coefficient of the image interpolation algorithm, and then uses the image interpolation algorithm to perform image interpolation on the first difference image block according to the difference coefficient to obtain the corresponding second difference image block.

[0088] 307. In the second reference image, the terminal determines the second reference image block that matches each first reference image block.

[0089] Optionally, for any first reference image block, the terminal determines the image block in the second reference image that best matches the first reference image block, and that image block is the second reference image block that matches the first reference image.

[0090] Optionally, the terminal uses an image patch matching algorithm to determine the second reference image block that matches each first reference image block in the second reference image. This image patch matching algorithm can be of any type.

[0091] Optionally, the first reference image is obtained by downsampling the second reference image. There is a mapping relationship between the pixels in the first reference image and the pixels in the second reference image. Accordingly, based on this mapping relationship, the terminal determines the second reference image block mapped to each first reference image block in the second reference image. Then, the second reference image block mapped to each first reference image block is the second reference image block matched for each first reference image block.

[0092] 308. The terminal superimposes the second difference image block corresponding to each first reference image block with the matched second reference image block to obtain the second target image block. The second difference image block corresponding to the first reference image block is: the second difference image block obtained based on the first difference image block corresponding to the first reference image block.

[0093] In this second target image block, the pixel value of each pixel is the sum of the pixel values ​​of the pixels in the second reference image block and the pixel values ​​of the pixels in the second difference image block. The second reference image block matches the first reference image block, and the second difference image block is obtained based on the first difference image block corresponding to the first reference image block. Therefore, the second difference image block corresponds to the second reference image block, representing the difference image block between the second target image block and the second reference image block in the second target image to be generated. Furthermore, the pixel value of each pixel in the second difference image block represents the difference between the pixel values ​​of the pixels in the second target image block and the pixel values ​​of the pixels in the second reference image block.

[0094] Figure 4 This is a schematic diagram illustrating the process of acquiring a second target image patch according to an embodiment of this application. Figure 4 The process of acquiring a second target image patch will be used as an example for illustration. (Reference) Figure 4 The terminal first determines a first target image block in a first target image, then determines a first reference image block in a first reference image that matches the first target image block. It calculates the difference image between the first target image block and the first reference image block to obtain a first difference image block. Then, it performs image interpolation on the first difference image to obtain a second difference image block. Furthermore, after determining the first reference image block, the terminal also determines a second reference image block in a second reference image that matches the first reference image block. This second reference image block is then superimposed on the second difference image block to obtain a second target image block. This second target image block is an image block within the second target image.

[0095] 309. The terminal stitches together multiple second target image blocks to obtain a second target image.

[0096] Since the pixels in the multiple first target image blocks cover all the pixels in the first target image, the pixels in the multiple high-resolution second target image blocks reconstructed based on these multiple first target image blocks actually cover all the pixels in the high-resolution image corresponding to the first target image block. Therefore, the second target image obtained by stitching together these multiple second target image blocks is an image with the same content as the first target image but with a higher resolution.

[0097] In one possible implementation, adjacent image blocks among the plurality of first target image blocks determined by the terminal in step 303 overlap, meaning these two image blocks include the same pixels. Therefore, the plurality of second target image blocks reconstructed from the plurality of first target image blocks overlap. Accordingly, the terminal stitches the plurality of second target image blocks to obtain a second target image, including: the terminal removing duplicate pixels from the plurality of second target image blocks; and stitching the deduplicated plurality of second target image blocks into a second target image. This avoids stitching the same pixel multiple times from the plurality of second target image blocks, thereby preventing the second target image blocks from including multiple duplicate pixels and ensuring the accuracy of the obtained second target image.

[0098] It should be noted that, as demonstrated by experiments, compared to the second target image generated when multiple first target image blocks do not overlap, the second target image generated when there is overlap between adjacent image blocks in multiple first target images has a more uniform transition between multiple image blocks, no obvious boundaries, and higher image quality.

[0099] In this embodiment, since the second difference image block obtained by increasing the resolution of the first difference image block between the first target image block and the matching first reference image block is the second difference image block corresponding to the first reference image block, superimposing the second difference image block corresponding to the first reference image block with the second reference image block matching the first reference image block is equivalent to superimposing the second difference image block with the corresponding second reference block to obtain a second target image block of the second resolution. The content of the second target image block is the same as the content of the first target image block matching the first reference image block. In this way, a second target image block with a higher resolution is constructed for multiple first target image blocks in the first reference image. Therefore, by stitching these multiple second target image blocks together, a high-resolution second target image can be obtained.

[0100] In one possible implementation, the image processing method described above is executed by a terminal, which includes a GPU and an HWA. Accordingly, the terminal generates a first target image via the GPU and sends it to the HWA. Through the HWA, it acquires a first reference image and a second reference image, as well as a first difference image between the first target image and the first reference image. Then, the terminal adjusts the resolution of the first difference image through the HWA to obtain a second difference image. The second reference image and the second difference image are then superimposed to obtain the second target image. Optionally, before generating the first target image via the GPU, the terminal generates the first and second reference images via the GPU, sends them to the HWA, and stores them through the HWA. Subsequently, after receiving the first target image, the HWA increases the resolution of the first target image based on the first and second reference images to obtain the second target image.

[0101] In this embodiment, considering the high power consumption of rendering high-resolution images with a GPU, after obtaining a reference image through GPU rendering, a low-resolution image is subsequently rendered with a GPU. The high-resolution image is then reconstructed using HWA based on the low-resolution image rendered by the GPU and the reference image. This can greatly reduce the GPU load, thereby reducing the power consumption of the terminal and improving the terminal performance.

[0102] Figure 5 This is a schematic diagram illustrating an image processing procedure based on a hardware accelerator, as provided in an embodiment of this application. Wherein, Figure 5 This explanation uses the generation of high-resolution images corresponding to multiple low-resolution images as an example. (Reference) Figure 5 The image processor generates a high-resolution image of frame T and downsamples it to obtain a low-resolution image of frame T. The image processor then sends both the high-resolution and low-resolution images of frame T to the hardware accelerator. The hardware accelerator stores the high-resolution and low-resolution images of frame T. Subsequently, the image processor generates a low-resolution image of frame T+1 and sends it to the hardware accelerator. The hardware accelerator then reconstructs the high-resolution image of frame T+1 based on the high-resolution and low-resolution images of frame T+1. The generation process of the high-resolution images of frames T+2 and T+3 is similar to that of the high-resolution image of frame T+1, and will not be repeated here. Here, T is a positive integer, high-resolution image refers to an image with a second resolution, and low-resolution image refers to an image with a first resolution. Optionally, Figure 5 The multiple hardware accelerators shown are the same hardware accelerator.

[0103] The solution provided in this application takes into account the relationship between a low-resolution target image and a low-resolution reference image, which is similar to the relationship between a high-resolution target image and a high-resolution reference image. Therefore, the difference image between the low-resolution target image and the low-resolution reference image is the same as the difference image between the high-resolution target image and the high-resolution reference image, only differing in resolution. Thus, the second difference image obtained by adjusting the resolution of the first difference image is the difference image between the second target image and the second reference image. The image obtained by superimposing the second reference image and the second difference image is the second target image with the same content as the first target image but at the second resolution. Therefore, this solution can improve image resolution. Furthermore, due to the simplicity of the process, this solution is not only highly efficient in improving image resolution but also consumes less power, thus improving the device's performance when processing images. Compared to providing image resolution through a neural network model, this application does not require training a neural network model and reduces the requirements for device computing power, memory bandwidth, and system power consumption, making it applicable to a wider range of scenarios.

[0104] Furthermore, based on the first target image block and the matched first reference image block, the pixel value of the pixel in the first difference image block is determined to be the difference between the pixel value of the pixel in the first target image block and the pixel value of the pixel in the matched first reference image block. Then, the second difference image block obtained from the first difference image block can represent the difference between the pixel value of the pixel in the second target image block and the pixel value of the pixel in the matched second reference image block. In this way, the pixel value of the pixel in the second target image block can be determined based on the pixel value of the pixel in the second reference image and the pixel values ​​of the pixel in multiple second difference image blocks, thereby generating a high-resolution second target image.

[0105] Furthermore, since the second difference image block obtained by increasing the resolution of the first difference image block between the first target image block and the matching first reference image block is the second difference image block corresponding to the first reference image block, superimposing the second difference image block corresponding to the first reference image block with the second reference image block matching the first reference image block is equivalent to superimposing the second difference image block with the corresponding second reference block. The content of the second target image block with the second resolution obtained is the same as the content of the first target image block matching the first reference image block. In this way, a high-resolution second target image block is constructed for multiple first target image blocks in the first reference image. Therefore, by stitching these multiple second target image blocks together, a high-resolution second target image can be obtained.

[0106] Furthermore, considering the high power consumption of rendering high-resolution images using the GPU, after obtaining a reference image through GPU rendering, a low-resolution image is subsequently rendered using the GPU. The high-resolution image is then reconstructed using HWA based on the low-resolution image rendered by the GPU and the reference image. This can greatly reduce the GPU load and thus reduce the power consumption of the terminal.

[0107] Furthermore, experiments have shown that, compared to the second target image generated when multiple first target image blocks do not overlap, the second target image generated when there is overlap between adjacent image blocks in multiple first target images has a more uniform transition between multiple image blocks, no obvious boundaries, and higher image quality.

[0108] The above Figure 2 The illustrated embodiment briefly describes a method for reconstructing a high-resolution image from a low-resolution image. The following... Figure 6 The illustrated embodiment describes the application of this method in a virtual scene update scenario.

[0109] See Figure 6 , Figure 6 A flowchart illustrating an exemplary embodiment of this application provides an image processing method, the method comprising:

[0110] 601. The terminal responds to the first scene update command and renders a first scene image of a first resolution based on the first scene update command and the currently displayed virtual scene image.

[0111] The first scene update instruction instructs the updating of the currently displayed virtual scene image. Optionally, the first scene update instruction carries scene update parameters, and accordingly, the terminal renders a first scene image of a first resolution based on the scene update parameters and the currently displayed virtual scene image. The scene update parameters include resolution, displacement parameters of objects in the virtual scene, shape change parameters, etc., which are not limited in this embodiment.

[0112] Optionally, the first scene update instruction is generated based on an interactive operation. For example, the terminal receives an interactive operation performed in a virtual scene and generates the first scene update instruction based on that interactive operation. For example, during gameplay, the terminal detects an interactive operation performed in a displayed virtual game scene and generates the first scene update instruction based on that interactive operation.

[0113] Optionally, the first scene update instruction is generated according to the scene update conditions of the virtual scene. The scene update conditions can include conditions of any type. For example, if the scene update conditions include adding a target element to the virtual scene at a target time, then when the target time is reached, the terminal generates a scene update instruction to instruct the rendering of a virtual scene image that includes the target element.

[0114] Optionally, the first scene update instruction is generated by the CPU in the terminal. For example, the terminal generates a scene update instruction through the CPU and sends the scene update instruction to the GPU, which then renders the virtual scene image based on the scene update instruction and the currently displayed virtual scene image.

[0115] 602. The terminal acquires a first reference image at a first resolution and a second reference image at a second resolution. The first reference image and the second reference image are scene images with the same content rendered in response to a second scene update command. The first resolution is lower than the second resolution, and the second scene update command is generated earlier than the first scene update command.

[0116] In response to a first scene update command, before rendering a first scene image at a first resolution based on the first scene update command and the currently displayed virtual scene image, the terminal renders a second resolution image in response to a second scene update command. After updating the virtual scene based on this second resolution image, the terminal also acquires a first resolution image with the same content as the first resolution image, using these two images as reference images. Subsequent scene update commands do not require rendering a high-resolution virtual scene image; instead, a low-resolution virtual scene image is rendered, and the corresponding high-resolution virtual scene image is reconstructed based on the two reference images. The acquisition method for the second scene update command is the same as that for the first scene update command, and will not be elaborated further here.

[0117] 603. The terminal determines the first difference image between the first scene image and the first reference image, and adjusts the resolution of the first difference image to obtain a second difference image with a second resolution.

[0118] 604. The terminal overlays the second reference image and the second difference image to obtain the second scene image, and updates the currently displayed virtual scene image based on the second scene image.

[0119] Please refer to the above for the implementation methods of steps 603-604. Figure 2 and Figure 3 The embodiments shown are not described in detail here.

[0120] Optionally, the terminal updates the currently displayed virtual scene image based on the second scene image in the following ways: the terminal directly displays the second scene image, or the terminal adjusts the resolution of the second scene image to the target resolution and then displays the virtual scene image at the target resolution. Optionally, the target resolution is the resolution of the terminal's display.

[0121] In this embodiment, considering that directly rendering a high-resolution virtual scene image consumes a lot of power, in response to the scene update command, instead of directly rendering a high-resolution virtual scene image, a low-resolution virtual scene image is rendered. Then, based on a previously rendered reference image, the resolution of the currently rendered virtual scene image is increased to obtain a high-resolution virtual scene image. This method of obtaining a high-resolution virtual scene image and updating the displayed virtual scene image based on it not only reduces terminal power consumption but also improves the smoothness of virtual scene updates and enhances the user experience.

[0122] Compared to the above Figure 6 The embodiments shown below Figure 7 The illustrated embodiment describes under what circumstances the terminal renders a first scene image and obtains a second scene image with a higher resolution based on the first scene image in response to a first scene update command, and under what circumstances the terminal directly renders the second scene image.

[0123] See Figure 7 , Figure 7 A flowchart illustrating an exemplary embodiment of this application provides an image processing method, the method comprising:

[0124] 701. The terminal generates a first scene update instruction based on the first interactive operation detected at the moment.

[0125] Please refer to step 601 above for how to implement this step; it will not be repeated here.

[0126] 702. The terminal responds to the first scene update instruction and determines the magnitude of the change in the operation of the first interactive operation relative to the second interactive operation, wherein the second interactive operation is the last interactive operation detected before the first interactive operation.

[0127] The interactive operation includes any type of interactive operation, such as an interactive operation to control the movement of a virtual object in the currently displayed virtual scene image, or an interactive operation to control the virtual object to release a skill, etc. This application embodiment does not limit this.

[0128] The magnitude of the change in the first interactive operation relative to the second interactive operation indicates the magnitude of change in the virtual scene. For example, the magnitude of this change is positively correlated with the magnitude of change in the virtual scene; the greater the magnitude of the change, the greater the magnitude of change in the virtual scene. For instance, if the first and second interactive operations are sliding operations performed in specific areas of the terminal interface to control the movement of virtual objects in the virtual scene image, and the greater the sliding distance, the greater the displacement of the virtual object in the virtual scene, and the greater the magnitude of change in the virtual scene.

[0129] 703. When the change in operation is less than the magnitude threshold, the terminal renders the first scene image based on the first scene update instruction and the currently displayed virtual scene image.

[0130] The amplitude threshold can be flexibly set according to actual needs.

[0131] In this embodiment, considering that the change in the first interactive operation detected at the moment is less than the magnitude threshold relative to the second interactive operation, that is, the change in the operation is small, it indicates that the change in the virtual scene is not large, and thus it indicates that the second scene image to be generated at the second resolution is not much different from the reference image. In this case, the low-resolution first scene image is rendered, and the resolution of the first scene image is increased based on the reference image to obtain the second scene image, thus ensuring the quality of the rendered second scene image.

[0132] 704. The terminal acquires a first reference image at a first resolution and a second reference image at a second resolution.

[0133] 705. The terminal determines the first difference image between the first scene image and the first reference image, and adjusts the resolution of the first difference image to obtain a second difference image with a second resolution.

[0134] 706. The terminal overlays the second reference image and the second difference image to obtain the second scene image.

[0135] Please refer to the above for the implementation methods of steps 704-706. Figure 2 and Figure 3 The embodiments shown are not described in detail here.

[0136] 707. When the change in operation is not less than the magnitude threshold, the terminal renders a second scene image of a second resolution based on the first scene update instruction and the currently displayed virtual scene image.

[0137] Considering that the change in the first interaction operation relative to the second interaction operation is less than the threshold, meaning the change is relatively large, it indicates that the virtual scene has a large change. This further indicates that the second scene image to be generated at the second resolution has a large change compared to the reference image. In this case, if a low-resolution first scene image is rendered and a high-resolution second scene image is reconstructed based on the reference image, the quality of the second scene image may be poor. Therefore, in this case, directly rendering the high-resolution second scene image can ensure the quality of the rendered second scene image.

[0138] 708. The terminal updates the currently displayed virtual scene image based on the second scene image.

[0139] Please refer to step 604 above for how to implement this step; it will not be repeated here.

[0140] Optionally, in step 707, after the terminal renders the second scene image at the second resolution, it also acquires a first scene image with the same content as the second scene image but at the first resolution. This second scene image and the first scene image are used as new reference images. When a scene update command is received again, a high-resolution virtual scene image is acquired based on the new reference image, thereby updating the virtual scene. In other words, during the virtual scene update process, the reference image is also continuously updated to ensure the quality of the high-resolution virtual scene image reconstructed based on the reference image.

[0141] The above Figure 7 The illustrated embodiment describes how, in response to a first scene update command, the terminal determines whether to render a first scene image or directly render a second scene image based on the magnitude of the operation change. Figure 8 The illustrated embodiment describes how, in response to a first scene update command, the terminal determines whether to render a first scene image or directly render a second scene image based on the similarity between the last rendered low-resolution image frame and a first reference image.

[0142] 801. The terminal responds to the first scene update command and determines the third scene image rendered after the first reference image, which is the last one with the first resolution.

[0143] 802. The terminal determines the similarity between the first reference image and the third scene image.

[0144] Optionally, the terminal performs feature extraction on the first reference image and the third scene image respectively to obtain the image features of the first reference image and the image features of the third scene image, and determines the similarity between the image features of the first reference image and the image features of the third scene image as the similarity between the first reference image and the third scene image.

[0145] 803. When the similarity is greater than the similarity threshold, the terminal renders the first scene image based on the first scene update instruction and the currently displayed virtual scene image.

[0146] The similarity threshold can be flexibly set according to actual needs.

[0147] In this embodiment, considering that the similarity between the third scene image and the first reference image is greater than the similarity threshold, it indicates that the change in the virtual scene is not significant. This further indicates that the second scene image to be generated at the second resolution is not significantly different from the reference image. In this case, the low-resolution first scene image is rendered, and the resolution of the first scene image is increased based on the reference image to obtain the second scene image, thus ensuring the quality of the rendered second scene image.

[0148] 804. The terminal acquires a first reference image at a first resolution and a second reference image at a second resolution.

[0149] 805. The terminal determines the first difference image between the first scene image and the first reference image, and adjusts the resolution of the first difference image to obtain a second difference image with a second resolution.

[0150] 806. The terminal overlays the second reference image and the second difference image to obtain the second scene image.

[0151] Please refer to the above for the implementation methods of steps 804-806. Figure 2 and Figure 3 The embodiments shown are not described in detail here.

[0152] 807. When the similarity is not greater than the similarity threshold, the terminal renders a second scene image with a second resolution based on the first scene update instruction and the currently displayed virtual scene image.

[0153] Considering that the similarity between the third scene image and the first reference image is no greater than the similarity threshold, it indicates that the virtual scene has a large range of changes. This means that the second scene image to be generated at the second resolution has a large change compared to the reference image. In this case, if the low-resolution first scene image is rendered and the high-resolution second scene image is reconstructed based on the reference image, the quality of the second scene image may be poor. Therefore, in this case, directly rendering the high-resolution second scene image can ensure the quality of the rendered second scene image.

[0154] 808. The terminal updates the currently displayed virtual scene image based on the second scene image.

[0155] Please refer to step 604 above for how to implement this step; it will not be repeated here.

[0156] Optionally, in step 807, after the terminal renders the second scene image at the second resolution, it also acquires a first scene image with the same content as the second scene image but at the first resolution. This second scene image and the first scene image are used as new reference images. When a scene update command is received again, a high-resolution virtual scene image is acquired based on the new reference image, thereby updating the virtual scene. In other words, during the virtual scene update process, the reference image is also continuously updated to ensure the quality of the high-resolution virtual scene image reconstructed based on the reference image.

[0157] One point that needs to be clarified is that, in addition to the above... Figure 7 and Figure 8 In addition to the embodiments shown, in response to a scene image update command, it is also possible to determine whether a low-resolution or high-resolution virtual scene image is being generated at present through other means. For example, in response to a first scene update command, the terminal determines the number of first-resolution virtual scene images that have been generated from the generation of the first reference image up to the current time. If the number reaches a threshold, a second-resolution virtual scene image is directly rendered. If the number does not reach the threshold, the first-resolution virtual scene image is rendered, and the resolution of the first-resolution virtual scene image is increased based on the reference image to obtain a second-resolution virtual scene image.

[0158] Another point to note is that, optionally, the pixel value of a pixel in the image described in the above embodiments is the value of the luminance signal (Y) of the pixel after the image has been converted to the YUV color space. Here, U and V are the two signal components that constitute color. Alternatively, the pixel value of a pixel is the value of the luminance signal (Y) of the pixel after the image has been converted to the YCbCr color space. Here, Cb refers to the blue chromaticity signal, and Cr refers to the red chromaticity signal. Since users are more sensitive to changes in brightness in an image than to color difference, image processing based on the luminance signal Y value can both ensure the quality of the generated image and reduce the amount of data processing, thus improving image processing efficiency.

[0159] The following are embodiments of the apparatus described in this application, which can be used to execute the embodiments of the method described in this application. For details not disclosed in the apparatus embodiments of this application, please refer to the embodiments of the method described in this application.

[0160] Please refer to Figure 9 This illustration shows a structural block diagram of an image processing apparatus provided in an exemplary embodiment of this application. The image processing apparatus includes:

[0161] Image generation module 901 is used to generate a first target image, the resolution of which is a first resolution;

[0162] Image acquisition module 902 is used to acquire a first reference image and a second reference image. The content of the first reference image and the second reference image are the same, and the resolution of the first reference image is a first resolution, and the resolution of the second reference image is a second resolution. The first resolution is lower than the second resolution.

[0163] Image determination module 903 is used to determine a first difference image between a first target image and a first reference image;

[0164] The resolution adjustment module 904 is used to adjust the resolution of the first difference image to obtain a second difference image with a second resolution.

[0165] The image overlay module 905 is used to overlay the second reference image and the second difference image to obtain the second target image.

[0166] The solution provided in this application takes into account the relationship between a low-resolution target image and a low-resolution reference image, which is similar to the relationship between a high-resolution target image and a high-resolution reference image. Therefore, the difference image between the low-resolution target image and the low-resolution reference image is the same as the difference image between the high-resolution target image and the high-resolution reference image, only differing in resolution. Thus, the second difference image obtained by adjusting the resolution of the first difference image is the difference image between the second target image and the second reference image. The image obtained by superimposing the second reference image and the second difference image is the second target image with the same content as the first target image but at the second resolution. Therefore, this solution can improve image resolution. Furthermore, due to the simplicity of the process, this solution is not only highly efficient in improving image resolution but also consumes less power, thereby improving the performance of the device when processing images.

[0167] In one possible implementation, the first difference image includes a plurality of first difference image blocks, and the image determination module 903 is used to determine a plurality of first target image blocks in the first target image; in the first reference image, a first reference image block matching each first target image block is determined; and a first difference image block between each first target image block and the matching first reference image block is determined.

[0168] In one possible implementation, the second difference image includes multiple second difference image blocks, and the image overlay module 905 includes:

[0169] An image determination unit is used to determine, in the second reference image, a second reference image block that matches each first reference image block;

[0170] The image overlay unit is used to overlay the second difference image block corresponding to each first reference image block with the matched second reference image block to obtain a second target image block. The second difference image block corresponding to the first reference image block is: the second difference image block obtained based on the first difference image block corresponding to the first reference image block.

[0171] The image stitching unit is used to stitch together multiple second target image blocks to obtain a second target image.

[0172] In one possible implementation, there is overlap between two adjacent image blocks in a plurality of first target image blocks;

[0173] The image stitching unit is used to remove duplicate pixels from multiple second target image blocks and stitch the deduplicated second target image blocks into a second target image.

[0174] In one possible implementation, the resolution adjustment module 904 is used to determine the ratio of the second resolution to the first resolution as a resolution adjustment parameter; and adjust the resolution of the first difference image according to the resolution adjustment parameter to obtain the second difference image.

[0175] In one possible implementation, the device is configured on a terminal, which includes a graphics processor (GPU) and a hardware accelerator (HWA).

[0176] Image generation module 901 is used to generate a first target image through GPU and send the first target image to HWA;

[0177] Image acquisition module 902 is used to acquire a first reference image and a second reference image via HWA;

[0178] Image determination module 903 is used to determine a first difference image between a first target image and a first reference image through HWA;

[0179] The resolution adjustment module 904 is used to adjust the resolution of the first difference image through HWA to obtain the second difference image;

[0180] The image overlay module 905 is used to overlay the second reference image and the second difference image through HWA to obtain the second target image.

[0181] Please refer to Figure 10 This illustration shows a structural block diagram of a scene update apparatus provided in an exemplary embodiment of this application. The scene update apparatus includes:

[0182] The image rendering module 1001 is used to render a first scene image of a first resolution based on the first scene update command and the currently displayed virtual scene image in response to the first scene update command.

[0183] The image acquisition module 1002 is used to acquire a first reference image with a first resolution and a second reference image with a second resolution. The first reference image and the second reference image are scene images with the same content that are rendered in response to a second scene update instruction. The first resolution is lower than the second resolution, and the second scene update instruction is generated earlier than the first scene update instruction.

[0184] The resolution adjustment module 1003 is used to determine a first difference image between a first scene image and a first reference image, and adjust the resolution of the first difference image to obtain a second difference image with a second resolution.

[0185] The scene update module 1004 is used to overlay the second reference image and the second difference image to obtain the second scene image, and update the currently displayed virtual scene image based on the second scene image.

[0186] In this embodiment, considering that directly rendering a high-resolution virtual scene image consumes a lot of power, in response to the scene update command, instead of directly rendering a high-resolution virtual scene image, a low-resolution virtual scene image is rendered. Then, based on a previously rendered reference image, the resolution of the currently rendered virtual scene image is increased to obtain a high-resolution virtual scene image. This method of obtaining a high-resolution virtual scene image and updating the displayed virtual scene image based on it not only reduces terminal power consumption but also improves the smoothness of virtual scene updates and enhances the user experience.

[0187] In one possible implementation, the first scene update instruction is generated based on the currently detected first interaction operation, and the image rendering module 1001 includes:

[0188] The amplitude determination unit is used to determine the amplitude of the operation change of the first interactive operation relative to the second interactive operation in response to the first scene update instruction. The second interactive operation is the last interactive operation detected before the first interactive operation.

[0189] The first rendering unit is used to render the first scene image based on the first scene update instruction and the currently displayed virtual scene image when the magnitude of the operation change is less than the magnitude threshold.

[0190] In one possible implementation, the first rendering unit is further configured to render a second scene image of a second resolution based on a first scene update instruction and the currently displayed virtual scene image, provided that the magnitude of the operation change is not less than a magnitude threshold.

[0191] In one possible implementation, the image rendering module 1001 includes:

[0192] An image determination unit is configured to determine, in response to a first scene update instruction, the last third scene image rendered after the first reference image at a first resolution.

[0193] A similarity determination unit is used to determine the similarity between the first reference image and the third scene image;

[0194] The second rendering unit is used to render the first scene image based on the first scene update instruction and the currently displayed virtual scene image when the similarity is greater than the similarity threshold.

[0195] In one possible implementation, the second rendering unit is further configured to render a second scene image of a second resolution based on the first scene update instruction and the currently displayed virtual scene image, provided that the similarity is not greater than a similarity threshold.

[0196] It should be noted that the apparatus provided in the above embodiments is only illustrated by the division of the above functional modules when implementing its functions. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the computer device can be divided into different functional modules to complete all or part of the functions described above. In addition, the apparatus and method embodiments provided in the above embodiments belong to the same concept, and the specific implementation process can be found in the method embodiments, which will not be repeated here.

[0197] This application provides a computer device including a processor and a memory; the memory stores at least one piece of program code, which is executed by the processor to implement the methods provided in the above-described method embodiments.

[0198] In some embodiments, the computer device is a terminal; please refer to [reference needed]. Figure 11 This diagram illustrates a structural block diagram of a terminal provided in an exemplary embodiment of this application. In some embodiments, the terminal 1100 is a smartphone, tablet, wearable device, or other terminal capable of accessing a wireless local area network as a wireless station. The terminal 1100 in this application includes at least one or more of the following components: a processor 1110, a memory 1120, and at least two wireless links 1130.

[0199] In some embodiments, processor 1110 includes one or more processing cores. Processor 1110 connects to various parts within terminal 1100 using various interfaces and lines, and performs various functions and processes data of terminal 1100 by running or executing program code stored in memory 1120 and calling data stored in memory 1120. In some embodiments, processor 1110 is implemented using at least one hardware form of Digital Signal Processing (DSP), Field-Programmable Gate Array (FPGA), or Programmable Logic Array (PLA). Processor 1110 can integrate one or more of the following: Central Processing Unit (CPU), Graphics Processing Unit (GPU), Neural-network Processing Unit (NPU), and modem. Specifically, the CPU primarily handles the operating system, user interface, and applications; the GPU is responsible for rendering and drawing the content required for display on the screen; the NPU is used to implement Artificial Intelligence (AI) functions; and the modem is used for wireless communication. It is understandable that the aforementioned modem could also be implemented as a separate chip without being integrated into the processor 1110.

[0200] In some embodiments, the processor 1110 is used to control the operating status of at least two wireless links 1130. Accordingly, the processor 1110 is a processor integrating a Wireless Fidelity (Wi-Fi) chip. This Wi-Fi chip is a chip with dual Wi-Fi processing capabilities. For example, the Wi-Fi chip is a dual-band dual-concurrent (DBDC) chip, or a dual-band simultaneous (DBS) chip, etc.

[0201] In some embodiments, memory 1120 includes random access memory (RAM), and in some embodiments, memory 1120 includes read-only memory (ROM). In some embodiments, memory 1120 includes non-transitory computer-readable storage medium. Memory 1120 can be used to store program code. Memory 1120 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for at least one function (such as touch functionality, sound playback functionality, image playback functionality, etc.), instructions for implementing the various method embodiments described below, etc.; the data storage area may store data created based on the use of terminal 1100 (such as audio data, phonebook, etc.).

[0202] In some embodiments, the memory 1120 stores reception schemes for different wireless links 1130 receiving beacon frames, as well as identifiers of access nodes connected to different wireless links 1130, identifiers of wireless links 1130, etc.

[0203] The at least two wireless links 1130 are used to connect different access points (APs). They receive downlink data from the APs. These different access points can be access points within the same router or access points within different routers.

[0204] In some embodiments, the terminal 1100 further includes a display screen. The display screen is a display component used to display a user interface. In some embodiments, the display screen is a touch-enabled display screen, allowing users to perform touch operations on the display screen using fingers, styluses, or any suitable object. In some embodiments, the display screen is typically located on the front panel of the terminal 1100. In some embodiments, the display screen is designed as a full-screen, curved screen, irregularly shaped screen, dual-sided screen, or foldable screen. In some embodiments, the display screen is also designed as a combination of a full-screen and a curved screen, or a combination of an irregularly shaped screen and a curved screen, etc., which are not limited in this embodiment.

[0205] In addition, those skilled in the art will understand that the structure of the terminal 1100 shown in the above figures does not constitute a limitation on the terminal 1100. The terminal 1100 may include more or fewer components than shown, or combine certain components, or have different component arrangements. For example, the terminal 1100 may also include components such as a microphone, speaker, input unit, sensor, audio circuit, module, power supply, and Bluetooth module, which will not be described in detail here.

[0206] In some embodiments, the computer device is a server; please refer to [reference needed]. Figure 12 This diagram illustrates a structural block diagram of a server provided in an exemplary embodiment of this application. The server 1200 can vary significantly due to different configurations or performance characteristics. It may include one or more Central Processing Units (CPUs) 1201 and one or more memories 1202. The memories 1202 store at least one line of program code, which is loaded and executed by the processor 1201 to implement the methods provided in the various method embodiments described above. Of course, the server may also have wired or wireless network interfaces, a keyboard, and input / output interfaces for input and output. The server may also include other components for implementing device functions, which will not be elaborated upon here.

[0207] This application also provides a computer-readable storage medium storing at least one piece of program code, which is loaded and executed by the processor to implement the methods shown in the above embodiments.

[0208] This application also provides a chip including programmable logic circuitry and / or program instructions, which, when run on a terminal, implement the methods shown in the various embodiments above.

[0209] This application also provides a computer program product that stores at least one piece of program code for execution by a processor to implement the methods shown in the above embodiments.

[0210] The sequence numbers of the embodiments in this application are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0211] Those skilled in the art will understand that all or part of the steps in the image processing method of the above embodiments can be implemented by hardware, or by a program instructing related hardware. The program can be stored in a computer-readable storage medium, such as a read-only memory, a disk, or an optical disk. The above descriptions are merely optional embodiments of this application and are not intended to limit the application. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of this application should be included within the scope of protection of this application.

Claims

1. An image processing method, characterized in that, The method includes: Generate a first target image, wherein the resolution of the first target image is a first resolution; Acquire a first reference image and a second reference image, wherein the content of the first reference image and the second reference image are the same, and the resolution of the first reference image is the first resolution, the resolution of the second reference image is the second resolution, and the first resolution is lower than the second resolution; Determine a first difference image between the first target image and the first reference image; Based on the resolution adjustment parameters, the resolution of the first difference image is adjusted to obtain a second difference image with the second resolution. The resolution adjustment parameters are determined based on the ratio of the second resolution to the first resolution. The second reference image is superimposed on the second difference image to obtain the second target image.

2. The method according to claim 1, characterized in that, The first difference image includes a plurality of first difference image blocks, and determining the first difference image between the first target image and the first reference image includes: Identify multiple first target image blocks within the first target image; In the first reference image, a first reference image block matching each first target image block is determined; Each first target image block is determined to be a first difference image block between itself and the matched first reference image block.

3. The method according to claim 2, characterized in that, The second difference image includes multiple second difference image blocks. The step of overlaying the second reference image with the second difference image to obtain the second target image includes: In the second reference image, a second reference image block matching each of the first reference image blocks is determined; The second difference image block corresponding to each of the first reference image blocks is superimposed with the matching second reference image block to obtain the second target image block. The second difference image block corresponding to the first reference image block is: the second difference image block obtained based on the first difference image block corresponding to the first reference image block. Multiple second target image blocks are stitched together to obtain the second target image.

4. The method according to claim 3, characterized in that, There is overlap between two adjacent image blocks in the plurality of first target image blocks; The step of stitching together multiple second target image blocks to obtain the second target image includes: The pixels in the plurality of second target image blocks are deduplicated; The multiple deduplicated second target image blocks are stitched together to form the second target image.

5. The method according to any one of claims 1-4, characterized in that, The method is executed by a terminal, the terminal including a graphics processing unit (GPU) and a hardware accelerator (HWA), and the method further includes: The first target image is generated using the GPU and then sent to the HWA. The first reference image and the second reference image, as well as the first difference image between the first target image and the first reference image, are obtained through the HWA. The resolution of the first difference image is adjusted to obtain the second difference image. The second reference image and the second difference image are superimposed to obtain the second target image.

6. A scene update method, characterized in that, The method includes: In response to a first scene update command, a first scene image of a first resolution is rendered based on the first scene update command and the currently displayed virtual scene image; A first reference image at the first resolution and a second reference image at the second resolution are obtained. The first reference image and the second reference image are scene images with the same content rendered in response to a second scene update instruction. The first resolution is lower than the second resolution, and the second scene update instruction is generated earlier than the first scene update instruction. A first difference image is determined between the first scene image and the first reference image, and the resolution of the first difference image is adjusted based on a resolution adjustment parameter to obtain a second difference image with a second resolution. The resolution adjustment parameter is determined based on the ratio of the second resolution to the first resolution. The second reference image is superimposed on the second difference image to obtain the second scene image. Based on the second scene image, the currently displayed virtual scene image is updated.

7. The method according to claim 6, characterized in that, The first scene update instruction is generated based on the currently detected first interaction operation. The step of rendering a first scene image at a first resolution in response to the first scene update instruction, based on the first scene update instruction and the currently displayed virtual scene image, includes: In response to the first scene update instruction, the magnitude of the change in the first interactive operation relative to the second interactive operation is determined, wherein the second interactive operation is the last interactive operation detected before the first interactive operation; If the magnitude of the change in operation is less than the magnitude threshold, the first scene image is rendered based on the first scene update instruction and the currently displayed virtual scene image.

8. The method according to claim 7, characterized in that, The method further includes: If the magnitude of the operation change is not less than the magnitude threshold, the second scene image at the second resolution is rendered based on the first scene update instruction and the currently displayed virtual scene image.

9. The method according to claim 6, characterized in that, The step of responding to a first scene update command and rendering a first scene image of a first resolution based on the first scene update command and the currently displayed virtual scene image includes: In response to the first scene update instruction, determine the last third scene image rendered after the first reference image at the first resolution; Determine the similarity between the first reference image and the third scene image; If the similarity is greater than the similarity threshold, the first scene image is rendered based on the first scene update instruction and the currently displayed virtual scene image.

10. The method according to claim 9, characterized in that, The method further includes: If the similarity is not greater than the similarity threshold, the second scene image at the second resolution is rendered based on the first scene update instruction and the currently displayed virtual scene image.

11. An image processing apparatus, characterized in that, The device includes: An image generation module is used to generate a first target image, wherein the resolution of the first target image is a first resolution; An image acquisition module is used to acquire a first reference image and a second reference image, wherein the content of the first reference image and the second reference image are the same, and the resolution of the first reference image is the first resolution, the resolution of the second reference image is the second resolution, and the first resolution is lower than the second resolution; The image determination module is used to determine a first difference image between the first target image and the first reference image; A resolution adjustment module is used to adjust the resolution of the first difference image based on a resolution adjustment parameter to obtain a second difference image with a second resolution, wherein the resolution adjustment parameter is determined based on the ratio of the second resolution to the first resolution; An image overlay module is used to overlay the second reference image and the second difference image to obtain a second target image.

12. A scene update device, characterized in that, The device includes: An image rendering module is used to render a first scene image of a first resolution based on the first scene update command and the currently displayed virtual scene image in response to the first scene update command. The image acquisition module is used to acquire a first reference image at the first resolution and a second reference image at the second resolution. The first reference image and the second reference image are scene images with the same content rendered in response to a second scene update instruction. The first resolution is lower than the second resolution, and the second scene update instruction is generated earlier than the first scene update instruction. A resolution adjustment module is used to determine a first difference image between the first scene image and the first reference image, and adjust the resolution of the first difference image based on resolution adjustment parameters to obtain a second difference image with a second resolution. The resolution adjustment parameters are determined based on the ratio of the second resolution to the first resolution. The scene update module is used to overlay the second reference image and the second difference image to obtain a second scene image, and update the currently displayed virtual scene image based on the second scene image.

13. A computer device, characterized in that, The computer device includes a processor and a memory; the memory stores at least one piece of program code, which is executed by the processor to implement the method as claimed in any one of claims 1 to 10.

14. A computer-readable storage medium, characterized in that, The storage medium stores at least one piece of program code, which is executed by a processor to implement the method as described in any one of claims 1 to 10.

15. A chip, characterized in that, The chip includes programmable logic circuitry and / or program instructions, which, when the chip is run on a computer device, are used to implement the method as described in any one of claims 1 to 10.

16. A computer program product, comprising a computer program, characterized in that, The computer program product stores at least one piece of program code, which is executed by a processor to implement the method as described in any one of claims 1 to 10.

Images