Image processing method and related equipment

Abstract

The invention discloses an image processing method and related equipment based on the field of artificial intelligence. The method comprises the steps of obtaining a to-be-optimized first texture image; the first texture image is used for rendering a three-dimensional object in a game scene; calling a texture optimization model to perform feature extraction processing on the first texture image to obtain a texture feature plane of the first texture image; calling a texture optimization model to carry out texture optimization processing on the texture feature plane of the first texture image to obtain a second texture image; the second texture image is used for rendering a three-dimensional object, and the rendering performance of the second texture image is superior to that of the first texture image; adopting the second texture image to render the three-dimensional object in the game scene; the quality of the second texture image obtained through optimization can be improved, the quality of a rendering result when the second texture image is adopted for rendering is further improved, and the rendering performance of the second texture image is further improved.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence, and in particular to an image processing method, an image processing apparatus, a computer device, a computer-readable storage medium, and a computer program product. Background Art

[0002] In many game scenarios such as mixed reality, many three-dimensional objects (such as 3D game characters in games) suffer from poor rendering quality when using their texture images for rendering, and thus poor rendering performance. Based on this, how to improve the rendering performance of texture images of three-dimensional objects is a current research hotspot, but existing methods are usually based on geometry or camera angles, and are not very effective in improving the rendering performance of texture images. Summary of the Invention

[0003] The embodiments of the present application provide an image processing method and related equipment, which can improve the quality of the optimized second texture image and further improve the quality of the rendering result when rendering using the second texture image, that is, further improve the rendering performance of the second texture image.

[0004] In one aspect, an embodiment of the present application provides an image processing method, the method comprising:

[0005] Obtaining a first texture image to be optimized; the first texture image is used to render a three-dimensional object in a game scene;

[0006] Calling the texture optimization model to perform feature extraction processing on the first texture image to obtain a texture feature plane of the first texture image;

[0007] Calling a texture optimization model to perform texture optimization processing on a texture feature plane of the first texture image to obtain a second texture image; the second texture image is used to render a three-dimensional object, and the rendering performance of the second texture image is better than the rendering performance of the first texture image;

[0008] The second texture image is used to render the three-dimensional object in the game scene.

[0009] In one aspect, an embodiment of the present application provides an image processing device, comprising:

[0010] An acquisition unit, configured to acquire a first texture image to be optimized; the first texture image is used to render a three-dimensional object in a game scene;

[0011] a processing unit, configured to call a texture optimization model to perform feature extraction processing on the first texture image to obtain a texture feature plane of the first texture image;

[0012] The processing unit is further configured to call a texture optimization model to perform texture optimization processing on a texture feature plane of the first texture image to obtain a second texture image; the second texture image is used to render a three-dimensional object, and the rendering performance of the second texture image is better than the rendering performance of the first texture image;

[0013] The processing unit is further configured to render the three-dimensional object in the game scene using the second texture image.

[0014] In one aspect, an embodiment of the present application provides a computer device, the computer device including an input interface and an output interface, and the computer device further including:

[0015] a processor and a computer-readable storage medium;

[0016] a computer-readable storage medium for storing a computer program;

[0017] The processor is used to run the computer program to implement the above image processing method.

[0018] On the one hand, an embodiment of the present application provides a computer-readable storage medium, which stores a computer program, and the computer program is suitable for being loaded by a processor and executing the above-mentioned image processing method.

[0019] On the one hand, an embodiment of the present application provides a computer program product, which includes a computer program, and the computer program is suitable for being loaded by a processor and executing the above-mentioned image processing method.

[0020] In an embodiment of the present application, after obtaining the first texture image to be optimized for rendering a three-dimensional object in a game scene, the texture optimization model can be called to perform feature extraction processing on the first texture image to obtain a texture feature plane of the first texture image, and the texture optimization model can be called to perform texture optimization processing on the texture feature plane of the first texture image to obtain a second texture image; the texture feature plane extracted from the first texture image can be used to express the texture features of the three-dimensional object when the first texture image is used to render the three-dimensional object. In the process of optimizing the second texture image, by introducing the texture feature plane for optimization, the quality of the optimized second texture image can be improved, and the quality of the rendering result when the second texture image is used for rendering can be further improved, that is, the rendering performance of the second texture image can be further improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following is a brief introduction to the drawings required for use in the description of the embodiments. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0022] Figure 1a This is a structural diagram of a texture optimization model provided in an embodiment of the present application;

[0023] Figure 1b is a structural diagram of an image processing system provided in an embodiment of the present application;

[0024] Figure 2 This is a flowchart of an image processing method provided by an embodiment of the present application;

[0025] Figure 3 is a flowchart of another image processing method provided in an embodiment of the present application;

[0026] Figure 4a is a schematic diagram of image optimization based on a texture optimization model provided in an embodiment of the present application;

[0027] Figure 4b is a schematic diagram of a joint training of a second model provided in an embodiment of the present application;

[0028] Figure 5 This is a flowchart of another image processing method provided by an embodiment of the present application;

[0029] Figure 6 is a schematic diagram of a rendering performance optimization test provided by an embodiment of the present application;

[0030] Figure 7 is a structural diagram of an image processing device provided in an embodiment of the present application;

[0031] Figure 8 It is a structural diagram of a computer device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0032] The technical solutions in the embodiments of the present application will be described clearly and completely below in conjunction with the drawings in the embodiments of the present application.

[0033] The embodiments of the present application relate to the field of artificial intelligence (AI). Among them, artificial intelligence is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results. In other words, artificial intelligence is a comprehensive technology in computer science that attempts to understand the essence of intelligence and produce a new type of intelligent machine that can respond in a similar way to human intelligence. Artificial intelligence is to study the design principles and implementation methods of various intelligent machines, so that machines have the functions of perception, reasoning and decision-making.

[0034] Artificial intelligence (AI) technology is a comprehensive discipline encompassing a wide range of fields, encompassing both hardware and software technologies. Foundational AI technologies generally include sensors, specialized AI chips, cloud computing, distributed storage, big data processing, pre-trained models, operating / interaction systems, and mechatronics. Pre-trained models, also known as large models or basic models, can be fine-tuned and widely applied to downstream tasks across various AI disciplines. AI software technologies primarily encompass computer vision, speech processing, natural language processing, and machine learning / deep learning.

[0035] The embodiments of the present application mainly relate to computer vision technology (Computer Vision, CV) in the field of artificial intelligence. Among them, computer vision is a science that studies how to make machines "see". More specifically, it refers to machine vision such as using cameras and computers to replace human eyes to identify and measure targets, and further performs graphic processing so that the computer processing becomes an image that is more suitable for human eye observation or transmission to instrument detection. As a scientific discipline, computer vision studies related theories and technologies, and attempts to establish an artificial intelligence system that can obtain information from images or multidimensional data. Large model technology has brought important changes to the development of computer vision technology. Pre-trained models in the visual field such as swin-transformer (a self-attention neural network), ViT (vision transformer, a visual model), V-MOE (a sparse mixture model), MAE (an autoencoding model) can be quickly and widely applied to downstream specific tasks after fine-tuning. Among them, computer vision technology generally includes image processing, image recognition, image semantic understanding, image retrieval, OCR (Optical Character Recognition), video processing, video semantic understanding, video content / behavior recognition, three-dimensional object reconstruction, 3D (three-dimensional) technology, virtual reality, augmented reality, simultaneous positioning and map construction, and also includes common biometric recognition technologies such as face recognition and fingerprint recognition.

[0036] The embodiment of the present application provides an image processing solution based on the above-mentioned computer vision technology, which can obtain a first texture image to be optimized, and the first texture image is used to render a three-dimensional object in a game scene; call a texture optimization model to perform feature extraction processing on the first texture image to obtain a texture feature plane of the first texture image; call a texture optimization model to perform texture optimization processing on the texture feature plane of the first texture image to obtain a second texture image, wherein the second texture image is used to render a three-dimensional object, and the rendering performance of the second texture image is better than the rendering performance of the first texture image; the second texture image is used to render the three-dimensional object in the game scene. The texture feature plane extracted from the first texture image refers to: a plane that can be used to express the texture features of the three-dimensional object when the first texture image is used to render the three-dimensional object.

[0037] The above image processing solution mentions that a texture optimization model can be called to optimize the first texture image to be optimized into a second texture image. That is, the texture optimization model can be called to perform feature extraction processing on the first texture image to obtain the texture feature plane of the first texture image, and the texture optimization model can be called to perform texture optimization processing on the texture feature plane of the first texture image to obtain the second texture image. Figure 1a , which is a structural diagram of a texture optimization model exemplarily proposed in an embodiment of the present application. The texture optimization model can be a neural network model, which can include a feature extraction network and a feature optimization network. The related process of performing feature extraction processing on the first texture image to obtain the texture feature plane of the first texture image can be implemented by the feature extraction network in the texture optimization model. The related process of performing texture optimization processing on the texture feature plane of the first texture image to obtain the second texture image can be collaboratively implemented by the feature extraction network and the feature optimization network in the texture optimization model. The related process is introduced in the subsequent embodiments and will not be repeated here.

[0038] In a specific implementation, the above-mentioned image processing solution can be executed by a computer device, which can be a terminal device or a server. In other words, the above-mentioned image processing solution can be executed by the terminal device or the server. Alternatively, the above-mentioned image processing solution can also be executed collaboratively by the terminal device and the server; see Figure 1b , is a structural diagram of an image processing system provided in an embodiment of the present application. The image processing system may include a terminal device 101 and a server 102. The terminal device 101 and the server 102 can communicate with each other via wired or wireless means.

[0039] For example, when the above-mentioned image processing scheme is collaboratively executed by the terminal device 101 and the server 102, the terminal device 101 can obtain the first texture image to be optimized and send the first texture image to the server 102, wherein the first texture image is used to render the three-dimensional object in the game scene; after the server 102 receives the first texture image sent by the terminal device 101, it can call the texture optimization model to perform feature extraction processing on the first texture image to obtain the texture feature plane of the first texture image, call the texture optimization model to perform texture optimization processing on the texture feature plane of the first texture image to obtain the second texture image, and return the second texture image to the terminal device 101; after the terminal device 101 receives the second texture image returned by the server 102, it can use the second texture image to render the three-dimensional object in the game scene.

[0040] For another example, when the above-mentioned image processing solution is executed collaboratively by the terminal device 101 and the server 102, the server 102 can execute the relevant process of obtaining the first texture image to be optimized and optimizing the second texture image. After optimizing the second texture image, the server 102 can send the second texture image to the terminal device 101; when the terminal device 101 needs to render a three-dimensional object, it can use the second texture image to render the three-dimensional object in the game scene. It is worth noting that the image processing solution proposed in the embodiment of the present application can be executed by a computer device alone, or it can be executed collaboratively by a system composed of multiple computer devices. Figure 1b The image processing system shown is only an exemplary system. Figure 1b The implementation method of the image processing system shown in the figure cooperating to execute the image processing solution is only an exemplary implementation method and is not limited in the embodiments of the present application. For the sake of ease of explanation, the subsequent embodiments of the present application take the image processing solution being executed by a computer device as an example.

[0041] Among them, the terminal devices mentioned in the embodiments of the present application can be smart phones, computers (such as tablets, laptops, desktop computers, etc.), smart wearable devices (such as smart watches, smart glasses), smart voice interaction devices, smart home appliances (such as smart TVs), car terminals or aircraft, etc. The server mentioned in the embodiments of the present application can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN (Content Delivery Network), and basic cloud computing services such as big data and artificial intelligence platforms, etc. Optionally, the terminal devices and servers can be located within or outside the blockchain network, which is not limited in the embodiments of the present application; the terminal devices and servers can also upload internally stored data to the blockchain network for storage to prevent the internally stored data from being tampered with and improve data security.

[0042] The collection and processing of relevant data in this application (such as the first texture image, etc.) should be strictly in accordance with the requirements of laws and regulations when applied in practice, and the informed consent or separate consent of the personal information subject should be obtained. Subsequent data use and processing should be carried out within the scope of authorization of laws and regulations and the personal information subject.

[0043] Based on the above description, the present invention provides an image processing method. Figure 2 , is a flowchart of an image processing method provided in an embodiment of the present application; the image processing method can be executed by a computer device, and the image processing method may include the following steps S201-S204:

[0044] S201, obtaining a first texture image to be optimized; the first texture image is used to render a three-dimensional object in a game scene.

[0045] The 3D objects in the game scene are three-dimensional objects in the game scene, i.e., objects in a three-dimensional space determined by the X-axis, Y-axis, and Z-axis. For example, they may include a 3D game map, a 3D game character, etc. in the game scene. The first texture image may include: an original texture image with poor rendering performance among the original texture images of the 3D object used to render the 3D object in the game scene. Based on this, when obtaining the first texture image to be optimized, the computer device may use the original texture image with poor rendering performance among the original texture images of the 3D object as the first texture image to be optimized. For example, the poor rendering performance of a texture image may include: poor quality of a rendering result obtained by rendering using the texture image. The poor quality of the rendering result may include, for example, one or more of the following: artifacts in the rendering result, camera drift in the rendering result, ghosting in the rendering result, blurry rendering result, etc. Artifacts refer to images that should not appear in the rendering result but do appear. Camera drift refers to balance drift caused by color correction when simulating camera color addition during the rendering process.

[0046] S202: Calling a texture optimization model to perform feature extraction processing on the first texture image to obtain a texture feature plane of the first texture image.

[0047] In a feasible embodiment, the texture feature plane of the first texture image can be obtained by calling the texture optimization model to perform texture feature extraction processing on the first texture image; the texture feature plane extracted from the first texture image refers to: a plane that can be used to express the texture features of the three-dimensional object when the first texture image is used to render the three-dimensional object, that is, a two-dimensional feature plane jointly determined by the X-axis and the Y-axis, and the texture features of the three-dimensional object can be captured based on the texture feature plane. In other words, the texture features of the game scene can be captured based on the texture feature plane, so that in the subsequent process of optimizing the second texture image, the introduced texture feature plane can be used for optimization, which can improve the quality of the optimized second texture image and further improve the rendering quality of the second texture image, that is, further improve the rendering performance of the second texture image.

[0048] S203 , calling a texture optimization model to perform texture optimization processing on the texture feature plane of the first texture image to obtain a second texture image.

[0049] The second texture image is used to render the three-dimensional object, and the rendering performance of the second texture image is better than the rendering performance of the first texture image; for example, the case where the rendering performance of the second texture image is better than the rendering performance of the first texture image may include: the quality of the rendering result obtained by rendering using the second texture image is higher than the quality of the rendering result obtained by rendering using the first texture image.

[0050] S204: Rendering a three-dimensional object in the game scene using the second texture image.

[0051] The process of rendering the three-dimensional object using the second texture image in the game scene is a process of re-rendering the three-dimensional object using the second texture image.

[0052] In an embodiment of the present application, after obtaining the first texture image to be optimized for rendering a three-dimensional object in a game scene, the texture optimization model can be called to perform feature extraction processing on the first texture image to obtain a texture feature plane of the first texture image, and the texture optimization model can be called to perform texture optimization processing on the texture feature plane of the first texture image to obtain a second texture image; the texture feature plane extracted from the first texture image can be used to express the texture features of the three-dimensional object when the first texture image is used to render the three-dimensional object. In the process of optimizing the second texture image, by introducing the texture feature plane for optimization, the quality of the optimized second texture image can be improved, and the quality of the rendering result when the second texture image is used for rendering can be further improved, that is, the rendering performance of the second texture image can be further improved.

[0053] Based on the above description, this embodiment of the application provides another image processing method, see Figure 3 , is a flow chart of another image processing method provided in an embodiment of the present application; the image processing method can be executed by a computer device, and the image processing method may include the following steps S301-S306:

[0054] S301, obtaining a first texture image to be optimized; the first texture image is used to render a three-dimensional object in a game scene.

[0055] The process of step S301 is similar to that of step S201, and will not be described in detail here.

[0056] S302: Calling a texture optimization model to perform feature extraction processing on the first texture image to obtain a texture feature plane of the first texture image.

[0057] Among them, the relevant process of step S302 is similar to the relevant process of the above-mentioned step S202; in a feasible implementation manner, the computer device can call the feature extraction network in the texture optimization model to perform feature extraction processing on the first texture image to obtain the texture feature plane of the first texture image.

[0058] In a feasible implementation manner, in the process of a computer device calling a texture optimization model to perform feature extraction processing on a first texture image and obtaining a texture feature plane of the first texture image, the following steps can be performed: ① calling the texture optimization model to perform color domain texture feature extraction processing on the first texture image to obtain a color domain texture feature plane group of the first texture image; the color domain texture feature plane group includes multiple color domain texture feature planes of different resolutions; ② calling the texture optimization model to perform density domain feature extraction processing on the first texture image to obtain a density domain texture feature plane group of the first texture image; the density domain texture feature plane group includes multiple density domain texture feature planes of different resolutions; wherein, the texture feature planes of the first texture image include a color domain texture feature plane group and a density domain texture feature plane group. That is to say, the computer device can call the texture optimization model to extract the texture features corresponding to the color dimension and the texture features corresponding to the density dimension from the color dimension and the density dimension respectively, so that in the subsequent process of optimizing the second texture image, the introduced color domain texture feature planes of different resolutions and the density domain texture feature planes of different resolutions can be used for optimization to obtain the predicted color and predicted density of each pixel point in the first texture image respectively, and further use the predicted color and predicted density of each pixel point in the first texture image for volume rendering processing to obtain the second texture image.

[0059] In a feasible embodiment, color domain texture feature extraction processing of the first texture image and density domain feature extraction processing of the first texture image can be achieved through a multi-resolution pyramid model. Based on this, the feature extraction network may include a first multi-resolution pyramid model and a second multi-resolution pyramid model. The resolutions adopted by the first multi-resolution pyramid model and the second multi-resolution pyramid model can be set according to specific needs, and the embodiment of the present application does not limit this. Illustratively, after obtaining the first texture image to be optimized, the computer device can call the first multi-resolution pyramid model in the feature extraction network, perform color domain texture feature extraction processing on the first texture image, and obtain a color domain texture feature plane group (also called a multi-resolution color domain texture feature plane) of the first texture image; and call the second multi-resolution pyramid model in the feature extraction network, perform density domain feature extraction processing on the first texture image, and obtain a density domain texture feature plane group (also called a multi-resolution density domain texture feature plane) of the first texture image.

[0060] S303: Calling a texture optimization model to determine a grid feature of each pixel in the first texture image according to the texture feature plane of the first texture image.

[0061] In a feasible implementation, the computer device may call a feature extraction network in the texture optimization model to determine the grid features of each pixel in the first texture image based on the texture feature plane of the first texture image.

[0062] In a feasible embodiment, the three-dimensional object is an object in a three-dimensional space jointly determined by the X-axis, Y-axis and Z-axis, and the texture feature plane is a two-dimensional feature plane jointly determined by the X-axis and Y-axis; when the computer device determines the grid features of each pixel point in the first texture image based on the texture feature plane of the first texture image, it can include: performing feature fusion processing on the texture feature plane and the feature vector of the Z-axis to obtain a feature grid of the three-dimensional object; performing feature interpolation processing on the feature grid of the three-dimensional object to obtain the grid features of each voxel in the three-dimensional object; and determining the grid features of each pixel point in the first texture image based on the first texture image and the grid features of each voxel in the three-dimensional object.

[0063] In a feasible embodiment, since the texture feature planes of the first texture image include a color domain texture feature plane group and a density domain texture feature plane group, the color domain texture feature plane group includes multiple color domain texture feature planes of different resolutions, and the density domain texture feature plane group includes multiple density domain texture feature planes of different resolutions; based on this, when the computer device performs feature fusion processing on the texture feature planes and the feature vectors of the Z axis to obtain the feature grid of the three-dimensional object, it can include: (1) performing feature fusion processing on each color domain texture feature plane in the color domain texture feature plane group and the feature vector of the Z axis to obtain a color domain feature grid group; the color domain feature grid group includes multiple color domain feature grids; (2) performing feature fusion processing on each density domain texture feature plane in the density domain texture feature plane group and the feature vector of the Z axis to obtain a density domain feature grid group; the density domain feature grid group includes multiple density domain feature grids; wherein the feature grid of the three-dimensional object includes the color domain feature grid group and the density domain feature grid group. Optionally, the color domain feature grid group can also be called a multi-resolution color domain feature network, and the density domain feature grid group can also be called a multi-resolution density domain feature network. The feature fusion processing method can be selected from the series connection, dot product, outer product and other methods, and the embodiment of the present application does not limit this. The embodiment of the present application proposes that by constructing a multi-resolution plane-vector feature space, the texture features of the three-dimensional object can be expressed by the texture feature plane. Compared with the feature plane obtained from the full 3D grid, performing texture compression can provide more information (for example, from 3D model information to spatial information conversion, and even vector compression). Through feature fusion methods such as series connection and dot product, 3D information is restored from the 2D texture feature plane to obtain a feature grid. For example, the outer product operation is used from the perspective of tensor decomposition of low-rank approximation, which can achieve more compressed memory usage while maintaining high quality.

[0064] Please refer to the following formula 1.1, which is an exemplary embodiment of the present application, which illustrates a method for performing feature fusion processing on a color domain texture feature plane and a feature vector of the Z axis. Also, please refer to the following formula 1.2, which is an exemplary embodiment of the present application, which illustrates a method for performing feature fusion processing on a density domain texture feature plane and a feature vector of the Z axis:

[0065]

[0066]

[0067] where, for a specific resolution n, represents the density domain feature grid corresponding to the resolution, Represents the color domain feature grid corresponding to the resolution, R σ Represents the density domain texture feature plane corresponding to the resolution, R c V represents the color domain texture feature plane corresponding to the resolution; z Represents the eigenvector of the Z axis, M xy Represents a matrix spanning the XY plane, r represents the channel, and for each channel r∈R σ and R c , we can use the density domain texture feature plane R σ and color domain texture feature plane R c The channel outer product of and the feature vector of the global encoding z-axis are used to approximate the full density and appearance (ie, color) grid features, where Represents a series operation; by combining XYZ multi-plane features in series, the feature range is wider. Based on this, we can see that the density (σ∈R + ) and view-dependent colors (i.e., colors, c∈R 3 ) are learned separately, and the density domain feature grid group (i.e., multi-resolution density domain feature grid) can be expressed as The color domain feature grid group (i.e., multi-resolution color domain feature grid) can be expressed as Through the multi-resolution density domain feature grid G σ and the multi-resolution color domain feature grid G c It can capture different degrees of local complexity of the scene and contains features of different granularities to describe the scene.

[0068] In a feasible embodiment, when a computer device performs feature interpolation processing on a feature grid of a three-dimensional object, a feature interpolation method can be selected according to specific needs, for example, linear interpolation, bilinear interpolation, nonlinear interpolation, etc. can be selected, and the embodiment of the present application does not limit this. Since the feature grid of the three-dimensional object includes a color domain feature grid group and a density domain feature grid group, the color domain feature grid group includes multiple color domain feature grids, and the density domain feature grid group includes multiple density domain feature grids; based on this, when a computer device performs feature interpolation processing on the feature grid of the three-dimensional object, feature interpolation processing can be performed on multiple color domain feature grids in the color domain feature grid group to obtain the grid features of each voxel in the three-dimensional object in the color domain, and feature interpolation processing can be performed on multiple density domain feature grids in the density domain feature grid group to obtain the grid features of each voxel in the three-dimensional object in the density domain. The grid features of each voxel in the three-dimensional object include the grid features of each voxel in the three-dimensional object in the color domain and the grid features of each voxel in the three-dimensional object in the density domain. By performing feature interpolation on the feature grid of the three-dimensional object, the grid features of each voxel in the three-dimensional object are obtained. At a sufficiently high resolution, this mechanism can produce detailed reconstruction results, thereby recovering subtle changes in the scene.

[0069] In a feasible embodiment, when a computer device determines the grid features of each pixel point in the first texture image based on the grid features of each voxel in the first texture image and the three-dimensional object, the method may include: performing point sampling on a light beam in the observation direction corresponding to the target pixel point in the first texture image to obtain multiple sampling points corresponding to the target pixel point; sampling the grid features corresponding to the multiple sampling points from the grid features of each voxel in the three-dimensional object based on the coordinates of the multiple sampling points; and using the grid features corresponding to the multiple sampling points as the grid features of the target pixel point. The target pixel point can be any pixel point in the first texture image; the sampling accuracy used for point sampling on the light beam in the observation direction corresponding to the target pixel point can be called the target sampling accuracy. The target sampling accuracy can be set according to specific needs and is not limited in this embodiment of the present application. Since the grid features of each voxel in the three-dimensional object include the grid features of each voxel in the three-dimensional object in the color domain and the grid features of each voxel in the three-dimensional object in the density domain, based on this, when the computer device samples the grid features of each voxel in the three-dimensional object according to the coordinates of multiple sampling points to obtain grid features corresponding to multiple sampling points, it can sample the grid features of the color domain corresponding to multiple sampling points from the grid features of each voxel in the three-dimensional object in the color domain, and sample the grid features of the density domain corresponding to multiple sampling points from the grid features of each voxel in the three-dimensional object in the density domain. The grid features corresponding to a sampling point include the grid features of the color domain corresponding to the sampling point and the grid features of the density domain corresponding to the sampling point.

[0070] In another optional embodiment, in the process of determining the grid features of each pixel point in the first texture image based on the texture feature plane of the first texture image, the computer device can perform point sampling on the light beam in the observation direction corresponding to the target pixel point in the first texture image to obtain multiple sampling points corresponding to the target pixel point; based on the projection of the coordinates of the multiple sampling points in the texture feature plane of the first texture image, multi-resolution texture features corresponding to the multiple sampling points are sampled from the texture feature plane of the first texture image, and the grid features of the multiple sampling points corresponding to the target pixel point are obtained through feature fusion processing with the feature vector of the Z axis and feature interpolation processing, that is, the grid features of the target pixel point are obtained.

[0071] S304 , predicting a predicted texture feature of each pixel in the first texture image according to the grid feature of each pixel in the first texture image; the predicted texture feature includes a predicted color and a predicted density.

[0072] In a feasible embodiment, the texture optimization model includes a feature optimization network; when the computer device predicts the predicted texture features of each pixel point in the first texture image based on the grid features of each pixel point in the first texture image, it can include: obtaining the position code of each pixel point in the first texture image; using the grid features of each pixel point in the first texture image and the position code of each pixel point in the first texture image as input features of the feature optimization network; calling the feature optimization network to perform prediction processing on the input features to obtain the predicted texture features of each pixel point in the first texture image. Among them, the grid features of any pixel point in the first texture image include: the grid features corresponding to multiple sampling points of the pixel point, that is, the grid features of the color domain corresponding to multiple sampling points of the pixel point and the grid features of the density domain corresponding to multiple sampling points of the pixel point; the predicted texture features of any pixel point in the first texture image include: the predicted texture features of multiple sampling points of the pixel point, that is, the predicted color and predicted density of multiple sampling points of the pixel point. That is to say, the computer device calls the texture optimization model to perform texture optimization processing on the texture feature plane of the first texture image. In the process of obtaining the second texture image, the texture feature plane of the first texture image can be processed based on the target sampling accuracy to obtain the grid features of each pixel point in the first texture image under the target sampling accuracy, and combined with the position coding of each pixel point in the first texture image, the predicted texture features of each pixel point in the first texture image are predicted.

[0073] In a feasible embodiment, the feature optimization network can be any one of a grid branch network and a neural radiation field branch network; the predicted texture features include predicted color and predicted density; if the feature optimization network is a grid branch network, the input features are input into the grid branch network for prediction processing to obtain the predicted density and predicted color of each pixel in the first texture image; if the feature optimization network is a neural radiation field branch network, the input features are input into the neural radiation field branch network for a first prediction processing to obtain intermediate features and the predicted density of each pixel in the first texture image; the neural radiation field branch network then performs a second prediction processing based on the intermediate features to obtain the predicted color of each pixel in the first texture image.

[0074] Please see the following formula 2.1, which shows the processing logic of predicting texture features via the grid branch network prediction, and please see the following formula 2.2, which shows the processing logic of predicting texture features via the neural radiation field branch network prediction:

[0075] σ,c=F σ (G σ (X)),F c (G c (X),PE(d)) (2.1)

[0076] (σ′,c′)=F′(G σ (X),G c (X),PE(X),PE(d)) (2.2)

[0077] Wherein, σ represents the predicted density of any pixel point predicted by the grid branch network, c represents the predicted color of the pixel point predicted by the grid branch network, σ′ represents the predicted density of the pixel point predicted by the neural radiation field branch network, c′ represents the predicted color of the pixel point predicted by the neural radiation field branch network; X represents the position information corresponding to the pixel point, which may include the coordinates of multiple sampling points of the pixel point, d∈S 2 Indicates the observation direction corresponding to the pixel point, G σ (X) represents the grid features of the density domain corresponding to multiple sampling points of the pixel point (also called the grid features of the pixel point in the density domain), G c (X) represents the grid features of the color domain corresponding to multiple sampling points of the pixel (also called the grid features of the pixel in the color domain), PE represents the position code, PE(X) represents the position code of the position information corresponding to the pixel, PE(d) represents the position code of the observation direction corresponding to the pixel, and the position code PE can be calculated using the exp imaginary exponent to achieve efficient coding. The relevant coding process can be expressed as: ((exp j(.) ,…,exp j2L-1(.) ),exp j2L-1(.) ). Among them, F σ represents the fusion function used when the grid branch network predicts the predicted density, F c represents the fusion function used when the grid branch network predicts the predicted color, and F′ represents the neural radiation field branch network.

[0078] Based on the above, it can be seen that if the feature optimization network is a grid branch network, the position coding of each pixel point in the first texture image included in the input feature should be the position coding of each pixel point in the first texture image corresponding to the observation direction; if the feature optimization network is a neural radiation field branch network, the position coding of each pixel point in the first texture image included in the input feature should be the position coding of each pixel point in the first texture image corresponding to the observation direction, and the position coding of the corresponding position information of each pixel point in the first texture image; if the feature optimization network is a neural radiation field branch network, the grid features of each pixel point in the first texture image and the position coding of the corresponding position information of each pixel point in the first texture image are input as input features of the feature optimization network to the neural radiation field branch network for a first prediction processing to obtain intermediate features and the predicted density of each pixel point in the first texture image, and then the neural radiation field branch network performs a second prediction processing based on the intermediate features and the position coding of each pixel point in the first texture image corresponding to the observation direction to obtain the predicted color of each pixel point in the first texture image.

[0079] The grid branch network may include a density prediction subnetwork for predicting the predicted density (corresponding to F in formula 2.1). σ ), and the color prediction subnetwork for predicting the predicted color (corresponding to F in formula 2.1 c ), optionally, the density prediction subnetwork and the color prediction subnetwork can be fully connected networks combining multiple multilayer perceptrons (MLPs), and the number of multilayer perceptrons can be set according to specific needs, which is not limited in the embodiments of the present application. The neural radiation field branch network can include a first prediction subnetwork for performing a first prediction process, and a second prediction subnetwork for performing a second prediction process, wherein the first prediction subnetwork and the second prediction subnetwork can be fully connected networks combining multiple multilayer perceptrons, and the number of multilayer perceptrons can be set according to specific needs, which is not limited in the embodiments of the present application.

[0080] The nodes between fully connected network layers are fully connected, and the smallest unit of a fully connected network is called a perceptron. For each perceptron in layer L, the linear combination layer of the input is the (L-1) layer. The output of the (L-1) layer and the weights work together to produce the output. The established formula model can be shown as the following formula 3:

[0081]

[0082] Where L represents the number of layers, represents the output of the i-th perceptron in the L-th layer, represents the output of the j-th perceptron in the (L-1)-th layer, represents the weight between the j-th perceptron in the (L-1)-th layer and the i-th perceptron in the L-th layer, is the activation function, and b is the bias added at each layer. In one possible implementation, the nonlinear function The fully connected network is able to capture the nonlinear relationship between input and output. The L layer here can be a combination of different MLP groups. During the training process of the fully connected network, the fully connected network can be used for super-resolution based on supervised machine learning. The training process of the supervised machine learning model is converted into an optimization problem to determine the weight w of the model F. The weight w can be optimized by minimizing the loss function ε. This optimization process is expressed as the following optimization logic: w = argmin w ε(w).

[0083] S305 , performing volume rendering processing using the predicted texture features of each pixel point in the first texture image to obtain a second texture image.

[0084] The second texture image is used for rendering a three-dimensional object, and the rendering performance of the second texture image is better than the rendering performance of the first texture image.

[0085] See Figure 4a , is a schematic diagram of image optimization based on a texture optimization model provided in an embodiment of the present application, wherein the texture optimization model includes a feature extraction network and a feature optimization network, and the feature extraction network includes a first multi-resolution pyramid model and a second multi-resolution pyramid model.

[0086] After obtaining the first texture image to be optimized, the computer device can call the feature extraction network to perform feature extraction processing on the first texture image to obtain the texture feature plane of the first texture image; that is, the first multi-resolution pyramid model in the feature extraction network can be called to perform color domain texture feature extraction processing on the first texture image to obtain the color domain texture feature plane group of the first texture image, and the second multi-resolution pyramid model in the feature extraction network can be called to perform density domain feature extraction processing on the first texture image to obtain the density domain texture feature plane group of the first texture image. The texture feature plane of the first texture image includes a color domain texture feature plane group and a density domain texture feature plane group.

[0087] The computer device can call the feature extraction network to determine the grid features of each pixel point in the first texture image based on the texture feature plane of the first texture image, that is, obtain the grid features of each pixel point in the first texture image in the color domain based on the color domain texture feature plane group of the first texture image (that is, the grid features of the color domain corresponding to the sampling points of each pixel point), and obtain the grid features of each pixel point in the first texture image in the density domain based on the density domain texture feature plane group of the first texture image (that is, the grid features of the density domain corresponding to the sampling points of each pixel point).

[0088] The computer device can obtain the position code of each pixel point in the first texture image, and use the grid features of each pixel point in the first texture image and the position code of each pixel point in the first texture image as input features of the feature optimization network, call the feature optimization network to perform prediction processing on the input features, and obtain the predicted texture features of each pixel point in the first texture image, the predicted texture features include predicted color and predicted density; then the computer device can use the predicted texture features of each pixel point in the first texture image to perform volume rendering processing to obtain a second texture image.

[0089] S306: Rendering the three-dimensional object in the game scene using the second texture image.

[0090] The following describes the process of constructing a texture optimization model using a computer device as an example.

[0091] In a feasible embodiment, the method for constructing a texture optimization model may include: constructing a first model using a grid branch network, and pre-training the first model to obtain a pre-trained first model; constructing a second model using the pre-trained first model and a neural radiation field branch network, and training the second model to obtain a trained second model; wherein the grid branch network and the neural radiation field branch network in the second model are jointly trained; constructing a texture optimization model based on the trained second model; the texture optimization model includes the grid branch network or the neural radiation field branch network in the trained second model. wherein the first model may be constructed using an initialized feature extraction network and an initialized grid branch network, and the second model may be constructed using the pre-trained first model and the initialized neural radiation field branch network.

[0092] In a feasible embodiment, the way in which the computer device pre-trains the first model may include the following steps: obtaining a pre-trained texture image; calling the first model to perform feature extraction processing on the pre-trained texture image to obtain a texture feature plane of the pre-trained texture image; performing texture optimization processing on the texture feature plane of the pre-trained texture image by calling the grid branch network in the first model to obtain a target predicted texture image; and training the first model based on the difference between the target predicted texture image and the pre-trained texture image. Among them, the pre-trained texture image can be a texture image with good rendering performance. The related process of calling the first model to process the pre-trained texture image to obtain the target predicted texture image is similar to the related process of calling the texture optimization model containing the grid branch network to process the first texture image to obtain the second texture image, which will not be repeated here; based on the difference between the target predicted texture image and the pre-trained texture image, when training the first model, the training can be carried out with the goal of reducing the difference between the target predicted texture image and the pre-trained texture image; the difference between the target predicted texture image and the pre-trained texture image can be the difference in pixels between the target predicted texture image and the pre-trained texture image, or it can be the difference between the feature quantization values obtained after feature quantization of the target predicted texture image and the pre-trained texture image. The loss function used in the corresponding pre-training process can be designed according to specific needs. For example, the total square error of pixels between the target predicted texture image and the pre-trained texture image can be used. The embodiments of the present application do not limit this.

[0093] See Figure 4b , which is a schematic diagram of a joint training of a second model provided in an embodiment of the present application; the way in which the computer device trains the second model may include the following steps: obtaining a sample texture image; calling the second model to perform feature extraction processing on the sample texture image to obtain a texture feature plane of the sample texture image; performing texture optimization processing on the texture feature plane of the sample texture image by calling the grid branch network in the second model to obtain a first predicted texture image; performing texture optimization processing on the texture feature plane of the sample texture image by calling the neural radiation field branch network in the second model to obtain a second predicted texture image; training the second model based on the difference between the first predicted texture image and the sample texture image, and the difference between the second predicted texture image and the sample texture image.

[0094] Among them, the sample texture image can be a texture image with good rendering performance, and the process of calling the second model to process the sample texture image to obtain the first predicted texture image is similar to the process of calling the texture optimization model containing a grid branch network to process the first texture image to obtain the second texture image. The process of calling the second model to process the sample texture image to obtain the second predicted texture image is similar to the process of calling the texture optimization model containing a neural radiation field branch network to process the first texture image to obtain the second texture image, which will not be repeated here. Based on the difference between the first predicted texture image and the sample texture image, and the difference between the second predicted texture image and the sample texture image, when training the second model, the training goal can be to reduce the difference between the first predicted texture image and the sample texture image, and to reduce the difference between the second predicted texture image and the sample texture image; the difference between the first predicted texture image and the sample texture image, and the difference between the second predicted texture image and the sample texture image, can be the difference between the pixels between the images, or the difference between the feature quantization values obtained after the images are feature quantized. The loss function used in the corresponding training process can be designed according to specific needs, and the embodiments of the present application are not limited to this. For example, the loss function can be designed as a combination of a grid loss function and a neural radiation field loss function, wherein the grid loss function is used to determine the grid loss, which can be used to characterize the difference between the first predicted texture image and the sample texture image, and the neural radiation field loss function is used to determine the neural radiation field loss, which can be used to characterize the difference between the second predicted texture image and the sample texture image; taking the feature quantization value as an example, the corresponding loss function can be expressed as: Loss = || F mesh -gt mesh ||+||F neural -gt neural ||, where F mesh It can represent the feature quantization value of the first predicted texture image, F neural It can represent the feature quantization value of the second predicted texture image, and gt can represent the feature quantization value of the sample texture image.

[0095] The sampling accuracy used in the above-mentioned process of calling the texture optimization model containing the grid branch network to process the first texture image to obtain the second texture image is the target sampling accuracy. The target sampling accuracy is also the sampling accuracy used in the process of training the second model. The sampling accuracy used in the process of calling the first model to process the pre-trained texture image to obtain the target predicted texture image can be called the reference sampling accuracy. The reference sampling accuracy can be set according to specific needs. In an optional embodiment, the reference sampling accuracy can be set lower than the target sampling accuracy, that is, coarse sampling is performed during the pre-training process and fine sampling is performed during the joint training process; that is, the scene can be captured with a pyramid model in the pre-training stage, and based on the coarse sampling of light points, the radiation value (including predicted density and predicted color) is predicted by the MLP renderer (grid branch network), which is supervised by the loss of image color difference. The pre-training process can generate a set of information-rich multi-resolution texture feature planes (including multi-resolution density domain texture feature planes and multi-resolution color domain texture feature planes); in the joint training stage, based on the fine sampling of light points, the learned feature grid can be used to guide the sampling of the neural radiation field branch network so that it focuses on the scene surface, and the grid features of the sampling points can be determined based on the feature interpolation processing of the texture feature plane. In the joint training stage, the output of the grid branch network is supervised together with the sample texture image and the output from the neural radiation field branch network.

[0096] This solution proposes that the sampling space of the neural radiation field can be compressed using pre-trained feature grid density, and the grid features obtained in the pre-training stage can be used to enrich the pure coordinate input of the neural radiation field. This is because the pre-trained grid features can already provide an approximation of the scene, which can be used to guide the point sampling of the neural radiation field and provide grid features as a supplement to the coordinate input of the neural radiation field; the neural radiation field can focus on the approximate scene surface to achieve more efficient and denser point sampling, and through position encoding, high-frequency Fourier features can be evoked to recover finer details, making the prediction of the neural radiation field more accurate. The introduction of multi-resolution texture feature planes can provide information about the scene at multiple granularities (i.e., multi-resolution), reducing the assembly burden of the position encoding of the neural radiation field, allowing it to focus on refining the details of the scene.

[0097] The above describes the relevant processing logic for constructing a texture optimization network based on a pre-training process. In an optional implementation, the pre-training process may not be introduced, that is, an initialized feature extraction network, an initialized grid branch network, and an initialized neural radiation field branch network may be used to construct a second model, and the second model may be trained to construct a texture optimization model based on the trained second model.

[0098] In an embodiment of the present application, after obtaining the first texture image to be optimized for rendering a three-dimensional object in a game scene, the texture optimization model can be called to perform feature extraction processing on the first texture image to obtain a texture feature plane of the first texture image, and the texture optimization model can be called to perform texture optimization processing on the texture feature plane of the first texture image to obtain a second texture image; the texture feature plane extracted from the first texture image can be used to express the texture features of the three-dimensional object when the first texture image is used to render the three-dimensional object, and the texture feature plane of the first texture image can be a multi-resolution texture feature plane. In the process of optimizing the second texture image, the introduction of the multi-resolution texture feature plane can provide information about the scene at multiple granularities (i.e., multi-resolutions), so that the neural radiation field branch network can focus on refining the details of the scene, thereby improving the quality of the optimized second texture image, and further improving the quality of the rendering result when rendering using the second texture image, that is, further improving the rendering performance of the second texture image.

[0099] In the process of constructing a texture optimization model, a grid branch network can be used to construct a first model, and the first model can be pre-trained to obtain a pre-trained first model. The pre-trained first model and the neural radiation field branch network can be used to construct a second model, and the second model can be trained to obtain a trained second model. A texture optimization model is constructed based on the trained second model; wherein, the grid branch network and the neural radiation field branch network in the second model are jointly trained, and the texture optimization model includes the grid branch network or the neural radiation field branch network in the trained second model; by introducing the pre-training process, the training effect of the second model can be improved.

[0100] Based on the above description, the present application provides another image processing method, see Figure 5 , which is a flowchart of another image processing method provided in an embodiment of the present application; the image processing method can be executed by a computer device, and the image processing method may include the following steps S501-S506:

[0101] S501: Exporting an original texture image of a 3D object from a game scene, wherein the original texture image is used to render the 3D object in the game scene.

[0102] S502: Render the three-dimensional object using the original texture image and obtain corresponding first rendering performance parameters.

[0103] The rendering performance parameters are used to characterize rendering performance. Comparison of rendering performance can be achieved by comparing the rendering performance parameters. The parameters included in the rendering performance parameters can be designed based on specific needs. In one feasible implementation, the rendering performance parameters may include at least one of the following: rendering result information, the amount of resources consumed by the rendering process, and the quality of the rendering result. The rendering result information may indicate rendering success or failure, and the amount of resources consumed by the rendering process may indicate the amount of processing resources, storage resources, and other resources consumed by the rendering process. In an optional embodiment, the quality of the rendering result obtained by rendering included in the rendering performance parameters can be obtained by quantifying the evaluation results of preset quality evaluation items; wherein, the quality evaluation items can be set according to specific needs, and the quantization method can also be set according to specific needs, and the embodiments of the present application do not limit this; for example, the quality evaluation items are set as: there are artifacts in the rendering results, there is camera drift in the rendering results, there are ghosting in the rendering results, and the rendering results are blurred. When the evaluation result of any quality evaluation item is true, it is represented by the label "-1", otherwise, it is represented by the label "1". The evaluation results of the preset quality evaluation items can be quantified by the number of quantified labels "-1" or the sum of quantified labels "-1", that is, the quality of the rendering results can be represented by numerical values, which is convenient for quality comparison.

[0104] S503: If the first rendering performance parameter indicates that the rendering performance of the original texture image meets the optimization condition, the original texture image is determined as a first texture image to be optimized.

[0105] In a feasible embodiment, since the rendering performance parameter is used to characterize the rendering performance, the rendering performance parameter includes at least one of the following: rendering result information, the amount of resources consumed by the rendering process, and the quality of the rendering result obtained by rendering; then the first rendering performance parameter indicating that the rendering performance of the original texture image meets the optimization condition may include at least one of the following: if the first rendering performance parameter includes rendering result information, and the corresponding rendering result information indicates that the rendering of the stereo object fails, then the first rendering performance parameter indicates that the rendering performance of the original texture image meets the optimization condition; if the first rendering performance parameter includes the amount of resources consumed by the rendering process, and the corresponding amount of resources is greater than a resource consumption threshold, then the first rendering performance parameter indicates that the rendering performance of the original texture image meets the optimization condition; if the first rendering performance parameter includes the quality of the rendering result obtained by rendering, and the corresponding quality is lower than a quality threshold, then the first rendering performance parameter indicates that the rendering performance of the original texture image meets the optimization condition. Optionally, the rendering performance parameter may also include rendering duration. Based on this, if the first rendering performance parameter includes rendering duration, and the corresponding rendering duration is greater than the duration threshold, then the first rendering performance parameter indicates that the rendering performance of the original texture image meets the optimization condition. Among them, the resource consumption threshold, quality threshold, and duration threshold can all be set according to specific needs; optionally, the above-mentioned process of detecting whether the rendering performance of the original texture image meets the optimization conditions can be implemented by calling a performance detection tool, for example, calling the DEBUG tool in the performance detection tool.

[0106] That is, in one feasible embodiment, the computer device may determine that the rendering performance of the original texture image meets the optimization condition when rendering using the original texture image fails; in another feasible embodiment, the computer device may determine that the rendering performance of the original texture image meets the optimization condition when the amount of resources consumed by rendering using the original texture image is greater than a resource consumption threshold; in yet another feasible embodiment, the computer device may determine that the rendering performance of the original texture image meets the optimization condition when the quality of the rendering result obtained by rendering using the original texture image is lower than a quality threshold. In yet another feasible embodiment, the computer device may determine that the rendering performance of the original texture image meets the optimization condition when the rendering time consumed by rendering using the original texture image is greater than a time threshold.

[0107] S504: Calling a texture optimization model to perform feature extraction processing on the first texture image to obtain a texture feature plane of the first texture image.

[0108] S505 , calling a texture optimization model to perform texture optimization processing on the texture feature plane of the first texture image to obtain a second texture image.

[0109] The second texture image is used to render the three-dimensional object, and the rendering performance of the second texture image is better than that of the first texture image; the relevant processes from step S504 to step S505 are similar to the relevant processes from step S202 to step S203 and step S302 to step S305 mentioned above.

[0110] In another feasible embodiment, the rendering performance of the first texture image is characterized by a first rendering performance parameter; when the computer device calls the texture optimization model to perform texture optimization processing on the texture feature plane of the first texture image to obtain the second texture image, it can include: calling the texture optimization model to perform texture optimization processing on the texture feature plane of the first texture image to obtain the target texture image; using the target texture image to pre-render the three-dimensional object and obtain the corresponding second rendering performance parameter; the second rendering performance parameter is used to characterize the rendering performance of the target texture image; if the rendering performance characterized by the second rendering performance parameter is better than the rendering performance characterized by the first rendering performance parameter, the target texture image is determined to be the second texture image. Among them, the process in which the computer device calls the texture optimization model to perform texture optimization processing on the texture feature plane of the first texture image to obtain the target texture image is related to the above Figure 2 as well as Figure 3 In the method embodiment, the texture optimization model is called to perform texture optimization processing on the texture feature plane of the first texture image to obtain the relevant process of the second texture image, which is similar and will not be described here in detail.

[0111] In a feasible implementation, since the rendering performance of the first texture image is characterized by a first rendering performance parameter, and the rendering performance of the second texture image is characterized by a second rendering performance parameter; the rendering performance parameter includes at least one of the following: rendering result information, the amount of resources consumed by the rendering process, and the quality of the rendering result obtained by rendering; then the rendering performance of the second texture image is better than the rendering performance of the first texture image, which may include at least one of the following: if the rendering performance parameter includes rendering result information, and the rendering result information of the second rendering performance parameter indicates that the rendering of the stereo object is successful, and the rendering result information of the first rendering performance parameter indicates that the rendering of the stereo object fails, then the rendering performance of the second texture image is better than the rendering performance of the first texture image; if the rendering performance parameter includes the amount of resources consumed by the rendering process, and the amount of resources consumed by the rendering process indicated by the second rendering performance parameter is lower than that of the first rendering performance parameter, If the rendering performance parameter includes the amount of resources consumed by the rendering process indicated by the second rendering performance parameter, the rendering performance of the second texture image is better than the rendering performance of the first texture image; if the rendering performance parameter includes the quality of the rendering result obtained by rendering, and the quality of the rendering result indicated by the second rendering performance parameter is higher than the quality of the rendering result indicated by the first rendering performance parameter, the rendering performance of the second texture image is better than the rendering performance of the first texture image; if the rendering performance parameter includes the amount of resources consumed by the rendering process and the quality of the rendering result obtained by rendering, and the quality of the rendering result indicated by the second rendering performance parameter is higher than the quality of the rendering result indicated by the first rendering performance parameter, and the difference between the amount of resources consumed by the rendering process indicated by the second rendering performance parameter and the amount of resources consumed by the rendering process indicated by the first rendering performance parameter is less than a preset difference threshold, the rendering performance of the second texture image is better than the rendering performance of the first texture image. Optionally, the rendering performance parameter may further include rendering time. Based on this, if the rendering performance parameter includes rendering time, and the rendering time indicated by the second rendering performance parameter is less than the rendering time indicated by the first rendering performance parameter, the rendering performance of the second texture image is better than the rendering performance of the first texture image. Among them, the preset difference threshold can be set according to specific needs, and the embodiment of the present application is not limited. Optionally, the difference between the amount of resources consumed by the rendering process indicated by the second rendering performance parameter and the amount of resources consumed by the rendering process indicated by the first rendering performance parameter can be represented by the absolute value of the difference between the amount of resources consumed by the rendering process indicated by the second rendering performance parameter and the amount of resources consumed by the rendering process indicated by the first rendering performance parameter.

[0112] In other words, in a feasible embodiment, the computer device may determine that the rendering performance of the second texture image is better than the rendering performance of the first texture image when rendering with the first texture image fails and rendering with the second texture image succeeds; in another feasible embodiment, the computer device may determine that the rendering performance of the second texture image is better than the rendering performance of the first texture image when the amount of resources consumed by rendering with the second texture image is lower than the amount of resources consumed by rendering with the first texture image; in yet another feasible embodiment, the computer device may determine that the rendering performance of the second texture image is better than the rendering performance of the first texture image when the quality of the rendering result obtained by rendering with the second texture image is higher than the quality of the rendering result obtained by rendering with the first texture image; in yet another feasible embodiment, the computer device may determine that the rendering performance of the second texture image is better than the rendering performance of the first texture image when the quality of the rendering result obtained by rendering with the second texture image is higher than the quality of the rendering result obtained by rendering with the first texture image, and the difference between the amount of resources consumed by rendering with the second texture image and the amount of resources consumed by rendering with the first texture image is less than a preset difference threshold. In another feasible embodiment, the computer device may determine that the rendering performance of the second texture image is superior to the rendering performance of the first texture image when the rendering time consumed by rendering the second texture image is less than the rendering time consumed by rendering the first texture image. It should be understood that the above is only an exemplary description of the condition for determining whether the rendering performance of the second texture image is superior to the rendering performance of the first texture image. The determination condition can also be determined by combining parameter items in the rendering performance parameter according to specific needs.

[0113] See Figure 6 , which is a schematic diagram of a rendering performance optimization detection provided by an embodiment of the present application, the computer device can export the original texture image of the three-dimensional object from the game scene, use the original texture image to render the three-dimensional object and obtain the corresponding first rendering performance parameter; when the first rendering performance parameter indicates that the rendering performance of the original texture image meets the optimization condition, the original texture image is determined as the first texture image to be optimized, and the texture optimization model is called to optimize the first texture image to obtain the target texture image; the three-dimensional object is pre-rendered using the target texture image and the corresponding second rendering performance parameter is obtained; by comparing the first rendering performance parameter and the second rendering performance parameter, it is determined whether the rendering performance is optimized, that is, whether the rendering performance represented by the second rendering performance parameter is better than the rendering performance represented by the first rendering performance parameter, so that when the rendering performance represented by the second rendering performance parameter is better than the rendering performance represented by the first rendering performance parameter, the target texture image can be determined as the second texture image, and the second texture image can be used to render the three-dimensional object in the game scene.

[0114] S506: Rendering the three-dimensional object in the game scene using the second texture image.

[0115] In an embodiment of the present application, an original texture image of a three-dimensional object can be exported from a game scene, and a first rendering performance parameter when the original texture image is used to render the three-dimensional object is obtained. When the first rendering performance parameter indicates that the rendering performance of the original texture image meets the optimization condition, the texture optimization model is called to optimize the original texture image to obtain a target texture image, and a second rendering performance parameter when the target texture image is used to pre-render the three-dimensional object is obtained. By comparing the first rendering performance parameter and the second rendering performance parameter, it is possible to accurately determine whether the rendering performance is optimized, so that the target texture image can be used to render the three-dimensional object in the game scene in the subsequent case where the rendering performance represented by the second rendering performance parameter is better than the rendering performance represented by the first rendering performance parameter, so as to improve the rendering performance.

[0116] Based on the description of the above method embodiment, the embodiment of the present application further discloses an image processing device; the image processing device can be a computer program running in a computer device, and the image processing device can execute Figure 2 、 Figure 3 or Figure 5 The steps in the method flow are shown. Figure 7 , is a schematic structural diagram of an image processing device provided in an embodiment of the present application, the image processing device may include an acquisition unit 701 and a processing unit 702, wherein:

[0117] The acquisition unit 701 is used to acquire a first texture image to be optimized; the first texture image is used to render a three-dimensional object in a game scene;

[0118] The processing unit 702 is configured to call a texture optimization model to perform feature extraction processing on the first texture image to obtain a texture feature plane of the first texture image;

[0119] The processing unit 702 is further configured to call a texture optimization model to perform texture optimization processing on the texture feature plane of the first texture image to obtain a second texture image; the second texture image is used to render a three-dimensional object, and the rendering performance of the second texture image is better than the rendering performance of the first texture image;

[0120] The processing unit 702 is further configured to render the three-dimensional object in the game scene using the second texture image.

[0121] In one embodiment, when the acquiring unit 701 is used to acquire the first texture image to be optimized, it can be specifically used to:

[0122] Export the original texture images of the three-dimensional objects from the game scene;

[0123] Rendering the three-dimensional object using the original texture image and obtaining corresponding first rendering performance parameters;

[0124] If the first rendering performance parameter indicates that the rendering performance of the original texture image meets the optimization condition, the original texture image is determined as the first texture image to be optimized.

[0125] In one embodiment, the rendering performance parameter is used to characterize the rendering performance, and the rendering performance parameter includes at least one of the following: rendering result information, the amount of resources consumed by the rendering process, and the quality of the rendering result obtained by rendering;

[0126] The first rendering performance parameter indicates that the rendering performance of the original texture image meets the optimization condition, including at least one of the following:

[0127] If the first rendering performance parameter includes rendering result information, and the corresponding rendering result information indicates that the rendering of the three-dimensional object fails, then the first rendering performance parameter indicates that the rendering performance of the original texture image meets the optimization condition;

[0128] If the first rendering performance parameter includes an amount of resources consumed by the rendering process, and the corresponding amount of resources is greater than a consumed resource amount threshold, then the first rendering performance parameter indicates that the rendering performance of the original texture image meets the optimization condition;

[0129] If the first rendering performance parameter includes the quality of the rendering result obtained by rendering, and the corresponding quality is lower than the quality threshold, the first rendering performance parameter indicates that the rendering performance of the original texture image meets the optimization condition.

[0130] In one embodiment, the rendering performance of the first texture image is characterized by a first rendering performance parameter;

[0131] When the processing unit 702 is used to call the texture optimization model to perform texture optimization processing on the texture feature plane of the first texture image to obtain the second texture image, it can be specifically used to:

[0132] Calling a texture optimization model to perform texture optimization processing on a texture feature plane of the first texture image to obtain a target texture image;

[0133] Pre-rendering the three-dimensional object using the target texture image and obtaining corresponding second rendering performance parameters; the second rendering performance parameters are used to characterize the rendering performance of the target texture image;

[0134] If the rendering performance represented by the second rendering performance parameter is better than the rendering performance represented by the first rendering performance parameter, the target texture image is determined to be the second texture image.

[0135] In one embodiment, the rendering performance of the first texture image is characterized by a first rendering performance parameter, and the rendering performance of the second texture image is characterized by a second rendering performance parameter; the rendering performance parameter includes at least one of the following: rendering result information, the amount of resources consumed by the rendering process, and the quality of the rendering result obtained by rendering;

[0136] The rendering performance of the second texture image is better than the rendering performance of the first texture image, including at least one of the following:

[0137] If the rendering performance parameter includes rendering result information, and the rendering result information of the second rendering performance parameter indicates that the three-dimensional object rendering is successful, and the rendering result information of the first rendering performance parameter indicates that the three-dimensional object rendering fails, then the rendering performance of the second texture image is better than the rendering performance of the first texture image;

[0138] If the rendering performance parameter includes an amount of resources consumed by a rendering process, and the amount of resources consumed by the rendering process indicated by the second rendering performance parameter is lower than the amount of resources consumed by the rendering process indicated by the first rendering performance parameter, then the rendering performance of the second texture image is better than the rendering performance of the first texture image;

[0139] If the rendering performance parameter includes the quality of a rendering result obtained by rendering, and the quality of the rendering result indicated by the second rendering performance parameter is higher than the quality of the rendering result indicated by the first rendering performance parameter, then the rendering performance of the second texture image is better than the rendering performance of the first texture image;

[0140] If the rendering performance parameters include the amount of resources consumed by the rendering process and the quality of the rendering results obtained by rendering, and the quality of the rendering results indicated by the second rendering performance parameters is higher than the quality of the rendering results indicated by the first rendering performance parameters, and the difference between the amount of resources consumed by the rendering process indicated by the second rendering performance parameters and the amount of resources consumed by the rendering process indicated by the first rendering performance parameters is less than a preset difference threshold, then the rendering performance of the second texture image is better than the rendering performance of the first texture image.

[0141] In one embodiment, when the processing unit 702 is used to call the texture optimization model to perform feature extraction processing on the first texture image to obtain the texture feature plane of the first texture image, it can be specifically used to:

[0142] Calling the texture optimization model to perform color domain texture feature extraction processing on the first texture image to obtain a color domain texture feature plane group of the first texture image; the color domain texture feature plane group includes a plurality of color domain texture feature planes with different resolutions;

[0143] Calling the texture optimization model to perform density domain feature extraction processing on the first texture image to obtain a density domain texture feature plane group of the first texture image; the density domain texture feature plane group includes a plurality of density domain texture feature planes of different resolutions;

[0144] The texture feature planes of the first texture image include a color domain texture feature plane group and a density domain texture feature plane group.

[0145] In one embodiment, when the processing unit 702 is used to call the texture optimization model to perform texture optimization processing on the texture feature plane of the first texture image to obtain the second texture image, it can be specifically used to:

[0146] Invoking a texture optimization model to determine a grid feature of each pixel in the first texture image according to a texture feature plane of the first texture image;

[0147] Predicting predicted texture features of each pixel in the first texture image based on the grid features of each pixel in the first texture image; the predicted texture features include predicted color and predicted density;

[0148] Volume rendering is performed using the predicted texture features of each pixel in the first texture image to obtain a second texture image.

[0149] In one embodiment, the three-dimensional object is an object in a three-dimensional space determined by an X-axis, a Y-axis, and a Z-axis, and the texture feature plane is a two-dimensional feature plane determined by an X-axis and a Y-axis;

[0150] When the processing unit 702 is used to determine the grid features of each pixel point in the first texture image according to the texture feature plane of the first texture image, it can be specifically used to:

[0151] The texture feature plane and the feature vector of the Z axis are fused to obtain the feature grid of the three-dimensional object;

[0152] Performing feature interpolation processing on the feature grid of the three-dimensional object to obtain the grid features of each voxel in the three-dimensional object;

[0153] Based on the first texture image and the grid features of each voxel in the three-dimensional object, the grid features of each pixel point in the first texture image are determined.

[0154] In one embodiment, when the processing unit 702 is configured to determine the grid features of each pixel in the first texture image based on the first texture image and the grid features of each voxel in the three-dimensional object, it can be specifically configured to:

[0155] For a target pixel point in the first texture image, point sampling is performed on a light beam in an observation direction corresponding to the target pixel point to obtain a plurality of sampling points corresponding to the target pixel point;

[0156] According to the coordinates of the multiple sampling points, the grid features corresponding to the multiple sampling points are sampled from the grid features of each voxel in the three-dimensional object;

[0157] The grid features corresponding to multiple sampling points are used as the grid features of the target pixel points.

[0158] In one embodiment, the texture feature planes of the first texture image include a color domain texture feature plane group and a density domain texture feature plane group, the color domain texture feature plane group includes a plurality of color domain texture feature planes of different resolutions, and the density domain texture feature plane group includes a plurality of density domain texture feature planes of different resolutions;

[0159] When the processing unit 702 is used to perform feature fusion processing on the texture feature plane and the feature vector of the Z axis to obtain the feature grid of the three-dimensional object, it can be specifically used to:

[0160] Performing feature fusion processing on each color domain texture feature plane in the color domain texture feature plane group and the feature vector of the Z axis to obtain a color domain feature grid group; the color domain feature grid group includes multiple color domain feature grids;

[0161] Performing feature fusion processing on each density domain texture feature plane in the density domain texture feature plane group and the feature vector of the Z axis to obtain a density domain feature grid group; the density domain feature grid group includes multiple density domain feature grids;

[0162] The feature grids of the three-dimensional object include a color domain feature grid group and a density domain feature grid group.

[0163] In one embodiment, the texture optimization model includes a feature optimization network; when the processing unit 702 is used to predict the predicted texture features of each pixel in the first texture image based on the grid features of each pixel in the first texture image, it can be specifically used to:

[0164] Obtaining a position code of each pixel in the first texture image;

[0165] Encoding the grid features of each pixel point in the first texture image and the position of each pixel point in the first texture image as input features of the feature optimization network;

[0166] The feature optimization network is called to perform prediction processing on the input features to obtain the predicted texture features of each pixel in the first texture image.

[0167] In one embodiment, the feature optimization network is any one of a grid branch network and a neural radiation field branch network; the predicted texture features include predicted color and predicted density;

[0168] If the feature optimization network is a grid branch network, the input feature is input into the grid branch network for prediction processing to obtain a predicted density and a predicted color of each pixel in the first texture image;

[0169] If the feature optimization network is a neural radiation field branch network, the input features are input into the neural radiation field branch network for a first prediction process to obtain intermediate features and the predicted density of each pixel in the first texture image; the neural radiation field branch network then performs a second prediction process based on the intermediate features to obtain the predicted color of each pixel in the first texture image.

[0170] In one embodiment, when used to construct a texture optimization model, the processing unit 702 may be specifically configured to:

[0171] A first model is constructed using a grid branch network, and the first model is pre-trained to obtain a pre-trained first model;

[0172] constructing a second model using the pre-trained first model and the neural radiation field branch network, and training the second model to obtain a trained second model; wherein the grid branch network and the neural radiation field branch network in the second model are jointly trained;

[0173] A texture optimization model is constructed based on the trained second model; the texture optimization model includes a grid branch network or a neural radiation field branch network in the trained second model.

[0174] In one embodiment, when the processing unit 702 is used to train the second model, it can be specifically used to:

[0175] Get the sample texture image;

[0176] Calling the second model to perform feature extraction processing on the sample texture image to obtain a texture feature plane of the sample texture image;

[0177] By calling the grid branch network in the second model, texture optimization processing is performed on the texture feature plane of the sample texture image to obtain a first predicted texture image;

[0178] By calling the neural radiation field branch network in the second model, texture optimization processing is performed on the texture feature plane of the sample texture image to obtain a second predicted texture image;

[0179] The second model is trained based on the difference between the first predicted texture image and the sample texture image, and the difference between the second predicted texture image and the sample texture image.

[0180] According to another embodiment of the present application, Figure 7The various units in the image processing device shown can be individually or all combined into one or several other units to constitute, or one (or some) of the units can be further divided into multiple smaller units in function to constitute, which can achieve the same operation without affecting the realization of the technical effects of the embodiments of the present application. The above-mentioned units are divided based on logical functions. In actual applications, the functions of one unit can also be implemented by multiple units, or the functions of multiple units can be implemented by one unit. In other embodiments of the present application, the image processing device may also include other units. In actual applications, these functions can also be implemented with the assistance of other units, and can be implemented by the collaboration of multiple units.

[0181] According to another embodiment of the present application, the program can be executed by running on a general computing device such as a computer including a central processing unit (CPU), a random access memory (RAM), a read-only memory (ROM) and other processing elements and storage elements. Figure 2 、 Figure 3 or Figure 5 The computer program of each step involved in the corresponding method shown in is constructed as follows Figure 7 The image processing apparatus shown in and the image processing method of the embodiment of the present application are implemented. The computer program can be recorded on, for example, a computer-readable storage medium, and loaded into the above-mentioned computing device through the computer-readable storage medium and run therein.

[0182] In the embodiments of the present application, the term "module", "unit" or "network" refers to a computer program or a part of a computer program that has a predetermined function and works together with other related parts to achieve a predetermined goal, and can be implemented in whole or in part by using software, hardware (such as processing circuits or memories) or a combination thereof. Similarly, a processor (or multiple processors or memories) can be used to implement one or more modules, units or networks. In addition, each module, unit or network can be part of an overall module, unit or network that includes the function of the module, unit or network.

[0183] In an embodiment of the present application, after obtaining the first texture image to be optimized for rendering a three-dimensional object in a game scene, the texture optimization model can be called to perform feature extraction processing on the first texture image to obtain a texture feature plane of the first texture image, and the texture optimization model can be called to perform texture optimization processing on the texture feature plane of the first texture image to obtain a second texture image; the texture feature plane extracted from the first texture image can be used to express the texture features of the three-dimensional object when the first texture image is used to render the three-dimensional object. In the process of optimizing the second texture image, by introducing the texture feature plane for optimization, the quality of the optimized second texture image can be improved, and the quality of the rendering result when the second texture image is used for rendering can be further improved, that is, the rendering performance of the second texture image can be further improved.

[0184] Based on the description of the above method embodiment and apparatus embodiment, the present application embodiment also provides a computer device. Figure 8 , the computer device includes at least a processor 801, an input interface 802, an output interface 803 and a computer-readable storage medium 804. The processor 801, input interface 802, output interface 803 and computer-readable storage medium 804 in the computer device can be connected via a bus or other means. The computer-readable storage medium 804 can be stored in the memory of the computer device. The computer-readable storage medium 804 is used to store computer programs, and the processor 801 is used to execute the computer programs stored in the computer-readable storage medium 804. The processor 801 (or CPU (Central Processing Unit)) is the computing core and control core of the computer device, which is suitable for running computer programs to implement corresponding method processes or corresponding functions.

[0185] In one embodiment, the processor 801 proposed in the embodiment of the present application can be used to execute related processes such as calling a texture optimization model to optimize a texture image and using the optimized texture image for rendering, which may include: obtaining a first texture image to be optimized; using the first texture image to render a three-dimensional object in a game scene; calling the texture optimization model to perform feature extraction processing on the first texture image to obtain a texture feature plane of the first texture image; calling the texture optimization model to perform texture optimization processing on the texture feature plane of the first texture image to obtain a second texture image; the second texture image is used to render a three-dimensional object, and the rendering performance of the second texture image is better than the rendering performance of the first texture image; using the second texture image in the game scene The texture image renders the three-dimensional object; for example, the processor 801 proposed in the embodiment of the present application can also be used to execute related processes for constructing a texture optimization model, such as: using a grid branch network to construct a first model, and pre-training the first model to obtain a pre-trained first model; using the pre-trained first model and a neural radiation field branch network to construct a second model, and training the second model to obtain a trained second model; wherein the grid branch network and the neural radiation field branch network in the second model are jointly trained; constructing a texture optimization model based on the trained second model; the texture optimization model includes the grid branch network or the neural radiation field branch network in the trained second model; and so on.

[0186] The embodiment of the present application also provides a computer-readable storage medium (Memory), which is a memory device in a computer device for storing computer programs and data. It is understandable that the computer-readable storage medium here can include both built-in storage media in the computer device and, of course, extended storage media supported by the computer device. The computer-readable storage medium provides a storage space that stores the operating system of the computer device. In addition, a computer program is also stored in the storage space, which is suitable for being loaded and executed by the processor 801 to implement the corresponding method flow provided in the embodiment of the present application. It should be noted that the computer-readable storage medium here can be a high-speed RAM memory, or a non-volatile memory (non-volatile memory), such as at least one disk storage; optionally, it can also be at least one computer-readable storage medium located away from the aforementioned processor.

[0187] In one embodiment, a computer program stored in a computer-readable storage medium may be loaded and executed by a processor to implement the above-mentioned Figure 2 、 Figure 3 or Figure 5 The corresponding steps in the method embodiment shown; in a specific implementation, the computer program in the computer-readable storage medium can be loaded by the processor and execute the following steps:

[0188] Obtaining a first texture image to be optimized; the first texture image is used to render a three-dimensional object in a game scene;

[0189] Calling the texture optimization model to perform feature extraction processing on the first texture image to obtain a texture feature plane of the first texture image;

[0190] Calling a texture optimization model to perform texture optimization processing on a texture feature plane of the first texture image to obtain a second texture image; the second texture image is used to render a three-dimensional object, and the rendering performance of the second texture image is better than the rendering performance of the first texture image;

[0191] The second texture image is used to render the three-dimensional object in the game scene.

[0192] In one embodiment, when the processor 801 is used to obtain the first texture image to be optimized, it can be specifically used to:

[0193] Export the original texture images of the three-dimensional objects from the game scene;

[0194] Rendering the three-dimensional object using the original texture image and obtaining corresponding first rendering performance parameters;

[0195] If the first rendering performance parameter indicates that the rendering performance of the original texture image meets the optimization condition, the original texture image is determined as the first texture image to be optimized.

[0196] In one embodiment, the rendering performance parameter is used to characterize the rendering performance, and the rendering performance parameter includes at least one of the following: rendering result information, the amount of resources consumed by the rendering process, and the quality of the rendering result obtained by rendering;

[0197] The first rendering performance parameter indicates that the rendering performance of the original texture image meets the optimization condition, including at least one of the following:

[0198] If the first rendering performance parameter includes rendering result information, and the corresponding rendering result information indicates that the rendering of the three-dimensional object fails, then the first rendering performance parameter indicates that the rendering performance of the original texture image meets the optimization condition;

[0199] If the first rendering performance parameter includes an amount of resources consumed by the rendering process, and the corresponding amount of resources is greater than a consumed resource amount threshold, then the first rendering performance parameter indicates that the rendering performance of the original texture image meets the optimization condition;

[0200] If the first rendering performance parameter includes the quality of the rendering result obtained by rendering, and the corresponding quality is lower than the quality threshold, the first rendering performance parameter indicates that the rendering performance of the original texture image meets the optimization condition.

[0201] In one embodiment, the rendering performance of the first texture image is characterized by a first rendering performance parameter;

[0202] When the processor 801 is used to call the texture optimization model to perform texture optimization processing on the texture feature plane of the first texture image to obtain the second texture image, it can be specifically used to:

[0203] Calling a texture optimization model to perform texture optimization processing on a texture feature plane of the first texture image to obtain a target texture image;

[0204] Pre-rendering the three-dimensional object using the target texture image and obtaining corresponding second rendering performance parameters; the second rendering performance parameters are used to characterize the rendering performance of the target texture image;

[0205] If the rendering performance represented by the second rendering performance parameter is better than the rendering performance represented by the first rendering performance parameter, the target texture image is determined to be the second texture image.

[0206] In one embodiment, the rendering performance of the first texture image is characterized by a first rendering performance parameter, and the rendering performance of the second texture image is characterized by a second rendering performance parameter; the rendering performance parameter includes at least one of the following: rendering result information, the amount of resources consumed by the rendering process, and the quality of the rendering result obtained by rendering;

[0207] The rendering performance of the second texture image is better than the rendering performance of the first texture image, including at least one of the following:

[0208] If the rendering performance parameter includes rendering result information, and the rendering result information of the second rendering performance parameter indicates that the three-dimensional object rendering is successful, and the rendering result information of the first rendering performance parameter indicates that the three-dimensional object rendering fails, then the rendering performance of the second texture image is better than the rendering performance of the first texture image;

[0209] If the rendering performance parameter includes an amount of resources consumed by a rendering process, and the amount of resources consumed by the rendering process indicated by the second rendering performance parameter is lower than the amount of resources consumed by the rendering process indicated by the first rendering performance parameter, then the rendering performance of the second texture image is better than the rendering performance of the first texture image;

[0210] If the rendering performance parameter includes the quality of a rendering result obtained by rendering, and the quality of the rendering result indicated by the second rendering performance parameter is higher than the quality of the rendering result indicated by the first rendering performance parameter, then the rendering performance of the second texture image is better than the rendering performance of the first texture image;

[0211] If the rendering performance parameters include the amount of resources consumed by the rendering process and the quality of the rendering results obtained by rendering, and the quality of the rendering results indicated by the second rendering performance parameters is higher than the quality of the rendering results indicated by the first rendering performance parameters, and the difference between the amount of resources consumed by the rendering process indicated by the second rendering performance parameters and the amount of resources consumed by the rendering process indicated by the first rendering performance parameters is less than a preset difference threshold, then the rendering performance of the second texture image is better than the rendering performance of the first texture image.

[0212] In one embodiment, when the processor 801 is used to call the texture optimization model to perform feature extraction processing on the first texture image to obtain the texture feature plane of the first texture image, it can be specifically used to:

[0213] Calling the texture optimization model to perform color domain texture feature extraction processing on the first texture image to obtain a color domain texture feature plane group of the first texture image; the color domain texture feature plane group includes a plurality of color domain texture feature planes with different resolutions;

[0214] Calling the texture optimization model to perform density domain feature extraction processing on the first texture image to obtain a density domain texture feature plane group of the first texture image; the density domain texture feature plane group includes a plurality of density domain texture feature planes of different resolutions;

[0215] The texture feature planes of the first texture image include a color domain texture feature plane group and a density domain texture feature plane group.

[0216] In one embodiment, when the processor 801 is used to call the texture optimization model to perform texture optimization processing on the texture feature plane of the first texture image to obtain the second texture image, it can be specifically used to:

[0217] Invoking a texture optimization model to determine a grid feature of each pixel in the first texture image according to a texture feature plane of the first texture image;

[0218] Predicting predicted texture features of each pixel in the first texture image based on the grid features of each pixel in the first texture image; the predicted texture features include predicted color and predicted density;

[0219] Volume rendering is performed using the predicted texture features of each pixel in the first texture image to obtain a second texture image.

[0220] In one embodiment, the three-dimensional object is an object in a three-dimensional space determined by an X-axis, a Y-axis, and a Z-axis, and the texture feature plane is a two-dimensional feature plane determined by an X-axis and a Y-axis;

[0221] When the processor 801 is configured to determine the grid feature of each pixel point in the first texture image according to the texture feature plane of the first texture image, it may be specifically configured to:

[0222] The texture feature plane and the feature vector of the Z axis are fused to obtain the feature grid of the three-dimensional object;

[0223] Performing feature interpolation processing on the feature grid of the three-dimensional object to obtain the grid features of each voxel in the three-dimensional object;

[0224] Based on the first texture image and the grid features of each voxel in the three-dimensional object, the grid features of each pixel point in the first texture image are determined.

[0225] In one embodiment, when the processor 801 is configured to determine the grid features of each pixel in the first texture image based on the first texture image and the grid features of each voxel in the three-dimensional object, it may be specifically configured to:

[0226] For a target pixel point in the first texture image, point sampling is performed on a light beam in an observation direction corresponding to the target pixel point to obtain a plurality of sampling points corresponding to the target pixel point;

[0227] According to the coordinates of the multiple sampling points, the grid features corresponding to the multiple sampling points are sampled from the grid features of each voxel in the three-dimensional object;

[0228] The grid features corresponding to multiple sampling points are used as the grid features of the target pixel points.

[0229] In one embodiment, the texture feature planes of the first texture image include a color domain texture feature plane group and a density domain texture feature plane group, the color domain texture feature plane group includes a plurality of color domain texture feature planes of different resolutions, and the density domain texture feature plane group includes a plurality of density domain texture feature planes of different resolutions;

[0230] When the processor 801 is used to perform feature fusion processing on the texture feature plane and the feature vector of the Z axis to obtain the feature grid of the three-dimensional object, it can be specifically used to:

[0231] Performing feature fusion processing on each color domain texture feature plane in the color domain texture feature plane group and the feature vector of the Z axis to obtain a color domain feature grid group; the color domain feature grid group includes multiple color domain feature grids;

[0232] Performing feature fusion processing on each density domain texture feature plane in the density domain texture feature plane group and the feature vector of the Z axis to obtain a density domain feature grid group; the density domain feature grid group includes multiple density domain feature grids;

[0233] The feature grids of the three-dimensional object include a color domain feature grid group and a density domain feature grid group.

[0234] In one embodiment, the texture optimization model includes a feature optimization network; when the processor 801 is used to predict the predicted texture features of each pixel in the first texture image based on the grid features of each pixel in the first texture image, it can be specifically used to:

[0235] Obtaining a position code of each pixel in the first texture image;

[0236] Encoding the grid features of each pixel point in the first texture image and the position of each pixel point in the first texture image as input features of the feature optimization network;

[0237] The feature optimization network is called to perform prediction processing on the input features to obtain the predicted texture features of each pixel in the first texture image.

[0238] In one embodiment, the feature optimization network is any one of a grid branch network and a neural radiation field branch network; the predicted texture features include predicted color and predicted density;

[0239] If the feature optimization network is a grid branch network, the input feature is input into the grid branch network for prediction processing to obtain a predicted density and a predicted color of each pixel in the first texture image;

[0240] If the feature optimization network is a neural radiation field branch network, the input features are input into the neural radiation field branch network for a first prediction process to obtain intermediate features and the predicted density of each pixel in the first texture image; the neural radiation field branch network then performs a second prediction process based on the intermediate features to obtain the predicted color of each pixel in the first texture image.

[0241] In one embodiment, when used to construct a texture optimization model, the processor 801 may be specifically configured to:

[0242] A first model is constructed using a grid branch network, and the first model is pre-trained to obtain a pre-trained first model;

[0243] constructing a second model using the pre-trained first model and the neural radiation field branch network, and training the second model to obtain a trained second model; wherein the grid branch network and the neural radiation field branch network in the second model are jointly trained;

[0244] A texture optimization model is constructed based on the trained second model; the texture optimization model includes a grid branch network or a neural radiation field branch network in the trained second model.

[0245] In one embodiment, when the processor 801 is used to train the second model, it can be specifically used to:

[0246] Get the sample texture image;

[0247] Calling the second model to perform feature extraction processing on the sample texture image to obtain a texture feature plane of the sample texture image;

[0248] By calling the grid branch network in the second model, texture optimization processing is performed on the texture feature plane of the sample texture image to obtain a first predicted texture image;

[0249] By calling the neural radiation field branch network in the second model, texture optimization processing is performed on the texture feature plane of the sample texture image to obtain a second predicted texture image;

[0250] The second model is trained based on the difference between the first predicted texture image and the sample texture image, and the difference between the second predicted texture image and the sample texture image.

[0251] In an embodiment of the present application, after obtaining the first texture image to be optimized for rendering a three-dimensional object in a game scene, the texture optimization model can be called to perform feature extraction processing on the first texture image to obtain a texture feature plane of the first texture image, and the texture optimization model can be called to perform texture optimization processing on the texture feature plane of the first texture image to obtain a second texture image; the texture feature plane extracted from the first texture image can be used to express the texture features of the three-dimensional object when the first texture image is used to render the three-dimensional object. In the process of optimizing the second texture image, by introducing the texture feature plane for optimization, the quality of the optimized second texture image can be improved, and the quality of the rendering result when the second texture image is used for rendering can be further improved, that is, the rendering performance of the second texture image can be further improved.

[0252] The present invention provides a computer program product, which includes a computer program stored in a computer-readable storage medium; a processor of a computer device reads the computer program from the computer-readable storage medium, and the processor executes the computer program, so that the computer device performs the above-mentioned Figure 2 、 Figure 3 or Figure 5 It should be understood that the above disclosure is only a preferred embodiment of the present application and certainly cannot be used to limit the scope of the present application. Therefore, equivalent changes made according to the claims of the present application are still within the scope of the present application.

Claims

1. An image processing method, characterized in that: include: Obtaining a first texture image to be optimized; The first texture image is used to render a three-dimensional object in a game scene; Calling a texture optimization model to perform feature extraction processing on the first texture image to obtain a texture feature plane of the first texture image; Calling the texture optimization model to perform texture optimization processing on the texture feature plane of the first texture image to obtain a second texture image; the second texture image is used to render the three-dimensional object, and the rendering performance of the second texture image is better than the rendering performance of the first texture image; The second texture image is used to render the three-dimensional object in the game scene.

2. The method according to claim 1, wherein The obtaining of the first texture image to be optimized includes: Export the original texture images of the three-dimensional objects from the game scene; Rendering the three-dimensional object using the original texture image and obtaining a corresponding first rendering performance parameter; If the first rendering performance parameter indicates that the rendering performance of the original texture image meets the optimization condition, the original texture image is determined as the first texture image to be optimized.

3. The method according to claim 2, wherein The rendering performance parameter is used to characterize the rendering performance, and the rendering performance parameter includes at least one of the following: rendering result information, the amount of resources consumed by the rendering process, and the quality of the rendering result obtained by rendering; The first rendering performance parameter indicates that the rendering performance of the original texture image meets the optimization condition, including at least one of the following: If the first rendering performance parameter includes rendering result information, and the corresponding rendering result information indicates that the rendering of the three-dimensional object fails, then the first rendering performance parameter indicates that the rendering performance of the original texture image meets the optimization condition; If the first rendering performance parameter includes an amount of resources consumed by the rendering process, and the corresponding amount of resources is greater than a consumed resource amount threshold, then the first rendering performance parameter indicates that the rendering performance of the original texture image meets the optimization condition; If the first rendering performance parameter includes the quality of the rendering result obtained by rendering, and the corresponding quality is lower than the quality threshold, then the first rendering performance parameter indicates that the rendering performance of the original texture image meets the optimization condition.

4. The method according to any one of claims 1 to 3, wherein The rendering performance of the first texture image is characterized by a first rendering performance parameter; The calling of the texture optimization model to perform texture optimization processing on the texture feature plane of the first texture image to obtain a second texture image includes: Calling the texture optimization model to perform texture optimization processing on the texture feature plane of the first texture image to obtain a target texture image; Pre-rendering the three-dimensional object using the target texture image and obtaining a corresponding second rendering performance parameter; the second rendering performance parameter is used to characterize the rendering performance of the target texture image; If the rendering performance represented by the second rendering performance parameter is better than the rendering performance represented by the first rendering performance parameter, the target texture image is determined to be the second texture image.

5. The method according to any one of claims 1 to 3, wherein The rendering performance of the first texture image is characterized by a first rendering performance parameter, and the rendering performance of the second texture image is characterized by a second rendering performance parameter; The rendering performance parameters include at least one of the following: rendering result information, the amount of resources consumed by the rendering process, and the quality of the rendering result obtained by rendering; The rendering performance of the second texture image is better than the rendering performance of the first texture image, including at least one of the following: If the rendering performance parameter includes rendering result information, and the rendering result information of the second rendering performance parameter indicates that the rendering of the three-dimensional object is successful, and the rendering result information of the first rendering performance parameter indicates that the rendering of the three-dimensional object fails, then the rendering performance of the second texture image is better than the rendering performance of the first texture image; If the rendering performance parameter includes an amount of resources consumed by a rendering process, and the amount of resources consumed by the rendering process indicated by the second rendering performance parameter is lower than the amount of resources consumed by the rendering process indicated by the first rendering performance parameter, then the rendering performance of the second texture image is better than the rendering performance of the first texture image; If the rendering performance parameter includes the quality of a rendering result obtained by rendering, and the quality of the rendering result indicated by the second rendering performance parameter is higher than the quality of the rendering result indicated by the first rendering performance parameter, then the rendering performance of the second texture image is better than the rendering performance of the first texture image; If the rendering performance parameters include the amount of resources consumed by the rendering process and the quality of the rendering results obtained by rendering, and the quality of the rendering results indicated by the second rendering performance parameters is higher than the quality of the rendering results indicated by the first rendering performance parameters, and the difference between the amount of resources consumed by the rendering process indicated by the second rendering performance parameters and the amount of resources consumed by the rendering process indicated by the first rendering performance parameters is less than a preset difference threshold, then the rendering performance of the second texture image is better than the rendering performance of the first texture image.

6. The method according to claim 1, wherein The calling of the texture optimization model to perform feature extraction processing on the first texture image to obtain a texture feature plane of the first texture image includes: Calling the texture optimization model to perform color domain texture feature extraction on the first texture image to obtain a color domain texture feature plane group of the first texture image; the color domain texture feature plane group includes a plurality of color domain texture feature planes with different resolutions; calling the texture optimization model to perform density domain feature extraction processing on the first texture image to obtain a density domain texture feature plane group of the first texture image; the density domain texture feature plane group includes a plurality of density domain texture feature planes of different resolutions; The texture feature planes of the first texture image include the color domain texture feature plane group and the density domain texture feature plane group.

7. The method according to claim 1, wherein The calling of the texture optimization model to perform texture optimization processing on the texture feature plane of the first texture image to obtain a second texture image includes: Calling the texture optimization model to determine the grid features of each pixel in the first texture image according to the texture feature plane of the first texture image; Predicting predicted texture features of each pixel in the first texture image based on grid features of each pixel in the first texture image; the predicted texture features include predicted color and predicted density; Volume rendering is performed using the predicted texture features of each pixel point in the first texture image to obtain the second texture image.

8. The method according to claim 7, wherein The three-dimensional object is an object in a three-dimensional space determined by the X-axis, the Y-axis and the Z-axis, and the texture feature plane is a two-dimensional feature plane determined by the X-axis and the Y-axis; The determining, based on the texture feature plane of the first texture image, the grid feature of each pixel point in the first texture image includes: Performing feature fusion processing on the texture feature plane and the feature vector of the Z axis to obtain a feature grid of the three-dimensional object; Performing feature interpolation processing on the feature grid of the three-dimensional object to obtain grid features of each voxel in the three-dimensional object; Based on the first texture image and the grid features of each voxel in the three-dimensional object, a grid feature of each pixel point in the first texture image is determined.

9. The method according to claim 8, wherein The determining, based on the first texture image and the grid features of each voxel in the three-dimensional object, the grid features of each pixel in the first texture image includes: For a target pixel point in the first texture image, point sampling is performed on a light beam in an observation direction corresponding to the target pixel point to obtain a plurality of sampling points corresponding to the target pixel point; According to the coordinates of the plurality of sampling points, sampling from the grid features of each voxel in the three-dimensional object to obtain grid features corresponding to the plurality of sampling points; The grid features corresponding to the multiple sampling points are used as the grid features of the target pixel point.

10. The method according to claim 8, wherein The texture feature planes of the first texture image include a color domain texture feature plane group and a density domain texture feature plane group, the color domain texture feature plane group includes a plurality of color domain texture feature planes of different resolutions, and the density domain texture feature plane group includes a plurality of density domain texture feature planes of different resolutions; The step of performing feature fusion processing on the texture feature plane and the feature vector of the Z axis to obtain the feature grid of the three-dimensional object includes: Performing feature fusion processing on each color domain texture feature plane in the color domain texture feature plane group and the feature vector of the Z axis to obtain a color domain feature grid group; the color domain feature grid group includes a plurality of color domain feature grids; Performing feature fusion processing on each density domain texture feature plane in the density domain texture feature plane group and the feature vector of the Z axis to obtain a density domain feature grid group; the density domain feature grid group includes a plurality of density domain feature grids; The feature grids of the three-dimensional object include the color domain feature grid group and the density domain feature grid group.

11. The method according to claim 7, wherein The texture optimization model includes a feature optimization network; and predicting the predicted texture features of each pixel in the first texture image based on the grid features of each pixel in the first texture image includes: Obtaining a position code of each pixel in the first texture image; encoding the grid features of each pixel point in the first texture image and the position of each pixel point in the first texture image as input features of the feature optimization network; The feature optimization network is called to perform prediction processing on the input features to obtain predicted texture features of each pixel point in the first texture image.

12. The method according to claim 11, wherein The feature optimization network is any one of a grid branch network and a neural radiation field branch network; the predicted texture features include predicted color and predicted density; If the feature optimization network is the grid branch network, the input feature is input into the grid branch network for prediction processing to obtain a predicted density and a predicted color of each pixel in the first texture image; If the feature optimization network is the neural radiation field branch network, the input feature is input into the neural radiation field branch network for a first prediction processing to obtain an intermediate feature and a predicted density of each pixel point in the first texture image; and then the neural radiation field branch network performs a second prediction processing based on the intermediate feature to obtain a predicted color of each pixel point in the first texture image.

13. The method according to claim 1, wherein The texture optimization model is constructed in the following manner: Constructing a first model using a grid branch network, and pre-training the first model to obtain a pre-trained first model; Constructing a second model using the pre-trained first model and the neural radiation field branch network, and training the second model to obtain a trained second model; wherein the grid branch network and the neural radiation field branch network in the second model are jointly trained; The texture optimization model is constructed based on the trained second model; the texture optimization model includes a grid branch network or a neural radiation field branch network in the trained second model.

14. The method according to claim 13, wherein The method of training the second model includes: Get the sample texture image; Calling the second model to perform feature extraction processing on the sample texture image to obtain a texture feature plane of the sample texture image; By calling the grid branch network in the second model, performing texture optimization processing on the texture feature plane of the sample texture image, a first predicted texture image is obtained; By calling the neural radiation field branch network in the second model, performing texture optimization processing on the texture feature plane of the sample texture image, a second predicted texture image is obtained; The second model is trained based on the difference between the first predicted texture image and the sample texture image, and the difference between the second predicted texture image and the sample texture image.

15. An image processing device, characterized in that: include: an acquiring unit, configured to acquire a first texture image to be optimized; The first texture image is used to render a three-dimensional object in a game scene; a processing unit, configured to call a texture optimization model to perform feature extraction processing on the first texture image to obtain a texture feature plane of the first texture image; The processing unit is further configured to call the texture optimization model to perform texture optimization processing on the texture feature plane of the first texture image to obtain a second texture image; the second texture image is used to render the three-dimensional object, and the rendering performance of the second texture image is better than the rendering performance of the first texture image; The processing unit is further configured to render the three-dimensional object in the game scene using the second texture image.

16. A computer device comprising an input interface and an output interface, characterized in that: Also includes: a processor and a computer-readable storage medium; The computer-readable storage medium is used to store a computer program; The processor is configured to run the computer program to implement the image processing method according to any one of claims 1 to 14.

17. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, and the computer program is suitable for being loaded by a processor and executing the image processing method according to any one of claims 1 to 14.

18. A computer program product, characterized in that The computer program product comprises a computer program, and the computer program is suitable for being loaded by a processor and executing the image processing method according to any one of claims 1 to 14.

Images