Texture image processing method and device, electronic equipment and storage medium

Abstract

The invention discloses a texture image processing method and device, electronic equipment and a storage medium, and belongs to the technical field of computers. The method comprises the steps of obtaining a first texture image; extracting a first image feature from the first texture image, wherein the first image feature is used for representing the image content of the first texture image; compressing the first image feature to obtain a second image feature, the second image feature being used for representing texture content obtained by compressing the image content; and based on the second image feature, reconstructing to obtain a target texture image. According to the method, information is compressed on the feature level, redundant content in the image content is removed, and the texture content is reserved, so that after the target texture image is obtained based on the second image feature, the texture quality of the target texture image can be ensured, and the rendering performance when the electronic equipment renders the target texture image is improved.

Description

TECHNICAL FIELD

[0001] Embodiments of the present application relate to the technical field of computer, in particular to a texture image processing method and device, electronic equipment and storage medium. BACKGROUND

[0002] In the field of computer technology, scenes such as games and animations often involve texture images. Texture images are images that show the visual effects of textures, where textures refer to regular changes in pixel color, brightness, etc. Generally, textures include multiple sets of repeating units, which can include lines, dots, graphics, etc. These units can appear regularly or randomly.

[0003] Generally, texture images are closely related to rendering performance. Therefore, how to process texture images has become a problem that needs to be solved. SUMMARY

[0004] The present application provides a texture image processing method and device, electronic equipment and storage medium, which can be used to reconstruct a target texture image that can guarantee texture quality, and improve the rendering performance of the electronic equipment when rendering the target texture image. The technical solution includes the following contents.

[0005] In one aspect, a texture image processing method is provided, which includes:

[0006] obtaining a first texture image;

[0007] extracting a first image feature from the first texture image, the first image feature being used to represent the image content of the first texture image;

[0008] compressing the first image feature to obtain a second image feature, the second image feature being used to represent texture content obtained by compressing the image content;

[0009] reconstructing a target texture image based on the second image feature, the target texture image being used to carry the texture content.

[0010] In another aspect, a texture image processing device is provided, which includes:

[0011] an acquisition module configured to obtain a first texture image;

[0012] an extraction module configured to extract a first image feature from the first texture image, the first image feature being used to represent the image content of the first texture image;

[0013] a compression module, configured to compress the first image feature to obtain a second image feature, the second image feature being used to represent texture content obtained by compressing the image content;

[0014] a reconstruction module, configured to reconstruct a target texture image based on the second image feature, the target texture image being used to carry the texture content.

[0015] In a possible implementation, the extraction module is configured to split the first texture image to obtain a plurality of image blocks; for any image block, extract an image feature of the any image block from the any image block by using a feature extraction network; and determine the first image feature based on the image features of the plurality of image blocks.

[0016] In a possible implementation, the acquisition module is further configured to acquire a sample texture image.

[0017] The extraction module is further configured to extract a sample image feature from the sample texture image by using a first network.

[0018] The reconstruction module is further configured to reconstruct a first prediction image based on the sample image feature.

[0019] The apparatus further includes:

[0020] a training module, configured to train the first network based on the sample texture image and the first prediction image to obtain the feature extraction network.

[0021] In a possible implementation, the first image feature includes a plurality of first sub-features of different dimensions, and each first sub-feature of a dimension includes a plurality of first initial elements.

[0022] The compression module is configured to determine, for each first sub-feature of a dimension, a first reference element from the plurality of first initial elements, the first reference element including a first initial element greater than a first threshold value and a first initial element less than a second threshold value, the first threshold value being greater than the second threshold value; and determine the second image feature based on the first reference element corresponding to each first sub-feature of the plurality of dimensions.

[0023] In a possible implementation, the compression module is configured to, for each first sub-feature of a dimension, interpolate a plurality of first candidate elements based on the first reference element, select a first target element corresponding to each first initial element from the plurality of first candidate elements; and determine the second image feature based on the first reference element corresponding to each first sub-feature of the plurality of dimensions and each first target element.

[0024] In a possible implementation, the compression module is configured to quantize the first reference element to obtain a first quantized element; and interpolate a plurality of first candidate elements based on the first quantized element.

[0025] In a possible implementation, the compression module is configured to, for each dimension of the first sub-feature, quantize each first target element based on a first quantized element corresponding to the first reference element to obtain a second quantized element; and determine the second image feature based on the first quantized elements and the second quantized elements corresponding to the first sub-feature of each dimension.

[0026] In a possible implementation, the reconstruction module is configured to determine a third image feature based on the second image feature, the third image feature being used to represent a target content obtained by decompressing the texture content; and reconstruct a target texture image based on the third image feature.

[0027] In a possible implementation, the reconstruction module is configured to determine, based on the second image feature, a second reference element and a plurality of reference elements corresponding to each dimension; for each dimension, determine a plurality of second candidate elements based on the second reference element, and select a second target element corresponding to each reference element from the plurality of second candidate elements; and determine a third image feature based on the second target elements corresponding to each dimension.

[0028] In a possible implementation, the second image feature includes a plurality of first quantized elements corresponding to each dimension.

[0029] The reconstruction module is configured to, for each dimension, dequantize a first quantized element corresponding to the dimension based on a first reference element corresponding to the dimension to obtain a second reference element corresponding to the dimension.

[0030] In a possible implementation, the reconstruction module is configured to determine a target adjustment factor based on the first texture image and the third image feature, the target adjustment factor being used to represent a difference between the image content and the target content; adjust the third image feature based on the target adjustment factor to obtain a target image feature; and reconstruct a target texture image based on the target image feature.

[0031] In a possible implementation, the target adjustment factor is determined by an adjustment network; and the apparatus further includes:

[0032] The determination module is configured to determine a reconstruction image feature based on a sample image feature, the reconstruction image feature being used to represent reconstruction information obtained by compressing and decompressing image content of a sample texture image; and determine the target adjustment factor based on the reconstruction image feature.

[0033] The determining module is further configured to determine, by the second network, a predicted adjustment factor based on the sample texture image and the reconstructed image feature, the predicted adjustment factor being used to represent a difference between image content of the sample texture image and the reconstructed information.

[0034] The reconstructing module is further configured to reconstruct a second predicted image based on the predicted adjustment factor and the reconstructed image feature.

[0035] The training module is configured to train the second network based on the sample texture image and the second predicted image, to obtain the adjustment network.

[0036] In a possible implementation, the apparatus further includes:

[0037] The determining module is configured to determine resources occupied in a process of loading the first texture image and resources occupied in a process of loading the target texture image.

[0038] The selecting module is configured to select a texture image with least occupied resources from the first texture image and the target texture image.

[0039] In another aspect, an electronic device is provided, which includes a processor and a memory, and the memory stores at least one computer program, and the at least one computer program is loaded and executed by the processor, so that the electronic device implements the texture image processing method described above.

[0040] In another aspect, a computer readable storage medium is also provided, which stores at least one computer program, and the at least one computer program is loaded and executed by a processor, so that an electronic device implements the texture image processing method described above.

[0041] In another aspect, a computer program is also provided, which is at least one, and the at least one computer program is loaded and executed by a processor, so that an electronic device implements the texture image processing method described above.

[0042] In another aspect, a computer program product is also provided, which stores at least one computer program, and the at least one computer program is loaded and executed by a processor, so that an electronic device implements the texture image processing method described above.

[0043] The technical solutions provided in the present application at least bring the following beneficial effects:

[0044] In the technical scheme provided in the present application, the first image feature of the first texture image is compressed to obtain the second image feature, so that the image content is compressed at the feature level, the texture content is retained, and the redundant content irrelevant to the texture is removed. After the target texture image is reconstructed based on the second image feature, on the one hand, since the texture content is retained, it can be ensured that the target texture image has high texture quality, and on the other hand, since the redundant content is removed, the electronic device has higher rendering efficiency when rendering the target texture image. BRIEF DESCRIPTION OF DRAWINGS

[0045] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed in the embodiment description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0046] Figure 1 is a computer system schematic diagram of a texture image processing method provided by an embodiment of the present application;

[0047] Figure 2 is a flowchart of a texture image processing method provided by an embodiment of the present application;

[0048] Figure 3 is a reconstruction schematic diagram of a target texture image provided by an embodiment of the present application;

[0049] Figure 4 is a determination schematic diagram of a texture image used by a game picture provided by an embodiment of the present application;

[0050] Figure 5 is a schematic diagram of loading a texture image provided by an embodiment of the present application;

[0051] Figure 6 is a flowchart of another texture image processing method provided by an embodiment of the present application;

[0052] Figure 7 is a texture image comparison schematic diagram provided by an embodiment of the present application;

[0053] Figure 8 is a structure schematic diagram of a texture image processing apparatus provided by an embodiment of the present application;

[0054] Figure 9 is a structure schematic diagram of a terminal device provided by an embodiment of the present application;

[0055] Figure 10 is a structure schematic diagram of a server provided by an embodiment of the present application. DETAILED DESCRIPTION

[0056] In order to make the purposes, technical solutions and advantages of the present application clearer, the embodiments of the present application will be further described in detail below with reference to the drawings.

[0057] It should be noted that the terms "first", "second", and the like in the present application are used to distinguish similar objects, and do not necessarily have to describe a specific order or sequence. It should be understood that the terms used in this way can be interchanged under appropriate circumstances, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present application. Rather, they are merely examples of devices and methods consistent with some aspects of the present application as detailed in the appended claims.

[0058] In the field of games, animations and the like involving texture images, electronic devices need to render texture images. Texture images are closely related to rendering performance, and unreasonable texture images can cause a decrease in rendering performance. For example, a texture image that is too large in size can cause an electronic device to consume too many resources such as a central processing unit (CPU), a graphic processing unit (GPU), and a memory when loading the texture image, thereby affecting the rendering performance. Based on this, the embodiments of the present application provide a texture image processing method, which can obtain a second image feature by compressing a first image feature of a texture image, thereby removing redundant content in the texture image and retaining texture content. After obtaining a target texture image based on the second image feature, the rendering performance of the electronic device when rendering the target texture image is improved while ensuring the texture quality of the target texture image.

[0059] As shown in Figure 1 Figure 1 is a computer system schematic diagram of a texture image processing method provided by the embodiments of the present application, which includes a terminal device 101 and a server 102. The terminal device 101 can run a client that applies a texture image, such as a game client, a photo album client, and the like. The texture image processing method provided by the embodiments of the present application can be executed by the terminal device 101, or by the server 102, or by the terminal device 101 and the server 102 together.

[0060] ​The server 102 is configured to provide background services for the client installed on the terminal device 101. In a possible implementation, the server 102 undertakes the main computing work, and the terminal device 101 undertakes the secondary computing work. Alternatively, the server 102 undertakes the secondary computing work, and the terminal device 101 undertakes the main computing work. Alternatively, the terminal device 101 and the server 102 jointly perform the computing in a distributed computing architecture.

[0061] Optionally, the terminal device 101 can be any electronic device product that can interact with a user through one or more of a keyboard, a touchpad, a remote controller, voice interaction, or a handwriting device. For example, the terminal device 101 can be a smartphone, a tablet computer, a notebook computer, a desktop computer, a smart speaker, a smart watch, a PC (Personal Computer), a mobile phone, a PDA (Personal Digital Assistant), a wearable device, a PPC (Pocket PC), a smart car machine, a smart television, or the like.

[0062] The terminal device 101 can be used to refer to one of a plurality of terminal devices, and the terminal device 101 is used as an example in this embodiment. It can be understood by those skilled in the art that the number of terminal devices 101 can be more or less. For example, the terminal device 101 can be only one, or the terminal device 101 can be dozens or hundreds, or more, and the number and type of the terminal device 101 are not limited in this embodiment.

[0063] The server 102 can be one server, or a server cluster composed of a plurality of servers, or any one of a cloud computing platform and a virtualization center, and the number of servers is not limited in this embodiment. The server 102 and the terminal device 101 are directly or indirectly connected in communication through a wired network or a wireless network. The server 102 has a data receiving function, a data processing function, and a data sending function. Of course, the server 102 can also have other functions, and the functions are not limited in this embodiment.

[0064] In this embodiment, the target object 103 can use the terminal device 101. The terminal device 101 is configured to obtain a texture image from the server 102 and display the texture image. The server 102 can process the texture image according to the method of this embodiment to obtain a processed texture image, and the processing process includes the following steps.

[0065] In step 104, a first image feature of the texture image is extracted.

[0066] In step 105, the first image feature is compressed to obtain a second image feature.

[0067] Step 106: Decompress the second image features to obtain the third image features.

[0068] Step 107: Determine the target adjustment factor based on the texture image and the features of the third image.

[0069] Step 108: Adjust the third image features based on the target adjustment factor to obtain the target image features.

[0070] Step 109: Reconstruct a new texture image based on the features of the target image.

[0071] The implementation principles of steps 104 to 109 can be found in... Figure 2 The relevant descriptions of the embodiments shown will not be repeated here.

[0072] After receiving the processed texture image, server 102 sends the texture image to terminal device 101 via a wired or wireless network, so that the processed texture image can be displayed on terminal device 101. Figure 1 As shown, the processed texture image 110 can be displayed on the terminal device 101.

[0073] The solutions in the various optional embodiments of this application can be implemented based on artificial intelligence (AI) technology. Artificial intelligence is the theory, method, technology, and application system that uses digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to obtain optimal results. In other words, artificial intelligence is a comprehensive technology within computer science that attempts to understand the essence of intelligence and produce a new type of intelligent machine that can react in a way similar to human intelligence. Artificial intelligence also studies the design principles and implementation methods of various intelligent machines, enabling them to possess perception, reasoning, and decision-making functions.

[0074] Artificial intelligence (AI) is a comprehensive discipline encompassing a wide range of fields, including both hardware and software technologies. Fundamental AI technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing, pre-trained model technology, operating / interactive systems, and mechatronics. Among these, pre-trained models, also known as large-scale models or foundational models, can be widely applied to downstream tasks across various AI fields after fine-tuning. AI software technologies primarily include computer vision, speech processing, natural language processing, and machine learning / deep learning.

[0075] like Figure 2 As shown, Figure 2is a flowchart of a texture image processing method provided by an embodiment of the present application. The method can be applied to a computer system as shown in Figure 1 For ease of description, the terminal device 101 or the server 102 that performs the texture image processing method in the embodiments of the present application is referred to as an electronic device, that is, the method of the embodiments of the present application can be performed by an electronic device. As shown in Figure 2 The method includes the following steps.

[0076] Step 201: Obtain a first texture image.

[0077] The first texture image is an image for showing any texture, for example, the texture shown by the first texture image includes at least one of the texture of clothes, the texture of accessories, the texture of trees, etc. The embodiments of the present application do not limit the content of the first texture image and the way in which the electronic device obtains the first texture image. For example, the electronic device can obtain the first texture image input by a target object, or the electronic device can read the first texture image from a database, a U disk or the like, or the electronic device can obtain the first texture image through the Internet, or the electronic device can obtain the first texture image whose resource occupation is higher than a resource occupation threshold set by a human being, etc.

[0078] Step 202: Extract a first image feature from the first texture image, the first image feature being used to represent the image content of the first texture image.

[0079] For example, the electronic device can extract the feature of the first texture image by using an algorithm such as Principal Component Analysis (PCA), Scale-Invariant Feature Transform (SIFT), Histogram Of Oriented Gradient (HOG), etc., to obtain the first image feature, which is used to represent the image content such as color, structure, shape, semantics, etc. of the first texture image. Alternatively, step 202 includes steps 2021 to 2023 (not shown in the figure), and the feature of the first texture image is extracted according to the content of steps 2021 to 2023.

[0080] Step 2021: Segment the first texture image to obtain a plurality of image blocks.

[0081] In the embodiments of the present application, the electronic device can split the first texture image into a plurality of image blocks. The number of image blocks can be a pre-set value or a randomly generated value. Any two image blocks can have the same size or different sizes, and any two image blocks can have pixel overlap or no pixel overlap. If a pixel point is located in two image blocks at the same time, the pixel point is the same pixel point included in the two image blocks. Two image blocks have pixel overlap means that the two image blocks include at least one same pixel point. Two image blocks have no pixel overlap means that the two image blocks do not include the same pixel point. For example, the electronic device can split the first texture image into a plurality of 4x4 image blocks, and each 4x4 image block includes 4 rows and 4 columns of pixel points.

[0082] In step 2022, for any image block, the image feature of any image block is extracted from any image block by the feature extraction network.

[0083] Generally, the operation efficiency of the feature extraction network is related to the image size input to the feature extraction network. By splitting the first texture image into image blocks and extracting the features of the image blocks by the feature extraction network, the image size input to the feature extraction network is reduced, and the feature extraction efficiency is improved.

[0084] The embodiments of the present application do not limit the structure, size, etc. of the feature extraction network. For example, the feature extraction network includes an input layer and an output layer, and can further include at least one of a linear layer, a convolution layer, a normalization layer, an activation layer, etc. The structure of the feature extraction network is different, and the way of extracting the features of any image block by the feature extraction network is also different. In the following, the process of feature extraction is described by taking the feature extraction network as a convolution network.

[0085] In the present example, the feature extraction network comprises an input layer, at least one convolutional layer, and an output layer. The input layer is a linear layer, and the input of the input layer is an image block, which comprises information of a plurality of pixel points, and the information of any pixel point comprises at least one of the position, color, brightness, etc. of the pixel point. The information of each pixel point is linearly mapped to a first vector of the pixel point by the linear layer, and the first vector of the pixel point represents the information of the pixel point. The first vectors of the pixel points form an embedded feature of the image block, based on which the embedded feature can represent the content of the first texture image, and the output of the input layer is the embedded feature of the image block. The first convolutional layer in the at least one convolutional layer is configured to perform convolutional processing on the embedded feature of the image block to obtain an output result of the first convolutional layer; and the non-first convolutional layer in the at least one convolutional layer is configured to perform convolutional processing on the output result of the previous convolutional layer to obtain an output result of the non-first convolutional layer. Optionally, any convolutional layer comprises an activation function, which includes but is not limited to ReLU (Rectified Linear Unit), Tanh (hyperbolic tangent function), Sigmoid function (S-shaped growth curve), Leaky Relu (leaky linear rectifier), etc. Any convolutional layer is configured to sequentially perform convolutional processing and activation processing on the embedded feature of the image block or the output result of the previous convolutional layer. The output layer is a linear layer, and the output layer is configured to linearly map the output result of the last convolutional layer to obtain the image feature of the image block.

[0086] Suppose the first texture image is divided into a plurality of 4x4 image blocks, and each image block is independently input into the feature extraction network. The image block is denoted as I(x, y), where I represents the information of each pixel point in the image block, and x and y represent the pixel coordinates of each pixel point in the image block, x is the horizontal coordinate in the pixel coordinates, and y is the vertical coordinate in the pixel coordinates. I(x, y) is mapped to F(x, y, z) by the feature extraction network, where F(x, y, z) represents the image feature of the image block, and z is the dimension of the image feature. Since the 4x4 image block comprises 4 rows and 4 columns of pixel points, and the information of the pixel points is assumed to be color values, since the color values comprise data in three dimensions of red, green, and blue, the size of the image block is 4x4x3. The feature extraction network comprises an input layer, three convolutional layers (convolutional layers 1 to 3), and an output layer, and the contents of each layer of the feature extraction network are shown in Table 1.

[0087] Table 1

[0088] Layers of the feature extraction network Input size Output size Activation function Input layer 4×4×3 4×4×3 None Convolutional layer 1 4×4×3 4×4×16 ReLU Convolutional layer 2 4×4×16 4×4×32 ReLU Convolutional layer 3 4×4×32 4×4×k Tanh Output layer 4×4×k 4×4×k None

[0089] The feature of the image block is extracted by the feature extraction network shown in Table 1 to obtain an image feature of the image block, and the image feature has a size of 4*4*k, where k is a dimension of the image feature. Generally, k is a positive integer greater than 3. When k is greater than 3, because k is the dimension of the image feature and 3 is the dimension of the embedded feature output by the input layer, the dimension of the image feature is higher than the dimension of the embedded feature. Based on this, the embedded feature is a feature in a low-dimensional space, and the image feature is a feature in a high-dimensional space.

[0090] In a possible implementation of the embodiment of the present application, before the feature of the image block is extracted by the feature extraction network, the feature extraction network needs to be trained to extract the feature of the image block. In the embodiment of the present application, the feature extraction network is trained in the following steps A1 to A4. That is, the method of the embodiment of the present application further includes steps A1 to A4 (not shown in the figure).

[0091] In step A1, a sample texture image is obtained.

[0092] In the embodiment of the present application, the content and the acquisition manner of the sample texture image are similar to the content and the acquisition manner of the first texture image, which can be seen from the description of step 201, and will not be repeated here. The sample texture image can be the first texture image or can not be the first texture image.

[0093] In step A2, a sample image feature is extracted from the sample texture image by the first network.

[0094] In the embodiment of the present application, the structure and size of the first network are similar to the structure and size of the feature extraction network, and the only difference between the two is the network parameters. The sample texture image can be divided into multiple image blocks, and the features of the image blocks can be extracted in the manner of step 2022, and the features of the image blocks can be spliced into the sample image feature.

[0095] In step A3, a first predicted image is reconstructed based on the sample image feature.

[0096] In the embodiment of the present application, the first predicted image can be reconstructed based on the sample image feature by the third network. The structure and size of the third network are not limited in the embodiment of the present application. For example, the third network includes at least one of a linear layer, a deconvolution layer, a normalization layer, and an activation layer, and the structure of the third network is different, and the way of reconstructing the sample image feature into the first predicted image by the third network is also different. Hereinafter, the process of image reconstruction is described by taking the third network as a deconvolution network.

[0097] In the example, the third network comprises at least one deconvolution layer and an output layer. The sample image features comprise features of a plurality of image blocks, a first deconvolution layer in the at least one deconvolution layer is configured to perform deconvolution processing on the features of any image block to obtain an output result of the first deconvolution layer, and a non-first deconvolution layer in the at least one deconvolution layer is configured to perform deconvolution processing on the output result of a previous deconvolution layer to obtain an output result of the non-first deconvolution layer. Optionally, any deconvolution layer comprises an activation function, and any deconvolution layer is configured to perform deconvolution processing and activation processing on the features of an image block or the output result of a previous deconvolution layer in sequence. The output layer is a linear layer, and the output layer is configured to perform linear mapping on the output result of the last deconvolution layer to obtain an image block. The image blocks are spliced to obtain the first predicted image.

[0098] In step A4, the first network is trained based on the sample texture image and the first predicted image to obtain the feature extraction network.

[0099] In the example, the loss of the first network can be determined based on the sample texture image and the first predicted image according to a loss function. The type of the loss function is not limited herein, and exemplarily, the loss function is any one of a mean square error (MSE), a root mean squared error (RMSE), or the like. The first network is trained once through the loss of the first network to obtain a trained first network. If the trained first network meets a first training end condition, the trained first network is taken as the feature extraction network; if the trained first network does not meet the first training end condition, the trained first network is taken as the first network for next training, and the first network is trained next time in the manner of steps A1 to A4 until the trained first network meets the first training end condition, thereby obtaining the feature extraction network.

[0100] The first network is trained based on the sample texture image and the first predicted image, so that the first network is optimized in a direction that the extracted image features can reconstruct the original image, thereby improving the accuracy of the feature extraction network, and enabling accurate image features to be extracted through the feature extraction network subsequently.

[0101] The example does not limit the content that the trained first network meets the first training end condition, and exemplarily, the trained first network meets the first training end condition, including that the number of training times of the trained first network reaches a set number, or the accuracy of the trained first network reaches a set accuracy, or the like. Optionally, the set number or the set accuracy is a value pre-set according to artificial experience.

[0102] Optionally, the training target of the first network is: min||I(x, y)-G(F(x, y, z))||. 2 . Wherein, ||I(x, y)-G(F(x, y, z))|| 2 characterizes the loss of the first network. I(x, y) characterizes the sample texture image, F(x, y, z) characterizes the sample image feature, and G(F(x, y, z)) characterizes the first predicted image. ||·|| characterizes the norm symbol. min characterizes the minimum symbol.

[0103] In the process of training the first network, training can be performed based on a stochastic gradient descent (SGD) algorithm. The SGD algorithm is one of the optimization algorithms of the neural network, which is used to minimize the loss function. In the SGD algorithm, the learning rate is a very important parameter, which is used to determine the step size of the network parameter update of the neural network each time the training is performed. Among them, the network parameter is a parameter used by the neural network to perform feature processing such as convolution, linear mapping, activation, etc., which includes at least one of a weight term or a bias term, the weight term is used to weight the feature, the bias term is used to add to the feature, or the bias term is used to add to the weighted feature. Based on this, in each process of training the first network, the gradient of the loss of the first network with respect to the network parameter of the first network can be calculated, and the network parameter of the first network is updated based on the gradient and the learning rate, to realize one training of the first network. Among them, the learning rate, the number of training times, etc. can be flexibly adjusted according to the actual situation, such as the learning rate is 0.01, the iteration number is 1000, etc.

[0104] Step 2023, based on the image features of the plurality of image blocks, determine the first image feature.

[0105] In the embodiments of the application, the image features of the plurality of image blocks can be spliced to obtain the first image feature. Alternatively, the image features of the plurality of image blocks are fused by, for example, addition, multiplication, etc. to obtain the first image feature.

[0106] Step 203, compress the first image feature to obtain the second image feature, and the second image feature is used to characterize the texture content obtained by compressing the image content.

[0107] In the embodiments of the present application, the electronic device can compress the first image feature based on at least one of the compression methods such as principal component analysis, linear discriminant analysis, and feature quantization. By compressing the first image feature at the feature level, part of the image content is removed. After reconstructing the target texture image based on the second image feature, the electronic device has higher rendering efficiency when rendering the target texture image because part of the content is removed. It can be understood that the compression method is different, and the way of compressing the first image feature is also different. In the following, taking the least square method as an example, the implementation of step 203 is described.

[0108] In the present example, the first image feature includes a plurality of first sub-features of multiple dimensions, and each first sub-feature of any dimension includes a plurality of first initial elements. For example, assuming that the first image feature includes first sub-features of k dimensions, the first sub-feature of the z-th dimension is a matrix including a plurality of first initial elements in rows and columns. Optionally, the first initial element in the i-th row and the j-th column is the feature of the pixel point in the i-th row and the j-th column of the first texture image. Wherein, i, j, k and z are positive integers, and z takes any value from 1 to k.

[0109] It should be noted that each dimension corresponds to at least one first sub-feature. For example, the first image feature includes image features of a plurality of image blocks. Then, for any image block, the image feature of the image block includes a plurality of first sub-features of multiple dimensions, and each first sub-feature of any dimension includes a plurality of first initial elements. Optionally, the first initial element in the i-th row and the j-th column is the feature of the pixel point in the i-th row and the j-th column of the image block. Wherein, i and j are positive integers. In this case, for the first image feature, the first image feature includes a plurality of first sub-features corresponding to multiple dimensions respectively. That is, the electronic device can divide the first sub-feature of any dimension corresponding to the first texture image into the first sub-feature of the dimension corresponding to a plurality of image blocks. Alternatively, for the first image feature, the first image feature includes one first sub-feature corresponding to each dimension.

[0110] For ease of description, the implementation of step 203 is described below by taking the image feature of the image block as an example, which includes one first sub-feature corresponding to each dimension. In the following, the image feature of the image block is also referred to as the first feature block of the image block. In the embodiments of the present application, step 203 includes step 2031 to step 2032 (not shown in the figure).

[0111] In step 2031, for any first sub-feature of a dimension, a first reference element is determined from a plurality of first initial elements, the first reference element includes a first initial element greater than a first threshold and a first initial element less than a second threshold, and the first threshold is greater than the second threshold.

[0112] In the embodiments of the present application, the first image features include first feature blocks of a plurality of image blocks, and the first feature block of each image block includes first sub-features of each dimension. For example, the first feature block of an image block is 4x4xk, where k is the number of dimensions.

[0113] The first sub-feature of any dimension includes a plurality of first initial elements, and the electronic device can determine a first initial element satisfying a numerical condition from the first initial elements, and take the determined first initial element as a first reference element. There is at least one first reference element. The first initial element satisfying the numerical condition includes: a first initial element greater than a first threshold value; and a first initial element less than a second threshold value. The first threshold value is greater than the second threshold value, and the first threshold value or the second threshold value can be a numerical value determined according to experience or some first initial elements.

[0114] In a possible implementation, assuming that the first sub-feature of the zth dimension is a matrix F(x, y, z), where x and y represent pixel coordinates of each pixel point in the image block. The maximum first initial element Fmax(z) in the first sub-feature of this dimension can be determined as Fmax(z) = max x,y F(x, y, z), and the minimum first initial element Fmin(z) in the first sub-feature of this dimension can be determined as Fmin(z) = min x,y F(x, y, z). Wherein, z = 1, 2, …, k. Fmax(z) and Fmin(z) are the first reference elements in this example. In the following, the maximum first initial element can be referred to as the maximum value, and the minimum first initial element can be referred to as the minimum value.

[0115] In an image, there is a concept of high-frequency information. High-frequency information refers to the information of a pixel point changing rapidly. For example, the information of a pixel point located on a line is obviously different from the information of a pixel point near the pixel point and not located on the line. For another example, for the boundary between a background region and a non-background region of an image, the information of a pixel point located at the boundary is obviously different. According to the characteristics of high-frequency information, high-frequency information can reflect the texture of an image.

[0116] The first reference element of any dimension includes a first initial element greater than a first threshold value and a first initial element less than a second threshold value, for example, the first reference element of any dimension includes the maximum value and the minimum value of any dimension, so that the first reference element can reflect the high-frequency information of the dimension. By determining the first reference element, the high-frequency information of the dimension is focused, so that when the second image feature is determined based on the first reference element, more content related to the texture can be retained, and content unrelated to the texture can be removed, to ensure the quality of the texture image.

[0117] In step 2032, the second image feature is determined based on the first reference elements corresponding to the first sub-features of the plurality of dimensions.

[0118] In the embodiments of the present application, the electronic device can splice the first reference elements corresponding to the plurality of dimensions to obtain the second image feature.

[0119] In a possible implementation, step 2032 includes steps S1-S2 (not shown in the figure).

[0120] In step S1, for the first sub-feature of any dimension, a plurality of first candidate elements are obtained by interpolation based on the first reference elements, and a first target element corresponding to each first initial element is selected from the plurality of first candidate elements.

[0121] When each first reference element of any dimension includes a maximum value and a minimum value, the plurality of first candidate elements can be calculated based on the maximum value and the minimum value in an interpolation manner. That is, the difference between the maximum value and the minimum value is calculated, and the difference is weighted by different weights to obtain a plurality of weighted calculation results. Each first candidate element is obtained by adding each weighted calculation result to the smallest first initial element.

[0122] In an exemplary embodiment, obtaining the plurality of first candidate elements by interpolation based on the first reference elements includes: quantizing the first reference elements to obtain first quantized elements; and obtaining the plurality of first candidate elements by interpolation based on the first quantized elements.

[0123] Quantization is a technique for converting floating-point numbers into integers. Generally, the first reference elements are floating-point numbers, and by quantizing the first reference elements, the first reference elements in the form of floating-point numbers can be quantized into the first quantized elements in the form of integers. Since the operation efficiency of the electronic device on integers is higher than that on floating-point numbers, by quantizing the first reference elements into the first quantized elements, it is beneficial for the electronic device to perform subsequent operations based on the first quantized elements and to speed up the operation efficiency. The way to quantize the first reference elements is various, and one possible quantization way is shown below.

[0124] It is assumed that the first reference element is a floating point number between 0 and 2. For any first reference element, the first reference element can be quantized into an integer. It is assumed that the first reference element includes a maximum first initial element Fmax(z) and a minimum first initial element Fmin(z), and that, on the one hand, the maximum first initial element Fmax(z) is quantized according to Qmax(z) = [15 x (Fmax(z) - Fmax(z) / 2)] to obtain a first quantized element Qmax(z), and on the other hand, the minimum first initial element Fmin(z) is quantized according to Qmin(z) = [15 x (Fmin(z) - Fmin(z) / 2)] to obtain a first quantized element Qmin(z). Wherein [·] represents the rounding symbol. Since Fmax(z) and Fmin(z) are floating point numbers between 0 and 2, after quantization according to the above formula, Qmax(z) and Qmin(z) obtained are integers between 0 and 15. When the electronic device operates Qmax(z) and Qmin(z), Qmax(z) and Qmin(z) can be converted into binary data of several bits, such as 4-bit binary data, and subsequent operations are performed on the binary data.

[0125] Next, the electronic device calculates a plurality of first candidate elements based on each first quantized element, and the calculation method is as described in "obtaining a plurality of first candidate elements based on the first reference element interpolation", and the implementation principles are similar, which will not be repeated here.

[0126] Optionally, when calculating a plurality of first candidate elements in an interpolation manner, a plurality of interpolation levels can be determined. For any interpolation level, based on the interpolation level, the difference between the first quantized element corresponding to the maximum value and the first quantized element corresponding to the minimum value is weighted calculated with a corresponding weight, and the sum of the first quantized element corresponding to the minimum value and the weighted calculation result is calculated to obtain a first candidate element.

[0127] For example, the first candidate element Fi(x, y, z) corresponding to the interpolation level i is calculated according to Fi(x, y, z) = [Qmax(z) - Qmin(z)] / 3 x i + Qmin(z). Wherein i = 0, 1, 2, 3, Qmax(z) is the first quantized element corresponding to the maximum value, and Qmin(z) is the first quantized element corresponding to the minimum value. As can be seen from the above formula, when i = 0, Fi(x, y, z) = Qmin(z); when i = 3, Fi(x, y, z) = Qmax(z); when i = 1 or 2, Fi(x, y, z) is the data located at the three-equal position on the numerical interval between Qmin(z) and Qmax(z).

[0128] The first candidate element is data between the first quantization element corresponding to the maximum value and the first quantization element corresponding to the minimum value. Since the first quantization element corresponding to the maximum value and the first quantization element corresponding to the minimum value can reflect high-frequency information, and the high-frequency information is related to texture, by interpolating to obtain the plurality of first candidate elements, high-frequency information of different levels is focused, and the high-frequency information can reflect details of the texture. Subsequently, when the second image feature is determined based on the plurality of first candidate elements, high-frequency information is supplemented, which is beneficial to solve the problem of texture blurring, so that the second image feature can reflect more texture details, so that the texture image reconstructed based on the second image feature has higher resolution and better visual effect, and the texture quality and rendering efficiency of the texture image are improved.

[0129] After the plurality of first candidate elements are calculated, for any first initial element, a first target element corresponding to the first initial element is selected from the plurality of first candidate elements. The manner of selecting the first target element is not limited herein, for example, a first candidate element can be randomly selected from the plurality of first candidate elements as the first target element corresponding to the first initial element, or a first candidate element closest to the first initial element can be selected from the plurality of first candidate elements as the first target element corresponding to the first initial element. The first candidate element closest to the first initial element refers to a first candidate element having the smallest absolute value difference from the first initial element. By determining the first target element, the texture details are improved while being as close to the original texture image as possible, and the consistency of the texture image is ensured.

[0130] For example, the first target element Fs(x, y, z) is calculated according to Fs(x, y, z) = arg min Fi |F(x, y, z)-Fi(x, y, z)|, where F(x, y, z) represents a first initial element, Fi(x, y, z) represents a first candidate element corresponding to an interpolation level i, i takes any integer value in 0 to 3. |·| represents the absolute value symbol. arg min represents determining Fi(x, y, z) that makes |F(x, y, z)-Fi(x, y, z)| the smallest.

[0131] In step S2, the second image feature is determined based on the first reference elements corresponding to the plurality of dimensions of the first sub-features and the plurality of first target elements.

[0132] In the embodiments of the present application, the first reference elements corresponding to the first sub-features of the plurality of dimensions can be spliced to obtain the first feature part. The first target elements corresponding to the first sub-features of the plurality of dimensions can be spliced to obtain the second feature part. The first feature part and the second feature part are spliced to obtain the second image feature. Since the first reference elements can reflect the content of the texture, and the first target elements can reflect the content of the texture details, the second image feature is determined by the first reference elements and the first target elements, thereby improving the ability of the second image feature to represent the texture.

[0133] In the example embodiments, step 2032 includes: for the first sub-feature of any dimension, quantizing each first target element based on the first quantization element corresponding to the first reference element to obtain each second quantization element; and determining the second image feature based on the first quantization element corresponding to the first sub-feature of the plurality of dimensions and each second quantization element.

[0134] In the embodiments of the present application, on the one hand, the first reference elements corresponding to the first sub-features of the plurality of dimensions can be quantized to obtain the first quantization element. The manner of quantizing to obtain the first quantization element is described above, and will not be described here.

[0135] Suppose that the first image feature includes a plurality of 4x4xk first sub-features, and k is the number of dimensions. The zth dimension in any first sub-feature corresponds to the first quantization element Qmax(z) and the first quantization element Qmin(z), z takes any integer from 1 to k, and k is a positive integer greater than 1. Splicing each first quantization element obtains the first feature part C1, and C1=Qmax(1)Qmin(1)Qmax(2)Qmin(2)…Qmax(k)Qmin(k). Suppose that the first quantization element is a bit string of several bits, for example, the first quantization element is a bit string of four bits, then the first feature part is also a bit string of multiple bits, for example, the first feature part is a bit string of 64 bits.

[0136] On the other hand, the first sub-feature of any dimension corresponds to a plurality of first target elements. Any first target element can be a floating-point number, based on which the first target element is quantized to improve the operation efficiency of the electronic device in subsequent operations of the first target element. There are various quantization methods, and one possible quantization method is shown below.

[0137] For the first sub-feature of any dimension, each first target element can be quantized into a second quantization element based on the corresponding first quantization element. Assuming that the first quantization elements include a first quantization element Qmax(z) and a first quantization element Qmin(z). Then: the first target element Fs(x, y, z) can be quantized into a second quantization element Qs(x, y, z) according to Qs(x, y, z) = [(Fs(x, y, z) - Qmin(z)) / (Qmax(z) - Qmin(z))] x 3. Assuming that, since Fs(x, y, z) is data between Qmax(z) and Qmin(z), Qs(x, y, z) is data between 0 and 3. Subsequently, the computer can convert Qs(x, y, z) into binary data of several bits, such as 2-bit binary data.

[0138] Next, the second feature part is obtained by splicing the second quantization elements corresponding to the first sub-feature of each dimension. There are many splicing methods. Alternatively, the first sub-feature of any dimension corresponds to multiple second quantization elements, and each second quantization element corresponds to a pixel point. Based on this, for any pixel point, the second quantization elements of each dimension corresponding to the pixel point are spliced to obtain the splicing part of the pixel point. The splicing parts of each pixel point are spliced in order to obtain the second feature part.

[0139] Assuming that the first image feature includes multiple first sub-features of 4x4xk, and k is the number of dimensions. Then, for the zth dimension in any first sub-feature, the second quantization elements corresponding to the dimension include: Qs(1, 1, z), Qs(1, 2, z), Qs(1, 3, z), Qs(1, 4, z), Qs(2, 1, z), Qs(2, 2, z), Qs(2, 3, z), Qs(2, 4, z), Qs(3, 1, z), Qs(3, 2, z), Qs(3, 3, z), Qs(3, 4, z), Qs(4, 1, z), Qs(4, 2, z), Qs(4, 3, z), Qs(4, 4, z). Based on this, the second quantization elements of each dimension corresponding to the pixel point in the ith row and jth column are spliced first to obtain the splicing part Qs(i, j, 1) Qs(i, j, 2) … Qs(i, j, k) of the pixel point. Then, the splicing parts of each pixel point are spliced in order to obtain the second feature part C2, C2 = Qs(1, 1, 1) Qs(1, 1, 2) … Qs(4, 4, k). Assuming that the second quantization element is a bit string of several bits, for example, the second quantization element is a two-bit bit string, then the second feature part is also a multi-bit bit string, for example, the second feature part is a 64-bit bit string.

[0140] After obtaining the first feature part and the second feature part, the first feature part and the second feature part are spliced to obtain a second feature block corresponding to the image block. For example, the first feature part C1 and the second feature part C2 are spliced to obtain a second feature block C, C=C1C2, and the second feature block is a bit string of multiple bits, such as a bit string of 128 bits, used to represent a 4x4xk texture image block. The second feature blocks corresponding to the image blocks are spliced to obtain a second image feature.

[0141] In step 204, a target texture image is reconstructed based on the second image feature, and the target texture image is used to carry texture content.

[0142] In the embodiments of the present application, the target texture image can be reconstructed based on the second image feature by using a reconstruction network, and the reconstruction network is trained by a third network. Optionally, the first network and the third network are spliced. In the process of training the first network in the manner of steps A1 to A4, the third network can be trained synchronously. That is, the first network and the third network are trained multiple times in the manner of steps A1 to A4 until a first training end condition is met. At this time, the trained first network is the feature extraction network mentioned above, and the trained third network is the reconstruction network.

[0143] Since the reconstruction network is trained based on the third network, the second image feature can be input into the reconstruction network according to the content shown in step A3, and the reconstruction network is used for deconvolution processing, activation processing, and linear mapping to reconstruct the target texture image. Since the target texture image is reconstructed from the second image feature, the content of the target texture image includes the texture content represented by the second image feature. Optionally, the second image feature includes the second feature blocks corresponding to the image blocks, and the second feature block corresponding to any image block is reconstructed into a new image block by using the reconstruction network according to the content shown in step A3. The reconstructed image blocks are spliced into the target texture image.

[0144] In a possible implementation, step 204 includes steps 2041 to 2042 (not shown in the figure).

[0145] In step 2041, a third image feature is determined based on the second image feature, and the third image feature is used to represent the target content obtained after the decompressed texture content.

[0146] Since the second image feature is obtained by compressing the first image feature, some information is lost in the compression process. Based on this, the second image feature can be decompressed into the third image feature, which realizes the restoration of the image content of the first texture image as much as possible on the basis of the lost information, so that the third image feature can represent more abundant information, so that the texture image reconstructed based on the third image feature has high-quality texture details.

[0147] In an example embodiment, step 2041 includes steps B1 to B3 (not shown in the figure).

[0148] Step B1, determining a plurality of second reference elements and a plurality of reference elements corresponding to a plurality of dimensions respectively based on the second image feature.

[0149] As known from the above, the second image feature includes a plurality of second feature blocks corresponding to a plurality of image blocks, and any second feature block corresponding to an image block includes a first feature part and a second feature part. In one possible implementation, the first feature part includes a plurality of first reference elements corresponding to a plurality of dimensions respectively, and any first reference element is a second reference element in the embodiment. In another possible implementation, the first feature part includes a plurality of first quantization elements corresponding to a plurality of dimensions respectively, and a second reference element in the embodiment is determined based on any first quantization element. Similarly, the second feature part includes a plurality of first target elements corresponding to a plurality of dimensions respectively, and a reference element in the embodiment is determined based on any first target element. Alternatively, the second feature part includes a plurality of second quantization elements corresponding to a plurality of dimensions respectively, and any second quantization element is a reference element in the embodiment.

[0150] For example, the second image feature C=C1C2. C1 is the first feature part, corresponding to k dimensions, the first quantization element Qmax(z) corresponding to the maximum value and the first quantization element Qmin(z) corresponding to the minimum value of the zth dimension, i.e. C1=Qmax(1)Qmin(1)Qmax(2)Qmin(2)……Qmax(k)Qmin(k). A second reference element can be determined based on any first quantization element. C2 is the second feature part, corresponding to k dimensions, C2=Qs(1,1,1)Qs(1,1,2)……Qs(4,4,k), and each second quantization element corresponding to the zth dimension can be determined as each reference element. Wherein, k is a positive integer, and z takes any integer from 1 to k.

[0151] Optionally, the second image feature includes a plurality of first quantized elements each corresponding to a respective dimension; and the determining, in step B1, of the second reference element corresponding to each respective dimension based on the second image feature includes, for each respective dimension, dequantizing the first quantized element corresponding to the respective dimension based on the first reference element corresponding to the respective dimension to obtain the second reference element corresponding to the respective dimension.

[0152] As mentioned above, since the first quantized element is obtained by quantizing the first reference element, the first reference element can be dequantized to obtain the second reference element. Through dequantization, data is recovered as much as possible, and loss of texture details caused by quantization is reduced. There are various ways of dequantization, and one possible way of dequantization is shown below.

[0153] For each respective dimension, two first reference elements are obtained, which are the maximum value and the minimum value, and the maximum value and the minimum value are quantized to obtain two first quantized elements. In this case, the two first quantized elements can be dequantized based on the minimum value of the two first reference elements to obtain two second reference elements.

[0154] Suppose the maximum value is Fmax(z), the first quantized element Qmax(z) is obtained by quantizing the maximum value, the minimum value is Fmin(z), and the first quantized element Qmin(z) is obtained by quantizing the minimum value. Then, according to Fmax1(z) = Qmax(z) / 15×2+Fmin(z), the first quantized element Qmax(z) is dequantized to obtain the second reference element Fmax1(z). Similarly, according to Fmin1(z) = Qmin(z) / 15×2+Fmin(z), the first quantized element Qmin(z) is dequantized to obtain the second reference element Fmin1(z). Wherein, z = 1, 2, …, k, and k is the number of dimensions.

[0155] In step B2, for each respective dimension, a plurality of second candidate elements are determined based on the second reference element, and a second target element corresponding to each respective reference element is selected from the plurality of second candidate elements.

[0156] The electronic device can calculate the plurality of second candidate elements based on each respective second reference element corresponding to each respective dimension, and the calculation manner is not limited herein. For example, the electronic device can calculate a statistical value such as a mean value or a sum value of each respective second reference element, and calculate the plurality of second candidate elements by weighted calculation of the statistical value with different weights. For another example, when the second reference element is two, the plurality of second candidate elements can be calculated based on the two second reference elements in an interpolation manner. That is, a difference value between the two second reference elements is calculated, and the plurality of weighted calculation results are obtained by weighted calculation of the difference value with different weights. Each respective second candidate element is obtained by adding or subtracting each respective weighted calculation result to or from one second reference element.

[0157] For example, the second candidate element corresponding to the interpolation level i is calculated according to Fi1(x, y, z) = (Fmax1(z) - Fmin1(z)) / 3 × i + Fmin1(z), where i = 0, 1, 2, 3, Fmax1(z) and Fmin1(z) are two second reference elements. As can be seen from the above formula, when i = 0, Fi1(x, y, z) = Fmin1(z); when i = 3, Fi1(x, y, z) = Fmax1(z); and when i = 1 or 2, Fi1(x, y, z) is the data at the three-equal position in the numerical interval between Fmin1(z) and Fmax1(z).

[0158] After the plurality of second candidate elements are calculated, for any reference element, a second target element corresponding to the reference element is selected from the second candidate elements. The manner of selecting the second target element is not limited herein, for example, a second candidate element can be randomly selected from the second candidate elements as the second target element corresponding to the reference element, or a second candidate element with the interpolation level i equal to the reference element can be selected from the second candidate elements, and the selected second candidate element is taken as the second target element corresponding to the reference element. For example, the second target element Fs1(x, y, z) is determined according to Fs1(x, y, z) = Fi1(x, y, z), i = Qs(x, y, z), where Fi1(x, y, z) is the second candidate element with the interpolation level i equal to the reference element Qs(x, y, z).

[0159] Similar to the first candidate element, by determining the plurality of second candidate elements, high-frequency information of different levels is focused, and the details of the texture are reflected through the high-frequency information. Subsequently, when the third image feature is determined based on the plurality of second candidate elements, the third image feature can reflect more texture details.

[0160] In step B3, the third image feature is determined based on the plurality of second target elements corresponding to the plurality of dimensions respectively.

[0161] In the embodiments of the present application, for any image block, the plurality of second target elements corresponding to the plurality of dimensions respectively can be spliced to obtain a third feature block corresponding to the image block. Optionally, the third feature block corresponding to the image block has the same size as the first feature block. For example, the size of the first feature block is 4 × 4 × k, and the size of the third feature block is also 4 × 4 × k. The third feature block is a high-dimensional feature of the image block, which can represent more quality texture information and provide more abundant texture details. Since there are a plurality of image blocks, the third feature blocks corresponding to the plurality of image blocks can be spliced to obtain the third image feature.

[0162] Step 2042: Based on the third image features, the target texture image is reconstructed.

[0163] In this embodiment, the third image features can be input into the reconstruction network as shown in step A3. The reconstruction network performs deconvolution, activation, and linear mapping processes to reconstruct the target texture image. Optionally, the third image features include third feature blocks corresponding to each image block. As shown in step A3, the reconstruction network reconstructs the third feature block corresponding to any image block into a new image block. The reconstructed image blocks are then stitched together to form the target texture image.

[0164] In one possible implementation, such as Figure 3 As shown in (1) and (2) in the diagram, the electronic device can acquire a first texture image and divide the first texture image into 4×4 image blocks. The division method can be found in the description of step 2021, and will not be repeated here. Next, the features of the image blocks are extracted by the feature extraction network F to obtain a 4×4×k first feature block. Multiple 4×4×k first feature blocks can be stitched together to form a first image feature. The determination method of the first image feature can be found in the descriptions of steps 2022 and 2023, and will not be repeated here. Then, the first feature block is compressed to obtain a second feature block C = C1C2. The compression method can be found in the description of step 203, and will not be repeated here. Multiple second feature blocks can be stitched together to form a second image feature. Next, the second feature block is decompressed to obtain a third feature block Fs(x, y, z). The size of the third feature block is 4×4×k. The decompression method can be found in the description of step 2041, and will not be repeated here. Multiple third feature blocks can be stitched together to form a third image feature. Then, the third feature block is processed by the reconstructed network G to obtain a new image block with a size of 4×4. The new 4×4 image blocks are then stitched together to obtain the target texture image G(Fs(x, y, z)). The process of determining the target texture image can be found in the description of step 2042. The implementation principles of the two are similar and will not be repeated here.

[0165] In an exemplary embodiment, step 2042 includes steps C1 to C3 (not shown in the figures).

[0166] Step C1: Based on the features of the first texture image and the third image, determine the target adjustment factor, which is used to characterize the difference between the image content and the target content.

[0167] In the embodiments of the present application, the electronic device can obtain the embedding feature of the first texture image or the first image feature. The obtaining manner of the embedding feature and the first image feature is described above, and will not be repeated here. Since the embedding feature and the first image feature can represent the original image content, and the third image feature can represent the recovered image content, by adjusting the network, comparing the embedding feature of the first texture image and the third image feature, or comparing the first image feature of the first texture image and the third image feature, the target adjustment factor can be obtained, which represents the difference between the original image content and the recovered image content.

[0168] The embodiments of the present application do not limit the structure, size, etc. of the adjustment network. For example, the adjustment network includes an input layer and an output layer, and in addition, at least one of a linear layer, a convolutional layer, a normalization layer, an activation layer, etc. The structure of the adjustment network is different, and the way of determining the target adjustment factor through the adjustment network also has differences. In the following, taking the adjustment network as a convolutional network as an example, the determination process of the target adjustment factor is described.

[0169] In the present example, the adjustment network includes an input layer, at least one convolutional layer, and an output layer. The input layer is a linear layer, and the input of the input layer is any image block in the first texture image and the third feature block corresponding to the image block in the third image feature. The image block includes information of a plurality of pixel points, and the third feature block includes feature values of each pixel point. The image block and the third feature block can be spliced to obtain a splicing block, and the splicing block includes splicing information of each pixel point. The splicing information of any pixel point includes the information of the pixel point and the feature value of the pixel point. The splicing block is mapped to a second vector of each pixel point through the linear layer, and the output of the input layer is the second vector of each pixel point. The first convolutional layer in the at least one convolutional layer is used to perform convolution processing on the output of the input layer to obtain the output result of the first convolutional layer; and the non-first convolutional layer in the at least one convolutional layer is used to perform convolution processing on the output result of the last convolutional layer to obtain the output result of the non-first convolutional layer. Optionally, any convolutional layer includes an activation function, and any convolutional layer is used to perform convolution processing and activation processing on the output of the input layer or the output result of the last convolutional layer in sequence. The output layer is a linear layer, and the output layer is used to perform linear mapping on the output result of the last convolutional layer to obtain the target adjustment factor.

[0170] Optionally, the adjustment network includes an input layer, three convolutional layers (convolutional layers 1 to 3), and an output layer, and the contents of each layer of the adjustment network are shown in Table 2.

[0171] Table 2

[0172] Layers of the conditioning network Input size Output size Activation function Input layer 4×4×(3+k) 4×4×(3+k) None Convolutional layer 1 4×4×(3+k) 4×4×64 ReLU Convolutional layer 2 4×4×64 4×4×128 ReLU Convolutional layer 3 4×4×128 4×4×k Tanh Output layer 4×4×k 4×4×k None

[0173] The target adjustment factor is determined by the adjustment network shown in Table 2. Assuming that the first texture image is cut into multiple image blocks, the input of the adjustment network includes the image block I(x, y) and the third feature block Fs(x, y, z), the size of the image block is 4x4x3, 3 is the number of color channels, and the size of the third feature block is 4x4xk, k is the dimension of the third feature block. The image block I(x, y) and the third feature block Fs(x, y, z) are spliced to obtain a spliced block [I(x, y), Fs(x, y, z)], and the size of the spliced block is 4x4x(3+k), (3+k) is the dimension of the spliced block. The target adjustment factor A(x, y, z) is output by the adjustment network, and the size of the target adjustment factor A(x, y, z) is 4x4xk, which is information used to adjust the corresponding third feature block of the 4x4 image block.

[0174] It can be understood that before the target adjustment factor is determined by the adjustment network, the adjustment network needs to be trained. In the embodiment of the present application, the adjustment network is trained in the following steps C4 to C7 to determine the target adjustment factor through the adjustment network. That is, the method of the embodiment of the present application further includes steps C4 to C7 (not shown in the figure), wherein steps C4 to C7 are executed before step C1.

[0175] In step C4, a reconstructed image feature is determined based on the sample image feature, and the reconstructed image feature is used to represent the reconstruction information obtained after the image content of the sample texture image is compressed and decompressed.

[0176] In the embodiment of the present application, the sample image feature can be compressed according to the implementation principle shown in step 203, and the reconstructed image feature can be determined based on the compressed sample image feature according to the implementation principle shown in step 2041. The implementation principle of step C4 can be found in the description of steps 203 and 2041, and the implementation principles of the two are similar, which will not be repeated here.

[0177] In step C5, a predicted adjustment factor is determined by the second network based on the sample texture image and the reconstructed image feature, and the predicted adjustment factor is used to represent the difference between the image content of the sample texture image and the reconstruction information.

[0178] In the embodiment of the present application, the structure, size, etc. of the second network are similar to those of the adjustment network, and the only difference between the two is the network parameters. Based on this, the predicted adjustment factor can be determined by the second network based on the sample texture image and the reconstructed image feature according to the implementation principle shown in step C1, and the implementation principles of the two are similar, which will not be repeated here.

[0179] In step C6, a second predicted image is reconstructed based on the predicted adjustment factor and the reconstructed image feature.

[0180] In the embodiments of the present application, the predicted adjustment factor and the reconstructed image feature can be weighted and calculated to adjust the reconstructed image feature by the predicted adjustment factor to obtain the adjusted reconstructed image feature. The adjusted reconstructed image feature is input into the reconstruction network, and the adjusted reconstructed image feature is input into the reconstruction network according to the content shown in step A3, and at least one processing such as deconvolution processing, activation processing and linear mapping is performed on the adjusted reconstructed image feature by the reconstruction network to reconstruct the second predicted image. Optionally, the adjusted reconstructed image feature includes reconstructed feature blocks corresponding to each image block, and any reconstructed feature block corresponding to an image block is reconstructed into a new image block by the reconstruction network according to the content shown in step A3. The reconstructed image blocks are spliced into the second predicted image.

[0181] In step C7, the second network is trained based on the sample texture image and the second predicted image to obtain an adjustment network.

[0182] In the embodiments of the present application, the loss of the second network can be determined based on the sample texture image and the second predicted image according to a loss function. The type of the loss function is not limited herein, and exemplarily, the loss function is any one of mean square error, root mean square error, etc. The second network is trained once by the loss of the second network to obtain a trained second network. If the trained second network meets a second training end condition, the trained second network is taken as the adjustment network; if the trained second network does not meet the second training end condition, the trained second network is taken as the second network for the next training, and the second network is trained next time according to the manner of step C4 to step C7 until the trained second network meets the second training end condition, thereby obtaining the adjustment network.

[0183] The embodiments of the present application do not limit the content that the trained second network meets the second training end condition, and exemplarily, the trained second network meets the second training end condition, including that the training times of the trained second network reach a set number of times, or the accuracy of the trained second network reaches a set accuracy, etc. Optionally, the set number of times or the set accuracy are values pre-set according to artificial experience.

[0184] Optionally, the training target of the second network is: min||I(x, y)-G(Fa(x, y, z))||. 2 . wherein ||I(x, y)-G(Fa(x, y, z))||. 2 represents the loss of the second network. I(x, y) represents the sample texture image. Fa(x, y, z) represents the adjusted reconstructed image feature. G(Fa(x, y, z)) represents the second predicted image.

[0185] In the process of training the second network, training can be performed based on a stochastic gradient descent algorithm. In each training process of the second network, the gradient of the loss of the second network with respect to the network parameters of the second network can be calculated, the network parameters of the second network are updated based on the gradient and the learning rate, and the second network is trained once. Wherein, the learning rate, the number of training times and the like can be flexibly adjusted according to actual conditions, for example, the learning rate is 0.01, the iteration number is 1000, etc.

[0186] Step C2, adjusting the third image feature based on the target adjustment factor to obtain a target image feature.

[0187] In the embodiments of the present application, the target adjustment factor and the third image feature can be weighted and calculated to achieve the adjustment of the third image feature by the target adjustment factor to obtain the target image feature. Since the target adjustment factor is used to represent the difference between the image content of the first texture image and the target content represented by the third image feature, the third image feature is adjusted by the target adjustment factor, so that the image content represented by the target image feature is more close to the image content of the first texture image, and the loss of image content caused by compression is reduced, so that the subsequent high-quality texture image can be reconstructed based on the target image feature.

[0188] Suppose the target adjustment factor is A(x, y, z), and the third image feature is Fs(x, y, z). According to Fa(x, y, z) = Fs(x, y, z) + A(x, y, z), the target image feature Fa(x, y, z) is determined. Optionally, A(x, y, z) and Fs(x, y, z) are both 4x4xk tensors, and based on this, Fa(x, y, z) is also a 4x4xk tensor, which is a high-dimensional feature of a 4x4 image block.

[0189] Step C3, reconstructing a target texture image based on the target image feature.

[0190] In the embodiments of the present application, the target image feature can be input into the reconstruction network, and the target image feature is input into the reconstruction network according to the content shown in step A3, and at least one of convolution processing, activation processing and linear mapping is performed on the target image feature by the reconstruction network to reconstruct the target texture image. Optionally, the target image feature includes target feature blocks corresponding to each image block, and any image block corresponding to the target feature block is reconstructed into a new image block by the reconstruction network according to the content shown in step A3. The reconstructed image blocks are spliced into a target texture image.

[0191] Optionally, the target feature block corresponding to any image block is Fa(x, y, z), and Fa(x, y, z) is sent to a reconstruction network G. The reconstruction network is a deconvolutional neural network, and based on this, the reconstruction network outputs a new image block Ia(x, y) = G(Fa(x, y, z)) = CNN -1 (Fa(x, y, z)). Ia(x, y) is a 4x4 matrix, which is an image block after adjusting color, texture and the like. CNN -1 The processing of the deconvolutional network is represented.

[0192] In the example embodiment, the method of the application further includes step 205, which is performed after step 204.

[0193] In step 205, the resources occupied in the process of loading the first texture image and the resources occupied in the process of loading the target texture image are determined, and the texture image with the least occupied resources is selected from the first texture image and the target texture image.

[0194] In the application embodiment, the first texture image can be loaded in the electronic device, and the resources occupied in the loading process are determined, which can be denoted as first resources. The first resources include but are not limited to CPU occupied resources, GPU occupied resources, memory occupied resources, network occupied resources, etc. In addition, the target texture image can be loaded in the electronic device, and the resources occupied in the loading process are determined, which can be denoted as second resources. The second resources include but are not limited to CPU occupied resources, GPU occupied resources, memory occupied resources, network occupied resources, etc.

[0195] By comparing the first resources and the second resources, the least occupied resources can be determined, and thus the texture image with the least occupied resources can be determined. That is, if the first resources are less than the second resources, the first resources are the least occupied resources, and the first texture image is the texture image with the least occupied resources; if the second resources are less than the first resources, the second resources are the least occupied resources, and the target texture image is the texture image with the least occupied resources.

[0196] Alternatively, the first resources include the occupation amounts of various resources, and the occupation amounts of various resources are weighted and summed to obtain a first calculation result. Similarly, the second resources include the occupation amounts of various resources, and the occupation amounts of various resources are weighted and summed to obtain a second calculation result. The resources corresponding to the minimum value of the first calculation result and the second calculation result are determined to obtain the least occupied resources, and thus the texture image with the least occupied resources is determined.

[0197] Subsequently, when the electronic device needs to load a texture resource, the texture image with the least occupied resources can be directly loaded, thereby improving the rendering efficiency when rendering the texture image. For example, in the field of games, any frame of game picture includes at least one texture image. According to the method of the embodiment of the present application, any texture image is processed to obtain a new texture image, and the texture image with the least occupied resources is determined from the original texture image and the new texture image. Subsequently, when the terminal device needs to render the game picture, the texture image with the least occupied resources is directly used, thereby improving the rendering efficiency and the performance of running the game.

[0198] As shown in Figure 4 , a single frame of game picture corresponds to a first texture image. In this example, the terminal device includes a client that can run an application texture image, for example, the client includes a debug tool (a program for obtaining the first texture image). Optionally, the first texture image is loaded by the debug tool, and the first resources occupied in the loading process are recorded. Then, the first texture image is reconstructed according to the method of the embodiment of the present application to obtain a target texture image, the target texture image is loaded, and the second resources occupied in the loading process are recorded. By comparing the first resources and the second resources, it is determined whether to use the first texture image to render the single frame of game picture, and whether to use the target texture image to render the single frame of game picture.

[0199] As shown in Figure 5 , Figure 5 is a display interface when loading multiple texture images. Among them, the texture images in the loading stage are shown by the label 401, the label 402 and the label 405, and the texture images that have been loaded are shown by the label 403, the label 404, the label 406 and the label 407. Figure 5 It can be seen that the texture images shown by the label 401, the label 402 and the label 405 are texture images with poor loading performance, and the texture images shown by the label 403, the label 404, the label 406 and the label 407 are texture images with good loading performance. In this example, at least one texture image with poor loading performance and / or at least one texture image with good loading performance can be reconstructed according to the method of the embodiment of the present application. It can be understood that the display interface also includes other information, for example, the information shown by the label 408 and the label 409.

[0200] It should be noted that the information (including but not limited to user device information, user personal information, etc.), data (including but not limited to data for analysis, stored data, displayed data, etc.) and signals involved in the present application are all authorized by the user or fully authorized by all parties, and the collection, use and processing of related data need to comply with relevant laws, regulations and standards in the relevant region. For example, the first texture image, sample texture image and the like involved in the present application are obtained under full authorization.

[0201] In the above method, the first image feature of the first texture image is compressed to obtain the second image feature, which realizes compression of image content at the feature level, retains texture content, and removes redundant content irrelevant to the texture. After reconstructing the target texture image based on the second image feature, on the one hand, since the texture content is retained, it can be ensured that the target texture image has high texture quality, and on the other hand, since the redundant content is removed, the electronic device has higher rendering efficiency when rendering the target texture image.

[0202] The above describes the method of processing the texture image provided by the embodiments of the present application from the perspective of method steps. The method of the embodiments of the present application can be applied to the fields of games, animations, virtual reality, augmented reality, extended reality, mixed reality, etc. Taking the game field as an example, generally, the terminal device needs to use the texture image corresponding to the game picture in real time to render the game picture in the process of running the game. If the texture image is not used reasonably, it will seriously affect the smoothness of the game running. Based on this, the texture image corresponding to each frame of game picture can be processed according to the method flow as shown in Figure 6 When rendering any frame of game picture, the texture image with better loading performance is selected to render the game picture according to the loading performance of the texture image before processing and the loading performance of the texture image after processing, so as to improve the smoothness of the game running. As shown in Figure 6 The method of the embodiments of the present application includes the steps as shown below.

[0203] Step 601: acquiring a first texture image corresponding to any frame of game picture. The implementation principle of step 601 is similar to that of step 201, which can be seen from the description of step 201, and will not be repeated here.

[0204] Step 602: dividing the first texture image into a plurality of image blocks, extracting features of each image block to obtain a plurality of first feature blocks. The implementation principle of step 602 is similar to that of step 202, which can be seen from the description of step 202, and will not be repeated here.

[0205] Step 603, for any dimension in any first feature block, determine a first reference element and quantize the first reference element to obtain a first quantized element, determine a plurality of first candidate elements based on the first quantized element, select a first target element similar to each element of the dimension from each first candidate element, and quantize the first target element to obtain a second quantized element. The implementation principle of step 603 is similar to that of step 203, and can be seen from the description of step 203, which will not be repeated here.

[0206] Step 604, for any first feature block, splice the first quantized elements of each dimension into a first feature part, splice the second quantized elements of each dimension into a second feature part, and splice the first feature part and the second feature part to obtain a second feature block. The implementation principle of step 604 is similar to that of step 203, and can be seen from the description of step 203, which will not be repeated here.

[0207] Step 605, for any dimension in any second feature block, dequantize the first quantized element to obtain a second reference element, determine a plurality of second candidate elements based on the second reference element, and select a second target element corresponding to each second quantized element from each second candidate element.

[0208] In step 605, the second quantized element corresponds to the reference element mentioned above. Based on this, the implementation principle of step 605 is similar to that of steps B1 and B2, and can be seen from the description of steps B1 and B2, which will not be repeated here.

[0209] Step 606, for any second feature block, splice the second target elements of each dimension into a third feature block. The implementation principle of step 606 is similar to that of step B3, and can be seen from the description of step B3, which will not be repeated here.

[0210] Step 607, for any third feature block, determine a target adjustment factor based on the image block and the third feature block, and add the third feature block and the target adjustment factor to obtain a target feature block. The implementation principle of step 607 is similar to that of steps C1 and C2, and can be seen from the description of steps C1 and C2, which will not be repeated here.

[0211] Step 608, determine a new image block based on the target feature block through a reconstruction network, and splice each image block to obtain a target texture image. The implementation principle of step 608 is similar to that of step C3, and can be seen from the description of step C3, which will not be repeated here.

[0212] Step 609: Record the resources occupied when loading the target texture image and the resources occupied when loading the first texture image, and select the texture image with the fewest resources to render the game frame. The implementation principle of step 609 is similar to that of step 205, as described in the description of step 205, and will not be repeated here.

[0213] pass Figure 6 The scheme shown can reconstruct a new texture image from the original texture image. By rendering the game screen using a texture image with better loading performance, it replaces a poorly loading texture image with a better one, thereby improving the rendering efficiency and overall game performance. Figure 7 As shown, the texture image labeled 701 is the unreconstructed texture image, which has poor loading performance, while the texture image labeled 702 is the reconstructed texture image, which has better loading performance. Furthermore, comparing the unreconstructed and reconstructed texture images reveals that the reconstructed texture image is clearer, has higher quality, and more detail.

[0214] Figure 8 The diagram shown is a structural schematic of a texture image processing apparatus provided in an embodiment of this application. Figure 8 As shown, the device includes:

[0215] The acquisition module 801 is used to acquire the first texture image;

[0216] Extraction module 802 is used to extract first image features from the first texture image, the first image features being used to characterize the image content of the first texture image;

[0217] The compression module 803 is used to compress the first image features to obtain the second image features, which are used to characterize the texture content obtained by compressing the image content.

[0218] The reconstruction module 804 is used to reconstruct the target texture image based on the second image features.

[0219] In one possible implementation, the extraction module 802 is used to segment the first texture image to obtain multiple image blocks; for any image block, the image features of any image block are extracted from the image block through a feature extraction network; and the first image features are determined based on the image features of the multiple image blocks.

[0220] In one possible implementation, the acquisition module 801 is also used to acquire a sample texture image;

[0221] The extraction module 802 is also used to extract sample image features from the sample texture image through the first network;

[0222] The reconstruction module 804 is further configured to reconstruct the first predicted image based on the sample image feature.

[0223] The apparatus further includes:

[0224] The training module is configured to train the first network based on the sample texture image and the first predicted image to obtain the feature extraction network.

[0225] In a possible implementation, the first image feature includes a plurality of first sub-features in a plurality of dimensions, and each first sub-feature in any dimension includes a plurality of first initial elements.

[0226] The compression module 803 is configured to determine, for each first sub-feature in any dimension, a first reference element from the plurality of first initial elements, the first reference element including a first initial element greater than a first threshold value and a first initial element less than a second threshold value, the first threshold value being greater than the second threshold value; and determine the second image feature based on the first reference element corresponding to each first sub-feature in the plurality of dimensions.

[0227] In a possible implementation, the compression module 803 is configured to, for each first sub-feature in any dimension, interpolate a plurality of first candidate elements based on the first reference element, and select a first target element corresponding to each first initial element from the plurality of first candidate elements; and determine the second image feature based on the first reference element corresponding to each first sub-feature in the plurality of dimensions and each first target element.

[0228] In a possible implementation, the compression module 803 is configured to quantize the first reference element to obtain a first quantized element; and interpolate a plurality of first candidate elements based on the first quantized element.

[0229] In a possible implementation, the compression module 803 is configured to, for each first sub-feature in any dimension, quantize each first target element based on the first quantized element corresponding to the first reference element to obtain each second quantized element; and determine the second image feature based on the first quantized element corresponding to each first sub-feature in the plurality of dimensions and each second quantized element.

[0230] In a possible implementation, the reconstruction module 804 is configured to determine a third image feature based on the second image feature, the third image feature being used to represent a target content obtained after decompression of the texture content; and reconstruct a target texture image based on the third image feature.

[0231] In a possible implementation, the reconstruction module 804 is configured to determine, based on the second image feature, a plurality of second reference elements and a plurality of reference elements corresponding to a plurality of dimensions respectively; for each dimension, determine a plurality of second candidate elements based on the second reference elements corresponding to the dimension, and select a second target element corresponding to each reference element from the plurality of second candidate elements; and determine a third image feature based on the second target elements corresponding to the plurality of dimensions respectively.

[0232] In a possible implementation, the second image feature includes a plurality of first quantized elements corresponding to a plurality of dimensions respectively.

[0233] The reconstruction module 804 is configured to, for each dimension, dequantize the first quantized element corresponding to the dimension based on the first reference element corresponding to the dimension to obtain a second reference element corresponding to the dimension.

[0234] In a possible implementation, the reconstruction module 804 is configured to determine, based on the first texture image and the third image feature, a target adjustment factor, where the target adjustment factor is used to represent a difference between the image content and the target content; adjust the third image feature based on the target adjustment factor to obtain a target image feature; and reconstruct a target texture image based on the target image feature.

[0235] In a possible implementation, the target adjustment factor is determined by an adjustment network; and the apparatus further includes:

[0236] The determination module is configured to determine, based on the sample image feature, a reconstruction image feature, where the reconstruction image feature is used to represent reconstruction information obtained after the image content of the sample texture image is compressed and decompressed;

[0237] The determination module is further configured to determine, based on the sample texture image and the reconstruction image feature by a second network, a prediction adjustment factor, where the prediction adjustment factor is used to represent a difference between the image content of the sample texture image and the reconstruction information;

[0238] The reconstruction module 804 is further configured to reconstruct a second prediction image based on the prediction adjustment factor and the reconstruction image feature.

[0239] The training module is configured to train the second network based on the sample texture image and the second prediction image to obtain the adjustment network.

[0240] In a possible implementation, the apparatus further includes:

[0241] The determination module is configured to determine resources occupied in a process of loading the first texture image and resources occupied in a process of loading the target texture image.

[0242] The selection module is configured to select, from the first texture image and the target texture image, a texture image with the least occupied resources.

[0243] In the device, the first image features of the first texture image are compressed to obtain the second image features, so that the image content is compressed at the feature level, the texture content is retained, and the redundant content irrelevant to the texture is removed. After the target texture image is reconstructed based on the second image features, on the one hand, since the texture content is retained, the target texture image can have high texture quality, and on the other hand, since the redundant content is removed, the electronic device has higher rendering efficiency when rendering the target texture image.

[0244] It should be understood that the above Figure 8 The device provided in the embodiments only takes the above division of functional modules as an example to implement the functions thereof, and in actual applications, the above functions can be completed by different functional modules according to requirements, that is, the internal structure of the device is divided into different functional modules to complete all or part of the above described functions. In addition, the device and method embodiments provided in the above embodiments belong to the same concept, and the specific implementation process is detailed in the method embodiments, which will not be described here.

[0245] Figure 9 A structure block diagram of a terminal device 900 provided in an example embodiment of the present application is shown. The terminal device 900 includes a processor 901 and a memory 902.

[0246] The processor 901 can include one or more processing cores, such as a 4-core processor, an 8-core processor, etc. The processor 901 can be implemented in at least one of a hardware form of a DSP (Digital Signal Processing), an FPGA (Field-Programmable Gate Array), and a PLA (Programmable Logic Array). The processor 901 can also include a main processor and a coprocessor. The main processor is a processor for processing data in an awake state, also known as a CPU (Central Processing Unit). The coprocessor is a low-power processor for processing data in a standby state. In some embodiments, the processor 901 can be integrated with a GPU (Graphics Processing Unit) that is responsible for rendering and drawing the content to be displayed by the display screen. In some embodiments, the processor 901 can also include an AI (Artificial Intelligence) processor for processing machine learning-related computing operations.

[0247] The memory 902 can include one or more computer-readable storage media. The computer-readable storage media can be non-transitory. The memory 902 can also include high-speed random access memory and can include non-volatile memory, such as one or more magnetic disk storage devices, optical storage devices, flash memory devices, or other non-volatile solid-state storage devices. In some embodiments, the non-transitory computer-readable storage medium of the memory 902 is used to store at least one computer program for being executed by the processor 901 to implement the processing method of the texture image provided by the method embodiments of the present application.

[0248] In some embodiments, the terminal device 900 can also optionally include a peripheral device interface 903 and at least one peripheral device. The processor 901, the memory 902, and the peripheral device interface 903 can be connected through a bus or a signal line. Each peripheral device can be connected to the peripheral device interface 903 through a bus, a signal line, or a circuit board. Specifically, the peripheral device includes at least one of a radio frequency circuit 904, a display screen 905, a camera component 906, an audio circuit 907, and a power supply 908.

[0249] The peripheral device interface 903 can be used to connect at least one peripheral device related to I / O (Input / Output) to the processor 901 and the memory 902. In some embodiments, the processor 901, the memory 902, and the peripheral device interface 903 are integrated on the same chip or circuit board; in some other embodiments, any one or two of the processor 901, the memory 902, and the peripheral device interface 903 can be implemented on a separate chip or circuit board, and the present embodiment does not limit this.

[0250] The radio frequency circuit 904 is used to receive and transmit RF (Radio Frequency) signals, also known as electromagnetic signals. The radio frequency circuit 904 communicates with a communication network and other communication devices through electromagnetic signals. The radio frequency circuit 904 converts electrical signals into electromagnetic signals for transmission, or converts received electromagnetic signals into electrical signals. Optionally, the radio frequency circuit 904 includes an antenna system, an RF transceiver, one or more amplifiers, a tuner, an oscillator, a digital signal processor, a codec chipset, a subscriber identity module card, and the like. The radio frequency circuit 904 can communicate with other terminals through at least one wireless communication protocol. The wireless communication protocol includes but is not limited to the World Wide Web, a metropolitan area network, an intranet, various generations of mobile communication networks (2G, 3G, 4G, and 5G), a wireless local area network, and / or a WiFi (Wireless Fidelity) network. In some embodiments, the radio frequency circuit 904 can also include NFC (Near Field Communication) related circuitry, and the present application does not limit this.

[0251] The display screen 905 is configured to display a UI (User Interface). The UI can include graphics, text, icons, video, and any combination thereof. When the display screen 905 is a touch display screen, the display screen 905 is further configured to capture touch signals on or above the surface of the display screen 905. The touch signals can be input to the processor 901 as control signals for processing. In this case, the display screen 905 can also be configured to provide virtual buttons and / or virtual keyboard, also known as soft buttons and / or soft keyboard. In some embodiments, the display screen 905 can be one, disposed on the front panel of the terminal device 900; in other embodiments, the display screen 905 can be at least two, respectively disposed on different surfaces of the terminal device 900 or in a folding design; in other embodiments, the display screen 905 can be a flexible display screen, disposed on a curved surface or a folding surface of the terminal device 900. Even, the display screen 905 can also be disposed in an irregular shape, i.e., a special-shaped screen. The display screen 905 can be made of LCD (Liquid Crystal Display), OLED (Organic Light-Emitting Diode), etc.

[0252] The camera assembly 906 is configured to capture images or videos. Optionally, the camera assembly 906 includes a front camera and a rear camera. Typically, the front camera is disposed on the front panel of the terminal, and the rear camera is disposed on the back of the terminal. In some embodiments, the rear camera is at least two, which are any one of a main camera, a depth-of-field camera, a wide-angle camera, and a telephoto camera, to realize the background blur function by fusing the main camera and the depth-of-field camera, the panoramic shooting and VR (Virtual Reality) shooting function by fusing the main camera and the wide-angle camera, or other fusion shooting functions. In some embodiments, the camera assembly 906 can further include a flash. The flash can be a single-color temperature flash or a dual-color temperature flash. The dual-color temperature flash refers to the combination of a warm light flash and a cold light flash, which can be used for light compensation under different color temperatures.

[0253] The audio circuit 907 can include a microphone and a speaker. The microphone is used to collect sound waves of the user and the environment, and convert the sound waves into an electrical signal input to the processor 901 for processing, or input to the radio frequency circuit 904 to realize voice communication. For the purpose of stereo sound collection or noise reduction, the microphone can be multiple, respectively arranged at different parts of the terminal device 900. The microphone can also be an array microphone or an omnidirectional collection type microphone. The speaker is used to convert the electrical signal from the processor 901 or the radio frequency circuit 904 into sound waves. The speaker can be a traditional diaphragm speaker, or a piezoelectric ceramic speaker. When the speaker is a piezoelectric ceramic speaker, not only can it convert electrical signals into sound waves that humans can hear, but it can also convert electrical signals into sound waves that humans cannot hear for ranging purposes. In some embodiments, the audio circuit 907 can also include a headphone jack.

[0254] The power supply 908 is used to supply power to each component in the terminal device 900. The power supply 908 can be alternating current, direct current, disposable battery or rechargeable battery. When the power supply 908 includes a rechargeable battery, the rechargeable battery can be a wired charging battery or a wireless charging battery. The wired charging battery is a battery charged through a wired line, and the wireless charging battery is a battery charged through a wireless coil. The rechargeable battery can also be used to support fast charging technology.

[0255] In some embodiments, the terminal device 900 further includes one or more sensors 909. The one or more sensors 909 include, but are not limited to, an acceleration sensor 911, a gyroscope sensor 912, a pressure sensor 913, an optical sensor 914, and a proximity sensor 915.

[0256] The acceleration sensor 911 can detect the acceleration in three coordinate axes of the coordinate system established by the terminal device 900. For example, the acceleration sensor 911 can be used to detect the components of the gravitational acceleration in three coordinate axes. The processor 901 can control the display screen 905 to display the user interface in a landscape view or a portrait view according to the gravitational acceleration signal collected by the acceleration sensor 911. The acceleration sensor 911 can also be used for game or user motion data collection.

[0257] The gyroscope sensor 912 can detect the body direction and rotation angle of the terminal device 900, and the gyroscope sensor 912 can collect 3D actions of the user on the terminal device 900 in cooperation with the acceleration sensor 911. The processor 901 can realize the following functions according to the data collected by the gyroscope sensor 912: motion sensing (such as changing the UI according to the user's tilt operation), image stabilization when shooting, game control, and inertial navigation.

[0258] The pressure sensor 913 can be disposed on the side bezel of the terminal device 900 and / or the lower layer of the display screen 905. When the pressure sensor 913 is disposed on the side bezel of the terminal device 900, it can detect the user's grip signal on the terminal device 900, and the processor 901 can perform left / right hand recognition or quick operation based on the grip signal collected by the pressure sensor 913. When the pressure sensor 913 is disposed on the lower layer of the display screen 905, the processor 901 can control the operable controls on the UI interface based on the user's pressure operation on the display screen 905. The operable controls include at least one of button controls, scroll bar controls, icon controls, and menu controls.

[0259] An optical sensor 914 is used to collect ambient light intensity. In one embodiment, the processor 901 can control the display brightness of the display screen 905 based on the ambient light intensity collected by the optical sensor 914. Specifically, when the ambient light intensity is high, the display brightness of the display screen 905 is increased; when the ambient light intensity is low, the display brightness of the display screen 905 is decreased. In another embodiment, the processor 901 can also dynamically adjust the shooting parameters of the camera assembly 906 based on the ambient light intensity collected by the optical sensor 914.

[0260] The proximity sensor 915, also known as a distance sensor, is typically located on the front panel of the terminal device 900. The proximity sensor 915 is used to detect the distance between the user and the front of the terminal device 900. In one embodiment, when the proximity sensor 915 detects that the distance between the user and the front of the terminal device 900 is gradually decreasing, the processor 901 controls the display screen 905 to switch from a screen-on state to a screen-off state; when the proximity sensor 915 detects that the distance between the user and the front of the terminal device 900 is gradually increasing, the processor 901 controls the display screen 905 to switch from a screen-off state to a screen-on state.

[0261] Those skilled in the art will understand that Figure 9 The structure shown does not constitute a limitation on the terminal device 900, and may include more or fewer components than shown, or combine certain components, or use different component arrangements.

[0262] Figure 10A server structure diagram provided by the embodiments of the present application is shown in FIG. 10. The server 1000 can have great differences due to different configurations or performances, and can include one or more processors 1001 and one or more memories 1002, wherein the one or more memories 1002 store at least one computer program, the at least one computer program is loaded and executed by the one or more processors 1001 to implement the texture image processing method provided by the above-mentioned various method embodiments. For example, the processor 1001 is a CPU. Of course, the server 1000 can also have a wired or wireless network interface, a keyboard, an input and output interface, and other components for realizing the functions of the device, and the details are not described herein.

[0263] In an exemplary embodiment, a computer readable storage medium is also provided, and the storage medium stores at least one computer program, the at least one computer program is loaded and executed by a processor to enable an electronic device to implement any of the above-mentioned texture image processing methods.

[0264] Optionally, the above-mentioned computer readable storage medium can be a read-only memory (ROM), a random access memory (RAM), a compact disc read-only memory (CD-ROM), a magnetic tape, a floppy disk, and an optical data storage device, etc.

[0265] In an exemplary embodiment, a computer program is also provided, and the computer program is at least one, the at least one computer program is loaded and executed by a processor to enable an electronic device to implement any of the above-mentioned texture image processing methods.

[0266] In an exemplary embodiment, a computer program product is also provided, and the computer program product stores at least one computer program, the at least one computer program is loaded and executed by a processor to enable an electronic device to implement any of the above-mentioned texture image processing methods.

[0267] It should be understood that "multiple" mentioned herein refers to two or more. The association relationship of "and / or" describing the associated objects means that there can be three relationships, for example, A and / or B can mean that there are three cases of A alone, A and B together, and B alone. The character " / " generally represents that the associated objects before and after are in an "or" relationship.

[0268] The above-mentioned serial numbers of the embodiments of the present application are only for description, and do not represent the advantages or disadvantages of the embodiments.

[0269] The above merely illustrates the embodiments of the present application, and is not intended to limit the present application. Any modification, equivalent replacement, improvement, etc. made within the principles of the present application shall be included in the protection scope of the present application.

Claims

1. A method for processing texture images, characterized in that, The method includes: Obtain the first texture image; A first image feature is extracted from the first texture image, and the first image feature is used to characterize the image content of the first texture image; The first image feature is compressed to obtain a second image feature, which is used to characterize the texture content obtained by compressing the image content. Based on the second image features, a target texture image is reconstructed, which is used to carry the texture content.

2. The method according to claim 1, characterized in that, The step of extracting the first image features from the first texture image includes: The first texture image is segmented to obtain multiple image blocks; For any given image patch, the image features of that image patch are extracted from the given image patch using a feature extraction network; Based on the image features of the multiple image blocks, a first image feature is determined.

3. The method according to claim 2, characterized in that, The method further includes training the feature extraction network, wherein training the feature extraction network includes: Obtain the sample texture image; Sample image features are extracted from the sample texture image using a first network; The first predicted image is reconstructed based on the features of the sample image; The first network is trained based on the sample texture image and the first predicted image to obtain the feature extraction network.

4. The method according to claim 1, characterized in that, The first image feature includes multiple dimensions of first sub-features, and any one dimension of the first sub-feature includes multiple first initial elements; compressing the first image feature to obtain the second image feature includes: For any one dimension of the first sub-feature, a first reference element is determined from the plurality of first initial elements. The first reference element includes a first initial element that is greater than a first threshold and a first initial element that is less than a second threshold. The first threshold is greater than the second threshold. The second image feature is determined based on the first reference element corresponding to the first sub-feature of each of the multiple dimensions.

5. The method according to claim 4, characterized in that, The determination of the second image features based on the first reference elements corresponding to the first sub-features of the multiple dimensions includes: For any first sub-feature of any dimension, multiple first candidate elements are obtained by interpolation based on the first reference element, and the first target element corresponding to each first initial element is selected from the multiple first candidate elements. The second image feature is determined based on the first reference element and each first target element corresponding to the first sub-feature of the multiple dimensions.

6. The method according to claim 5, characterized in that, The interpolation based on the first reference element yields multiple first candidate elements, including: Quantize the first reference element to obtain the first quantized element; Multiple first candidate elements are obtained based on the interpolation of the first quantized element.

7. The method according to claim 5, characterized in that, The determination of the second image features based on the first reference elements and each first target element corresponding to the first sub-features of the multiple dimensions includes: For any first sub-feature of any dimension, based on the first quantization element corresponding to the first reference element, each first target element is quantized to obtain each second quantization element; The second image feature is determined based on the first quantization element and each second quantization element corresponding to the first sub-feature of the multiple dimensions.

8. The method according to claim 1, characterized in that, The process of reconstructing the target texture image based on the second image features includes: A third image feature is determined based on the second image feature, and the third image feature is used to characterize the target content obtained after decompressing the texture content; Based on the third image feature, the target texture image is reconstructed.

9. The method according to claim 8, characterized in that, The step of determining the third image feature based on the second image feature includes: Based on the second image features, second reference elements and multiple baseline elements corresponding to each of the multiple dimensions are determined; For any dimension, multiple second candidate elements are determined based on the second reference element, and the second target element corresponding to each benchmark element is selected from the multiple second candidate elements; The third image features are determined based on the second target elements corresponding to each of the multiple dimensions.

10. The method according to claim 9, characterized in that, The second image feature includes first quantization elements corresponding to multiple dimensions; determining second reference elements corresponding to multiple dimensions based on the second image feature includes: For any dimension, based on the first reference element corresponding to the given dimension, the first quantization element corresponding to the given dimension is dequantized to obtain the second reference element corresponding to the given dimension.

11. The method according to claim 8, characterized in that, The process of reconstructing the target texture image based on the third image features includes: Based on the features of the first texture image and the third image, a target adjustment factor is determined, which is used to characterize the difference between the image content and the target content; The third image feature is adjusted based on the target adjustment factor to obtain the target image feature; Based on the features of the target image, the target texture image is reconstructed.

12. The method according to claim 11, characterized in that, The target regulation factor is determined through a regulation network; the method further includes: Based on the features of the sample image, the reconstructed image features are determined. These reconstructed image features are used to characterize the reconstructed information obtained after compressing and decompressing the image content of the sample texture image. A second network determines a prediction adjustment factor based on the features of the sample texture image and the reconstructed image. The prediction adjustment factor is used to characterize the difference between the image content of the sample texture image and the reconstructed information. Based on the prediction adjustment factor and the reconstructed image features, a second prediction image is reconstructed; Based on the sample texture image and the second predicted image, the second network is trained to obtain the adjustment network.

13. The method according to any one of claims 1 to 12, characterized in that, The method further includes: Determine the resources used during the loading of the first texture image and the resources used during the loading of the target texture image; Select the texture image that occupies the least resources from the first texture image and the target texture image.

14. A texture image processing apparatus, characterized in that, The device includes: The acquisition module is used to acquire the first texture image; The extraction module is used to extract a first image feature from the first texture image, wherein the first image feature is used to characterize the image content of the first texture image; A compression module is used to compress the first image feature to obtain a second image feature, wherein the second image feature is used to characterize the texture content obtained by compressing the image content; The reconstruction module is used to reconstruct a target texture image based on the second image features, and the target texture image is used to carry the texture content.

15. An electronic device, characterized in that, The electronic device includes a processor and a memory, the memory storing at least one computer program, which is loaded and executed by the processor to enable the electronic device to implement the texture image processing method as described in any one of claims 1 to 13.

16. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores at least one computer program, which is loaded and executed by a processor to enable an electronic device to implement the texture image processing method as described in any one of claims 1 to 13.

17. A computer program product, characterized in that, The computer program product stores at least one computer program, which is loaded and executed by a processor to enable the electronic device to implement the texture image processing method as described in any one of claims 1 to 13.

Images