Processing method and device for three-dimensional model and storage medium

Abstract

The invention discloses a processing method and device for a three-dimensional model and a storage medium, and the method comprises the steps: extracting the texture information of a to-be-processed three-dimensional model, so as to obtain a texture image; determining an object category of each target object in the texture image; determining a compression ratio for the texture image according to all object categories of the texture image; compressing and reconstructing the texture image according to the compression ratio to obtain a compressed texture image and a reconstructed texture image; determining a target reconstruction error function corresponding to the compression ratio; determining a reconstruction error between the texture image and the reconstructed texture image according to the target reconstruction error function; and outputting the compressed texture image under the condition that the reconstruction error is smaller than or equal to a preset error threshold value. According to the scheme, the texture image can be compressed in a self-adaptive mode, the storage space of the texture data is greatly reduced on the premise that the visual quality is guaranteed, and the rendering efficiency of the three-dimensional model is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of three-dimensional model, in particular to a processing method and device for three-dimensional model and a storage medium. BACKGROUND

[0002] Texture reconstruction technology has been widely used in surveying and mapping, positioning, navigation, autonomous driving, VR / AR, industrial manufacturing, and consumer product fields, and relevant methods have emerged in an endless stream. However, the texture generated by the traditional reconstruction technology has many problems such as large memory occupation, long loading time, high storage requirement, and low transmission efficiency. Traditional texture compression is mostly based on mesh simplification or point cloud downsampling, which causes a lot of loss of texture details and cannot effectively reduce storage requirements and improve transmission efficiency. Most texture compression methods do not consider the actual content of the texture, that is, they do not distinguish different object types (such as trees, buildings, etc.). This "one-size-fits-all" approach can lead to unnecessary information loss, especially in cases where specific features need to be preserved. SUMMARY

[0003] The purpose of the embodiments of the present application is to provide a processing method, device and storage medium for three-dimensional model, to solve the problem that the texture information in the prior art causes a lot of detail loss after compression.

[0004] To achieve the above-mentioned purpose, the first aspect of the present application provides a processing method for three-dimensional model, the processing method comprising: extracting texture information of a three-dimensional model to be processed to obtain a texture image; determining an object category of each target object in the texture image; determining a compression ratio for the texture image according to all object categories of the texture image; compressing and reconstructing the texture image according to the compression ratio to obtain a compressed texture image and a reconstructed texture image; determining a target reconstruction error function corresponding to the compression ratio; determining a reconstruction error between the texture image and the reconstructed texture image according to the target reconstruction error function; in the case where the reconstruction error is less than or equal to a preset error threshold, outputting the compressed texture image.

[0005] In the embodiments of the present application, the compressing and reconstructing the texture image according to the compression ratio to obtain the compressed texture image and the reconstructed texture image comprises: inputting the texture image into the autoencoder to compress the texture image according to the compression ratio through the encoding module of the autoencoder to obtain the compressed texture image; reconstructing the compressed texture image through the decoding module of the autoencoder to obtain the reconstructed image; the method further comprises: in the case that the reconstruction error is greater than the preset error threshold, optimizing the model parameters of the autoencoder until the reconstruction error is less than or equal to the preset error threshold.

[0006] In the embodiments of the present application, the method further comprises: determining the peak signal-to-noise ratio and the structural similarity index between the reconstructed image and the texture image; determining the loss function of the autoencoder according to the target reconstruction error function, the peak signal-to-noise ratio and the structural similarity index; optimizing the model parameters of the autoencoder based on the loss function to obtain the trained autoencoder.

[0007] In the embodiments of the present application, determining the target reconstruction error function corresponding to the compression ratio comprises: determining a key object category in all object categories; in the case that the difference between the compression factor of the key object category and the compression ratio is less than or equal to the preset threshold, determining the first preset reconstruction error function as the target reconstruction error function; in the case that the difference between the compression factor of the key object category and the compression ratio is greater than the preset threshold, determining the second preset reconstruction error function as the target reconstruction error function; wherein the first preset reconstruction error function is used to define the difference between the input data and the reconstructed data, and the second preset reconstruction error function is used to define the difference between the input data and the reconstructed data after regularization constraint.

[0008] In the embodiments of the present application, the function expression of the first preset reconstruction error function is as shown in the following formula (1): (1) Wherein, is the first preset reconstruction error function, is the first parameterized function corresponding to the encoder of the autoencoder, is the second parameterized function corresponding to the decoder of the autoencoder, and I is the texture image, is the parameter set of the first parameterized function, is the parameter set of the second parameterized function, is the reconstruction error of the texture image I.

[0009] In the embodiments of the present application, the function expression of the second preset reconstruction error function is as shown in the following formula (2): (2) Wherein, is a second preset reconstruction error function, is a first parameterized function corresponding to an encoder of the autoencoder, is a second parameterized function corresponding to a decoder of the autoencoder, and I is a texture image, is a parameter set of the first parameterized function, is a parameter set of the second parameterized function, is a reconstruction error of the texture image I, is a regularization term for constraining the texture image to be compressed, is a coefficient for balancing the regularization term and the reconstruction error.

[0010] In an embodiment of the present application, determining the compression ratio for the texture image according to all object categories of the texture image comprises: determining a compression factor corresponding to each object category; determining a mean value of all compression factors; and determining the mean value as the compression ratio of the texture image.

[0011] In an embodiment of the present application, determining the object category of each target object in the texture image comprises: mapping each vertex of the three-dimensional model to the texture image and determining an image coordinate of each vertex in the texture image; extracting a first feature vector corresponding to each image coordinate; creating a data group corresponding to each vertex, the data group comprising the image coordinate and the first feature vector corresponding to the vertex; generating a data set corresponding to the three-dimensional model according to all data groups; performing feature extraction on the data set based on a convolutional neural network to obtain a second feature vector of the data set; and performing detection on the second feature vector based on a target detection algorithm to output the object category of each target object in the texture image.

[0012] The second aspect of the present application provides a processing device for a three-dimensional model, comprising: a memory configured to store instructions; a processor configured to call the instructions from the memory and capable of implementing the processing method for a three-dimensional model according to the above when executing the instructions.

[0013] The third aspect of the present application provides a machine-readable storage medium having instructions stored thereon, the instructions being used to cause a machine to execute the processing method for a three-dimensional model according to the above.

[0014] By the technical solution, the texture information of the three-dimensional model to be processed is extracted to obtain a texture image; an object category of each target object in the texture image is determined; a compression ratio for the texture image is determined according to all object categories of the texture image; the texture image is compressed and reconstructed according to the compression ratio to obtain a compressed texture image and a reconstructed texture image; a target reconstruction error function corresponding to the compression ratio is determined; a reconstruction error between the texture image and the reconstructed texture image is determined according to the target reconstruction error function; and the compressed texture image is output in a case where the reconstruction error is less than or equal to a preset error threshold. The above solution can realize adaptive compression of the texture image, greatly reduce the storage space of the texture data on the premise of ensuring the visual quality, and improve the rendering efficiency of the three-dimensional model.

[0015] Other features and advantages of the embodiments of the present application will be described in detail in the following specific implementation part. BRIEF DESCRIPTION OF DRAWINGS

[0016] The accompanying drawings are included to provide a further understanding of the embodiments of the present application, and constitute a part of the specification, and are used to explain the embodiments of the present application together with the specific implementation below, but do not constitute a limitation on the embodiments of the present application. In the drawings: Figure 1 A flowchart of a processing method for a three-dimensional model according to an embodiment of the present application is schematically shown; Figure 2 A structural block diagram of a processing device for a three-dimensional model according to an embodiment of the present application is schematically shown; Figure 3 A structural schematic diagram of a computer device according to an embodiment of the present application is schematically shown. DETAILED DESCRIPTION

[0017] To make the purpose, technical solutions and advantages of the embodiments of the present application more clear, the technical solutions in the embodiments of the present application will be clearly and completely described below in combination with the drawings in the embodiments of the present application. It should be understood that the specific implementation described herein is only used to explain and explain the embodiments of the present application, and is not used to limit the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without making creative efforts fall within the scope of protection of the present application.

[0018] It should be noted that if the embodiments of the present application involve directional indications (such as up, down, left, right, front, back, etc.), the directional indications are only used to explain the relative position relationship, movement condition, etc. between components in a certain posture (as shown in the drawings), and if the certain posture changes, the directional indications also change accordingly.

[0019] In addition, if the description of "first", "second", etc. is involved in the embodiments of the present application, the description of "first", "second", etc. is only for the purpose of description, and cannot be understood as indicating or implying the relative importance of the indicated technical features or implicitly indicating the number of the indicated technical features. Therefore, the features limited by "first", "second" can be explicitly or implicitly included at least one of the features. In addition, the technical solutions of various embodiments can be combined with each other, but it must be based on the realization of ordinary skilled in the art, when the combination of technical solutions appears contradictory or unachievable, it should be considered that the combination of technical solutions does not exist, nor within the protection scope required by the present application.

[0020] Figure 1 The flowchart of the processing method for a three-dimensional model according to the embodiments of the present application is schematically shown. As shown in Figure 1 The embodiments of the present application provide a processing method for a three-dimensional model, which can include the following steps.

[0021] S102, extracting the texture information of the three-dimensional model to be processed to obtain a texture image.

[0022] It can be understood that the three-dimensional model is a virtual object representation constructed in three-dimensional space by computer technology, which can be a real entity or a fictional object formed by connecting geometric bodies (such as polygons, lines, etc.). Texture information is a kind of image data containing color, effect and pattern, usually in bitmap form. It provides rich color and surface features for the model by mapping to the surface of the three-dimensional model, thereby increasing the visual realism and detail level of the model. Specifically, the processor can preprocess the three-dimensional model to be processed and extract the texture information of its surface, including RGB color, normal map and roughness, etc. physical properties, so as to obtain a texture image.

[0023] S104, determining the object category of each target object in the texture image.

[0024] It can be understood that the target object refers to a specific object in the texture image, and the object category includes but is not limited to trees, buildings, ground, vehicles and other objects. Specifically, a deep learning model can be used to identify the texture image to extract high-level texture features, and then identify the target object based on different texture features. And the class label and its position information of each detected object can be output.

[0025] In the embodiments of the present application, determining the object category of each target object in the texture image comprises: mapping each vertex of the three-dimensional model to the texture image and determining the image coordinates of each vertex in the texture image; extracting a first feature vector corresponding to each image coordinate; creating a data group corresponding to each vertex, the data group comprising the image coordinates and the first feature vector corresponding to the vertex; generating a data set corresponding to the three-dimensional model according to all the data groups; performing feature extraction on the data set based on a convolutional neural network to obtain a second feature vector of the data set; and performing detection on the second feature vector based on a target detection algorithm to output the object category of each target object in the texture image.

[0026] It can be understood that in the three-dimensional model, a vertex is a position point defined by three coordinate axes (x, y, z). The vertex is not only a basic element constituting the model, but also forms edges and faces through connection, thereby constituting a complex geometric shape. The vertex has a coordinate value and can be endowed with other attributes such as color and texture coordinates. Specifically, after obtaining the texture image of the three-dimensional model, the image coordinates (UV coordinates) of each vertex in the texture image are obtained by extracting the UV mapping. The image coordinates define the relationship of the texture image mapping onto the surface of the three-dimensional model. Further, a first feature vector in the texture image can be extracted, and the first feature vector is combined with the UV coordinates to form a unified data structure, i.e., a data group. For each vertex, a tuple or list is created, containing its UV coordinates and the corresponding first feature vector. A data set corresponding to the three-dimensional model is generated according to all the data groups, and the entire data set is a list, each element in the list representing a vertex and its related information. Further, feature extraction is performed on the data set based on a convolutional neural network to obtain a second feature vector of the data set. Assuming that the input texture image is I, after a series of convolutional layers and pooling layers are processed, the second feature vector F is obtained. It can be understood that the first feature vector and the second feature vector are relative, the first feature vector refers to the feature vector of the image coordinates corresponding to each vertex, and the second feature vector refers to the feature vector of the texture image. Further, detection is performed on the second feature vector based on a target detection algorithm to output the object category of each target object in the texture image. Specifically, the category label and the position information of each detected object can be output.

[0027] In one specific embodiment, the three-dimensional model texture is preprocessed, its surface texture information is extracted, and it is converted into a form suitable for a deep learning model. Specifically, the three-dimensional model to be processed can come from a b3dm file. b3dm is a standard format for 3D Tiles, mainly used for efficient transmission and rendering of large-scale 3D geospatial data. The b3dm file is loaded using the cesium / engine library in CesiumJS, the geometry data (vertex coordinates, normals, etc.) and texture information (texture images, UV mapping, etc.) are extracted. The texture image is obtained from the b3dm file, the UV coordinates corresponding to each vertex are obtained, and the UV mapping is extracted, where the UV coordinates define the relationship between the texture image and the model surface. Further, the texture image can be standardized to ensure that all texture images have the same size and format (RGB 256x256 pixels) for subsequent processing. The first layer of the VGG16 convolutional neural network can be used to automatically extract features, including color histograms, gradient direction histograms, etc. The first feature vector corresponding to the UV coordinates of each vertex is extracted from the texture image. The feature vector is combined with the UV coordinates to form a unified data structure for subsequent deep learning model processing. Each vertex corresponds to a set of two-dimensional coordinates (u, v) that define the way the texture is mapped to the 3D model surface. The first feature vector extracted from the texture image is obtained through the VGG16 convolutional neural network and is represented as a fixed-length vector. For each vertex, a tuple or list is created containing its UV coordinates and corresponding feature vector. The entire dataset is a list, where each element represents a vertex and its related information.

[0028] Specifically, vertex_data = [ # For the i-th vertex { 'uv': (ui, vi),# UV coordinates of the i-th vertex 'features': [f1, f2,..., fF]# Feature vector corresponding to the i-th vertex }, ... ] Further, high-level features are extracted from the texture image using a pre-trained VGG16 convolutional neural network. Assuming that the input texture image is I, after a series of convolutional layers and pooling layers, the second feature vector F is obtained. where, represents the convolutional neural network mapping function.

[0029] Further, to identify specific objects (trees, buildings, ground, vehicles) in the texture, the second feature vector is analyzed using the YOLOv5 object detection algorithm, which outputs the class label and location information of each detected object. Let the detection result be a set wherein, is the class label of the i-th target object, is the location box of the target object. For each target object, there is a confidence score which is considered reliable when the confidence score is greater than 0.7.

[0030] S106, determine the compression ratio for the texture image according to all object classes of the texture image.

[0031] Most texture compression methods do not consider the actual content of the texture, i.e. they do not distinguish different object types (such as trees, buildings, etc.). This "one-size-fits-all" approach can result in unnecessary loss of information, especially in cases where specific features need to be preserved. Based on this, the embodiments of the present application can apply different compression ratios to the texture image based on the detected object classes, achieving adaptive texture compression.

[0032] In the embodiments of the present application, determining the compression ratio for the texture image according to all object classes of the texture image includes: determining a compression factor corresponding to each object class; determining the mean value of all compression factors; and determining the mean value as the compression ratio of the texture image.

[0033] Specifically, adaptive texture compression can dynamically adjust the compression ratio according to the texture content. The processor can determine a compression factor corresponding to each object class. For visually important target objects (such as detailed parts of buildings), a lower compression ratio is used to preserve more details. For less critical target objects (such as sky or grass), a higher compression ratio can be used. This targeted compression strategy can greatly reduce the storage space of texture data while ensuring visual quality. Further, a compression factor function is defined , the input is the object class, and the corresponding compression factor is returned. The compression ratio of the overall image is represented as the average of the compression factors of all detected target objects. Specifically, the compression ratio is calculated according to formula (3): (3) wherein, is the final compression ratio used, is the compression factor for the i-th object class, and D refers to the total number of object classes of the texture image.

[0034] ​The compression factors of the common texture object categories are shown in Table 1 below. The greater the value of the compression factor, the higher the compression degree.

[0035] Table 1 Compression factors of common texture object categories

[0036] S108, compress and reconstruct the texture image according to the compression ratio to obtain a compressed texture image and a reconstructed texture image.

[0037] In the embodiments of the present application, compressing and reconstructing the texture image according to the compression ratio to obtain a compressed texture image and a reconstructed texture image includes: inputting the texture image into the autoencoder to compress the texture image according to the compression ratio by the encoding module of the autoencoder to obtain the compressed texture image; and reconstructing the compressed texture image by the decoding module of the autoencoder to obtain the reconstructed image.

[0038] Specifically, the method of compressing and reconstructing the texture image based on the compression ratio is adopted to obtain the compressed texture image and the reconstructed texture image. The specific operation steps are as follows: first, input the original texture image into an autoencoder. The autoencoder is a special type of neural network, which realizes the compression of the input image through an encoding module. In this process, the encoding module will encode the texture image according to the preset compression ratio to obtain the compressed texture image. Then, in order to verify the compression effect and restore the details of the original image as much as possible, the decoding module of the autoencoder will decode the compressed texture image, that is, reconstruct it. Through the processing of the decoding module, we can obtain a reconstructed image that is as close to the original texture image as possible. The whole process not only realizes the compression of image data, but also guarantees the preservation of image quality to a certain extent, which is of great significance for image storage and transmission and other application scenarios.

[0039] S110, determining a target reconstruction error function corresponding to the compression ratio.

[0040] It can be understood that the reconstruction error function refers to, in machine learning and deep learning, a difference or loss between reconstructed data and original input data when a model learned by a model reconstructs input data. The reconstruction error can be used to evaluate the restoration ability of the model to the input data, and a smaller reconstruction error indicates that the model can accurately reconstruct the input data, while a larger reconstruction error means that the model cannot completely restore the original data. In the embodiments of the present application, after the texture image is compressed, the reconstructed image can be restored, and the error between the restored image and the original texture image can be determined by using the reconstruction error function. When the compression ratio of the texture image is close to the compression factor of the object class marked as “important”, a stricter reconstruction error limit can be applied to it. Therefore, the corresponding target reconstruction error function can be selected based on the compression ratio to adapt to different compression requirements.

[0041] In the embodiments of the present application, determining the target reconstruction error function corresponding to the compression ratio comprises: determining a key object class in all object classes; determining the first preset reconstruction error function as the target reconstruction error function in the case that the difference between the compression factor of the key object class and the compression ratio is less than or equal to a preset threshold; determining the second preset reconstruction error function as the target reconstruction error function in the case that the difference between the compression factor of the key object class and the compression ratio is greater than the preset threshold; wherein the first preset reconstruction error function is used to define the difference between the input data and the reconstructed data, and the second preset reconstruction error function is used to define the difference between the input data after the regularization constraint and the reconstructed data.

[0042] It can be understood that the key object class can be an important object class defined by a user. The preset threshold is an error range set by a technician according to experience. In the compression process, different reconstruction error functions can be selected to adapt to different requirements. Specifically, after determining the compression factor corresponding to all object classes in the texture image as the compression ratio of the texture image, if the difference between the compression factor of the key object class and the compression ratio is less than or equal to the preset threshold, the first preset reconstruction error function is determined as the target reconstruction error function. If the difference between the compression factor of the key object class and the compression ratio is greater than the preset threshold, the second preset reconstruction error function is determined as the target reconstruction error function. It can be understood that the second preset reconstruction error function is obtained by introducing an additional constraint condition or modification to the first preset reconstruction error function, which can enable the model to achieve efficient compression while preserving important information.

[0043] In the embodiments of the present application, the function expression of the first preset reconstruction error function is as shown in the following formula (1): (1) wherein, is the first preset reconstruction error function, is a first parameterized function corresponding to an encoder of the autoencoder, is a second parameterized function corresponding to a decoder of the autoencoder, and I is a texture image, is a parameter set of the first parameterized function, is a parameter set of the second parameterized function, is a reconstruction error of the texture image I.

[0044] In an embodiment of the present application, a function expression of the second preset reconstruction error function is as shown in the following formula (2): (2) wherein, is the second preset reconstruction error function, is a first parameterized function corresponding to an encoder of the autoencoder, is a second parameterized function corresponding to a decoder of the autoencoder, and I is a texture image, is a parameter set of the first parameterized function, is a parameter set of the second parameterized function, is a reconstruction error of the texture image I, is a regularization term for constraining the texture image to be compressed, is a coefficient for balancing the regularization term and the reconstruction error.

[0045] S112, determining a reconstruction error between the texture image and the reconstructed texture image according to the target reconstruction error function.

[0046] S114, outputting the compressed texture image in a case where the reconstruction error is less than or equal to a preset error threshold.

[0047] It can be understood that the reconstruction error refers to a difference between the compressed texture image reconstructed back to the original space and the original texture image. The reconstruction error indicates that the less information lost by the autoencoder, the better the reconstruction ability and learning effect. In a case where the reconstruction error is less than or equal to a preset error threshold, the obtained compressed texture image can be outputted.

[0048] By the technical solution, the texture information of the three-dimensional model to be processed is extracted to obtain a texture image; the object category of each target object in the texture image is determined; the compression ratio for the texture image is determined according to all object categories of the texture image; the texture image is compressed and reconstructed according to the compression ratio to obtain a compressed texture image and a reconstructed texture image; a target reconstruction error function corresponding to the compression ratio is determined; the reconstruction error between the texture image and the reconstructed texture image is determined according to the target reconstruction error function; and the compressed texture image is output in a case where the reconstruction error is less than or equal to a preset error threshold. The above solution can realize adaptive compression of the texture image, greatly reduce the storage space of the texture data under the premise of ensuring the visual quality, and improve the rendering efficiency of the three-dimensional model.

[0049] In an embodiment of the present application, the method further comprises: in a case where the reconstruction error is greater than the preset error threshold, optimizing the model parameters of the autoencoder until the reconstruction error is less than or equal to the preset error threshold.

[0050] Specifically, the model parameters of the autoencoder include but are not limited to neural network weights, biases, latent variable distribution parameters, etc. corresponding to the encoder and the decoder respectively. These parameters are optimized through a back propagation algorithm in the training process to minimize the reconstruction error and the KL divergence of the latent distribution, so as to output the compressed texture image.

[0051] In another embodiment, if the reconstruction error is greater than the preset error threshold, the compression ratio can also be adaptively adjusted until the reconstruction error is less than or equal to the preset error threshold, and the compressed texture image is output.

[0052] In an embodiment of the present application, the method further comprises: determining the peak signal-to-noise ratio and the structural similarity index between the reconstructed image and the texture image; determining the loss function of the autoencoder according to the target reconstruction error function, the peak signal-to-noise ratio and the structural similarity index; and optimizing the model parameters of the autoencoder based on the loss function to obtain the trained autoencoder.

[0053] Specifically, the peak signal-to-noise ratio, the structural similarity index and other indicators are used as quality evaluation vectors, and the training process of the autoencoder is adjusted through these indicators to ensure that the compressed texture can significantly reduce the storage requirement and maintain a high visual fidelity.

[0054] Wherein, the peak signal-to-noise ratio (PSNR) is an indicator for measuring image distortion, and the higher the value, the better the image quality. The peak signal-to-noise ratio is calculated according to the following formula (4): (4) Wherein, is the peak signal-to-noise ratio, is the maximum possible value of a pixel (for 8-bit images, ), is the mean squared error, I and are the original and reconstructed texture images, respectively, and are the height and width of the image, respectively.

[0055] The MSE is calculated according to the following equation (5): (5) where, is the mean squared error, and are the height and width of the image, respectively, I and are the original and reconstructed texture images, respectively, i and j are the pixel coordinates in the image.

[0056] The structural similarity index (SSIM) is an index that measures the structural similarity between two images, with a value closer to 1 indicating better image quality. The structural similarity index is calculated according to the following equation (6): (6) where, and are the mean values of the original and reconstructed images, respectively, and are the variances, is the covariance, and are constants used to stabilize the denominator.

[0057] In order to achieve the purpose of reducing storage requirements and maintaining high visual fidelity during compression, the above quality evaluation indicators are added as part of the loss function, or as additional regularization terms in the training process. On the basis of the traditional autoencoder loss function, the quality evaluation indicators are added as additional terms. The following formula (7) shows the composite loss function: (7) where, is the reconstruction error, is the quality evaluation loss, is the composite loss function, is the weight coefficient corresponding to the quality evaluation loss.

[0058] (8) where, is the loss corresponding to the peak signal-to-noise ratio, is the loss corresponding to the structural similarity index.

[0059] (9) (10) wherein, denotes a first parameterized function corresponding to an encoder of the autoencoder, denotes a second parameterized function corresponding to a decoder of the autoencoder, I denotes a texture image, denotes a parameter set of the first parameterized function, denotes a parameter set of the second parameterized function.

[0060] In addition to adding a quality evaluation index in the loss function, the compression strategy can also be dynamically adjusted according to actual needs. For example, important area protection: for areas containing key visual elements (such as windows or doors of buildings), a lower compression ratio is used to maintain details. For background areas (such as sky, grass, forest, etc.), a higher compression ratio is used. A larger compression ratio is used in the preliminary training stage to quickly converge, and the compression ratio is gradually reduced and the weight of the quality evaluation index is increased in the subsequent training stage, so as to improve the visual fidelity while ensuring the compression rate. By introducing the quality evaluation vector (PSNR, SSIM, etc.) as part of the loss function, the model training is guided during the compression process to ensure that the compressed texture can significantly reduce storage requirements while maintaining high visual fidelity. The final generated compressed texture not only occupies less storage space, but also provides high-quality visual effects.

[0061] Figure 1 A flowchart of a processing method for a three-dimensional model in an embodiment is shown. It should be understood that although Figure 1 the steps in the flowchart of the processing method are shown in sequence according to the arrows, these steps are not necessarily executed in sequence according to the arrows. Unless otherwise specified herein, the execution of these steps is not strictly limited in sequence, and these steps can be executed in other sequences. Moreover, Figure 1 At least part of the steps in the processing method can include multiple sub-steps or multiple stages, which are not necessarily executed at the same time, but can be executed at different times, and the execution sequence of these sub-steps or stages is not necessarily sequential, but can be executed in rotation or alternation with at least part of other steps or sub-steps or stages of other steps.

[0062] Figure 2 A structural block diagram of a processing apparatus for a three-dimensional model according to an embodiment of the present application is schematically shown. As Figure 2 shown, the processing apparatus for a three-dimensional model according to an embodiment of the present application can include: a memory configured to store instructions; The processor is configured to call instructions from the memory and implement the above processing method for a three-dimensional model when the instructions are executed.

[0063] Specifically, in the embodiments of the present application, the processor can be configured to: extract texture information of the three-dimensional model to be processed to obtain a texture image; determine an object category of each target object in the texture image; determine a compression ratio for the texture image according to all object categories of the texture image; compress and reconstruct the texture image according to the compression ratio to obtain a compressed texture image and a reconstructed texture image; determine a target reconstruction error function corresponding to the compression ratio; determine a reconstruction error between the texture image and the reconstructed texture image according to the target reconstruction error function; and output the compressed texture image in a case where the reconstruction error is less than or equal to a preset error threshold.

[0064] In the embodiments of the present application, the processor can be further configured to: input the texture image into the autoencoder to compress the texture image according to the compression ratio through an encoding module of the autoencoder to obtain a compressed texture image; and reconstruct the compressed texture image through a decoding module of the autoencoder to obtain a reconstructed image; the method further comprising: in a case where the reconstruction error is greater than the preset error threshold, optimizing model parameters of the autoencoder until the reconstruction error is less than or equal to the preset error threshold.

[0065] In the embodiments of the present application, the processor can be further configured to: determine a peak signal-to-noise ratio and a structural similarity index between the reconstructed image and the texture image; determine a loss function of the autoencoder according to the target reconstruction error function, the peak signal-to-noise ratio and the structural similarity index; and optimize the model parameters of the autoencoder based on the loss function to obtain a trained autoencoder.

[0066] In the embodiments of the present application, the processor can be further configured to: determine a key object category among all object categories; in a case where a difference between a compression factor of the key object category and the compression ratio is less than or equal to a preset threshold, determine a first preset reconstruction error function as the target reconstruction error function; and in a case where the difference between the compression factor of the key object category and the compression ratio is greater than the preset threshold, determine a second preset reconstruction error function as the target reconstruction error function; wherein the first preset reconstruction error function is used to define a difference between input data and reconstructed data, and the second preset reconstruction error function is used to define a difference between the input data and the reconstructed data after a regularization constraint.

[0067] In the embodiments of the present application, a function expression of the first preset reconstruction error function is as shown in the following formula (1): (1) wherein, denotes a first preset reconstruction error function, denotes a first parameterized function corresponding to an encoder of the autoencoder, denotes a second parameterized function corresponding to a decoder of the autoencoder, and I denotes a texture image, denotes a parameter set of the first parameterized function, denotes a parameter set of the second parameterized function, denotes a reconstruction error of the texture image I.

[0068] In an embodiment of the present application, a function expression of the second preset reconstruction error function is shown in the following formula (2): (2) wherein, denotes a second preset reconstruction error function, denotes a first parameterized function corresponding to an encoder of the autoencoder, denotes a second parameterized function corresponding to a decoder of the autoencoder, and I denotes a texture image, denotes a parameter set of the first parameterized function, denotes a parameter set of the second parameterized function, denotes a reconstruction error of the texture image I, denotes a regularization term for constraining the texture image to be compressed, denotes a coefficient for balancing the regularization term and the reconstruction error.

[0069] In an embodiment of the present application, the processor can be further configured to: determine a compression factor corresponding to each object category; determine a mean value of all compression factors; and determine the mean value as a compression ratio of the texture image.

[0070] In an embodiment of the present application, the processor can be further configured to: map each vertex of the three-dimensional model to the texture image and determine an image coordinate of each vertex in the texture image; extract a first feature vector corresponding to each image coordinate; create a data group corresponding to each vertex, the data group including the image coordinate and the first feature vector corresponding to the vertex; generate a data set corresponding to the three-dimensional model according to all data groups; perform feature extraction on the data set based on a convolutional neural network to obtain a second feature vector of the data set; and perform detection on the second feature vector based on a target detection algorithm to output an object category of each target object in the texture image.

[0071] The embodiments of the present application also provide a machine readable storage medium, which stores instructions for causing a machine to execute the above-mentioned processing method for a three-dimensional model.

[0072] In one embodiment, a computer device, which can be a server, is provided. An internal structure diagram of the computer device can be as shown in FIG. 1. Figure 3 The computer device includes a processor A01, a network interface A02, a memory (not shown in the figure) and a database (not shown in the figure) connected through a system bus. The processor A01 of the computer device is configured to provide computing and control capabilities. The memory of the computer device includes an internal memory A03 and a non-volatile storage medium A04. The non-volatile storage medium A04 stores an operating system B01, a computer program B02 and a database (not shown in the figure). The internal memory A03 provides an environment for running the operating system B01 and the computer program B02 in the non-volatile storage medium A04. The database of the computer device is configured to store processing data for a three-dimensional model. The network interface A02 of the computer device is configured to communicate with an external terminal through a network connection. The computer program B02 is configured to be executed by the processor A01 to implement a processing method for a three-dimensional model.

[0073] Those skilled in the art can understand that Figure 3 The structure shown in FIG. 1 is only a block diagram of part of the structure related to the scheme of the present application, and does not constitute a limitation on the computer device to which the scheme of the present application is applied. The specific computer device can include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0074] Those skilled in the art can understand that the embodiments of the present application can be provided as a method, a system or a computer program product. Therefore, the present application can take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Moreover, the present application can take the form of a computer program product implemented on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0075] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), computer program products according to the embodiments of the present application. It should be understood that each flow and / or block in the flowcharts and / or block diagrams, and the combination of flows and / or blocks in the flowcharts and / or block diagrams can be implemented by computer program instructions. These computer program instructions can be provided to a general-purpose computer, a special-purpose computer, an embedded processor or other programmable data processing apparatus to produce a machine, so that the instructions executed by the computer or other programmable data processing apparatus produce a device that implements the functions specified in the flowcharts and / or block diagrams. Figure 1 The functions specified in one flow or multiple flows and / or blocks Figure 1 The functions specified in one flow or multiple flows and / or blocks

[0076] These computer program instructions can also be stored in a computer- readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instructions which implement the Figure 1 function specified in the flow or flows and / or blocks Figure 1 of the multiple flows or blocks.

[0077] These computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the Figure 1 function specified in the flow or flows and / or blocks Figure 1 of the multiple flows or blocks.

[0078] In a typical configuration, a computing device includes one or more processors (CPUs), input / output interfaces, network interfaces, and memory.

[0079] The memory can include non-persistent memory and / or volatile memory, such as random access memory (RAM) about which the computer stores information such as computer program instructions. RAM allows information to be loaded quickly into the computer rather than having to be read off of a persistent memory such as a hard disk, for example. Non-volatile memory can be used for program instructions to be loaded into the computer when it is booted or re-booted, for example. Persistent storage, non-volatile memory, or additional memory (e.g., read-only memory (ROM), EPROM, EEPROM, flash memory or other memory) can be used for storing programs, modules, and data, when such information is not needed for immediate use. The memory is an example of computer readable media.

[0080] Computer readable media includes permanent and non-permanent, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disc read-only memory (CD-ROM), digital versatile discs (DVDs) or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information accessible to a computing device. According to the definition herein, computer readable media does not include transitory media, such as modulated data signals and carrier waves.

[0081] It should also be noted that the terms "comprising", "comprises" or other variations thereof are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but can also include other elements not expressly listed or inherent to such process, method, article, or apparatus.

[0082] The above embodiments are only used to illustrate the present application, but not to limit it. Instead of the above, various modifications and changes can be made to the application by those skilled in the art. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the application shall fall into the scope of the claims of the application.

Claims

1. A processing method for a three-dimensional model, characterized in that: The processing method comprises: Extracting texture information of the three-dimensional model to be processed to obtain a texture image; determining an object category of each target object in the texture image; determining a compression ratio for the texture image according to all object categories of the texture image; compressing and reconstructing the texture image according to the compression ratio to obtain a compressed texture image and a reconstructed texture image; determining a target reconstruction error function corresponding to the compression ratio; determining a reconstruction error between the texture image and the reconstructed texture image according to the target reconstruction error function; When the reconstruction error is less than or equal to a preset error threshold, a compressed texture image is output.

2. The processing method for a three-dimensional model according to claim 1, characterized in that: The compressing and reconstructing the texture image according to the compression ratio to obtain a compressed texture image and a reconstructed texture image includes: Inputting the texture image into an autoencoder, so as to compress the texture image according to the compression ratio through an encoding module of the autoencoder to obtain a compressed texture image; Reconstructing the compressed texture image through the decoding module of the autoencoder to obtain a reconstructed image; The method further comprises: When the reconstruction error is greater than a preset error threshold, the model parameters of the autoencoder are optimized until the reconstruction error is less than or equal to the preset error threshold.

3. The processing method for a three-dimensional model according to claim 2, characterized in that: The method further comprises: determining a peak signal-to-noise ratio and a structural similarity index between the reconstructed image and the texture image; Determining a loss function of the autoencoder according to the target reconstruction error function, the peak signal-to-noise ratio, and the structural similarity index; The model parameters of the autoencoder are optimized based on the loss function to obtain a trained autoencoder.

4. The processing method for a three-dimensional model according to claim 1, characterized in that: Determining the target reconstruction error function corresponding to the compression ratio includes: Identify key object categories among all object categories; In a case where the difference between the compression factor of the key object category and the compression ratio is less than or equal to a preset threshold, determining a first preset reconstruction error function as the target reconstruction error function; In a case where the difference between the compression factor of the key object category and the compression ratio is greater than a preset threshold, determining a second preset reconstruction error function as the target reconstruction error function; The first preset reconstruction error function is used to define the difference between the input data and the reconstructed data, and the second preset reconstruction error function is used to define the difference between the input data and the reconstructed data after regularization constraints.

5. The processing method for a three-dimensional model according to claim 4, characterized in that: The functional expression of the first preset reconstruction error function is shown in the following formula (1): （1） in, refers to the first preset reconstruction error function, Refers to the first parameterized function corresponding to the encoder of the autoencoder, refers to the second parameterized function corresponding to the decoder of the autoencoder, I refers to the texture image, refers to the parameter set of the first parameterized function, refers to the parameter set of the second parameterized function, It refers to the reconstruction error of the texture image I.

6. The processing method for a three-dimensional model according to claim 4, characterized in that: The functional expression of the second preset reconstruction error function is shown in the following formula (2): （2） in, refers to the second preset reconstruction error function, Refers to the first parameterized function corresponding to the encoder of the autoencoder, refers to the second parameterized function corresponding to the decoder of the autoencoder, I refers to the texture image, refers to the parameter set of the first parameterized function, refers to the parameter set of the second parameterized function, refers to the reconstruction error of the texture image I, refers to the regularization term that constrains the texture image to be compressed, It refers to the coefficient that balances the regularization term and the reconstruction error.

7. The processing method for a three-dimensional model according to claim 1, characterized in that: The determining of the compression ratio for the texture image according to all object categories of the texture image comprises: Determine the compression factor corresponding to each object category; Determine the mean of all compression factors; The mean value is determined as the compression ratio of the texture image.

8. The processing method for a three-dimensional model according to claim 1, characterized in that: Determining the object category of each target object in the texture image includes: Mapping each vertex of the three-dimensional model to the texture image, and determining the image coordinates of each vertex in the texture image; Extract the first eigenvector corresponding to each image coordinate; Creating a data set corresponding to each vertex, the data set including the image coordinates and the first eigenvector corresponding to the vertex; generating a data set corresponding to the three-dimensional model based on all data groups; Performing feature extraction on the data set based on a convolutional neural network to obtain a second feature vector of the data set; The second feature vector is detected based on a target detection algorithm to output an object category of each target object in the texture image.

9. A processing device for a three-dimensional model, characterized in that: include: a memory configured to store instructions; A processor is configured to call the instructions from the memory and implement the three-dimensional model processing method according to any one of claims 1 to 8 when executing the instructions.

10. A machine-readable storage medium, characterized in that The machine-readable storage medium stores instructions for causing a machine to execute the three-dimensional model processing method according to any one of claims 1 to 8.

Images