Feature Extraction Model Training, Texture Mapping Processing Method, Device and Electronic Device

Abstract

The present application provides a method, apparatus, and electronic device for training a feature extraction model and processing a material texture map. The method for training the feature extraction model includes: obtaining an initial sample data set, where the initial sample data set includes at least one sample data, and each sample data includes a sample material texture map and a hard label for characterizing the material type to which the sample material texture map belongs; converting the hard label of the sample material texture map in the initial sample data set into a soft label related to the sample-level clustering result of the sample material texture map to obtain a training sample data set; the soft label is used to characterize the probability that the sample material texture map belongs to each material type; the sample-level clustering result includes a color result and / or a texture clustering result; using the training sample data set to train a neural network model to obtain a feature extraction model, and the feature extraction model is used to extract the material features of an image. The present application improves the efficiency and accuracy of material texture map processing.

Description

Technical Field

[0001] This application relates to computer technology, and in particular, to a method, apparatus, and electronic device for training a feature extraction model and processing a texture map. Background Art

[0002] Taking three-dimensional (3D) modeling as an example, an electronic device can respond to an operation of constructing a 3D model triggered by a user through modeling software, construct a 3D model framework, and then assign a target texture map to the 3D model framework, so that the 3D model has visual representations such as color, texture, and roughness corresponding to the target texture map. During the modeling process, before assigning the target texture map to the 3D model framework, the electronic device first needs to determine a target texture map.

[0003] In the related art, the way to determine the target texture map is mainly for the user to distinguish and find the required target texture map among a large number of texture maps in the texture map library. However, this method has problems of low efficiency and poor accuracy. Summary of the Invention

[0004] This application provides a method, apparatus, and electronic device for training a feature extraction model and processing a texture map to solve the problems of low efficiency and poor accuracy in the existing texture map processing.

[0005] In a first aspect, this application provides a method for training a feature extraction model, and the method includes:

[0006] Obtain an initial sample data set, where the initial sample data set includes: at least one sample data, and each sample data includes: a sample texture map, and a hard label of the sample texture map; the hard label is used to represent the material type to which the sample texture map belongs;

[0007] Convert the hard label of the sample texture map in the initial sample data set into a soft label to obtain a training sample data set; the soft label is used to represent the probability that the sample texture map belongs to each material type; the soft label is related to the sample hierarchical clustering result of the sample texture map, and the sample hierarchical clustering result includes: a color result and / or a texture clustering result;

[0008] Use the training sample data set to train a neural network model to obtain a feature extraction model, and the feature extraction model is used to extract the material features of an image.

[0009] Optionally, the sample hierarchical clustering results include: a first sample clustering result obtained by clustering sample material texture maps based on color, a second sample clustering result obtained by clustering sample material texture maps based on texture, and a third sample clustering result obtained by clustering sample material texture maps based on color and texture; both the first sample clustering result and the second sample clustering result are first-level sample clustering results in the sample hierarchical clustering results, and the third sample clustering result is a second-level sample clustering result in the sample hierarchical clustering results;

[0010] The conversion of the hard labels of the sample material texture maps in the initial sample dataset into soft labels to obtain a training sample dataset includes:

[0011] According to the first sample clustering result, the second sample clustering result, the third sample clustering result, and the hard label of each sample material texture map, obtain the soft label of each sample material texture map;

[0012] Use each sample material texture map and the soft label of each sample material texture map to obtain the training sample dataset.

[0013] Optionally, the obtaining of the soft label of each sample material texture map according to the first sample clustering result, the second sample clustering result, the third sample clustering result, and the hard label of each sample material texture map includes:

[0014] For each sample material texture map, determine the initial soft label of the sample material texture map according to the hard label of the sample material texture map and a preset soft label probability distribution method;

[0015] Adjust the probability in the initial soft label of the sample material texture map according to the first sample clustering result, the second sample clustering result, and the third sample clustering result to obtain the soft label of the sample material texture map.

[0016] Optionally, the method further includes:

[0017] Obtain the first sample clustering result and the second sample clustering result;

[0018] According to the first sample clustering result and the second sample clustering result, obtain the third sample clustering result.

[0019] In a second aspect, the present application provides a method for processing material texture maps, and the method includes:

[0020] Obtain a target image;

[0021] Input the target image into the feature extraction model to obtain the material features of the target image; the feature extraction model is obtained by using the method described in any item of the first aspect.

[0022] According to the material features of the target image, the material features of each material texture map in the material texture map library extracted by using the feature extraction model, and the hierarchical clustering results of the material texture maps in the material texture map library based on color and texture, obtain K material texture maps that match the material of the target image; K is an integer greater than or equal to 1.

[0023] Optionally, the obtaining K material texture maps that match the material of the target image according to the material features of the target image, the material features of each material texture map in the material texture map library extracted by using the feature extraction model, and the hierarchical clustering results of the material texture maps in the material texture map library based on color and texture includes:

[0024] According to the material features of the target image and the material features of each material texture map in the material texture map library, obtain the similarity between the target image and each material texture map.

[0025] Obtain the first N material texture maps in the order of sorting the similarities from large to small; N is an integer greater than or equal to 2.

[0026] According to the hierarchical clustering results of the material texture maps in the material texture map library based on color and texture, obtain the clustering scores of the N material texture maps.

[0027] Obtain the first K material texture maps as the material texture maps that match the material of the target image in the order of sorting the clustering scores from large to small.

[0028] Optionally, the hierarchical clustering results include: the first clustering result obtained by clustering the material texture maps based on color, the second clustering result obtained by clustering the material texture maps based on texture, and the third clustering result obtained by clustering the material texture maps based on color and texture; the first clustering result and the second clustering result are both first-level clustering results in the hierarchical clustering results, and the third clustering result is a second-level clustering result in the hierarchical clustering results.

[0029] The obtaining the clustering scores of the N material texture maps according to the hierarchical clustering results of the material texture maps in the material texture map library based on color and texture includes:

[0030] According to the similarity sorting of the N material texture maps, obtain the initial clustering scores of the N material texture maps.

[0031] According to the initial clustering scores of the N material texture maps and the first clustering result, obtain the first clustering scores of the N material texture maps.

[0032] Obtain the second clustering scores of the N material texture maps according to the initial clustering scores of the N material texture maps and the second clustering result;

[0033] Obtain the third clustering scores of the N material texture maps according to the initial clustering scores of the N material texture maps and the third clustering result;

[0034] Obtain the clustering scores of the N material texture maps according to the initial clustering scores, the first clustering scores, the second clustering scores, and the third clustering scores of the N material texture maps.

[0035] Optionally, after obtaining the K material texture maps that match the material of the target image, the method further includes:

[0036] Output the K material texture maps.

[0037] Optionally, after obtaining the K material texture maps that match the material of the target image, the method further includes:

[0038] Determine a target material texture map from the K material texture maps;

[0039] Render the model framework of the target object using the target material texture map to obtain the model of the target object;

[0040] Output the model of the target object.

[0041] Optionally, before using the target material texture map to render the model framework of the target object to obtain the model of the target object, the method further includes:

[0042] Construct the model framework of the target object.

[0043] In a third aspect, the present application provides a feature extraction model training device, and the device includes:

[0044] An acquisition module, configured to acquire an initial sample data set, where the initial sample data set includes: at least one sample data, and each sample data includes: a sample material texture map and a hard label of the sample material texture map; the hard label is used to characterize the material type to which the sample material texture map belongs;

[0045] A processing module, configured to convert the hard label of the sample material texture map in the initial sample data set into a soft label to obtain a training sample data set; the soft label is used to characterize the probability that the sample material texture map belongs to each material type; the soft label is related to the sample hierarchical clustering result of the sample material texture map, and the sample hierarchical clustering result includes: a color result and / or a texture clustering result;

[0046] A training module for training a neural network model using the training sample data set to obtain a feature extraction model, where the feature extraction model is used to extract the material features of an image.

[0047] In a fourth aspect, the present application provides a material texture processing device, which includes:

[0048] An acquisition module for acquiring a target image;

[0049] A processing module for inputting the target image into the feature extraction model to obtain the material features of the target image; obtaining K material textures that match the material of the target image according to the material features of the target image, the material features of each material texture in the material texture library extracted by using the feature extraction model, and the hierarchical clustering result of the material textures in the material texture library based on color and texture; the feature extraction model is obtained by using the method described in any item of the first aspect; K is an integer greater than or equal to 1.

[0050] In a fifth aspect, the present application provides an electronic device, which includes a memory and a processor, and the memory is used to store a set of computer instructions;

[0051] The processor executes the set of computer instructions stored in the memory to execute the method described in any item of the first aspect or the second aspect.

[0052] In a sixth aspect, the present application provides a computer-readable storage medium, on which computer-executable instructions are stored, and when the computer-executable instructions are executed by a processor, the method described in any item of the first aspect or the second aspect is implemented.

[0053] In a seventh aspect, the present application provides a computer program product, including a computer program, and when the computer program is executed by a processor, the method described in any item of the first aspect or the second aspect is implemented.

[0054] The feature extraction model training, texture mapping processing method, device and electronic device provided by the present application convert the hard labels of each sample texture mapping into soft labels related to the sample-level clustering results of the sample texture mapping, and train the neural network model based on the soft labels of the sample texture mapping. Considering the situation where the material features of various sample texture mappings are relatively similar, compared with training the neural network model using hard labels, the training process of the present application is more in line with the actual judgment of the material features of various sample texture mappings, improving the accuracy of the feature extraction model obtained by training the neural network model based on the above training sample data set, and thus improving the accuracy of texture mapping processing based on the feature extraction model. Through the above clustering results based on color and texture, the probabilities of each material type to which the sample texture mapping represented by the soft label belongs are consistent with human artistic intuition, improving the accuracy of determining the soft label of the sample texture mapping, further improving the accuracy of the feature extraction model obtained by training the neural network model based on the above training sample data set, and thus further improving the accuracy of texture mapping processing based on the feature extraction model. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] In order to more clearly illustrate the technical solutions in the present application or the prior art, the following will briefly introduce the drawings required for the description of the embodiments or the prior art. Obviously, the drawings in the following description are some embodiments of the present application. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0056] Figure 1 It is an example diagram of a standard texture mapping;

[0057] Figure 2 It is an example diagram of a non-standard texture mapping;

[0058] Figure 3a It is a schematic diagram of a 3D modeling scene;

[0059] Figure 3b It is a schematic diagram of the application scenario of a texture mapping processing system provided by the present application;

[0060] Figure 3c It is a schematic diagram of the application scenario of another texture mapping processing system provided by the present application;

[0061] Figure 3d It is a schematic diagram of the hardware structure of the electronic device 10 deployed with the texture mapping processing system;

[0062] Figure 4 It is a schematic diagram of the flow of a feature extraction model training method provided by the present application;

[0063] Figure 5 Flow diagram of a method for obtaining a soft label of a sample material texture map provided by this application;

[0064] Figure 6 Flow diagram of a method for obtaining a sample hierarchical clustering result provided by this application;

[0065] Figure 7 Structural diagram of a neural network model provided by this application;

[0066] Figure 8 Flow diagram of a method for processing a material texture map provided by this application;

[0067] Figure 9 Flow diagram of a method for obtaining K material texture maps for target image material matching provided by this application;

[0068] Figure 10 Interface diagram of a material texture map processing system provided by this application;

[0069] Figure 11 Structural diagram of a feature extraction model training device provided by this application;

[0070] Figure 12 Structural diagram of a material texture map processing device provided by this application.

[0071] Through the above-mentioned drawings, specific embodiments of this application have been shown, and there will be more detailed descriptions hereinafter. These drawings and textual descriptions are not intended to limit the scope of the concept of this application in any way, but to illustrate the concept of this application to those skilled in the art by referring to specific embodiments. Detailed implementation manners

[0072] To make the objectives, technical solutions and advantages of this application clearer, the technical solutions in this application will be clearly and completely described below in conjunction with the drawings in this application. Obviously, the described embodiments are some but not all of the embodiments of this application. All other embodiments obtained by those of ordinary skill in the art based on the embodiments in this application without making creative efforts shall fall within the scope of protection of this application.

[0073] First, the noun concepts related to this application will be explained below:

[0074] Material texture map: A material texture map is an image. It is a visual representation of the characteristics of an object, such as color, transparency, specular reflectance, specular intensity, self-illumination, and roughness. The material referred to in this application can refer to the combination of material and texture. Exemplarily, the material can be, for example, any one of wood, plastic, leather, etc. The texture can be, for example, a visual representation of various characteristics of an object, such as color, texture, transparency, specular reflectance, and roughness. The material and texture of the same material texture map are the same. Two different material texture maps with different materials and / or textures are two different material texture maps.

[0075] The material texture map can include a standard material texture map and a non-standard material texture map. Exemplarily, Figure 1 is an example diagram of a standard material texture map. As Figure 1 shown, the material texture Figure 1 、the material texture Figure 2 、and the material texture map 3 are three standard material texture maps with different forms of expression. The material texture map 3 and the material texture Figure 4 are of the same form of expression and correspond to standard material texture maps of two material types respectively. The standard material texture map mentioned here is a material texture map presented in the form of a material sphere. A material sphere is a relatively complete display of a material texture map.

[0076] Exemplarily, Figure 2 is an example diagram of a non-standard material texture map. As Figure 2 shown, the material texture Figure 5 、the material texture Figure 6 、the material texture Figure 7 、and the material texture Figure 8 are non-standard material texture maps of four different materials. Exemplarily, the above non-standard material texture maps can be obtained, for example, by taking screenshots of an image including an object.

[0077] Material texture map library: The material texture map library can include material texture maps of multiple material types. In some embodiments, the material texture maps included in the material texture map library are mainly the above-mentioned standard material texture maps. Or, the material texture map library can also include the above-mentioned standard material texture maps and non-standard material texture maps. In some embodiments, the material texture map library can also be one that only includes non-standard material texture maps.

[0078] Hierarchical classification: Hierarchical classification refers to dividing an aggregate of objects to be classified into corresponding several hierarchical categories successively according to a selected attribute or feature as the classification criterion or classification mark, and arranging them into a hierarchical and gradually expanding classification system. The forms of expression of hierarchical classification are mainly major categories, middle categories, minor categories, detailed categories, etc. It is successively divided into several levels, and each level can also be divided into several categories. The same-level categories of the same branch can form a parallel relationship, and different-level categories can form a subordinate relationship.

[0079] Hard label: A hard label is used to characterize the material map category uniquely determined by the material map, that is, the probability that this material map belongs to this category is 100%.

[0080] Soft label: A soft label is used to represent the probabilities of a material map belonging to various material map categories. For example, the soft label of material map A can be used to represent that material map A has a 60% probability of being material map category 1, a 20% probability of being material map category 2, and a 20% probability of being material map category 3.

[0081] Exemplarily, Figure 3a is a schematic diagram of a 3D modeling scene. As Figure 3a shown, the electronic device can respond to the modeling operation triggered by the user and construct a 3D model framework (such as Figure 3a the cuboid shown therein) through 3D modeling software. Then, the electronic device can endow the 3D model framework with a target material map through this modeling software, so that the 3D model has visual performances such as color, texture, roughness, etc. corresponding to the target material map. Taking the above target material map as the material map shown in Figure 1 as an example, by endowing the cuboid 3D model framework shown in Figure 2 with this material map, Figure 3a it can make the surface of the cuboid have the same visual performances as the texture, color, specular reflectance, specular intensity, roughness, etc. presented by the material map. Figure 2 shown in Figure 2

[0082] During the modeling process, before endowing the 3D model framework with a target material map, the electronic device first needs to determine a target material map.

[0083] In the related art, the way to determine the target material map is mainly that the user distinguishes and searches for the required target material map among the massive material maps in the material map library. However, because the number of material maps in the material map library is large, the user may need to spend a long time to obtain the target material map they need. Moreover, because there are many small differences between material maps, for example, two material maps may have the same color and texture, and the differences lie in the visual performances such as the roughness and specular reflectance of the presented materials, which may lead to the user choosing the wrong target material map. Therefore, the above method has problems of low efficiency and poor accuracy.

[0084] ​The related art also proposes a method for searching for a target material map from a material map library based on the name of the material map. However, this method requires the user to clearly understand the names of all material maps in order to find the target material map they need. If the user has little knowledge of material maps, for example, the user may not know the names of various textures (such as lychee pattern, mass flow pattern, flowing water pattern, mountain pattern, etc.) and the names of materials (such as marble, leather, cotton cloth, silk, wool, etc.). Therefore, this method still has the problems of low efficiency and poor accuracy.

[0085] Considering that the reason for the above problems in the existing material map processing method is that it needs to rely on the user's ability to distinguish many material maps, therefore, this application proposes a method for automatically retrieving and obtaining the material map required by the user without the user having to distinguish the material map, so as to improve the efficiency and accuracy of obtaining the material map required by the user.

[0086] It should be understood that this application does not limit the application scenario of this material map processing method. For example, this material map processing method can be used in a modeling service as part of the modeling process. For example, the material map determined by this material map processing method can be used to render a 3D model framework as described above to obtain a 3D model. Or, this material map can also be used to render a 2D model framework to obtain a 2D model. Or, the above material map processing method can also be used in any scenario where a material map can be used other than the modeling service. For example, this material map processing method can be applied to a material map providing service. This material map providing service can output a material map that matches the material of the target image provided by the user according to the target image provided by the user.

[0087] In addition, it should be understood that this application does not limit the execution subject of this material map processing method. Optionally, the execution subject of this material map processing method can also be a material map processing system. It should be understood that this application does not limit whether this material map processing system can provide other services (such as the 3D modeling service described above) in addition to providing the material map processing service.

[0088] Figure 3b For the application scenario schematic diagram of a material map processing system provided by this application, as Figure 3bAs shown, in one embodiment, the texture mapping processing system can be fully deployed in a cloud environment. A cloud environment is an entity that provides cloud services to users using basic resources under the cloud computing model. The cloud environment includes a cloud data center and a cloud service platform. The cloud data center includes a large number of basic resources (including computing resources, storage resources, and network resources) owned by the cloud service provider. The computing resources included in the cloud data center can be a large number of electronic devices (such as servers). For example, taking the computing resources included in the cloud data center as servers running virtual machines, the texture mapping processing system can be independently deployed on the servers or virtual machines in the cloud data center, or can be distributedly deployed on multiple servers in the cloud data center, or distributedly deployed on multiple virtual machines in the cloud data center, or further distributedly deployed on the servers and virtual machines in the cloud data center.

[0089] As Figure 3b shown, the texture mapping processing system can, for example, be abstracted by the cloud service provider into a texture mapping processing service on the cloud service platform and provided to users. When using the texture mapping processing service, users can specify a target image through an application program interface (API) or a graphical user interface (GUI). The texture mapping processing system in the cloud environment receives the target image input by the user, performs the operation of texture mapping processing, and the texture mapping processing system returns to the user the automatically determined texture map that matches the target image through the API or GUI. This texture map can be downloaded or used online by the user to complete specific tasks (such as rendering a 3D model framework to obtain a three-dimensional model).

[0090] Figure 3c This is a schematic diagram of an application scenario of another texture mapping processing system provided by this application. The deployment of the texture mapping processing system provided by this application is relatively flexible. As Figure 3cAs shown, in another embodiment, the texture mapping processing system provided in this application can also be deployed distributively in different environments. The texture mapping processing system provided in this application can be logically divided into multiple parts, each part having different functions. Each part of the texture mapping processing system can be separately deployed in any two or three of the terminal electronic devices (located on the user side), the edge environment, and the cloud environment. The terminal electronic devices located on the user side can include, for example, at least one of the following: terminal servers, smartphones, laptops, tablets, personal desktop computers, etc. The edge environment is an environment including a set of edge electronic devices relatively close to the terminal electronic devices. The edge electronic devices include: edge servers, edge stations with computing power, etc. Each part of the texture mapping processing system deployed in different environments or devices collaborates to provide the function of automatic texture mapping processing for users. It should be understood that this application does not restrictively divide which parts of the texture mapping processing system are specifically deployed in what environment. In actual applications, it can be adaptively deployed according to the computing power of the terminal electronic devices, the resource occupancy of the edge environment and the cloud environment, or specific application requirements. Figure 3c It is a schematic diagram of an application scenario taking the texture mapping processing system separately deployed in the edge environment and the cloud environment as an example.

[0091] The texture mapping processing system can also be separately deployed on an electronic device in any environment (for example: separately deployed on an edge server in the edge environment). Figure 3d It is a schematic hardware structure diagram of the electronic device 10 on which the texture mapping processing system is deployed. Figure 3d The shown electronic device 10 includes a memory 11, a processor 12, and a communication interface 13. The memory 11, the processor 12, and the communication interface 13 are communicatively connected to each other. For example, the memory 11, the processor 12, and the communication interface 13 can be communicatively connected by means of a network connection. Alternatively, the above-mentioned electronic device 10 may further include a bus 14. The memory 11, the processor 12, and the communication interface 13 are communicatively connected to each other through the bus 14. Figure 3d It is the electronic device 10 in which the memory 11, the processor 12, and the communication interface 13 are communicatively connected to each other through the bus 14.

[0092] The memory 11 can be a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 11 can store a program. When the program stored in the memory 11 is executed by the processor 12, the processor 12 and the communication interface 13 are used to execute the method for the texture mapping processing system to automatically provide texture mapping for users.

[0093] The processor 12 may be a general-purpose processor (Central Processing Unit, CPU), a microprocessor, an Application Specific Integrated Circuit (ASIC), a graphics processing unit (GPU), or one or more integrated circuits.

[0094] The processor 12 may also be an integrated circuit chip with signal processing capabilities. In the implementation process, the functions of the texture mapping processing system of this application can be completed by the integrated logic circuit in the hardware of the processor 12 or instructions in the form of software. The above-mentioned processor 12 may also be a general-purpose processor, a digital signal processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field Programmable Gate Array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, and can implement or execute the various methods, steps, and logic block diagrams disclosed in the embodiments below of this application. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc. The steps of the method disclosed in combination with the embodiments below of this application can be directly embodied as being executed and completed by a hardware decoding processor, or executed and completed by a combination of hardware and software modules in the decoding processor. The software module may be located in a mature storage medium in the art such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, or an electrically erasable programmable memory, a register, etc. This storage medium is located in the memory 11, and the processor 12 reads the information in the memory 11 and combines its hardware to complete the functions of the texture mapping processing system of this application.

[0095] The communication interface 13 uses a transceiver module such as, but not limited to, a transceiver to implement communication between the electronic device 10 and other devices or communication networks. For example, a data set can be obtained through the communication interface 13.

[0096] When the above-mentioned electronic device 10 includes a bus 14, the bus 14 may include a path for transmitting information between various components of the electronic device 10 (for example, the memory 11, the processor 12, the communication interface 13).

[0097] The texture mapping processing method provided in this application determines a texture map that matches the material of the target image based on the material characteristics of the target image and the material characteristics of each texture map in the texture map library. Among them, the material characteristics of the target image and the material characteristics of each texture map in the texture map library are both obtained by feature extraction based on a feature extraction model.

[0098] Therefore, the technical solution for training the feature extraction model provided by the present application will be described in detail below in combination with specific embodiments. It should be understood that the execution subject of the feature extraction model training method may be the aforementioned material texture processing system, or any electronic device with processing functions independent of the material texture processing system. The following is a detailed description of the technical solution for training the feature extraction model with the execution subject of the feature extraction model training method being any electronic device with processing functions. The following specific embodiments may be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments.

[0099] Figure 4 It is a schematic flowchart of a method for training a feature extraction model provided by the present application. As Figure 4 shown, the method includes the following steps:

[0100] S101. Obtain an initial sample data set.

[0101] The initial sample data set may include: at least one sample data. Wherein, each sample data may include: a sample material texture map, and a hard label of the sample material texture map. The hard label is used to characterize the material type to which the sample material texture map belongs.

[0102] Optionally, the above sample material texture map may be a standard material texture map or a non-standard material texture map. Taking some of the sample material texture maps included in the initial sample data set as standard material texture maps and the rest as non-standard material texture maps as an example, it should be understood that the present application does not limit the quantity of the standard material texture maps and the non-standard material texture maps.

[0103] Exemplarily, taking the above material types including five material types as an example, the hard labels of the sample material texture maps may be as shown in Table 1 below:

[0104] Table 1

[0105] Sample material texture map Hard label Class 1 (1，0，0，0，0) Class 2 (0，1，0，0，0) Class 3 (0，0，1，0，0) Class 4 (0，0，0，1，0) Class 5 (0，0，0，0，1)

[0106] As shown in Table 1, taking the hard label (1, 0, 0, 0, 0) as an example, the hard label of the sample material texture map being (1, 0, 0, 0, 0) characterizes that the material type to which the first type of sample material texture map belongs is the first type of material type.

[0107] Optionally, the electronic device may receive the above initial sample data set input by the user through, for example, an API or a GUI. Or, the electronic device may also obtain the initial sample data set from a server or database storing the above initial sample data set, etc. Or, the initial sample data set may also be pre-stored in the electronic device.

[0108] S102. Convert the hard labels of the sample material texture maps in the initial sample dataset into soft labels to obtain a training sample dataset.

[0109] Among them, the soft label is used to represent the probability that the sample material texture map belongs to each material type, and this soft label is related to the sample hierarchical clustering result of the sample material texture map. Among them, the sample hierarchical clustering result includes: a color result and / or a texture clustering result. Each sample data in the training sample dataset may include: the above sample material texture map, and the soft label of the sample material texture map.

[0110] Exemplarily, still taking the above material types including five material types as an example, the soft label of the sample material texture map can be as shown in Table 2 below:

[0111] Table 2

[0112] Sample material texture map Soft label Class 1 (0.8，0.1，0.05，0.05，0) Class 2 (0.1，0.9，0，0，0) Class 3 (0，0.1，0.9，0，0) Class 4 (0.05，0.1，0.05，0.8，0) Class 5 (0，0.1，0.05，0.05，0.8)

[0113] As shown in Table 2, taking the soft label (0.8, 0.1, 0.05, 0.05, 0) as an example, the soft label of the sample material texture map being (0.8, 0.1, 0.05, 0.05, 0) represents that the probability of the first-class sample material texture map belonging to the first-class material type is 80%, belonging to the second-class material type is 10%, belonging to the third-class material type is 5%, belonging to the fourth-class material type is 5%, and belonging to the fifth-class material type is 0.

[0114] In some embodiments, the sample hierarchical clustering result may include: a first-level sample clustering result, and a second-level sample clustering result. The first sample clustering result obtained by clustering the sample material texture maps based on color, and the second sample clustering result obtained by clustering the sample material texture maps based on texture can both be used as the first-level sample clustering result of the sample hierarchical clustering result. In the first sample clustering result, the sample material texture maps with the same color can be clustered into one category. In the second sample clustering result, the sample material texture maps with the same texture can be clustered into one category. The third sample clustering result obtained by clustering the sample material texture maps based on color and texture can be used as the second-level sample clustering result in the sample hierarchical clustering result. In the third sample clustering result, the sample material texture maps with the same color and the same texture can be clustered into one category.

[0115] Optionally, in this application, the first-level sample clustering result may include: the first sample clustering result and the second sample clustering result. Or, in some embodiments, the first-level sample clustering result may also only include: one of the first sample clustering result and the second sample clustering result.

[0116] Optionally, an electronic device may, for example, adjust the hard labels of each sample material map in the initial sample dataset according to the above sample-level clustering results to obtain the soft labels of each sample material map, and further obtain a training sample dataset.

[0117] S103. Use the training sample dataset to train a neural network model to obtain a feature extraction model.

[0118] Among them, the feature extraction model is used to extract the material features of an image. It should be understood that the present application does not limit the above neural network model. Exemplarily, the neural network model may be, for example, a Residual Network (ResNet).

[0119] It should be understood that the present application does not limit how the electronic device uses the above training sample dataset to train the neural network model. Optionally, an existing neural network model training method may be referred to to train the neural network model to obtain a feature extraction model.

[0120] In this embodiment, by converting the hard labels of each sample material map into soft labels related to the sample-level clustering results of the sample material map and training the neural network model based on the soft labels of the sample material map, the situation where the material features of various sample material maps are relatively similar is considered. Compared with training the neural network model using hard labels, the training process of the present application is more in line with the actual judgment of the material features of various sample material maps, improving the accuracy of the feature extraction model obtained by training the neural network model based on the above training sample dataset, and thus improving the accuracy of material map processing based on the feature extraction model. Through the above clustering results based on color and texture, the probabilities of each material type to which the sample material map represented by the soft label belongs are consistent with human artistic intuition, improving the accuracy of determining the soft labels of the sample material map, further improving the accuracy of the feature extraction model obtained by training the neural network model based on the above training sample dataset, and thus further improving the accuracy of material map processing based on the feature extraction model.

[0121] The following will detail how the electronic device converts the hard labels of the sample material maps in the initial sample dataset into soft labels to obtain a training sample dataset:

[0122] As a possible implementation, taking the sample-level clustering results including: the first sample clustering result, the second sample clustering result, and the third sample clustering result as an example, the electronic device can first obtain the soft label of each sample texture map according to the first sample clustering result, the second sample clustering result, the third sample clustering result, and the hard label of each sample texture map, and then use each sample texture map and the soft label of each sample texture map to obtain the training sample data set.

[0123] Optionally, after obtaining the soft label of each sample texture map, the electronic device can directly use each sample texture map and the soft label of each sample texture map as the training sample data set.

[0124] Alternatively, the electronic device, for example, can also use the pre-processed sample texture map and the soft label of each sample texture map to obtain the training sample data set after pre-processing the sample texture map. Exemplarily, the above pre-processing can be operations such as translating and rotating the sample texture map to improve the diversity of the training sample data set, thereby improving the accuracy of the feature extraction model trained based on the training sample data set, and thus realizing the improvement of the accuracy of extracting the material features of the image for material texture map processing.

[0125] Through this implementation method, the electronic device can determine the soft label of each sample texture map based on the first sample clustering result corresponding to the primary color clustering of the sample texture map, the second sample clustering result corresponding to the primary texture clustering of the sample texture map, and the third sample clustering result corresponding to the secondary color and texture clustering of the sample texture map, fully considering the importance of color and texture in differentiating between sample texture maps, and improving the accuracy of determining the soft label of the sample texture map.

[0126] As another possible implementation, taking the sample-level clustering results including: the primary sample clustering result and the secondary sample clustering result, and the primary sample clustering result including one of the first sample clustering result and the second sample clustering result as an example, the specific implementation method for the electronic device to obtain the training sample data set according to the sample-level clustering result and the hard label of each sample texture map can refer to the above embodiment and will not be elaborated here.

[0127] Next, taking the sample-level clustering results including: the first sample clustering result, the second sample clustering result, and the third sample clustering result as an example, a detailed description will be given on how the electronic device obtains the soft label of each sample texture map according to the first sample clustering result, the second sample clustering result, the third sample clustering result, and the hard label of each sample texture map:

[0128] As a first possible implementation, Figure 5 is a schematic flowchart of a method for obtaining a soft label of a sample material map provided by this application. As Figure 5 shown, the method may include the following steps:

[0129] S201. For each sample material map, determine the initial soft label of the sample material map according to the hard label of the sample material map and a preset soft label probability distribution method.

[0130] Optionally, the above-mentioned preset soft label probability distribution method may be, for example, pre-stored in the electronic device by the user. Optionally, the soft label probability distribution methods corresponding to different types of sample material maps may be the same or different. Exemplarily, taking the hard label of the sample material map shown in Table 1 above as an example, assuming that the soft label probability distribution methods corresponding to different types of sample material maps are the same, and the soft label probability distribution method is: reduce the probability of 1 in the hard label to 0.8, then the initial soft label of the sample material map may be as shown in Table 3 below:

[0131] Table 3

[0132] Sample material texture map Initial soft label Class 1 (0.8，0，0，0，0) Class 2 (0，0.8，0，0，0) Class 3 (0，0，0.8，0，0) Class 4 (0，0，0，0.8，0) Class 5 (0，0，0，0，0.8)

[0133] If the soft label probability distribution methods corresponding to different types of sample material maps may be different, for example, assuming that the soft label probability distribution method corresponding to the 5th type of sample material map shown in Table 3 is different from that of the 1st - 4th types, and the soft label probability distribution method corresponding to the 5th type of sample material map is: reduce the probability of 1 in the hard label to 0.7, then the initial soft label of the 5th type of sample material map may be (0, 0, 0, 0, 0.7).

[0134] S202. Adjust the probabilities in the initial soft label of the sample material map according to the first sample clustering result, the second sample clustering result, and the third sample clustering result to obtain the soft label of the sample material map.

[0135] Optionally, the electronic device may first adjust the probabilities in the initial soft label of the sample material map according to the first sample clustering result to obtain the first initial soft label of the sample material map. Then, the electronic device may adjust the probabilities in the first initial soft label of the sample material map according to the second sample clustering result to obtain the second initial soft label of the sample material map. Then, the electronic device may adjust the probabilities in the second initial soft label of the sample material map according to the third sample clustering result to obtain the soft label of the sample material map. Or, the electronic device may also adjust the probabilities in the initial soft label of the sample material map in the order of the second sample clustering result, the first sample clustering result, and the third sample clustering result, which will not be elaborated herein in this application.

[0136] In this implementation manner, the hard label of the sample material map is processed through a preset soft label probability distribution method, and the initial soft label of the sample material map can be obtained. Based on the above first sample clustering result, second sample clustering result, and third sample clustering result, the probability in the initial soft label can be adjusted, so that the soft label of the sample material map conforms to the actual situation of mainly distinguishing materials according to color and texture, improving the accuracy of determining the soft label of the sample material map.

[0137] As a second possible implementation manner, for each sample material map, the electronic device may, for example, first adjust the probability in the hard label of the sample material map according to the above first sample clustering result, second sample clustering result, and third sample clustering result to obtain the first soft label of the sample material map. If the electronic device determines that the sum of the probability values in the first soft label is greater than 1, the electronic device may proportionally reduce the probability values so that the sum of the probability values in the first soft label is equal to 1, thereby obtaining the soft label of the sample material map.

[0138] Exemplarily, taking the hard label (1, 0, 0, 0, 0) shown in Table 1 above as an example, assume that the electronic device determines, according to the first sample clustering result, that the sample material map with the same color as the first-class sample material map is the second-class sample material map, determines, according to the second sample clustering result, that the sample material maps with the same texture as the first-class sample material map are the second-class and fourth-class sample material maps, and determines, according to the third sample clustering result, that the sample material map with the same color and texture as the first-class sample material map is the second-class sample material map. Optionally, according to the first sample clustering result, the electronic device may, for example, increase the probability that the sample map belongs to the second-class sample material map in the hard label (1, 0, 0, 0, 0) by a preset value, for example, increase it by 0.1, to obtain (1, 0.1, 0, 0, 0).

[0139] Similarly, according to the second sample clustering result, the electronic device can, for example, further increase the probabilities that the sample texture map in (1, 0.1, 0, 0, 0) belongs to the sample texture map of the second type and the fourth type by a preset value, such as 0.1, to obtain (1, 0.2, 0, 0.1, 0). Similarly, according to the third sample clustering result, the electronic device can, for example, further increase the probability that the sample texture map in (1, 0.2, 0, 0.1, 0) belongs to the sample texture map of the second type by a preset value, such as 0.1, to obtain (1, 0.3, 0, 0.1, 0). Since 1 + 0.3 + 0.1 = 1.4, the electronic device can divide each probability in (1, 0.3, 0, 0.1, 0) by 1.4 to proportionally reduce the probability values, so that the sum of the probability values in the first soft label is equal to 1, thereby obtaining the soft label of the sample texture map.

[0140] The following gives an exemplary description of how the electronic device adjusts the probabilities in the initial soft label of the sample texture map according to the above first sample clustering result, second sample clustering result, and third sample clustering result to obtain the soft label of the sample texture map:

[0141] Optionally, the electronic device can, for example, first adjust the probabilities in the initial soft label of the sample texture map according to the first sample clustering result to obtain the first soft label of the sample texture map.

[0142] In this implementation manner, as a possible implementation, the electronic device can, for example, first determine the X - type sample texture maps that have the same color as the sample texture map of the i - th type represented by the first sample clustering result. Here, both X and i are integers greater than or equal to 1. Then, the electronic device can adjust the probabilities that belong to the X - type material type and the i - th type material type in the initial soft label of the sample texture map of the i - th type according to the aforementioned preset soft label probability distribution method to obtain the first soft label of the sample texture map of the i - th type.

[0143] Exemplarily, taking the initial soft labels shown in Table 3 as an example, when i equals 1, assuming that the electronic device can determine, according to the first sample clustering result, that the sample material map with the same color as the material map of the first-class samples is the material map of the second-class samples, the electronic device can adjust the probability of the initial soft label of the material map of the first-class samples belonging to the second material type and the probability of belonging to the first material type according to the aforementioned preset soft label probability distribution method to obtain the first soft label of the material map of the first-class samples. In the example shown in Table 3, the preset soft label probability distribution method is: reducing the probability of 1 in the hard label to 0.8, then through 1 - 0.8 = 0.2, the electronic device can determine that through this 0.2, adjust the probability of the initial soft label of the material map of the first-class samples belonging to the second material type and the probability of belonging to the first material type. For example, the electronic device can evenly distribute this 0.2 to the probability of the initial soft label of the material map of the first-class samples belonging to the second material type and the probability of belonging to the first material type, and then obtain the first soft label of the material map of the first-class samples as (0.8 + 0.1, 0 + 0.1, 0, 0, 0), that is, (0.9, 0.1, 0, 0, 0).

[0144] When i equals any value in 2 - 5, the first soft labels of various sample material maps can be determined with reference to the method described in the above embodiments, which will not be elaborated here.

[0145] After the electronic device adjusts the probabilities in the initial soft labels of the sample material maps according to the first sample clustering result to obtain the first soft labels of the sample material maps, it can adjust the probabilities in the first soft labels of the sample material maps according to the second sample clustering result to obtain the second soft labels of the sample material maps.

[0146] In this implementation manner, as a possible implementation manner, the electronic device can, for example, first determine the Y-class sample material maps with the same texture as the material map of the i-class samples represented by the second sample clustering result. Where Y is an integer greater than or equal to 1. Then, the electronic device can adjust the probability of the first soft label of the material map of the i-class samples belonging to the Y material type and the probability of belonging to the i material type according to the preset soft label probability distribution method to obtain the second soft label of the material map of the i-class samples.

[0147] Exemplarily, when i equals 1, taking the first soft label of the first - type sample material texture map shown in the above - mentioned embodiment as (0.9, 0.1, 0, 0, 0) as an example, assuming that the electronic device can determine, according to the second - sample clustering result, that the sample material texture maps with the same texture as the first - type sample material texture map are of the second type and the fourth type. Then, the electronic device can adjust the probabilities of the first soft label of the first - type sample material texture map belonging to the second - type material type, the fourth - type material type, and the first - type material type according to a preset soft - label probability distribution method to obtain the second soft label of the first - type sample material texture map. Assuming that the preset soft - label probability distribution method is: subtracting 0.3 from the probability that the sample material texture map in the first soft label belongs to the first - type material type. Then the electronic device can determine that through this 0.3, it adjusts the probabilities of the first soft label of the first - type sample material texture map belonging to the second - type material type, the fourth - type material type, and the first - type material type. For example, the electronic device can evenly distribute this 0.3 to the probabilities of the first soft label belonging to the second - type material type, the fourth - type material type, and the first - type material type, and then obtain the second soft label of the first - type sample material texture map as (0.6 + 0.1, 0.1+0.1, 0, 0.1, 0), that is, (0.7, 0.2, 0, 0.1, 0).

[0148] When i equals any value in the range of 2 - 5, the method described in the above - mentioned embodiment can be referred to determine the second soft label of each type of sample material texture map, which will not be elaborated here.

[0149] After the electronic device adjusts the probabilities in the first soft label of the sample material texture map according to the second - sample clustering result to obtain the second soft label of the sample material texture map, it can adjust the probabilities in the second soft label of the sample material texture map according to the third - sample clustering result to obtain the soft label of the sample material texture map.

[0150] In this implementation manner, as a possible implementation, the electronic device can, for example, first determine the Z - type sample material texture maps whose color and texture are both the same as those of the i - type sample material texture map represented by the third - sample clustering result. Where Z is an integer greater than or equal to 1. Then, the electronic device can adjust the probabilities of the second soft label of the i - type sample material texture map belonging to the Z - type material type and the i - type material type according to a preset soft - label probability distribution method to obtain the soft label of the i - type sample material texture map.

[0151] Exemplarily, when i is equal to 1, taking the second soft label of the first type of sample material texture map exemplified in the above embodiment as (0.7, 0.2, 0, 0.1, 0) as an example, assuming that the electronic device can determine, according to the third sample clustering result, that the sample material texture map with the same color and texture as the first type of sample material texture map is the second type of sample material texture map, then the electronic device can adjust the probability belonging to the second type of material type and the probability belonging to the first type of material type in the second soft label of the first type of sample material texture map according to the preset soft label probability distribution method, so as to obtain the soft label of the first type of sample material texture map. Assuming that the preset soft label probability distribution method is: subtracting 0.2 from the probability that the sample material texture map in the second soft label belongs to the first type of material type, then the electronic device can determine that through this 0.2, the probability belonging to the second type of material type and the probability belonging to the first type of material type in the second soft label of the first type of sample material texture map are adjusted. For example, the electronic device can evenly distribute this 0.2 to the probability belonging to the second type of material type and the probability belonging to the first type of material type in the second soft label, and then obtain the soft label of the first type of sample material texture map as (0.5 + 0.1, 0.2 + 0.1, 0, 0.1, 0), that is, (0.6, 0.3, 0, 0.1, 0).

[0152] When i is equal to any value from 2 to 5, the soft labels of various types of sample material texture maps can be determined with reference to the method described in the above embodiment, which will not be elaborated here.

[0153] The following will elaborate in detail on how the electronic device obtains the above first sample clustering result, second sample clustering result, and third sample clustering result:

[0154] As a first possible implementation manner, the electronic device can, for example, first obtain the first sample clustering result and the second sample clustering result, and then obtain the third sample clustering result according to the first sample clustering result and the second sample clustering result.

[0155] Exemplarily, Figure 6 is a flowchart of a method for obtaining a sample hierarchical clustering result provided by this application. As Figure 6 shown, for sample material texture maps of multiple material types, the user can manually determine offline whether the colors of the sample material texture maps of each material type are the same, and whether the textures are the same. The electronic device can respond to the user's classification operation on each sample material texture map, classify the sample material texture maps of each material type with the same color as one category in the first sample clustering result, and classify the sample material texture maps of each material type with the same texture as one category in the second sample clustering result, so as to obtain the first sample clustering result and the second sample clustering result.

[0156] Exemplarily, in the first sample clustering result, sample material texture maps of each material type with the same color can be placed in the same folder, or the identifiers of sample material texture maps of each material type with the same color can be the same. In the second sample clustering result, sample material texture maps with the same texture can be placed in the same folder, or the identifiers of sample material texture maps of each material type with the same texture can be the same. Exemplarily, the identifier of the above sample material texture map can be, for example, the name of the sample material texture map, or the field at the target position in the name of the sample material texture map.

[0157] Then, the electronic device can receive the first sample clustering result and the second sample clustering result input by the user. After obtaining the first sample clustering result and the second sample clustering result, exemplarily, if the electronic device determines that the sample material texture map of material type 1 is not only in the same folder as the sample material texture map of material type 2 in the first sample clustering result, but also in the same folder as the sample material texture map of material type 2 in the second sample clustering result, then the electronic device can determine that the sample material texture map of material type 1 has the same color and texture as the sample material texture map of material type 2, and the electronic device can cluster the sample material texture map of material type 1 and the sample material texture map of material type 2 into one category, and further obtain the third sample clustering result.

[0158] As a second possible implementation manner, the above third sample clustering result can also be the third sample clustering result calibrated offline by the user according to the first sample clustering result and the second sample clustering result. The electronic device can receive the first sample clustering result, the second sample clustering result, and the third sample clustering result input by the user through, for example, an API or a GUI.

[0159] Taking the above neural network model as the ResNet network model as an example, an exemplary description will be given below on how the electronic device uses the training sample data set to train the neural network model to obtain the feature extraction model:

[0160] Exemplarily, Figure 7 This is a schematic structural diagram of a neural network model provided by the present application. The neural network model can include a material feature extraction part and a non-material feature extraction part. The electronic device can perform feature extraction on the sample material texture map through the material feature extraction part; through the non-material feature extraction part, determine the probabilities of the sample material texture maps predicted by the material features belonging to each material type.

[0161] Exemplarily, as Figure 7As shown, the input of the ResNet network model is a sample material texture map. The material feature extraction part of the ResNet network model may include three convolutional layers, one pooling layer, and two fully connected layers. The non-material feature extraction part may include a Softmax activation function layer. The electronic device can extract features from the sample material texture map through the convolutional layer, pooling layer, and fully connected layer to obtain the material features of the sample material texture map and output them to the Softmax activation function layer. The electronic device can output the probability p of each material type to which the sample material texture map predicted by the neural network model belongs through the Softmax activation function according to the material features of the above sample material texture map.

[0162] Optionally, the loss function used to train the neural network model may be shown, for example, by the following formula (1):

[0163]

[0164] Where CE() represents the cross-entropy loss function, Label j represents the soft label of the j-th sample material texture map, and N represents the number of sample material texture maps in a training batch. Predict j represents the probability of each material type to which the j-th sample material texture map predicted by the neural network model belongs.

[0165] It should be understood that the above cross-entropy loss function is only a possible implementation of the loss function provided in this application. This application does not limit the loss function used to train the neural network model.

[0166] Optionally, after the electronic device finishes training the neural network model and obtains the trained neural network model, it can respond to the user's operation of deleting the non-material feature extraction part in the trained neural network model, delete the non-material feature extraction part in the trained neural network model, and retain the feature extraction part in the trained neural network model as a feature extraction model. Through the above method, the feature extraction model can be used to extract features from the image input to the model to obtain the material features of the input image.

[0167] In this embodiment, as described above, training the neural network model based on the soft label of the sample material texture map can improve the accuracy of obtaining the feature extraction model. Although there are also some existing methods for training neural networks based on soft labels of images, however, in the prior art, the main role of training the network model based on soft labels is to reduce the scale of the neural network to improve the network training efficiency. That is to say, the role of the existing network model training based on soft labels is to improve the network training efficiency, which is different from the role of this application for improving the accuracy of network training.

[0168] In addition, there are also some image feature extraction methods based on deep learning in the prior art. However, the existing image feature extraction methods mainly extract features from images including obvious objects (such as cars, people, animals, etc.), while the material texture map of this application does not have obvious objects. Therefore, the existing image feature extraction methods based on deep learning are also not applicable to this application.

[0169] After obtaining the feature extraction model through the feature extraction model training method described in any of the foregoing embodiments, the material texture map processing system can perform material texture map processing based on this feature extraction model.

[0170] The technical solution of the material texture map processing provided by this application will be described in detail below in conjunction with specific embodiments. These specific embodiments below can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments.

[0171] Figure 8 It is a schematic flowchart of a material texture map processing method provided by this application. As Figure 8 shown, the method includes the following steps:

[0172] S301. Obtain a target image.

[0173] In some embodiments, a target image may include one type of material, that is, a target image may include one color and one texture. In this implementation manner, before obtaining the target image, the material texture map processing system may, for example, also output a prompt message for prompting the user to upload a target image including only one type of material, so as to improve the accuracy and efficiency of the material texture map processing system in obtaining the target image, and further improve the accuracy of the material texture map processing system in performing material texture map processing.

[0174] In some embodiments, the target image may also be an image after preprocessing such as image enhancement, etc., so as to further improve the accuracy of the material texture map processing system in performing material texture map processing based on this target image.

[0175] Optionally, the material texture map processing system may, for example, receive the target image input by the user through an API or a GUI, etc.

[0176] S302. Input the target image into the feature extraction model to obtain the material features of the target image.

[0177] Among them, the feature extraction model is obtained by using the method described in any of the foregoing embodiments.

[0178] S303. Obtain K material maps that match the material of the target image according to the material characteristics of the target image, the material characteristics of each material map in the material map library extracted by using the feature extraction model, and the hierarchical clustering results of the material maps in the material map library based on color and texture.

[0179] Wherein, K is an integer greater than or equal to 1.

[0180] In some embodiments, the above hierarchical clustering results may include: a first-level clustering result and a second-level clustering result. The first clustering result obtained by clustering the material maps based on color and the second clustering result obtained by clustering the material maps based on texture may both be used as the first-level clustering results of the hierarchical clustering results. In the first clustering result, the material maps with the same color may be clustered into one category. In the second clustering result, the material maps with the same texture may be clustered into one category. The third clustering result obtained by clustering the material maps based on color and texture may be used as the second-level clustering result in the hierarchical clustering results. In the third clustering result, the material maps with the same color and the same texture may be clustered into one category.

[0181] Optionally, in this application, the first-level clustering result may include: the first clustering result and the second clustering result. Or, in some embodiments, the first-level clustering result may also only include: one of the first clustering result and the second clustering result.

[0182] The material characteristics of each material map in the material map library extracted by using the feature extraction model and the hierarchical clustering results of the material maps in the material map library based on color and texture may be, for example, pre-stored in the material map processing system. It should be understood that the execution entity for performing the operation of extracting the material characteristics of each material map in the material map library by using the feature extraction model may be the same as or different from the execution entity for performing the material map processing method, and this application does not limit this.

[0183] In this embodiment, the feature extraction model trained based on the soft labels of the sample material maps is used to extract the features of the target image, which improves the accuracy of obtaining the material characteristics of the target image, and further improves the accuracy of material map processing based on the material characteristics of the target image. Through the material characteristics of the target image, the material characteristics of each material map in the material map library extracted by using the feature extraction model, and the hierarchical clustering results of the material maps in the material map library based on color and texture, K material maps that match the material of the target image can be determined. Through the above method, the automatic determination of the material maps that match the material of the target image is realized, which improves the efficiency and accuracy of determining the material maps required by the user compared with the existing method that relies on the user's discrimination ability of numerous material maps.

[0184] In addition, the method of determining K material maps that match the material of the target image based on the hierarchical clustering results of the material maps based on color and texture as described above fully considers the importance of texture and color in material map retrieval, further improving the accuracy of determining the material maps required by the user. In addition, the execution of the material map processing method provided in this application only requires the use of a feature extraction model trained based on a neural network model, thus further improving the efficiency of material map processing and reducing the training cost.

[0185] The following will detail how the material map processing system obtains K material maps that match the material of the target image based on the material characteristics of the target image, the material characteristics of each material map in the material map library extracted using the feature extraction model, and the hierarchical clustering results of the material maps in the material map library based on color and texture:

[0186] Figure 9 It is a flowchart of a method for K material maps that match the material of the target image provided in this application. As Figure 9 shown, as a first possible implementation, the foregoing step S303 may include the following steps:

[0187] S401. Obtain the similarity between the target image and each material map based on the material characteristics of the target image and the material characteristics of each material map in the material map library.

[0188] Optionally, the material map processing system may calculate the similarity between the material characteristics of the target image and the material characteristics of each material map in the material map library through any existing similarity calculation method such as the Euclidean distance similarity calculation method or the cosine similarity calculation method, as the similarity between the target image and each material map.

[0189] S402. Obtain the top N material maps in the order of similarity from large to small.

[0190] Wherein, N is an integer greater than or equal to 2.

[0191] The greater the similarity between the target image and the material map, the closer the material type included in the target image is to the material type corresponding to the material map. The smaller the similarity between the target image and the material map, the greater the difference between the material type included in the target image and the material type corresponding to the material map. Therefore, the material map processing system may obtain the top N material maps from the material map library in the order of similarity from large to small.

[0192] S403. Obtain the clustering scores of the N material maps based on the hierarchical clustering results of the material maps in the material map library based on color and texture.

[0193] The clustering scores of the N texture maps are obtained based on the hierarchical clustering results of the texture maps in the texture map library based on color and texture. Therefore, the clustering score of each texture map takes into account the possibility that the texture maps "belonging to the same cluster as this texture map" are texture maps that match the material of the target image.

[0194] Taking one of the first-level clustering results including the first clustering result and the second clustering result as an example, the texture map processing system can calculate the clustering scores of the N texture maps based on this first-level clustering result and the third clustering result. Taking the hierarchical clustering result including the first clustering result, the second clustering result, and the third clustering result as an example, the texture map processing system can calculate the clustering scores of the N texture maps based on the first clustering result, the second clustering result, and the third clustering result.

[0195] S404. Obtain the top K texture maps as the texture maps that match the material of the target image in descending order of the clustering scores.

[0196] The larger the clustering score of a texture map, the more it indicates that this texture map is more matched with the color and texture materials of the target image compared to other texture maps. Therefore, the texture map processing system can obtain the top K texture maps as the texture maps that match the material of the target image in descending order of the clustering scores.

[0197] In this embodiment, by determining the similarity between the target image and each texture map according to the material characteristics of the target image and the material characteristics of each texture map in the texture map library, and first determining the top N texture maps with higher similarity according to this similarity. Through the above method, the texture map processing system does not need to calculate the clustering scores of all texture maps, further improving the efficiency of texture map retrieval. Through the above hierarchical clustering result, the clustering scores of the N texture maps are determined, and according to the order of the sizes of these clustering scores, K texture maps are determined. Through the above method, the importance of texture and color in texture map retrieval is fully considered, and the top N texture maps with higher similarity are re-sorted according to the texture and color characteristics of the texture maps, further improving the accuracy of texture map retrieval.

[0198] As a second possible implementation manner, after the texture map processing system obtains the similarity between the target image and each texture map, it can use at least one texture map with a similarity greater than a preset similarity threshold as the initial texture map. Then, the texture map processing system can obtain the clustering scores of the at least one initial texture map according to the hierarchical clustering result of the texture maps in the texture map library based on color and texture. Then, the texture map processing system can use the initial texture map with a clustering score greater than the preset clustering score threshold as the target texture map.

[0199] If the number of the above-mentioned initial material maps is less than or equal to K, optionally, the material map processing system may use the at least one initial material map as the material map matching the target image material. If the number of the above-mentioned initial material maps is greater than K and the number of the target material maps is less than or equal to K, optionally, the material map processing system may use the target material map as the material map matching the target image material. If the number of the above-mentioned target material maps is greater than K, optionally, the material map processing system may obtain the top K material maps from the multiple target material maps as the material maps matching the target image material in the order of the clustering scores from large to small.

[0200] Taking the hierarchical clustering result including the above-mentioned first clustering result, second clustering result, and third clustering result as an example, the following details how the material map processing system obtains the clustering scores of N material maps based on the hierarchical clustering result of the material maps in the material map library according to color and texture:

[0201] As a possible implementation manner, the foregoing step S403 may include the following steps:

[0202] Step 1: The material map processing system may obtain the initial clustering scores of the N material maps according to the similarity sorting of the foregoing N material maps.

[0203] Exemplarily, the material map processing system may set the initial clustering score of the r-th material map in the similarity sorting from large to small to be 1 / r. Taking the above-mentioned N equal to 20 as an example, in this implementation manner, the material map processing system may obtain that the initial clustering score of the 1st material map in the similarity sorting from large to small is 1, the initial clustering score of the 3rd material map is 1 / 2, the initial clustering score of the 3rd material map is 1 / 3, and so on, and the initial clustering score of the 20th material map is 1 / 20.

[0204] Step 2: The material map processing system may obtain the first clustering scores of the N material maps according to the initial clustering scores of the foregoing N material maps and the first clustering result.

[0205] Exemplarily, according to the above first clustering result, the texture map processing system can determine multiple texture maps with the same color from the above N texture maps. Then, the texture map processing system can determine the first clustering score of the multiple texture maps according to the initial clustering scores of the multiple texture maps. Among them, the first clustering scores of each texture map in the multiple texture maps with the same color can be the same. Exemplarily, for any one of the multiple texture maps with the same color, the texture map processing system can use the sum of the initial clustering scores of the multiple texture maps with the same color as the first clustering score of this texture map.

[0206] For example, still taking the above N equal to 20 as an example, assuming that the texture map processing system determines according to the above first clustering result that the colors of the 1st, 4th, 8th, and 16th texture maps among the 20 texture maps are the same in the sorting from large to small similarity, then the texture map processing system can make the first clustering scores of the 1st, 4th, 8th, and 16th texture maps all equal to: 1 / 1 + 1 / 4 + 1 / 8 + 1 / 16 = 23 / 16. Referring to the above method, the texture map processing system can determine the first clustering scores corresponding to the other texture maps among the 20 texture maps.

[0207] Step 3: The texture map processing system can obtain the second clustering scores of the N texture maps according to the initial clustering scores of the N texture maps and the second clustering result.

[0208] Exemplarily, according to the above second clustering result, the texture map processing system can determine multiple texture maps with the same texture from the above N texture maps. Then, the texture map processing system can determine the second clustering score of the multiple texture maps according to the initial clustering scores of the multiple texture maps. Among them, the second clustering scores of each texture map in the multiple texture maps with the same texture can be the same. Exemplarily, for any one of the multiple texture maps with the same texture, the texture map processing system can use the sum of the initial clustering scores of the multiple texture maps with the same texture as the second clustering score of this texture map.

[0209] For example, still taking the above N equal to 20 as an example, assuming that the texture map processing system determines according to the above second clustering result that the textures of the 1st and 4th texture maps among the 20 texture maps are the same in the sorting from large to small similarity, then the texture map processing system can make the second clustering scores of the 1st and 4th texture maps all equal to: 1 / 1 + 1 / 4 = 5 / 4. Referring to the above method, the texture map processing system can determine the second clustering scores corresponding to the other texture maps among the 20 texture maps.

[0210] Step 4: Obtain the third clustering scores of the N texture maps according to the initial clustering scores of the N texture maps and the third clustering result.

[0211] Exemplarily, according to the above third clustering result, the material texture processing system can determine, from the above N material textures, a plurality of material textures with the same color and texture. Then, the material texture processing system can determine the third clustering score of the plurality of material textures according to the initial clustering scores of the plurality of material textures. Among them, the third clustering scores of the material textures in the plurality of material textures with the same color and texture can be the same. Exemplarily, for any one of the plurality of material textures with the same color and texture, the material texture processing system can use the sum of the initial clustering scores of the plurality of material textures with the same color and texture as the third clustering score of the material texture.

[0212] For example, still taking the above N equal to 20 as an example, assuming that the material texture processing system determines according to the above third clustering result that the 1st and 4th material textures in the 20 material textures sorted from large to small in similarity have the same color and texture, then the material texture processing system can make the third clustering scores of the 1st and 4th material textures both equal to: 1 / 1 + 1 / 4 = 5 / 4. Referring to the above method, the material texture processing system can determine the third clustering scores corresponding to the other material textures in the 20 material textures.

[0213] It should be understood that the present application does not limit the order in which the material texture processing system executes the above steps 2, 3, and 4. For example, the material texture processing system can first execute step 2, then execute step 3, and then execute step 4. Or, the material texture processing system can also first execute step 3, then execute step 4, and then execute step 2.

[0214] Step 5: Obtain the clustering scores of the N material textures according to the initial clustering scores, the first clustering scores, the second clustering scores, and the third clustering scores of the N material textures.

[0215] Optionally, for any one of the N material textures, the material texture processing system can directly use the sum of the initial clustering score, the first clustering score, the second clustering score, and the third clustering score as the clustering score of the material texture.

[0216] Or, for any one of the N material textures, the material texture processing system can also use the weighted sum of the initial clustering score, the first clustering score, the second clustering score, and the third clustering score as the clustering score of the material texture.

[0217] Among them, the weights corresponding to each item in the initial clustering score, the first clustering score, the second clustering score, and the third clustering score can be pre-stored by the user in the texture mapping processing system, for example. The user can adjust the values of the weights according to their own needs. For example, if the texture mapping that the user needs to obtain needs to match the target image more in terms of color, the user can increase the weight corresponding to the first clustering score to improve the accuracy of the K texture mappings determined by the texture mapping processing system in terms of color. If the texture mapping that the user needs to obtain needs to match the target image more in terms of texture, the user can increase the weight corresponding to the second clustering score to improve the accuracy of the K texture mappings determined by the texture mapping processing system in terms of texture. Through the above method, the flexibility and accuracy of determining K texture mappings by the texture mapping processing system are improved, and the user experience is also improved.

[0218] In this embodiment, according to the similarity ranking of the texture mappings, the initial clustering score of the texture mappings is determined. According to the first clustering result based on color clustering, the second clustering result based on texture clustering, and the third clustering result based on color and texture clustering, the first clustering score, the second clustering score, and the first clustering score of the texture mappings are respectively determined. Based on the above initial clustering score, the first clustering score, the second clustering score, and the third clustering score, the clustering scores of the N texture mappings not only consider the similarity between the target image and the texture mappings, but also combine the similarity of color and texture between the texture mappings, improving the accuracy of determining the clustering scores of the N texture mappings. Therefore, the accuracy of determining K texture mappings based on the clustering scores of the N texture mappings is further improved.

[0219] Taking the hierarchical clustering result including the above first clustering result and the second clustering result as an example, how the texture mapping processing system obtains the clustering scores of the N texture mappings according to the hierarchical clustering result can refer to the method described in the above embodiment, and will not be elaborated here.

[0220] Furthermore, as a possible implementation manner, after the texture mapping processing system obtains K texture mappings that match the material of the target image, it can also output the K texture mappings so that the user can view the K texture mappings.

[0221] Exemplarily, Figure 10 is a schematic diagram of the interface of a texture mapping processing system provided by this application. As Figure 10 shown, the texture mapping processing system can receive the target image input by the user through the model building interface, for example. Then, the texture mapping processing system can respond to the request of the user to obtain "K texture mappings that match the material of the target image" and output the K texture mappings. For example, the user can click as Figure 10The "Obtain Material Map" control on the model building interface as shown triggers this request. In response to this request, the material map processing system can output K material maps through the model building interface.

[0222] It should be understood that this application does not limit whether the model building interface further includes other content. Exemplarily, as Figure 10 shown, the model building interface may further include a model framework, for example.

[0223] As a possible implementation, after the material map processing system obtains K material maps that match the target image material, the material map processing system can first determine the target material map from the K material maps, and then use the target material map to render the model framework of the target object to obtain the model of the target object. Then, the material map processing system can output the model of the target object.

[0224] Optionally, the model framework of the above target object may be a three-dimensional model framework or a two-dimensional model framework, and this application does not limit this. Optionally, the above target object can be any object, such as a sofa, a table, etc.

[0225] In some embodiments, the material map processing system can, for example, receive the model framework of the above target object input by the user.

[0226] In some embodiments, the material map processing system can build the model framework of the target object before using the target material map to render the model framework of the target object to obtain the model of the target object. Optionally, the specific implementation of the material map processing system for building the model framework of the target object can refer to existing model building methods, which will not be elaborated here. Through this implementation, the material map processing system can, after building the model framework of the target object, render the model framework of the target object through the target material map, improving the efficiency of the material map processing system from building the model framework of the target object to obtaining the model of the target object, and improving the user experience.

[0227] Optionally, the material map processing system can, for example, use the material map with the highest clustering score among the above K material maps as the target material map. Or, the material map processing system can also randomly determine a material map from the K material maps as the target material map. Or, after the material map processing system obtains K material maps that match the target image material, it can output the K material maps. Then, the material map processing system can determine the target material map from the K material maps according to the user selection instruction. For example, as above Figure 10Taking the model building interface shown as an example, the material texture processing system can respond to the user's operation of clicking on the first material texture, and use this first material texture as the target material texture.

[0228] It should be understood that this application does not limit how the material texture processing system uses the target material texture to render the model framework of the target object. Optionally, the method of using the material texture to render the model of the target object by referring to the existing material texture processing system can be used, which will not be elaborated here.

[0229] Optionally, the material texture processing system can output the model of the above target object through a display device. Optionally, this display device can be the display device of the material texture processing system, or a display device connected to the material texture processing system. Or, the material texture processing system can also send the model of the target object to other devices.

[0230] As another possible implementation, taking the above K equal to 1 as an example, after the material texture processing system obtains 1 material texture that matches the target image material, it can directly use this material texture as the target material texture, and use this material texture to render the model framework of the target object, obtain the model of the target object and output it.

[0231] In this embodiment, by rendering the model framework of the target object with the target material texture determined from the K material textures, the model of the target object can be obtained, improving the accuracy of obtaining the model of the target object. By outputting the model of the target object, the user can view the model of the target object with higher accuracy and more in line with the material required by the user, improving the user experience.

[0232] Taking the above hierarchical clustering results including: the first-level clustering result, and, the second-level clustering result, and the first-level clustering result including: the first clustering result obtained by clustering the material textures based on color, and, the second clustering result obtained by clustering the material textures based on texture, and the second-level clustering result including: the third clustering result obtained by clustering the material textures based on color and texture as an example, as a possible implementation, this application also provides a material texture processing method, which may include the following steps:

[0233] Step 1: Obtain the target image input by the user.

[0234] Step 2: Input the target image into the feature extraction model to obtain the material features of the target image.

[0235] Step 3: According to the material features of the target image and the material features of each material texture in the material texture library extracted by using the feature extraction model, obtain the similarity between the target image and each material texture.

[0236] Step 4: Obtain the initial clustering scores of the top N texture maps in descending order of similarity.

[0237] Step 5: Obtain the first clustering scores of the N texture maps based on the initial clustering scores of the N texture maps and the first clustering result; obtain the second clustering scores of the N texture maps based on the initial clustering scores of the N texture maps and the second clustering result; obtain the third clustering scores of the N texture maps based on the initial clustering scores of the N texture maps and the third clustering result.

[0238] Step 6: For any one of the N texture maps, calculate the weighted sum of the initial clustering score, the first clustering score, the second clustering score, and the third clustering score as the clustering score of this texture map.

[0239] Step 7: Obtain the top K texture maps as the texture maps that match the target image material in descending order of clustering scores.

[0240] Step 8: Output the K texture maps.

[0241] Step 9: In response to the texture map selected by the user, determine the target texture map from the K texture maps.

[0242] Step 10: Render the model framework of the target object using the target texture map to obtain the model of the target object.

[0243] Step 11: Output the model of the target object.

[0244] The following Table 4 shows an example of the experimental results for testing the accuracy rate of the texture map processing method provided in this application:

[0245] Table 4

[0246] Experimental component comparison Base model + Soft label loss function + Re-ranking Total extraction points Top1 recall rate 62.33％ 65.72％ 65.72％ 3.39 Top2 recall rate 65.64％ 68.24％ 72.53％ 6.89 Top3 recall rate 66.58％ 70.39％ 83.46％ 16.88 Top4 recall rate 70.44％ 74.77％ 87.31％ 16.87

[0247] When testing the accuracy rate of the texture map processing method provided in this application, the texture maps in the test dataset can be used as the target images. For each texture map, for example, 5 pictures with different illuminations and different angles can be taken. As shown in Table 4, the Top1 recall rate represents the recall rate corresponding to this texture map processing method when K is equal to 1. The Top2 recall rate represents the recall rate corresponding to this texture map processing method when K is equal to 2. The Top3 recall rate represents the recall rate corresponding to this texture map processing method when K is equal to 3. The Top4 recall rate represents the recall rate corresponding to this texture map processing method when K is equal to 4.

[0248] As shown in Table 4, for any row recall rate, the accuracy of the feature extraction model obtained by training the neural network model based on the soft label loss function provided by this application is higher than that of the basic model (for example, the feature extraction model obtained without training the model based on soft labels). After reordering the N texture maps based on the hierarchical clustering results provided by this application, the accuracy of the obtained K texture maps is improved compared to the texture map processing method without the above reordering.

[0249] Figure 11 The following is a schematic structural diagram of a feature extraction model training device provided by this application. As Figure 11 shown, the device includes: an acquisition module 51, a processing module 52, and a training module 53. Among them,

[0250] The acquisition module 51 is used to acquire an initial sample data set. Among them, the initial sample data set includes: at least one sample data, and each sample data includes: a sample texture map, and a hard label of the sample texture map; the hard label is used to characterize the material type to which the sample texture map belongs.

[0251] The processing module 52 is used to convert the hard label of the sample texture map in the initial sample data set into a soft label to obtain a training sample data set. Among them, the soft label is used to characterize the probability that the sample texture map belongs to each material type; the soft label is related to the sample hierarchical clustering result of the sample texture map, and the sample hierarchical clustering result includes: a color result and / or a texture clustering result.

[0252] The training module 53 is used to train a neural network model using the training sample data set to obtain a feature extraction model. Among them, the feature extraction model is used to extract the material features of an image.

[0253] Taking the sample hierarchical clustering result including: a first sample clustering result obtained by clustering the sample texture maps based on color, a second sample clustering result obtained by clustering the sample texture maps based on texture, and a third sample clustering result obtained by clustering the sample texture maps based on color and texture as an example, optionally, the processing module 52 is specifically used to obtain the soft label of each sample texture map according to the first sample clustering result, the second sample clustering result, the third sample clustering result, and the hard label of each sample texture map; use each sample texture map and the soft label of each sample texture map to obtain the training sample data set. Among them, both the first sample clustering result and the second sample clustering result are first-level sample clustering results in the sample hierarchical clustering result, and the third sample clustering result is a second-level sample clustering result in the sample hierarchical clustering result.

[0254] Optionally, the processing module 52 is specifically configured to determine an initial soft label of each sample material map according to the hard label of the sample material map and a preset soft label probability distribution method; and adjust the probability in the initial soft label of the sample material map according to the first sample clustering result, the second sample clustering result, and the third sample clustering result to obtain the soft label of the sample material map.

[0255] Optionally, the obtaining module 51 is further configured to obtain the first sample clustering result and the second sample clustering result; and obtain the third sample clustering result according to the first sample clustering result and the second sample clustering result.

[0256] The feature extraction model training device provided in this application is used to execute the foregoing embodiments of the feature extraction model training method, and its implementation principle and technical effects are similar, which will not be elaborated here.

[0257] Figure 12 It is a schematic structural diagram of a material map processing device provided in this application. As Figure 12 shown, the device includes: an obtaining module 61 and a processing module 62. Among them,

[0258] The obtaining module 61 is configured to obtain a target image.

[0259] The processing module 62 is configured to input the target image into a feature extraction model to obtain the material feature of the target image; and obtain K material maps that match the material of the target image according to the material feature of the target image, the material features of each material map in the material map library extracted by using the feature extraction model, and the hierarchical clustering result of the material maps in the material map library based on color and texture. Wherein, the feature extraction model is obtained by using any of the foregoing feature extraction model training methods; K is an integer greater than or equal to 1.

[0260] Optionally, the processing module 62 is specifically configured to obtain the similarity between the target image and each material map according to the material feature of the target image and the material features of each material map in the material map library; obtain the first N material maps in the order of similarity from large to small; obtain the clustering scores of the N material maps according to the hierarchical clustering result of the material maps in the material map library based on color and texture; and obtain the first K material maps in the order of clustering scores from large to small as the material maps that match the material of the target image. Wherein, N is an integer greater than or equal to 2.

[0261] Taking the hierarchical clustering results including: the first clustering result obtained by clustering material maps based on color, the second clustering result obtained by clustering material maps based on texture, and the third clustering result obtained by clustering material maps based on color and texture as an example, optionally, the processing module 62 is specifically configured to obtain the initial clustering scores of N material maps according to the similarity ranking of the N material maps; obtain the first clustering scores of the N material maps according to the initial clustering scores of the N material maps and the first clustering result; obtain the second clustering scores of the N material maps according to the initial clustering scores of the N material maps and the second clustering result; obtain the third clustering scores of the N material maps according to the initial clustering scores of the N material maps and the third clustering result; and obtain the clustering scores of the N material maps according to the initial clustering scores, the first clustering scores, the second clustering scores, and the third clustering scores of the N material maps. Wherein, the first clustering result and the second clustering result are both first-level clustering results in the hierarchical clustering result, and the third clustering result is a second-level clustering result in the hierarchical clustering result.

[0262] Optionally, the apparatus may further include an output module 63, configured to output the K material maps after obtaining the K material maps that match the material of the target image.

[0263] Optionally, the processing module 62 is further configured to determine a target material map from the K material maps after obtaining the K material maps that match the material of the target image; use the target material map to render the model framework of the target object to obtain the model of the target object. The output module 63 is further configured to output the model of the target object.

[0264] Optionally, the processing module 62 is further configured to construct the model framework of the target object before using the target material map to render the model framework of the target object to obtain the model of the target object.

[0265] The material map processing apparatus provided in this application is used to execute the foregoing embodiments of the material map processing method, and its implementation principle and technical effects are similar, and will not be described in detail here.

[0266] This application also provides an electronic device 10 as Figure 3d shown. The processor 12 in the electronic device 10 reads a set of computer instructions stored in the memory 11 to execute the foregoing feature extraction model training method or material map processing method.

[0267] The present application also provides a computer-readable storage medium, which may include: various media that can store program codes, such as USB flash drives, mobile hard disks, read-only memories (ROM, Read-Only Memory), random access memories (RAM, Random Access Memory), magnetic disks, or optical discs. Specifically, the computer-readable storage medium stores program instructions, and the program instructions are used for the methods in the foregoing embodiments.

[0268] The present application also provides a program product, which includes execution instructions stored in a readable storage medium. At least one processor of an electronic device can read the execution instructions from the readable storage medium, and the execution of the execution instructions by at least one processor enables the electronic device to implement the feature extraction model training or texture mapping processing methods provided by the various embodiments described above.

[0269] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions described in the foregoing embodiments, or perform equivalent replacements on some or all of the technical features; and these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the various embodiments of the present application.

Claims

1. A method for training a feature extraction model, characterized in that, the method includes: Obtain an initial sample data set, the initial sample data set includes: at least one sample data, each sample data includes: a sample material map, and a hard label of the sample material map; the hard label is used to characterize the material type to which the sample material map belongs; Convert the hard label of the sample material map in the initial sample data set into a soft label to obtain a training sample data set; the soft label is used to characterize the probability that the sample material map belongs to each material type; the soft label is related to the sample hierarchical clustering result of the sample material map, and the sample hierarchical clustering result includes: color result and / or texture clustering result; Use the training sample data set to train a neural network model to obtain a feature extraction model, and the feature extraction model is used to extract the material features of an image; The sample hierarchical clustering result includes: a first sample clustering result obtained by clustering the sample material maps based on color, a second sample clustering result obtained by clustering the sample material maps based on texture, and a third sample clustering result obtained by clustering the sample material maps based on color and texture; the first sample clustering result and the second sample clustering result are both first-level sample clustering results in the sample hierarchical clustering result, and the third sample clustering result is a second-level sample clustering result in the sample hierarchical clustering result; The converting the hard label of the sample material map in the initial sample data set into a soft label to obtain a training sample data set includes: For each sample material map, determine the initial soft label of the sample material map according to the hard label of the sample material map and a preset soft label probability assignment method; Adjust the probabilities in the initial soft label of the sample material map according to the first sample clustering result, the second sample clustering result, and the third sample clustering result to obtain the soft label of the sample material map; Use each sample material map and the soft label of each sample material map to obtain the training sample data set.

2. The method according to claim 1, characterized in that, the method further includes: Obtain the first sample clustering result and the second sample clustering result; Obtain the third sample clustering result according to the first sample clustering result and the second sample clustering result.

3. A method for processing a material map, characterized in that, the method includes: Obtain a target image; Input the target image into the feature extraction model to obtain the material features of the target image; the feature extraction model is obtained by using the method according to any one of claims 1-2; According to the material features of the target image, the material features of each material map in the material map library extracted by using the feature extraction model, and the hierarchical clustering result of the material maps in the material map library based on color and texture, obtain K material maps that match the material of the target image; K is an integer greater than or equal to 1.

4. The method according to claim 3, characterized in that, Obtaining K material maps that match the material of the target image based on the material features of the target image, the material features of each material map in the material map library extracted by using the feature extraction model, and the hierarchical clustering results of the material maps in the material map library based on color and texture, includes: Obtaining the similarity between the target image and each material map according to the material features of the target image and the material features of each material map in the material map library; Obtaining the top N material maps in the order of similarity from large to small; N is an integer greater than or equal to 2; Obtaining the clustering scores of the N material maps according to the hierarchical clustering results of the material maps in the material map library based on color and texture; Obtaining the top K material maps in the order of clustering scores from large to small as the material maps that match the material of the target image.

5. The method according to claim 4, wherein, The hierarchical clustering results include: a first clustering result obtained by clustering material maps based on color, a second clustering result obtained by clustering material maps based on texture, and a third clustering result obtained by clustering material maps based on color and texture; the first clustering result and the second clustering result are both first-level clustering results in the hierarchical clustering results, and the third clustering result is a second-level clustering result in the hierarchical clustering results; The obtaining the clustering scores of the N material maps according to the hierarchical clustering results of the material maps in the material map library based on color and texture includes: Obtaining the initial clustering scores of the N material maps according to the similarity sorting of the N material maps; Obtaining the first clustering scores of the N material maps according to the initial clustering scores of the N material maps and the first clustering result; Obtaining the second clustering scores of the N material maps according to the initial clustering scores of the N material maps and the second clustering result; Obtaining the third clustering scores of the N material maps according to the initial clustering scores of the N material maps and the third clustering result; Obtaining the clustering scores of the N material maps according to the initial clustering scores, first clustering scores, second clustering scores, and third clustering scores of the N material maps.

6. The method according to any one of claims 3-5, wherein, After obtaining the K material maps that match the material of the target image, the method further includes: Outputting the K material maps.

7. The method according to any one of claims 3-5, wherein, After obtaining the K material maps that match the material of the target image, the method further includes: Determining a target material map from the K material maps; Rendering the model framework of the target object using the target material map to obtain the model of the target object; Outputting the model of the target object.

8. The method according to claim 7, wherein, Before using the target material map to render the model framework of the target object to obtain the model of the target object, the method further includes: Constructing the model framework of the target object.

9. A feature extraction model training device, It is characterized in that the device includes: an acquisition module, configured to acquire an initial sample data set, where the initial sample data set includes: at least one sample data, and each sample data includes: a sample material map, and a hard label of the sample material map; the hard label is used to characterize the material type to which the sample material map belongs; a processing module, configured to convert the hard label of the sample material map in the initial sample data set into a soft label to obtain a training sample data set; the soft label is used to characterize the probability that the sample material map belongs to each material type; the soft label is related to the sample hierarchical clustering result of the sample material map, and the sample hierarchical clustering result includes: a color result and / or a texture clustering result; a training module, configured to use the training sample data set to train a neural network model to obtain a feature extraction model, where the feature extraction model is used to extract the material features of an image; the sample hierarchical clustering result includes: a first sample clustering result obtained by clustering the sample material maps based on color, a second sample clustering result obtained by clustering the sample material maps based on texture, and a third sample clustering result obtained by clustering the sample material maps based on color and texture; the first sample clustering result and the second sample clustering result are both first-level sample clustering results in the sample hierarchical clustering result, and the third sample clustering result is a second-level sample clustering result in the sample hierarchical clustering result; the processing module is specifically configured to, for each sample material map, determine an initial soft label of the sample material map according to the hard label of the sample material map and a preset soft label probability distribution method; adjust the probability in the initial soft label of the sample material map according to the first sample clustering result, the second sample clustering result, and the third sample clustering result to obtain the soft label of the sample material map; use each sample material map and the soft label of each sample material map to obtain the training sample data set.

10. A device for processing a material map, It is characterized in that the device includes: an acquisition module, configured to acquire a target image; a processing module, configured to input the target image into the feature extraction model to obtain the material features of the target image; according to the material features of the target image, the material features of each material map in the material map library extracted by using the feature extraction model, and the hierarchical clustering result of the material maps in the material map library based on color and texture, obtain K material maps that match the material of the target image; the feature extraction model is obtained by using the method according to any one of claims 1-2; K is an integer greater than or equal to 1.

11. An electronic device, It is characterized in that the electronic device includes a memory and a processor, and the memory is used to store a set of computer instructions; the processor executes the set of computer instructions stored in the memory to execute the method according to any one of claims 1 to 8 above.

12. A computer-readable storage medium, It is characterized in that Computer-executable instructions are stored on the computer-readable storage medium, and when the computer-executable instructions are executed by a processor, the method according to any one of claims 1-8 is implemented.

Images