Methods and apparatus for training models

Abstract

This application discloses a method and apparatus for training a model, relating to the field of computer technology. The method includes: acquiring sample data and an initial classification model, and training the initial classification model through multiple iterative operations; the iterative operations include: acquiring representations of the sample data, and determining features of the sample data based on these representations; using the features of the sample data to train the initial classification model based on contrastive learning, obtaining the contrastive loss function value output by the trained initial classification model, and generating feedback information based on the contrastive loss function value; updating the initial classification model based on the feedback information before executing the next iteration; and, in response to determining that the initial classification model has reached a preset convergence condition, determining the initial classification model in the last iteration as the target classification model. Using a model trained based on this method to classify 3D model data can improve the accuracy of classifying 3D model data.

Description

Technical Field

[0001] This disclosure relates to the field of computer technology, and more particularly to methods and apparatus for training models. Background Technology

[0002] A 3D model defines the shape of an object using a combination of points, edges, and faces in space. Its flexible geometric structure can efficiently and accurately describe the shape characteristics of an object. Existing methods for classifying objects based on 3D models mainly include: converting the 3D model into a 2D image and then classifying it based on the recognition of the 2D image; acquiring point cloud data of the 3D model and using the vertex coordinates in the point cloud data as input to a neural network model to classify the objects indicated by the 3D model; or using the geometric features of the 3D model as input to a neural network model to classify the objects indicated by the 3D model.

[0003] However, existing methods for classifying objects based on 3D models suffer from inaccurate classification. Summary of the Invention

[0004] This disclosure provides a method, apparatus, electronic device, and computer-readable storage medium for training models.

[0005] According to a first aspect of this disclosure, a method for training a model is provided, comprising: acquiring sample data and an initial classification model, and training the initial classification model through multiple rounds of iterative operations; the iterative operations include: acquiring representations of the sample data, and determining features of the sample data based on the representations of the sample data; using the features of the sample data to train the initial classification model based on contrastive learning, obtaining the contrastive loss function value output by the trained initial classification model, and generating feedback information based on the contrastive loss function value, and updating the initial classification model based on the feedback information before performing the next iteration operation; in response to determining that the initial classification model has reached a preset convergence condition, determining the initial classification model in the last round of iteration operation as the target classification model.

[0006] In some embodiments, the sample data includes 3D model data. Obtaining a representation of the sample data includes: obtaining a preset network, wherein the preset network includes a spatial feature descriptor and a structural feature descriptor; inputting the sample data into the preset network to obtain the spatial features of the sample data output by the spatial feature descriptor and the structural features of the sample data output by the structural feature descriptor; inputting the spatial features of the sample data into a preset convolutional network and obtaining a first output result of each convolutional layer in the preset convolutional network; inputting the structural features of the sample data into the preset convolutional network and obtaining a second output result of each convolutional layer in the preset convolutional network; for each convolutional layer in the preset convolutional network, aggregating the first output result and the second output result of the convolutional layer, inputting them into a self-attention network, and obtaining the output result of the self-attention network corresponding to that convolutional layer; aggregating the output results corresponding to each convolutional layer to obtain a representation of the sample data.

[0007] In some embodiments, training an initial classification model based on contrastive learning using features of the sample data includes: training an initial classification model based on unsupervised contrastive learning using features of the sample data.

[0008] In some embodiments, the method for training the model further includes: obtaining the labels of the sample data; and using the features of the sample data to train the initial classification model based on contrastive learning, including: using the features of the sample data and the labels of the sample data to train the initial classification model based on supervised contrastive learning.

[0009] In some embodiments, the iterative operation further includes: training an initial classification model using the representation and labels of the sample data to obtain the cross-entropy loss function value output by the trained initial classification model; and generating feedback information based on the contrastive loss function value, including: generating feedback information based on the contrastive loss function value and the cross-entropy loss function value.

[0010] According to a second aspect of this disclosure, a method for classifying three-dimensional models is provided, comprising: acquiring three-dimensional model data to be classified; and determining the category of the three-dimensional model data to be classified using a target classification model, wherein the target classification model is trained based on the method described in the first aspect.

[0011] According to a third aspect of this disclosure, an apparatus for training a model is provided, comprising: an acquisition unit configured to acquire sample data and an initial classification model, and to train the initial classification model through multiple rounds of iterative operations by a training unit; the training unit comprising: an acquisition module configured to acquire representations of the sample data, and to determine features of the sample data based on the representations of the sample data; a first update module configured to use the features of the sample data to train the initial classification model based on contrastive learning, to obtain a contrastive loss function value output by the trained initial classification model, and to generate feedback information based on the contrastive loss function value, wherein the initial classification model is updated based on the feedback information before performing the next iteration operation; and a determination module configured to, in response to determining that the initial classification model has reached a preset convergence condition, determine the initial classification model in the last round of iterative operations as the target classification model.

[0012] In some embodiments, the sample data includes 3D model data. The acquisition module includes: a network acquisition module configured to acquire a preset network, wherein the preset network includes a spatial feature descriptor and a structural feature descriptor; a feature extraction module configured to input the sample data into the preset network to obtain the spatial features of the sample data output by the spatial feature descriptor and the structural features of the sample data output by the structural feature descriptor; a first convolution module configured to input the spatial features of the sample data into the preset convolutional network and obtain a first output result of each convolutional layer in the preset convolutional network; a second convolution module configured to input the structural features of the sample data into the preset convolutional network and obtain a second output result of each convolutional layer in the preset convolutional network; a first aggregation module configured to aggregate the first output result and the second output result of each convolutional layer in the preset convolutional network, input them into a self-attention network, and obtain the output result of the self-attention network corresponding to that convolutional layer; and a second aggregation module configured to aggregate the output results corresponding to each convolutional layer to obtain a representation of the sample data.

[0013] In some embodiments, the first update module includes: a first update submodule configured to train an initial classification model based on unsupervised contrastive learning using features of the sample data.

[0014] In some embodiments, the apparatus for training the model further includes: a label acquisition module configured to acquire labels for the sample data; and a first update module including: a second update submodule configured to train the initial classification model based on supervised contrastive learning using features of the sample data and labels of the sample data.

[0015] In some embodiments, the training unit further includes: a second update module, configured to train an initial classification model using the representation of the sample data and the label of the sample data, to obtain the cross-entropy loss function value output by the trained initial classification model; and to generate feedback information based on the contrastive loss function value, including: generating feedback information based on the contrastive loss function value and the cross-entropy loss function value.

[0016] According to a fourth aspect of this disclosure, an apparatus for classifying three-dimensional models is provided, comprising: a three-dimensional model data acquisition unit configured to acquire three-dimensional model data to be classified; and a classification unit configured to determine the category of the three-dimensional model data to be classified using a target classification model, wherein the target classification model is trained based on the apparatus described in the third aspect.

[0017] According to a fifth aspect of this disclosure, embodiments of this disclosure provide an electronic device, including: one or more processors; and a storage device for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement the method for training a model as provided in the first aspect or the method for classifying a three-dimensional model as provided in the second aspect.

[0018] According to a sixth aspect of this disclosure, embodiments of this disclosure provide a computer-readable storage medium having a computer program stored thereon, wherein the program, when executed by a processor, implements either the method for training a model as provided in the first aspect or the method for classifying a three-dimensional model as provided in the second aspect.

[0019] The method and apparatus for training a model disclosed herein include: acquiring sample data and an initial classification model, and training the initial classification model through multiple rounds of iterative operations; the iterative operations include: acquiring representations of the sample data, and determining features of the sample data based on the representations of the sample data; using the features of the sample data to train the initial classification model based on contrastive learning, obtaining the contrastive loss function value output by the trained initial classification model, and generating feedback information based on the contrastive loss function value, and updating the initial classification model based on the feedback information before performing the next iteration operation; in response to determining that the initial classification model has reached a preset convergence condition, determining the initial classification model in the last round of iteration operation as the target classification model, and using the target classification model trained by this contrastive training method to classify 3D model data, which can improve the accuracy of classifying 3D model data.

[0020] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this disclosure, nor is it intended to limit the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0021] The accompanying drawings are provided for a better understanding of this solution and do not constitute a limitation of this application. Wherein:

[0022] Figure 1 This is an exemplary system architecture diagram in which embodiments of this application can be applied;

[0023] Figure 2 This is a flowchart of one embodiment of the method for training a model according to this application;

[0024] Figure 3 This is a flowchart of another embodiment of the method for training a model according to this application;

[0025] Figure 4 This is a flowchart illustrating the acquisition of a representation of sample data in an application scenario of the method for training a model according to this application;

[0026] Figure 5(a) is a simplified flowchart of the steps for obtaining the representation of sample data in an application scenario of the method for training a model according to this application;

[0027] Figure 5(b) is a simplified flowchart of the steps for obtaining the contrastive loss function value in one application scenario of the method for training a model according to this application;

[0028] Figure 5(c) is a simplified flowchart of the steps for obtaining the cross-entropy loss function value in one application scenario of the method for training a model according to this application;

[0029] Figure 6 This is a flowchart of an embodiment of a method for classifying three-dimensional models according to this application;

[0030] Figure 7 This is a schematic diagram of one embodiment of the apparatus for training a model according to this application;

[0031] Figure 8 This is a schematic diagram of one embodiment of the apparatus for classifying three-dimensional models according to this application;

[0032] Figure 9 This is a block diagram of an electronic device used to implement the method for training a model according to the embodiments of this application. Detailed Implementation

[0033] The following description, in conjunction with the accompanying drawings, illustrates exemplary embodiments of this application, including various details to aid understanding. These should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of this application. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description.

[0034] Figure 1 An exemplary system architecture 100 is shown, in which embodiments of the methods or apparatus for training models of this application may be applied.

[0035] like Figure 1 As shown, system architecture 100 may include terminal devices 101, 102, and 103, a network 104, and a server 105. Network 104 serves as the medium for providing communication links between terminal devices 101, 102, and 103 and server 105. Network 104 may include various connection types, such as wired or wireless communication links, or fiber optic cables, etc.

[0036] Users can use terminal devices 101, 102, and 103 to interact with server 105 via network 104 to receive or send messages, etc. Terminal devices 101, 102, and 103 can be user terminal devices, on which various client applications can be installed, such as image applications, video applications, shopping applications, chat applications, search applications, financial applications, etc.

[0037] Terminal devices 101, 102, and 103 can be various electronic devices with displays that support receiving messages from the server, including but not limited to smartphones, tablets, e-book readers, electronic players, laptops, and desktop computers.

[0038] Terminal devices 101, 102, and 103 can be either hardware or software. When terminal devices 101, 102, and 103 are hardware, they can be various electronic devices. When terminal devices 101, 102, and 103 are software, they can be installed in the electronic devices listed above. They can be implemented as multiple software programs or software modules (e.g., multiple software modules used to provide distributed services) or as a single software program or software module. No specific limitations are made here.

[0039] Server 105 can acquire sample data and an initial classification model, and train the initial classification model through multiple rounds of iterative operations. The iterative operations include: acquiring the representation of the sample data, and determining the features of the sample data based on the representation of the sample data; using the features of the sample data, training the initial classification model based on contrastive learning, obtaining the contrastive loss function value output by the trained initial classification model, and generating feedback information based on the contrastive loss function value. The initial classification model is updated based on the feedback information before executing the next iteration operation; if it is determined that the initial classification model has reached the preset convergence condition, the initial classification model in the last round of iteration operation is determined as the target classification model.

[0040] It should be noted that the method for training the model provided in the embodiments of this disclosure can be executed by the server 105, and correspondingly, the apparatus for training the model can be set in the server 105.

[0041] It should be understood that Figure 1 The number of terminal devices, networks, and servers shown is merely illustrative. Depending on implementation needs, any number of terminal devices, networks, and servers can be included.

[0042] Continue to refer to Figure 2 The flowchart 200 of one embodiment of the method for training a model according to the present disclosure is shown, including the following steps:

[0043] Step 201: Obtain sample data and an initial classification model, and train the initial classification model through multiple rounds of iterative operations.

[0044] In this embodiment, the execution entity of the method used to train the model (e.g. Figure 1 The server 105 shown can acquire sample data and an initial classification model, and then use the sample data to perform multiple rounds of iterative training on the initial classification model. The sample data may include the acquired data and the data after data augmentation operations have been performed on the acquired data.

[0045] Step 202, the iterative operation includes:

[0046] Step 2021: Obtain the representation of the sample data, and determine the features of the sample data based on the representation of the sample data.

[0047] In this embodiment, a representation of the sample data can be obtained based on the sample data, and features of the sample data can be extracted based on the representation of the sample data. Specifically, the representation of the sample data can be features extracted from the sample data using a convolutional network, and the features of the sample data can be features calculated using a self-attention network on the features extracted by the convolutional network.

[0048] Step 2022: Using the features of the sample data, the initial classification model is trained based on contrastive learning to obtain the contrastive loss function value output by the trained initial classification model. Feedback information is generated based on the contrastive loss function value, and the initial classification model is updated based on the feedback information before performing the next iteration operation.

[0049] In this embodiment, the features of the sample data can be used to train the initial classification model based on contrastive learning, obtain the contrastive loss function value output by the trained initial classification model, generate feedback information based on the contrastive loss function value, and update the initial classification model with the feedback information before performing the next iteration operation, so as to realize the iterative training of the initial classification model.

[0050] Step 203: In response to determining that the initial classification model has reached the preset convergence condition, the initial classification model in the last round of iteration is determined as the target classification model.

[0051] In this embodiment, if the initial classification model is determined to have reached the preset convergence condition, such as the comparison loss function value being less than a preset threshold or the number of iterations reaching a threshold, then the initial classification model in the last iteration operation is determined as the target classification model.

[0052] The method for training a model provided in this embodiment acquires sample data and an initial classification model, and trains the initial classification model through multiple rounds of iterative operations. The iterative operations include: acquiring the representation of the sample data, and determining the features of the sample data based on the representation of the sample data; using the features of the sample data, training the initial classification model based on contrastive learning to obtain the contrastive loss function value output by the trained initial classification model, and generating feedback information based on the contrastive loss function value. The initial classification model is updated based on the feedback information before executing the next iteration operation; in response to determining that the initial classification model has reached the preset convergence condition, the initial classification model in the last round of iteration operation is determined as the target classification model. Using the target classification model trained by this contrastive training method to classify 3D model data can improve the accuracy of classifying 3D model data.

[0053] Optionally, the sample data includes 3D model data. Obtaining the representation of the sample data includes: acquiring a preset network, wherein the preset network includes a spatial feature descriptor and a structural feature descriptor; inputting the sample data into the preset network to obtain the spatial features of the sample data output by the spatial feature descriptor and the structural features of the sample data output by the structural feature descriptor; inputting the spatial features of the sample data into a preset convolutional network and obtaining a first output result from each convolutional layer in the preset convolutional network; inputting the structural features of the sample data into the preset convolutional network and obtaining a second output result from each convolutional layer in the preset convolutional network; for each convolutional layer in the preset convolutional network, aggregating the first and second output results of that convolutional layer and inputting them into a self-attention network to obtain the output result of the self-attention network corresponding to that convolutional layer; and aggregating the output results corresponding to each convolutional layer to obtain the representation of the sample data.

[0054] In this embodiment, the sample data includes 3D model data. A preset network can be obtained, which includes a spatial feature descriptor and a structural feature descriptor. After the sample data is input into the preset network, the spatial features contained in the sample data (i.e., the sample 3D model data) output by the spatial feature descriptor and the structural features of the sample data output by the structural feature descriptor are obtained. The spatial features of the sample data are input into a preset convolutional network to obtain a first output result from each convolutional layer in the preset convolutional network. The structural features of the sample data are input into the preset convolutional network to obtain a second output result from each convolutional layer in the preset convolutional network.

[0055] Next, for each convolutional layer in the preset convolutional network, the first output and the second output of that convolutional layer are aggregated and then input into the self-attention network to obtain the output of the self-attention network for that convolutional layer. It can be understood that since the convolutional network contains multiple convolutional layers, multiple outputs can be obtained, each output being an aggregation of the spatial and structural features of the corresponding convolutional layer.

[0056] Finally, the multiple output results are aggregated, and the aggregated result is used as the representation of the sample data.

[0057] This embodiment processes sample data using convolutional and self-attention networks to obtain the global relationships between features of 3D model data. Using this processed sample data to train a classification model improves the accuracy of the trained model in classifying 3D model data. Furthermore, the complex geometry of 3D mesh models makes it impossible to directly apply convolutional neural networks to their processing. Existing methods, to overcome this problem, design numerous complex operations for convolution and pooling, making the neural network more susceptible to overfitting and unable to cope with attacks on the shape and structure of 3D mesh models, resulting in poor network robustness. The method used in this embodiment to obtain the representation of sample data avoids this overfitting problem and improves the network's robustness.

[0058] Optionally, the initial classification model can be trained using the features of the sample data based on contrastive learning, including: training the initial classification model using the features of the sample data based on unsupervised contrastive learning.

[0059] In this embodiment, the features of the sample data can be used to train the initial classification model based on unsupervised contrastive learning.

[0060] Optionally, the method for training the model further includes: obtaining the labels of the sample data; and using the features of the sample data to train the initial classification model based on contrastive learning, including: using the features of the sample data and the labels of the sample data to train the initial classification model based on supervised contrastive learning.

[0061] In this embodiment, the labels of the sample data can be obtained in advance, and the initial classification model can be trained based on supervised contrastive learning using the features and labels of the sample data.

[0062] Continue to refer to Figure 3 The diagram illustrates a flow 300 of another embodiment of a method for training a model according to the present disclosure, comprising the following steps:

[0063] Step 301: Obtain sample data and an initial classification model, and train the initial classification model through multiple rounds of iterative operations.

[0064] Step 302, the iterative operation includes:

[0065] Step 3021: Obtain the representation of the sample data, and determine the features of the sample data based on the representation of the sample data.

[0066] Step 3022: Using the features of the sample data, train the initial classification model based on contrastive learning to obtain the contrastive loss function value output by the trained initial classification model.

[0067] In this embodiment, the features of the sample data can be used to train the initial classification model based on contrastive learning, and the contrastive loss function value output by the trained initial classification model can be obtained.

[0068] Step 3023: Using the representation and labels of the sample data, train the initial classification model and obtain the cross-entropy loss function value output by the trained initial classification model.

[0069] In this embodiment, the initial classification model can be trained using the representation and labels of the sample data, and the cross-entropy loss function value output by the trained initial classification model can be obtained.

[0070] Step 3024: Based on the comparison loss function value and the cross-entropy loss function value, generate feedback information. The initial classification model is updated based on the feedback information before performing the next iteration.

[0071] In this embodiment, the contrastive loss function value and the cross-entropy loss function value obtained after this round of training can be weighted and calculated. Feedback information is generated based on the result of the weighted calculation. Before the next iteration, the parameters of the initial classification model are updated using this feedback information.

[0072] Step 303: In response to determining that the initial classification model has reached the preset convergence condition, the initial classification model in the last round of iteration is determined as the target classification model.

[0073] In this embodiment, the descriptions of steps 301, 3021, and 303 are consistent with those of steps 201, 2021, and 203, and will not be repeated here.

[0074] The method for training the model provided in this embodiment is compared to... Figure 2 The method described in the embodiment adds the steps of using the representation and labels of sample data to train an initial model and obtain the cross-entropy loss function of the model output. It also uses the weighted result of the contrastive loss function value and the cross-entropy loss function value obtained from the contrastive learning training to generate feedback information to complete the model training. This can improve the accuracy of the trained target classification model in classifying data.

[0075] In some application scenarios, the methods used to train the model include the following steps:

[0076] The first step is to acquire sample data, which consists of multiple sample 3D model data and augmented data after data augmentation operations. Two random augmentation operations can be performed on each sample 3D model data to change its features through random transformations, making the machine learning process more difficult and adding regularization terms to the neural network.

[0077] Data augmentation operations on the sample 3D model data can include pixel perturbation, translation, scaling, and rotation of the vertex coordinates, as well as model deformation and edge flipping operations. Model deformation refers to using Freeform Deformation (FFD) to assign a new position to each vertex in the 3D model, thereby deforming the entire model. Edge flipping involves changing the connectivity between points in the 3D model, separating previously connected points to form edges, and connecting previously disconnected points to form new edges. In essence, model deformation and edge flipping operations enable global feature transformations of the 3D model.

[0078] The second step, as Figure 4 As shown, a pre-defined network (such as MeshNet) is used to obtain the representation of the sample data. The pre-defined network may include a spatial feature descriptor and a structural feature descriptor. The sample data is input into the spatial feature descriptor to obtain the spatial features of the sample data. The spatial features of the sample data are then input into the pre-defined convolutional network, and the first output result of each convolutional layer in the pre-defined convolutional network is obtained.

[0079] The sample data is input into the structural feature descriptor to obtain the structural features of the sample data. The structural features of the sample data are then input into a preset convolutional network to obtain the second output result of each convolutional layer in the preset convolutional network.

[0080] For each convolutional layer in the preset convolutional network, the first output result of the convolutional network and the second output result of the convolutional layer are aggregated based on the fully connected layer, and then input into the self-attention network to obtain the output result of the self-attention network corresponding to the convolutional layer.

[0081] Since the pre-defined convolutional network has multiple convolutional layers, the output results corresponding to each of the multiple convolutional layers have been obtained. After aggregating the output results of each convolutional layer based on the fully connected layer, the aggregated result is used as the representation of the sample data.

[0082] The process of acquiring the representation of sample data can be understood as an encoding process, and the device / unit used to perform this encoding process of acquiring the representation of sample data can be called an encoder / encoder network.

[0083] A simplified flowchart of the first and second steps can be found in Figure 5(a).

[0084] The third step is to input the above sample data into the network (z). x =g(h x In the process of obtaining the features of the sample data output by the mapping network, the mapping network may include a multilayer perceptron and hidden layers.

[0085] z x =g(h x ) = W (2) ReLU(W (1) h x )

[0086] Among them, z x h represents the characteristics of the sample data x ReLU() represents the representation of sample data, and W represents the preset function in machine learning. (1) 、W (2) This represents the learning parameters.

[0087] The fourth step involves using the features of the sample data to train the initial classification model based on contrastive learning, and obtaining the contrastive loss function.

[0088] In unsupervised contrastive learning, since the sample data does not have class labels, positive samples come from the data augmentation results before and after data augmentation of the anchor sample. Negative samples are all samples other than positive samples and anchor samples. In this case, the contrastive loss function of unsupervised contrastive learning can be expressed by the following formula:

[0089]

[0090] Where i represents the anchor sample identifier; j represents the positive sample identifier; k represents the negative sample identifier; and τ represents the annealing coefficient.

[0091] In supervised contrastive learning, since the sample data has class labels and the class of each sample data is known, the contrastive loss function of supervised contrastive learning can be expressed by the following formula:

[0092]

[0093] Where i represents the anchor sample identifier; P(i) represents the set of positive samples in the anchor sample set, and p represents the positive sample identifier; A(i) represents the set of negative samples, and a represents the negative sample identifier. Since the loss function of supervised contrastive learning allows the encoder to provide closer representations for samples from the same class, it can achieve more robust representation space clustering results.

[0094] Furthermore, in classification tasks, the loss function used in contrastive learning is calculated using the feature mapping of the entire model. To bring more changes to the overall features of the 3D mesh model, all sample data can be randomly augmented twice, resulting in two augmented samples. For anchor sample i, its positive samples include the augmented samples and samples of the same class as the anchor sample; the negative samples of anchor sample i consist of samples of a different class. In segmentation tasks, the function used in contrastive learning is calculated using the feature mapping of a single facet. Processing the features of all faces requires very high computational overhead, so a sampling strategy is needed to select features from some faces to reduce the number of features. Since most faces with incorrect class predictions in the segmentation results come from boundary locations, a boundary-aware sampling method can be used to select boundary samples when choosing anchor samples, forcing the network to focus more on boundary samples. The following two methods can be used to select positive and negative samples:

[0095] The first method is based on comparing boundary samples with difficult samples. Difficult samples are those that are predicted incorrectly. Specifically, for a negative sample, it is considered difficult if its similarity to the anchor sample is close to 1; for a positive sample, it is considered difficult if its similarity to the anchor sample is close to 0. A dynamic storage space is constructed to store difficult samples. In the first training iteration, the similarity between the anchor sample and its neighboring samples (including k positive and k negative samples) is calculated, and the difficult samples among the neighboring samples are stored in the dynamic space. In subsequent training iterations, the similarity between the anchor sample and its neighboring samples, and the similarity between the anchor sample and the difficult samples, need to be calculated. When a neighboring sample n is more difficult than a difficult sample h in the dynamic space, n is stored in the dynamic space to replace h, thus ensuring that the samples in the dynamic space are always the most difficult.

[0096] The second method is based on the comparison of boundary samples. It compares boundary samples located in the same position (e.g., samples at the boundary between the head and body in a 3D human body model data), thereby bringing positive samples closer to each other and negative samples further apart.

[0097] A simplified flowchart of steps three and four can be found in Figure 5(b).

[0098] The fifth step is to calculate the overall loss function used to train the network, and then use the overall loss function to train the initial classification model to obtain the target classification model.

[0099] First, obtain the cross-entropy loss function used to train the network:

[0100]

[0101] Among them, hi The normalized output of the network (i.e., the model's predicted labels), y i The true label represents the sample data, and 'i' represents the label identifier.

[0102] Secondly, the overall loss function is a weighted sum of the contrastive loss function obtained in step four and the cross-entropy loss function obtained in this step:

[0103] L all =L ce +α·L cl

[0104] Among them, L ce Let L represent the contrastive loss function, and α represent the weights of the contrastive loss function. In actual training, due to L... ce and L cl The initial values ​​of the two components differ significantly, and their decreasing trends are not identical. Setting appropriate weights can balance this difference, ensuring that both play an effective role in training the model. Weights can be set based on their impact on the optimizer: First, calculate L... ce and L cl For variable z i The gradient of L is calculated, and the gradient values ​​of both are set to be pulled to the same order of magnitude; then, the scaling parameter n is adjusted to adjust L. ce and L cl The relationship between them:

[0105]

[0106] Training ends when the predetermined number of iterations is reached or the loss function is less than a preset threshold, and the target classification model is obtained.

[0107] A simplified flowchart for calculating the cross-entropy loss function in step 5 can be found in Figure 5(c).

[0108] Continue to refer to Figure 6 The flowchart 600 of an embodiment of a method for classifying 3D models according to the present disclosure is shown, including the following steps:

[0109] Step 601: Obtain the 3D model data to be classified.

[0110] In this embodiment, the execution entity of the method used to train the model (e.g. Figure 1 The server 105 shown can acquire the 3D model data to be classified. 3D model data refers to model data with three-dimensional spatial features and three-dimensional structural features.

[0111] Step 602: Determine the category of the 3D model data to be classified using a target classification model, wherein the target classification model is based on, for example... Figure 2 or Figure 3 The method described in the embodiments is used for training.

[0112] In this embodiment, a method based on, for example Figure 2 or Figure 3 The target classification model trained by the method in the described embodiments can classify the 3D model data to be classified.

[0113] The method for classifying 3D models provided in this embodiment uses a target classification model trained based on a contrastive training method to classify 3D model data, which can improve the accuracy of classifying 3D model data.

[0114] Further reference Figure 7 As an implementation of the methods shown in the above figures, this disclosure provides an embodiment of an apparatus for training a model, which is similar to... Figure 2 and Figure 3 Corresponding to the method embodiments shown, this device can be specifically applied to various electronic devices.

[0115] like Figure 7 As shown, the apparatus for training a model in this embodiment includes: an acquisition unit 701, a training unit 702, and a determination unit 703. The acquisition unit is configured to acquire sample data and an initial classification model, and train the initial classification model through multiple rounds of iterative operations by the training unit. The training unit includes: an acquisition module configured to acquire representations of the sample data and determine features of the sample data based on these representations; a first update module configured to use the features of the sample data to train the initial classification model based on contrastive learning, obtain the contrastive loss function value output by the trained initial classification model, and generate feedback information based on the contrastive loss function value. The initial classification model is updated based on the feedback information before the next iteration operation. The determination module is configured to, in response to determining that the initial classification model has reached a preset convergence condition, determine the initial classification model in the last round of iteration as the target classification model.

[0116] In some embodiments, the sample data includes 3D model data. The acquisition module includes: a network acquisition module configured to acquire a preset network, wherein the preset network includes a spatial feature descriptor and a structural feature descriptor; a feature extraction module configured to input the sample data into the preset network to obtain the spatial features of the sample data output by the spatial feature descriptor and the structural features of the sample data output by the structural feature descriptor; a first convolution module configured to input the spatial features of the sample data into the preset convolutional network and obtain a first output result of each convolutional layer in the preset convolutional network; a second convolution module configured to input the structural features of the sample data into the preset convolutional network and obtain a second output result of each convolutional layer in the preset convolutional network; a first aggregation module configured to aggregate the first output result and the second output result of each convolutional layer in the preset convolutional network, input them into a self-attention network, and obtain the output result of the self-attention network corresponding to that convolutional layer; and a second aggregation module configured to aggregate the output results corresponding to each convolutional layer to obtain a representation of the sample data.

[0117] In some embodiments, the first update module includes: a first update submodule configured to train an initial classification model based on unsupervised contrastive learning using features of the sample data.

[0118] In some embodiments, the apparatus for training the model further includes: a label acquisition module configured to acquire labels for the sample data; and a first update module including: a second update submodule configured to train the initial classification model based on supervised contrastive learning using features of the sample data and labels of the sample data.

[0119] In some embodiments, the training unit further includes: a second update module, configured to train an initial classification model using the representation of the sample data and the label of the sample data, to obtain the cross-entropy loss function value output by the trained initial classification model; and to generate feedback information based on the contrastive loss function value, including: generating feedback information based on the contrastive loss function value and the cross-entropy loss function value.

[0120] Each unit in the aforementioned device 700 and the reference Figure 2 and Figure 3 The steps described in the method correspond to those steps. Therefore, the operations, features, and technical effects described above for the method used to train the model also apply to device 700 and the units contained therein, and will not be repeated here.

[0121] Further reference Figure 8 As an implementation of the methods shown in the above figures, this disclosure provides an embodiment of an apparatus for classifying three-dimensional models, which is similar to... Figure 6 Corresponding to the method embodiments shown, this device can be specifically applied to various electronic devices.

[0122] like Figure 8 As shown, the apparatus for training a model in this embodiment includes a three-dimensional model data acquisition unit 801 and a classification unit 802. The three-dimensional model data acquisition unit is configured to acquire three-dimensional model data to be classified; the classification unit is configured to determine the category of the three-dimensional model data to be classified using a target classification model, wherein the target classification model is trained based on the apparatus described in the third aspect.

[0123] Each unit in the aforementioned device 800 and the reference Figure 6 The steps described in the method correspond to those steps. Therefore, the operations, features, and technical effects described above for the method of classifying 3D models are also applicable to the device 800 and the units contained therein, and will not be repeated here.

[0124] According to embodiments of this application, this application also provides an electronic device and a readable storage medium.

[0125] like Figure 9 The diagram shown is a block diagram of an electronic device 900 for a method of training a model according to an embodiment of this application. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the present application described and / or claimed herein.

[0126] like Figure 9 As shown, the electronic device includes one or more processors 901, a memory 902, and interfaces for connecting the components, including high-speed interfaces and low-speed interfaces. The components are interconnected via different buses and can be mounted on a common motherboard or otherwise as required. The processors can process instructions executed within the electronic device, including instructions stored in or on memory to display graphical information of a GUI on an external input / output device (such as a display device coupled to the interface). In other embodiments, multiple processors and / or multiple buses can be used with multiple memories and multiple memory modules, if desired. Similarly, multiple electronic devices can be connected, each providing some of the necessary operations (e.g., as a server array, a group of blade servers, or a multiprocessor system). Figure 9Take the 901 processor as an example.

[0127] The memory 902 is the non-transitory computer-readable storage medium provided in this application. This memory stores instructions executable by at least one processor to cause the at least one processor to perform the method for training a model provided in this application. The non-transitory computer-readable storage medium of this application stores computer instructions for causing a computer to perform the method for training a model provided in this application.

[0128] Memory 902, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs, non-transitory computer-executable programs, and modules, such as the program instructions / modules corresponding to the method for training a model in the embodiments of this application (e.g., appendix). Figure 7 The acquisition unit 701, training unit 702, and determination unit 703 are shown. The processor 901 executes various functional applications and data processing of the server by running non-transient software programs, instructions, and modules stored in the memory 902, thereby implementing the method for training the model in the above method embodiments.

[0129] Memory 902 may include a program storage area and a data storage area. The program storage area may store an operating system and applications required for at least one function; the data storage area may store data created based on the use of the electronic device for extracting video clips. Furthermore, memory 902 may include high-speed random access memory and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, memory 902 may optionally include memory remotely located relative to processor 901, and this remote memory may be connected to the electronic device for extracting video clips via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0130] The electronic device used for training the model may further include an input device 903, an output device 904, and a bus 905. The processor 901, memory 902, input device 903, and output device 904 may be connected via the bus 905 or other means. Figure 9 Taking the connection between China and Israel via bus 905 as an example.

[0131] Input device 903 can receive input digital or character information, and generate key signal inputs related to user settings and function control of the electronic device used to extract video clips, such as touch screens, keypads, mice, trackpads, touchpads, joysticks, one or more mouse buttons, trackballs, joysticks, etc. Output device 904 may include display devices, auxiliary lighting devices (e.g., LEDs), and haptic feedback devices (e.g., vibration motors). The display device may include, but is not limited to, liquid crystal displays (LCDs), light-emitting diode (LED) displays, and plasma displays. In some embodiments, the display device may be a touch screen.

[0132] Various implementations of the systems and techniques described herein can be implemented in digital electronic circuit systems, integrated circuit systems, application-specific integrated circuits (ASICs), computer hardware, firmware, software, and / or combinations thereof. These various implementations may include: implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transferring data and instructions to the storage system, the at least one input device, and the at least one output device.

[0133] These computational programs (also referred to as programs, software, software applications, or code) include machine instructions for a programmable processor and can be implemented using high-level procedural and / or object-oriented programming languages, and / or assembly / machine languages. As used herein, the terms “machine-readable medium” and “computer-readable medium” refer to any computer program product, device, and / or apparatus (e.g., disk, optical disk, memory, programmable logic device (PLD)) used to provide machine instructions and / or data to a programmable processor, including machine-readable media that receive machine instructions as machine-readable signals. The term “machine-readable signal” refers to any signal used to provide machine instructions and / or data to a programmable processor.

[0134] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0135] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as a data server), or computing systems that include middleware components (e.g., an application server), or computing systems that include frontend components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with embodiments of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., a communication network). Examples of communication networks include local area networks (LANs), wide area networks (WANs), and the Internet.

[0136] Computer systems can include clients and servers. Clients and servers are generally located far apart and typically interact through communication networks. Client-server relationships are created by computer programs running on the respective computers and having a client-server relationship with each other.

[0137] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in this application can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in this application can be achieved, and this is not limited herein.

[0138] The specific embodiments described above do not constitute a limitation on the scope of protection of this application. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this application should be included within the scope of protection of this application.

Claims

1. A method for training a model, comprising: Acquire sample data and an initial classification model, and train the initial classification model through multiple rounds of iterative operations. The sample data includes augmented data after performing data augmentation operations on the three-dimensional model data of the object in at least one of the following ways: pixel perturbation, translation, scaling, and rotation of the vertex coordinates of the three-dimensional model data, and model deformation and edge flipping operations on the sample three-dimensional model data. The three-dimensional model is defined by combining spatial points, edges, and faces to define the shape of the object. The iterative operation includes: Obtain the representation of the sample data, and determine the features of the sample data based on the representation of the sample data; Using the features of the sample data, the initial classification model is trained based on contrastive learning to obtain the contrastive loss function value output by the trained initial classification model. Feedback information is generated based on the contrastive loss function value, and the initial classification model is updated based on the feedback information before performing the next iteration operation. In response to determining that the initial classification model has reached the preset convergence condition, the initial classification model in the last round of iteration is determined as the target classification model; The sample data includes three-dimensional model data, and the process of acquiring the representation of the sample data includes: Obtain a preset network, wherein the preset network includes a spatial feature descriptor and a structural feature descriptor; The sample data is input into the preset network to obtain the spatial features of the sample data output by the spatial feature descriptor and the structural features of the sample data output by the structural feature descriptor. The spatial features of the sample data are input into a preset convolutional network, and the first output result of each convolutional layer in the preset convolutional network is obtained. The structural features of the sample data are input into a preset convolutional network, and the second output result of each convolutional layer in the preset convolutional network is obtained. For each convolutional layer in the preset convolutional network, the first output result and the second output result of the convolutional layer are aggregated and input into the self-attention network to obtain the output result of the self-attention network corresponding to the convolutional layer. The outputs corresponding to each convolutional layer are aggregated to obtain a representation of the sample data.

2. The method according to claim 1, wherein, The step of using the features of the sample data to train the initial classification model based on contrastive learning includes: The initial classification model is trained using the features of the sample data based on unsupervised contrastive learning.

3. The method according to claim 1, wherein, The method further includes: Obtain the labels of the sample data; Using the features of the sample data, the initial classification model is trained based on contrastive learning, including: The initial classification model is trained using supervised contrastive learning based on the features and labels of the sample data.

4. The method according to claim 1, wherein, The iterative operation also includes: Using the representation and labels of the sample data, the initial classification model is trained to obtain the cross-entropy loss function value output by the trained initial classification model; The step of generating feedback information based on the contrast loss function value includes: The feedback information is generated based on the contrast loss function value and the cross-entropy loss function value.

5. A method for classifying three-dimensional models, comprising: Obtain the 3D model data to be classified; The category of the three-dimensional model data to be classified is determined by using a target classification model, wherein the target classification model is trained based on the method described in any one of claims 1-4.

6. An apparatus for training a model, comprising: The acquisition unit is configured to acquire sample data and an initial classification model, and train the initial classification model through multiple rounds of iterative operations by the training unit. The sample data includes augmented data after performing data augmentation operations on the three-dimensional model data of the object in at least one of the following ways: pixel perturbation, translation, scaling, and rotation of the vertex coordinates of the three-dimensional model data, and model deformation and edge flipping operations on the sample three-dimensional model data. The three-dimensional model is defined by combining spatial points, edges, and faces to define the shape of the object. The training unit includes: The acquisition module is configured to acquire a representation of the sample data and determine features of the sample data based on the representation of the sample data. The first update module is configured to use the features of the sample data to train the initial classification model based on contrastive learning, obtain the contrastive loss function value output by the trained initial classification model, and generate feedback information based on the contrastive loss function value. The initial classification model is updated based on the feedback information before performing the next iteration operation. The determination module is configured to determine the initial classification model in the last iteration operation as the target classification model in response to determining that the initial classification model has reached a preset convergence condition; The sample data includes three-dimensional model data, and the acquisition module includes: The network acquisition module is configured to acquire a preset network, wherein the preset network includes a spatial feature descriptor and a structural feature descriptor; The feature extraction module is configured to input the sample data into the preset network to obtain the spatial features of the sample data output by the spatial feature descriptor and the structural features of the sample data output by the structural feature descriptor. The first convolution module is configured to input the spatial features of the sample data into a preset convolutional network and obtain the first output result of each convolutional layer in the preset convolutional network. The second convolution module is configured to input the structural features of the sample data into a preset convolutional network and obtain the second output result of each convolutional layer in the preset convolutional network. The first aggregation module is configured to aggregate the first output result and the second output result of each convolutional layer in the preset convolutional network, input them into the self-attention network, and obtain the output result of the self-attention network corresponding to the convolutional layer. The second aggregation module is configured to aggregate the output results corresponding to each convolutional layer to obtain a representation of the sample data.

7. The apparatus according to claim 6, wherein, The first update module includes: The first update submodule is configured to use the features of the sample data to train the initial classification model based on unsupervised contrastive learning.

8. The apparatus according to claim 6, wherein, The device further includes: The tag acquisition module is configured to acquire the tags of the sample data; The first update module includes: The second update submodule is configured to train the initial classification model based on supervised contrastive learning using the features and labels of the sample data.

9. The apparatus according to claim 6, wherein, The training unit also includes: The second update module is configured to use the representation of the sample data and the label of the sample data to train the initial classification model and obtain the cross-entropy loss function value output by the trained initial classification model. The step of generating feedback information based on the contrast loss function value includes: The feedback information is generated based on the contrast loss function value and the cross-entropy loss function value.

10. An apparatus for classifying three-dimensional models, comprising: The 3D model data acquisition unit is configured to acquire 3D model data to be classified. A classification unit is configured to determine the category of the three-dimensional model data to be classified using a target classification model, wherein the target classification model is trained based on the apparatus as described in any one of claims 6-9.

11. An electronic device, comprising: At least one processor; as well as A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-5.

12. A non-transitory computer-readable storage medium storing computer instructions, wherein, The computer instructions are used to cause the computer to perform the method according to any one of claims 1-5.

Images