Feature representation model loss function construction method and device, equipment and medium

Abstract

The application relates to the fields of artificial intelligence and intelligent medical treatment, and discloses a loss function construction method of a feature representation model, which comprises the following steps: obtaining training data and inputting the training data into an encoder to obtain a probability distribution of hidden variables corresponding to the training data; sampling a sampling vector in the probability distribution and inputting the sampling vector into a decoder to obtain reconstructed data corresponding to the training data; calculating a reconstruction loss of the training data and the reconstructed data; calculating a contrastive loss of the probability distribution and a prior distribution; calculating a consistency loss of the probability distribution and the prior distribution; generating a plurality of samples by using the reconstructed data, calculating edge distance between all sample pairs, and calculating an edge distance loss according to the edge distance; and determining a total loss function according to the reconstruction loss, the contrastive loss, the consistency loss and the edge distance loss. The method of the application solves the defects of deviation caused by noise distribution in the existing loss function, instability of a normalization exponential function, and design difficulty of a data enhancement or mask strategy.

Description

TECHNICAL FIELD

[0001] The present application relates to the fields of machine learning and smart medical treatment, and in particular to a feature representation model loss function construction method and device, equipment and medium. BACKGROUND

[0002] Unsupervised / self-supervised learning refers to a machine learning method of learning useful knowledge or patterns from data without label information. It can be widely used in medical treatment, finance and other fields. For example, through unsupervised / self-supervised learning, the ability of image recognition is obtained, so as to be used in the analysis of medical images, and to automatically analyze whether the tissue organs in the medical images are pathological tissue or healthy tissue.

[0003] An important goal of self-supervised learning is to learn a feature representation of high-dimensional data, that is, to map the original data to a low-dimensional or abstract space, so that the data in the space has better separability, interpretability and availability. The quality of feature representation directly affects subsequent self-supervised learning tasks such as clustering, dimensionality reduction, generation, etc.

[0004] At present, the commonly used feature representation methods in self-supervised learning mainly include two categories: generative self-supervised learning and contrastive learning. Among them, the generative self-supervised learning has defects such as complex model, difficult optimization, reconstruction error and feature representation quality not necessarily related. The loss function of contrastive learning is based on mutual information, which maximizes the mutual information between positive sample pairs and minimizes the mutual information between negative sample pairs. Mutual information can measure the correlation between two variables, the greater the correlation, the smaller the independence. Existing mainstream loss functions include Noise Contrastive Estimation (NCE), InfoNCE, Contrastive Predictive Coding (CPC), etc. NCE is an approximate method based on noise distribution, which needs to define a noise distribution to sample negative samples. This noise distribution may not conform to the real data distribution, resulting in estimation bias. InfoNCE is a method based on maximum likelihood estimation (MLE), which needs to calculate the normalized exponential function between positive samples and negative samples. This calculation may be affected by numerical stability and gradient disappearance. CPC is a method based on predicting future or missing information, which needs to design appropriate data augmentation or masking strategies to generate data pairs. This strategy may affect the difficulty and diversity of data pairs. SUMMARY

[0005] Therefore, the present application provides a feature representation model loss function construction method, device, medium and equipment, which solves the defects of bias caused by noise distribution in the existing loss function, instability of the normalized exponential function, and difficulty in designing data augmentation or masking strategies.

[0006] In a first aspect, a loss function construction method of a feature representation model is provided, and the method comprises:

[0007] obtaining training data and inputting the training data into an encoder to obtain a probability distribution of a latent variable corresponding to the training data;

[0008] sampling a sampling vector from the probability distribution and inputting the sampling vector into a decoder to obtain reconstruction data corresponding to the training data;

[0009] calculating a reconstruction loss between the training data and the reconstruction data;

[0010] calculating a contrastive loss between the probability distribution and a prior distribution;

[0011] calculating a consistency loss between the probability distribution and the prior distribution;

[0012] generating a plurality of samples using the reconstruction data, calculating edge distance between all sample pairs according to similarity between each two samples, and calculating an edge distance loss according to the edge distance;

[0013] determining a total loss function according to the reconstruction loss, the contrastive loss, the consistency loss, and the edge distance loss.

[0014] In a second aspect, a loss function construction device of a feature representation model is provided, and the device comprises:

[0015] an encoding module configured to obtain training data and input the training data into an encoder to obtain a probability distribution of a latent variable corresponding to the training data;

[0016] a decoding module configured to sample a sampling vector from the probability distribution and input the sampling vector into a decoder to obtain reconstruction data corresponding to the training data;

[0017] a first loss calculation module configured to calculate a reconstruction loss between the training data and the reconstruction data;

[0018] a second loss calculation module configured to calculate a contrastive loss between the probability distribution and a prior distribution;

[0019] a third loss calculation module configured to calculate a consistency loss between the probability distribution and the prior distribution;

[0020] a fourth loss calculation module configured to generate a plurality of samples using the reconstruction data, calculate edge distance between all sample pairs according to similarity between each two samples, and calculate an edge distance loss according to the edge distance.

[0021] a total loss calculation module configured to determine a total loss function according to the reconstruction loss, the contrastive loss, the consistency loss, and the margin distance loss.

[0022] In a third aspect, the present application provides a device comprising a storage medium, a processor, and instructions or code stored on the storage medium and executable on the processor, wherein the processor executes the instructions or code to implement the loss function construction method of the feature representation model.

[0023] In a fourth aspect, the present application provides a medium having instructions or code stored thereon, wherein the instructions or code are executable by a processor to implement the loss function construction method of the feature representation model.

[0024] The loss function construction method, device, equipment, and medium of the feature representation model achieve the scheme without the need to design specific data enhancement or mask strategies, but instead adopt a method based on a variational autoencoder (VAE) to generate data pairs using random latent variables, thereby improving the randomness and diversity between the data pairs. In addition, the embodiment does not need to predefine a noise distribution or calculate a normalization exponential function, but instead adopts a method based on maximum marginal likelihood estimation to directly optimize the margin distance between positive samples and negative samples, thereby improving the estimation efficiency and stability. Further, the embodiment also considers the difference between positive samples and negative samples, thereby improving the discrimination and representativeness between the data pairs.

[0025] The above description is only a summary of the technical solutions of the present application, in order to more clearly understand the technical means of the present application, the following specific embodiments of the present application can be implemented according to the content of the specification, and in order to make the above and other purposes, features and advantages of the present application more obvious and easy to understand, the following specific embodiments of the present application will be described. BRIEF DESCRIPTION OF DRAWINGS

[0026] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following will briefly introduce the drawings needed to be used in the description of the embodiments of the present application. Obviously, the drawings in the following description are only some embodiments of the present application, and those skilled in the art can obtain other drawings according to these drawings without any creative labor.

[0027] Figure 1 Fig. 1 shows a flowchart of a loss function construction method of a feature representation model according to an embodiment of the present application;

[0028] Figure 2 Fig. 2 shows a flowchart of another loss function construction method of a feature representation model according to an embodiment of the present application;

[0029] Figure 3 Fig. 1 shows a flow diagram of a method for constructing a loss function of a feature representation model according to an embodiment of the present application;

[0030] Figure 4 Fig. 2 shows a structural block diagram of a device for constructing a loss function of a feature representation model according to an embodiment of the present application;

[0031] Figure 5 Fig. 3 shows a structural block diagram of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0032] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative work fall within the scope of the present application.

[0033] The method for constructing a loss function of a feature representation model provided by the embodiments of the present application can be applied in a device composed of an electronic device having an instruction or program running capability. The electronic device can be, but is not limited to, various servers, workstations, personal computers, notebook computers, etc. Running on different computing devices is only a difference in the execution subject of the scheme, and those of ordinary skill in the art can predict that running in different computing devices can produce the same technical effects. The present application will be described in detail below through specific embodiments.

[0034] Referring to Figure 1 as shown, Figure 1 Fig. 1 shows a flow diagram of a method for constructing a loss function of a feature representation model according to an embodiment of the present application, which includes the following steps:

[0035] S101: Obtain training data and input the training data into an encoder to obtain a probability distribution of hidden variables corresponding to the training data.

[0036] The method provided by the present application can be used in a feature representation model, and the feature representation model is optimized by constructing a loss function to improve the feature extraction effect of the feature representation model. The feature representation model can be used to extract features in high-dimensional data and map to a low-dimensional or abstract space. For example, image features such as pixel values corresponding to pixel points in medical images are extracted, and multi-dimensional vector representation is used. The multi-dimensional vector representation of the image features can be used for image recognition and image processing. For example, the multi-dimensional representation of the image features can be input into a convolution model for classification processing to obtain a recognition result for the medical image, i.e., whether the tissue or organ in the medical image is a lesion tissue or a healthy tissue.

[0037] In this step, first, the training data is obtained and input into the encoder, and a high-dimensional training data is mapped to a low-dimensional latent variable space using an encoder network, that is, the probability distribution of the latent variable is obtained. The training data can be an image, text, or other formats of data.

[0038] The encoder can be a variational autoencoder (VAE). Unlike a normal autoencoder, the VAE does not directly output a determined latent variable, but outputs a probability distribution of the latent variable, usually assumed to be a Gaussian distribution. This embodiment uses a self-encoder, which uses random latent variables to normal data pairs, without the need to design specific data augmentation or masking strategies. The advantage of this is that it can increase the diversity and continuity of the latent variables, thereby improving the quality and diversity of the generated data.

[0039] S102: Sample a sampling vector in the probability distribution, and input the sampling vector into the decoder to obtain the reconstructed data corresponding to the training data.

[0040] In this step, the value or vector of the latent variable obtained after inputting the training data into the encoder is not a determined latent variable, but a probability distribution of the latent variable, that is, the value of the latent variable has multiple possibilities. Therefore, a decoder network can be used to sample one of the multiple possibilities, that is, a sampling vector, and then use the sampling vector to reconstruct the data to reduce the dimension of the data. The reconstructed data is not a specific value, but a conditional distribution.

[0041] S103: Calculate the reconstruction loss between the training data and the reconstructed data.

[0042] In this step, since the dimensionality of the data is reduced from the training data to the reconstructed data, there may be a certain deviation between the content expressed by the training data and the content expressed by the reconstructed data. Based on this, the reconstruction loss between the training data and the reconstructed data is calculated, and the reconstruction loss is used to measure the difference between the original training data and the reconstructed data. It can be understood that the smaller the reconstruction loss, the smaller the deviation between the content expressed by the reconstructed data and the content expressed by the training data.

[0043] Among them, the binary cross entropy (Binary Cross Entropy, BCE) or the mean squared error (Mean Squared Error, MSE) can be used as the loss function of the reconstruction loss.

[0044] S104: Calculate the contrastive loss between the probability distribution and the prior distribution.

[0045] In this step, the probability distribution of the latent variable ideally conforms to the prior distribution. Based on this, the contrast loss between the probability distribution of the latent variable and the prior distribution is calculated, and the contrast loss is used to measure the difference between the latent variable distribution output by the encoder and the prior distribution. The Kullback-Leibler divergence (KLD) can be used as the contrast loss. The prior distribution can be a normal distribution, and the smaller the contrast loss, the closer the probability distribution of the latent variable is to the normal distribution, and the better the effect of the encoder.

[0046] S105: Calculate the consistency loss between the probability distribution and the prior distribution.

[0047] In this step, the consistency loss between the probability distribution of the latent variable and the prior distribution is calculated. The maximum mean discrepancy (MMD) can be used as the consistency loss. The prior distribution can be a normal distribution, and the smaller the consistency loss, the closer the probability distribution of the latent variable is to the normal distribution, and the better the effect of the encoder.

[0048] S106: Use the reconstructed data to generate multiple samples, calculate the edge distance between all sample pairs according to the similarity between each two samples, and calculate the edge distance loss according to the edge distance.

[0049] In this step, samples are sampled in the conditional distribution of the reconstructed data, and the edge distance is calculated according to the similarity between the samples, and then the edge distance between all sample pairs is summarized to calculate the edge distance loss. The greater the difference between the positive samples and the negative samples, the greater the edge distance loss. By reducing the edge distance loss, the difference between the positive samples and the negative samples can be reduced, and the discrimination and representation between different data pairs can be improved.

[0050] S107: Determine the total loss function according to the reconstruction loss, the contrast loss, the consistency loss, and the edge distance loss.

[0051] In this step, the total loss function is obtained by combining the above-mentioned several losses. For example, the cumulative method or the weighted summation method can be used.

[0052] The embodiment does not require designing specific data enhancement or mask strategies, but adopts a method based on a variational autoencoder (VAE) to generate data pairs using random latent variables, thereby improving the randomness and diversity between data pairs. In addition, the embodiment does not require predefining a noise distribution or calculating a normalization exponential function, but adopts a method based on maximum marginal likelihood estimation to directly optimize the marginal distance between positive and negative samples, thereby improving the estimation efficiency and stability. Further, the embodiment also considers the difference between positive and negative samples, improving the discrimination and representativeness between data pairs.

[0053] Further, as a refinement and extension of the above embodiment, in order to fully describe the specific implementation process of the embodiment, another loss function construction method of feature representation model is provided, as shown in Figure 2 The method comprises the following steps:

[0054] S201: Obtain training data and input the training data into an encoder to obtain the probability distribution of the latent variable corresponding to the training data.

[0055] In this step, the variational autoencoder (VAE) is used to process the training data to obtain the probability distribution of the latent variable, specifically to obtain the mean vector μ and the variance vector σ of the latent variable Z. The specific formula used by the encoder is as follows:

[0056] q φ (|)=N(z;μ(x),σ())

[0057] Where φ is the parameter of the encoder, and μ(x) and σ(x) are the mean vector and variance vector output by the encoder.

[0058] It can be understood that VAE is a generative self-supervised learning model, and the loss function consists of two parts: one is the reconstruction loss, which is used to measure the difference between the original data and the reconstructed data, and usually uses indicators such as mean square error or cross entropy; the second is the contrast loss, which is used to measure the difference between the latent variable distribution output by the encoder and the prior distribution, and usually uses indicators such as KL divergence. The goal of VAE is to minimize the loss function, i.e. to make both the reconstruction error and the contrast error as small as possible.

[0059] S202: Using the reparameterization method, sampling in the probability distribution according to the mean vector and the variance vector to obtain a sampling vector.

[0060] In this step, a latent variable z is sampled from the probability distribution of the latent variable. By the reparameterization trick, the result of the sampling can be introduced into the network calculation to realize the derivability of the sampling process. Specifically, first sample from the standard normal distribution N(0, I), and then introduce the obtained μ and σ of the training data, that is, z = μ + σ ⊙ ∈, wherein ∈ is a noise vector sampled from the standard normal distribution N(0, I).

[0061] S203: Input the sampling vector into the decoder to obtain the reconstructed data corresponding to the training data.

[0062] In this step, the specific formula used by the decoder is as follows:

[0063] p θ (|)=f(z)

[0064] Where θ is the parameter of the decoder, p θ (|) is a conditional distribution, and f(z) is the reconstructed data output by the decoder. According to the conditional distribution p φ (|) of the reconstructed data, some new samples can be obtained, and then positive samples and negative samples are extracted from the samples for the calculation of the margin loss.

[0065] S204: Calculate the reconstruction loss between the training data and the reconstructed data.

[0066] In this step, the reconstruction loss L r The specific formula is as follows:

[0067]

[0068] Or

[0069]

[0070] Where n is the dimension of the input data, x i and x’ i are the i-th elements of the input training data and the reconstructed data, respectively.

[0071] S205: Calculate the contrast loss between the probability distribution and the prior distribution.

[0072] In this step, the contrast loss L d The specific calculation formula is as follows:

[0073]

[0074] Where d is the dimension of the latent variable, σ j and μ j are the j-th elements of the variance vector and the mean vector of the latent variable.

[0075] S206: Calculate the consistency loss between the probability distribution and the prior distribution.

[0076] In this step, the consistency loss L c The specific calculation formula is as follows:

[0077]

[0078] Where n is the number of input data, m is the number of sampled data, z i and z' i are the latent variables of input data and sampled data, k(z i ,z j ) is a Gaussian kernel function, defined as k(z i ,z j ) = exp(-γ||z i -z j || 2 ), where γ is a hyperparameter representing the kernel width.

[0079] S207: Sample in the conditional distribution corresponding to the reconstructed data to obtain multiple samples.

[0080] S208: Extract all positive sample pairs in the multiple samples, and calculate the similarity between each pair of positive sample pairs respectively.

[0081] S209: Extract all negative sample pairs in the multiple samples, and calculate the similarity between each pair of negative sample pairs respectively.

[0082] S210: Determine the edge distance between the positive sample pairs and the negative sample pairs according to the similarity between each pair of positive sample pairs and the similarity between each pair of negative sample pairs respectively.

[0083] In steps S207-S210, multiple samples are obtained by sampling in the conditional distribution of the reconstructed data. According to the different categories corresponding to the content of each sample, two samples can form a pair of positive sample pairs or negative sample pairs. All positive sample pairs and negative sample pairs are extracted from the multiple samples, and the similarity of the positive sample pairs and the negative sample pairs is calculated respectively, and then the edge distance between the positive sample pairs and the negative sample pairs is calculated according to the similarity. It can be understood that the positive sample refers to the sample belonging to a certain category, and the negative sample refers to the sample not belonging to a certain category, for example, in identifying medical images of lesion organs, if sample z i and sample z j correspond to data of medical images of lesion organs, sample z k corresponds to data of medical images of healthy organs, then sample z i and sample z j are a pair of positive sample pairs, and sample z i and sample z kis a pair of negative sample pairs. Specifically, first, the similarity s i between the positive sample pair z j and z ij is calculated. The cosine similarity or dot product can be used as the similarity function. Then, the similarity s i between the negative sample pair z k and z ik is calculated, and the calculation method is the same as that of the positive sample pair. Then, the edge distance between the positive sample pair and the negative sample pair is calculated according to the two similarities. It can be understood that the smaller the similarity between the positive sample pairs, the greater the similarity between the negative sample pairs, and then it can be considered that the difference between the positive sample and the negative sample is smaller, and it is more difficult to distinguish the positive sample and the negative sample, and therefore the edge distance loss is greater.

[0084] wherein the edge distance between the positive sample pair and the negative sample pair is determined according to the similarity between each positive sample pair and the similarity between each negative sample pair, and the edge distance comprises:

[0085] S2101: each positive sample pair is taken as a to-be-calculated positive sample pair, and a plurality of to-be-calculated negative sample pairs corresponding to the to-be-calculated positive sample pair are determined, wherein the to-be-calculated positive sample pair and the to-be-calculated negative sample pair have the same sample;

[0086] S2102: the difference between the similarity between the to-be-calculated positive sample pair and the similarity between each to-be-calculated negative sample pair is calculated, and the sum between the difference and a preset hyperparameter is calculated;

[0087] S2103: if the sum is greater than 0, the edge distance between the to-be-calculated positive sample pair and the to-be-calculated negative sample pair is determined as the sum;

[0088] S2104: if the sum is not greater than 0, the edge distance between the to-be-calculated positive sample pair and the to-be-calculated negative sample pair is determined as 0.

[0089] In steps S2101-S2104, the edge distance between two sample pairs is calculated according to the similarity between the positive sample pairs and the similarity between the negative sample pairs, wherein the two sample pairs should have the same sample. For example, for the positive sample pair z i and z j and the negative sample pair z i and z k , the same sample z i exists. The edge distance m ijk between the positive sample pair and the negative sample pair is calculated by the following formula:

[0090] m ijk = max(0, m + sik -s ij ), where m is a hyper-parameter representing the margin size, s ij and s ik are the similarity between positive sample pairs and the similarity between negative sample pairs, respectively.

[0091] S211: Average all margin distances and take the average as the margin distance loss.

[0092] In this step, the margin distance between each pair of positive sample pairs and negative sample pairs is comprehensively analyzed to obtain the total margin distance loss. The margin loss L m The specific calculation formula is as follows:

[0093]

[0094] where n is the number of input data, m ijk is the margin distance between positive sample pairs z i and z j and negative sample pairs z i and z k , s ij is the similarity function, defined as s ij = cos(z i , z j ) or represents the cosine similarity or dot product.

[0095] S212: Determine the first weight coefficient according to the contrast loss, and weight the contrast loss using the first weight coefficient to obtain the weighted contrast loss.

[0096] S213: Determine the second weight coefficient according to the margin distance loss, and weight the consistency loss using the second weight coefficient to obtain the weighted consistency loss.

[0097] S214: Accumulate the reconstruction loss, the weighted contrast loss, the weighted consistency loss, and the margin distance loss to obtain the total loss function.

[0098] In steps S212-S214, the first weight coefficient and the second weight coefficient are determined according to the contrast loss and the margin distance loss, respectively, and the contrast loss and the consistency loss are weighted using the weight coefficients, respectively, to balance the importance of reconstruction and contrast through the two weight coefficients. Specifically, the first weight coefficient a and the second weight coefficient b can be calculated using the following formula:

[0099] a = exp(-L d ), b = exp(-L m ), where L d and L mrespectively are contrast loss and consistency loss.

[0100] The total loss L can be calculated by the following formula:

[0101] L = L r + a x L d + b x L c + m

[0102] wherein L r is reconstruction loss, a x L d is weighted contrast loss, b x L c is weighted consistency loss, and L m is edge distance loss.

[0103] S215: According to the total loss value corresponding to the total loss function, the parameters of the encoder and the parameters of the decoder are adjusted, and the step of inputting the training data into the encoder is returned until the total loss value meets the preset optimization condition or the number of parameter adjustments reaches the preset cycle number.

[0104] In this step, the encoder and the decoder are optimized according to the total loss value of the total loss function, so as to better extract features. Specifically, the optimization algorithm is used to adjust the parameters of the encoder and the decoder in the direction of reducing the total loss value, and the step of inputting the training data into the encoder is returned. The new hidden variable probability distribution and the reconstructed data are obtained by using the adjusted encoder and decoder, and then the new total loss value is obtained. If the stop condition is still not met at this time, the step of adjusting the parameters of the encoder and the decoder is repeatedly executed until the stop condition is reached. The stop condition can be determined according to the total loss value, for example, the total loss value is less than a preset loss threshold, or the total loss value converges, or the number of cycles can be determined, for example, the number of parameter adjustments reaches a preset cycle number. The optimization method can use gradient descent method, etc. algorithm, or can use genetic algorithm, etc.

[0105] Figure 3 A flowchart of a loss function construction method of a feature representation model of an embodiment of the application is shown. As shown in the figure, the method comprises the following specific steps:

[0106] (1) Input data X, through the encoder Encoder, to obtain the mean vector μ and the variance vector σ of the hidden variable Z.

[0107] (2) According to μ and σ, using the reparameterization (Reparameterization) technique, a sampling vector z of the hidden variable Z is generated.

[0108] (3) According to z, through the decoder Decoder, the reconstructed data X' is obtained.

[0109] (4) Calculate the reconstruction loss L between X and X' r The loss function is either Binary Cross Entropy (BCE) or Mean Squared Error (MSE).

[0110] (5) Calculate the divergence loss L between μ and σ and the standard normal distribution N(0,1). d We use KL divergence (KLD) as the loss function.

[0111] (6) Calculate the consistency loss L between z and the standard normal distribution N(0,1). c MMD is used as the loss function.

[0112] (7) Calculate the positive sample pair z i and z j Similarity s between ij Cosine similarity or dot product is used as the similarity function.

[0113] (8) Calculate the negative sample pairs z i and z k Similarity s between ik Same as above.

[0114] (9) Calculate the edge distance m between positive sample pairs and negative sample pairs. ijk Using m ijk =max(0,m+s) ik -s ij ) is used as the edge distance function, where m is a hyperparameter representing the edge size.

[0115] (10) Calculate the average edge distance M between all data pairs, as the edge distance loss L. m The estimate.

[0116] (11) According to L d and L m Calculate the weighting coefficients α and β of the loss function, using α = exp(-L d ) and β = exp(-L m () is used as the weighting coefficient function.

[0117] (12) Calculate the total loss L = L r +α*L d +β*L c +L m It then backpropagates to update the parameters of the encoder and decoder.

[0118] It should be understood that the size of the serial number of each step in the above embodiment does not mean the order of execution, and the execution order of each process should be determined according to its function and internal logic, and should not constitute any limitation on the implementation process of the embodiment of the application.

[0119] It can be seen that, compared with the traditional loss function, the scheme combines the advantages of generative self-supervised learning and contrast learning, which can not only utilize the internal structure and diversity of data, but also effectively distinguish different categories of data. It can effectively utilize unlabeled data, learn high-level feature representation of data, and improve the performance of downstream tasks; in addition, the scheme can avoid some defects of existing generative contrast / self-supervised learning loss functions, such as bias of noise distribution, instability of normalization exponential function, difficulty in designing data enhancement or masking strategy, etc.; using variational autoencoder as the basic model, the random generation of hidden variables can improve the generalization ability of the network, and new data can also be generated; on this basis, the scheme also sets reconstruction loss, consistency loss, etc., which can consider the local similarity, global consistency and dynamic balance between data pairs, so as to improve the randomness, diversity, discrimination and representativeness between data pairs; finally, by flexibly adjusting the weight coefficient of the loss function, according to the marginal distance and maximum average difference between positive samples and negative samples, the flexibility and robustness of the loss function can be improved.

[0120] In an embodiment, a feature representation model loss function construction device is provided, which corresponds to the feature representation model loss function construction method in the above embodiment. As shown in the figure, the feature representation model loss function construction device comprises an encoding module, a decoding module, a first loss calculation module, a second loss calculation module, a third loss calculation module, a fourth loss calculation module and a total loss calculation module. The functions of each module are described in detail as follows: Figure 4

[0121] The encoding module is used to obtain training data and input the training data into the encoder to obtain the probability distribution of the hidden variable corresponding to the training data;

[0122] The decoding module is used to sample a sampling vector in the probability distribution and input the sampling vector into the decoder to obtain the reconstructed data corresponding to the training data;

[0123] The first loss calculation module is used to calculate the reconstruction loss between the training data and the reconstructed data;

[0124] The second loss calculation module is used to calculate the contrast loss between the probability distribution and the prior distribution;

[0125] The third loss calculation module is used to calculate the consistency loss between the probability distribution and the prior distribution;​

[0126] a fourth loss calculation module configured to generate a plurality of samples using the reconstructed data, calculate edge distances between all sample pairs according to similarities between each two samples, and calculate an edge distance loss according to the edge distances;

[0127] a total loss calculation module configured to determine a total loss function according to the reconstruction loss, the contrastive loss, the consistency loss, and the edge distance loss.

[0128] In an embodiment, the total loss calculation module is configured to:

[0129] weight the contrastive loss to obtain a weighted contrastive loss, and weight the consistency loss to obtain a weighted consistency loss;

[0130] accumulate the reconstruction loss, the weighted contrastive loss, the weighted consistency loss, and the edge distance loss to obtain the total loss function.

[0131] In an embodiment, the total loss calculation module is configured to:

[0132] determine a first weight coefficient according to the contrastive loss, and weight the contrastive loss using the first weight coefficient to obtain a weighted contrastive loss;

[0133] determine a second weight coefficient according to the edge distance loss, and weight the consistency loss using the second weight coefficient to obtain a weighted consistency loss.

[0134] In an embodiment, the fourth loss calculation module is configured to:

[0135] sample in a conditional distribution corresponding to the reconstructed data to obtain the plurality of samples;

[0136] extract all positive sample pairs in the plurality of samples, and calculate similarities between each pair of positive sample pairs respectively;

[0137] extract all negative sample pairs in the plurality of samples, and calculate similarities between each pair of negative sample pairs respectively;

[0138] determine edge distances between the positive sample pairs and the negative sample pairs according to the similarities between each pair of positive sample pairs and the similarities between each pair of negative sample pairs respectively;

[0139] average all the edge distances, and take the average as the edge distance loss.

[0140] In an embodiment, the fourth loss calculation module is configured to:

[0141] respectively take each positive sample pair as a to-be-calculated positive sample pair, and determine a plurality of to-be-calculated negative sample pairs corresponding to the to-be-calculated positive sample pair, wherein the to-be-calculated positive sample pair and the to-be-calculated negative sample pairs have the same sample.

[0142] Calculate the difference between the similarity between positive sample pairs to be calculated and the similarity between each negative sample pair to be calculated, and calculate the sum of the difference and the preset hyperparameters;

[0143] If the sum is greater than 0, then the marginal distance between the positive sample pair to be calculated and the negative sample pair to be calculated is determined as the sum.

[0144] If the sum is not greater than 0, then the marginal distance between the positive sample pair to be calculated and the negative sample pair to be calculated is determined to be 0.

[0145] In one embodiment, the apparatus further includes an optimization module for:

[0146] Based on the total loss value corresponding to the total loss function, adjust the parameters of the encoder and the decoder, and return to the step of inputting training data into the encoder until the total loss value meets the preset optimization conditions or the number of parameter adjustments reaches the preset number of loops.

[0147] In one embodiment, the decoding module is used for:

[0148] By using the reparameterization method, sampling vectors are obtained from the probability distribution based on the mean vector and variance vector.

[0149] In one embodiment, an electronic device is provided. This electronic device can be a mobile device such as a mobile phone, tablet, or in-vehicle mobile terminal, or other device with program execution capabilities. The internal structure diagram of this electronic device can be as follows: Figure 5 As shown, the electronic device includes a processor, memory, and a network module. The processor provides computing and control capabilities. The memory includes non-volatile and / or volatile storage media and internal memory. The non-volatile storage media stores an operating system, instructions, or code. The internal memory provides an environment for the operation of the operating system and instructions or code in the non-volatile storage media. When executed by the processor, the instructions or code implement the functions or steps of a loss function construction method for the aforementioned feature representation model. The network module may include a network interface and / or a wireless network module, allowing the electronic device to communicate with other devices or service platforms. Furthermore, the electronic device may also include a display screen and input devices.

[0150] In one embodiment, an electronic device is provided, including a memory, a processor, and instructions or code stored in the memory and executable on the processor, wherein the processor executes the instructions or code to perform the following steps:

[0151] Acquire training data and input the training data into the encoder to obtain the probability distribution of the latent variables corresponding to the training data;

[0152] sample a vector in the probability distribution, and input the sample vector into the decoder to obtain the reconstruction data corresponding to the training data;

[0153] calculate a reconstruction loss between the training data and the reconstruction data;

[0154] calculate a contrastive loss between the probability distribution and the prior distribution;

[0155] calculate a consistency loss between the probability distribution and the prior distribution;

[0156] generate a plurality of samples using the reconstruction data, calculate edge distances between all sample pairs according to the similarity between each two samples, and calculate an edge distance loss according to the edge distances;

[0157] determine a total loss function according to the reconstruction loss, the contrastive loss, the consistency loss, and the edge distance loss.

[0158] In an embodiment, a storage medium having instructions or codes stored thereon is provided, the instructions or codes being executed by a processor to implement the following steps:

[0159] obtain training data, and input the training data into an encoder to obtain a probability distribution of a latent variable corresponding to the training data;

[0160] sample a vector in the probability distribution, and input the sample vector into the decoder to obtain the reconstruction data corresponding to the training data;

[0161] calculate a reconstruction loss between the training data and the reconstruction data;

[0162] calculate a contrastive loss between the probability distribution and the prior distribution;

[0163] calculate a consistency loss between the probability distribution and the prior distribution;

[0164] generate a plurality of samples using the reconstruction data, calculate edge distances between all sample pairs according to the similarity between each two samples, and calculate an edge distance loss according to the edge distances;

[0165] determine a total loss function according to the reconstruction loss, the contrastive loss, the consistency loss, and the edge distance loss.

[0166] It should be noted that the functions or steps that the above storage medium or electronic device can implement can be referred to the related description in the foregoing method embodiments, and will not be described here again to avoid repetition.

[0167] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing relevant hardware through instructions or codes, and the instructions or codes can be stored in a non-volatile readable storage medium. When the instructions or codes are executed, the processes of the above-mentioned embodiments of each method can be included. Any reference to memory, storage, database or other medium used in each embodiment provided by the present application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration but not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0168] Those skilled in the art can clearly understand that, for the convenience and brevity of description, only the above-mentioned division of functional units and modules is exemplified, and in actual application, the above-mentioned functions can be completed by different functional units and modules according to needs, that is, the internal structure of the device is divided into different functional units or modules to complete all or part of the functions described above.

[0169] Those skilled in the art can understand that the units or processes in the drawings are not necessarily required for implementing the present application. Those skilled in the art can understand that the units in the system in the implementation scenario can be distributed in the system in the implementation scenario according to the description of the implementation scenario, or can be changed to be located in one or more systems different from the implementation scenario. The units of the above-mentioned implementation scenario can be combined into one unit, or can be further split into multiple sub-units.

[0170] The above-mentioned embodiments are only used to illustrate the technical solutions of the present application, but not to limit it; although the present application has been described in detail with reference to the above-mentioned embodiments, those skilled in the art should understand that the technical solutions recorded in the above-mentioned embodiments can be modified, or some technical features can be replaced by equivalents; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application, and should be included in the protection scope of the present application.

Claims

1. A method for constructing a loss function for a feature representation model, characterized in that, The method includes: Acquire training data and input the training data into an encoder to obtain the probability distribution of the latent variables corresponding to the training data; the training data is an image or text. A sampling vector is obtained by sampling from the probability distribution, and the sampling vector is input into the decoder to obtain the reconstructed data corresponding to the training data; Calculate the reconstruction loss between the training data and the reconstructed data; Calculate the contrast loss between the probability distribution and the prior distribution; Calculate the consistency loss between the probability distribution and the prior distribution; Multiple samples are obtained by sampling from the conditional distribution corresponding to the reconstructed data; all positive sample pairs are extracted from the multiple samples, and the similarity between each positive sample pair is calculated; all negative sample pairs are extracted from the multiple samples, and the similarity between each negative sample pair is calculated; the edge distance between the positive sample pair and the negative sample pair is determined based on the similarity between each positive sample pair and the similarity between each negative sample pair; the mean of all edge distances is calculated, and the mean is used as the edge distance loss; wherein: each positive sample pair is used as a positive sample pair to be calculated, and multiple negative sample pairs to be calculated are determined corresponding to the positive sample pair to be calculated, wherein there are identical samples in the positive sample pair to be calculated and the negative sample pair to be calculated; the difference between the similarity between the positive sample pair to be calculated and the similarity between each negative sample pair to be calculated is calculated, and the sum of the difference and the preset hyperparameter is calculated; if the sum is greater than 0, the edge distance between the positive sample pair to be calculated and the negative sample pair to be calculated is determined as the sum; otherwise, the edge distance between the positive sample pair to be calculated and the negative sample pair to be calculated is determined as 0; Based on the reconstruction loss, the contrast loss, the consistency loss, and the edge distance loss, a total loss function is determined. Based on the total loss value corresponding to the total loss function, the parameters of the encoder and the decoder are adjusted, and the process returns to the step of inputting the training data into the encoder until the total loss value meets the preset optimization conditions or the number of parameter adjustments reaches the preset number of iterations.

2. The method according to claim 1, characterized in that, The step of determining the total loss function based on the reconstruction loss, the contrast loss, the consistency loss, and the edge distance loss includes: The weighted comparison loss is obtained by weighting the comparison loss, and the weighted consistency loss is obtained by weighting the consistency loss. The total loss function is obtained by summing the reconstruction loss, the weighted comparison loss, the weighted consistency loss, and the edge distance loss.

3. The method according to claim 2, characterized in that, The step of weighting the comparison loss to obtain a weighted comparison loss and weighting the consistency loss to obtain a weighted consistency loss includes: A first weighting coefficient is determined based on the contrast loss, and the contrast loss is weighted using the first weighting coefficient to obtain a weighted contrast loss; The second weighting coefficient is determined based on the edge distance loss, and the consistency loss is weighted using the second weighting coefficient to obtain the weighted consistency loss.

4. The method according to claim 1, characterized in that, The sampling vector obtained by sampling from the probability distribution of the latent variable includes: Using a reparameterization method, the sampling vector is obtained by sampling from the probability distribution based on the mean vector and variance vector of the probability distribution.

5. A loss function construction apparatus for a feature representation model, characterized in that, The device includes: An encoding module is used to acquire training data and input the training data into an encoder to obtain the probability distribution of the latent variables corresponding to the training data; the training data is an image or text. A decoding module is used to sample a sampling vector from the probability distribution and input the sampling vector into a decoder to obtain reconstructed data corresponding to the training data. The first loss calculation module is used to calculate the reconstruction loss between the training data and the reconstructed data; The second loss calculation module is used to calculate the contrast loss between the probability distribution and the prior distribution; The third loss calculation module is used to calculate the consistency loss between the probability distribution and the prior distribution; The fourth loss calculation module is used to sample from the conditional distribution corresponding to the reconstructed data to obtain multiple samples; extract all positive sample pairs from the multiple samples and calculate the similarity between each positive sample pair; extract all negative sample pairs from the multiple samples and calculate the similarity between each negative sample pair; determine the edge distance between the positive sample pair and the negative sample pair based on the similarity between each positive sample pair and the similarity between each negative sample pair; calculate the mean of all the edge distances and use the mean as the edge distance loss; wherein: each positive sample pair is used as the edge distance loss. Calculate positive sample pairs and determine multiple negative sample pairs corresponding to the positive sample pairs to be calculated, wherein there are identical samples in the positive sample pairs and the negative sample pairs to be calculated; calculate the difference between the similarity between the positive sample pairs to be calculated and the similarity between each negative sample pair to be calculated, and calculate the sum of the difference and a preset hyperparameter; if the sum is greater than 0, determine the edge distance between the positive sample pairs to be calculated and the negative sample pairs to be calculated as the sum; otherwise, determine the edge distance between the positive sample pairs to be calculated and the negative sample pairs to be calculated as 0. The total loss calculation module is used to determine the total loss function based on the reconstruction loss, the contrast loss, the consistency loss, and the edge distance loss. The optimization module is used to adjust the parameters of the encoder and decoder according to the total loss value corresponding to the total loss function, and return to the step of inputting training data into the encoder until the total loss value meets the preset optimization conditions or the number of parameter adjustments reaches the preset number of loops.

6. The apparatus according to claim 5, characterized in that, The total loss calculation module is also used for: Weighted comparison loss is obtained by weighting the comparison loss, and weighted consistency loss is obtained by weighting the consistency loss. The total loss function is obtained by summing the reconstruction loss, weighted contrast loss, weighted consistency loss, and edge distance loss.

7. The apparatus according to claim 6, characterized in that, The total loss calculation module is also used for: The first weighting coefficient is determined based on the contrast loss, and the contrast loss is weighted using the first weighting coefficient to obtain the weighted contrast loss; The second weighting coefficient is determined based on the edge distance loss, and the consistency loss is weighted using the second weighting coefficient to obtain the weighted consistency loss.

8. The apparatus according to claim 5, characterized in that, The decoding module is used for: By using the reparameterization method, sampling vectors are obtained from the probability distribution based on the mean vector and variance vector.

9. A storage medium having a program or instructions stored thereon, characterized in that, When the program or instructions are executed by the processor, they implement the method as described in any one of claims 1 to 4.

10. An electronic device comprising a storage medium, a processor, and a computer program stored on the storage medium and executable on the processor, characterized in that, When the processor executes the program, it implements the method of any one of claims 1 to 4.

Images