Batch prior contrast learning framework and semantic structure alignment method

Abstract

The invention relates to the field of artificial intelligence, and discloses a batch prior comparison learning framework and a semantic structure alignment method, and the learning framework comprises a prior embedding generation module which carries out the processing of input data batches; the target prior distribution generation module is used for generating data-dependent target prior distribution for each anchor point sample in the data batch; the model embedding generation module is used for processing the data batches subjected to data enhancement; the model prediction distribution generation module is used for generating model prediction distribution for each anchor point sample in the data batch; and the model optimization module is used for calculating loss according to the difference between the target prior distribution and the model prediction distribution. According to the method, a fixed prior encoder is introduced, target prior distribution is generated for batch data to serve as a soft supervision signal, a main encoder is guided to align an internal semantic structure between the data, and the quality and generalization ability of finally obtained feature representation are improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of artificial intelligence, in particular to a batch prior contrast learning framework and a semantic structure alignment method. BACKGROUND

[0002] In the field of artificial intelligence, automatically learning the inherent laws and valuable patterns from massive unstructured data such as images and texts is the core foundation to realize various intelligent applications. High-quality data feature representation is the key to determine the upper limit of the performance of these applications. However, obtaining high-quality feature representation usually relies on a large amount of manually annotated data for model training, which not only has high cost, but also consumes time and effort. Therefore, developing self-supervised learning technology that can learn effective feature representation directly from unannotated data plays an important role in reducing the threshold of artificial intelligence applications and improving the popularity of technology.

[0003] Currently, a mainstream technology in the field of self-supervised learning is contrast learning. The core mechanism of the contrast learning framework represented by SimCLR and other methods is to construct a positive sample pair by performing different data augmentation operations on the same data sample, and at the same time, all other samples in the data batch are regarded as negative samples. Without any manual annotation, this technology trains a deep neural network model (encoder) to enable it to pull the representations of the positive sample pair closer and push the representations of the negative sample pair further apart in the embedding space. The advantage of existing contrast learning technology is that it successfully breaks away from the dependence on large-scale manually annotated data and can autonomously learn highly discriminative general-purpose feature representations from massive data. These features achieve comparable performance to supervised learning methods when transferred to downstream tasks such as image classification and object detection.

[0004] However, the effectiveness of existing contrast learning technology is based on a key assumption: in the data batch, except for the positive sample, the remaining samples are semantically unrelated to the anchor. In actual applications, especially when the batch capacity increases, this assumption is often violated, exposing its technical deficiencies. On the one hand, the randomly sampled data batch inevitably contains semantically similar samples (e.g., two pictures of different cars). According to the existing instance discrimination mechanism, these semantically similar samples are incorrectly designated as negative samples and pushed away by the model in the embedding space, which is the "pseudo-negative problem" that directly interferes with the model's correct learning of the inherent semantic structure of the data. On the other hand, its learning goal is defined as a hard instance discrimination task, and the supervision signal is one-hot, i.e., only requiring the model to identify the unique positive sample. This non-yes or no supervision method ignores the varying degrees of semantic affinity between samples (e.g., the similarity between "car" and "truck" is higher than that between "car" and "cat"), preventing the model from modeling the rich semantic relationships between samples. SUMMARY

[0005] In view of the deficiencies of the prior art, the present application provides a batch prior contrast learning framework and a semantic structure alignment method, which solves the problem of pseudo-negative examples caused by forcibly pushing away samples with similar semantics within a batch as negative samples when the existing contrast learning method performs instance discrimination.

[0006] To achieve the above object, the present application is implemented by the following technical solutions:

[0007] The present application provides a batch prior contrast learning framework in the first aspect, which comprises:

[0008] A prior embedding generation module is used to process an input data batch by using a pre-trained and weight-frozen prior encoder to generate corresponding prior embeddings;

[0009] A target prior distribution generation module receives the prior embeddings and generates a data-dependent target prior distribution for each anchor sample within the data batch based on the similarity between the prior embeddings;

[0010] A model embedding generation module is used to process the data batch after data augmentation by using a main encoder to be trained to generate corresponding model embeddings;

[0011] A model prediction distribution generation module receives the model embeddings and generates a model prediction distribution for each anchor sample within the data batch based on the similarity between the model embeddings;

[0012] A model optimization module is connected to the target prior distribution generation module and the model prediction distribution generation module to receive the target prior distribution and the model prediction distribution, and calculates a loss based on the difference between the target prior distribution and the model prediction distribution, and then updates the model parameters of the main encoder according to the loss.

[0013] Preferably, in the prior embedding generation module, the prior encoder is a pre-trained neural network model, and the parameters of the neural network model remain fixed during the entire training process of the main encoder.

[0014] In one specific embodiment, the target prior distribution generation module comprises:

[0015] A similarity calculation unit is used to calculate the cosine similarity between the prior embedding of an anchor sample and the prior embedding of other samples within the batch

[0016] ​a thresholding processing unit configured to process the computed similarity according to a preset similarity threshold the computed similarity to generate a prior , which is computed as follows:

[0017] ;

[0018] wherein, is a prior temperature coefficient; denotes the prior logit between the sample and the sample after thresholding processing; denotes a cosine similarity function for computing the similarity between two vectors; denotes the prior embedding vector generated by the prior encoder for the i-th sample; denotes the prior embedding vector generated by the prior encoder for the i-th sample; denotes the prior embedding vector generated by the prior encoder for the i-th sample; denotes a preset similarity threshold; denotes negative infinity; denotes a conditional judgment for defining different computing paths of the piecewise function.

[0019] a distribution generating unit configured to apply a Softmax function to the prior , thereby generating the target prior distribution .

[0020] In one specific embodiment, the model embedding generating module comprises:

[0021] a data augmentation unit configured to perform a preset data augmentation operation on the input data batch to generate the data-augmented data batch;

[0022] an embedding generating unit configured to input the data-augmented data batch into the to-be-trained main encoder to generate the model embedding.

[0023] In one specific embodiment, the model prediction distribution generating module comprises:

[0024] a computing unit configured to compute the cosine similarity between the model embedding of the anchor sample and the model embedding of other samples within the batch

[0025] a generating unit configured to scale the cosine similarity by a temperature coefficient and apply a Softmax function, thereby generating the model prediction distribution , which is computed as follows:​​

[0026] ;

[0027] wherein, is the model predicted distribution; is the exponential function in mathematics, i.e. the exponential operation with the natural constant as the base; is the function, representing the cosine similarity calculation; represents the model embedding vector generated by the main encoder to be trained for the i-th sample; represents the model embedding vector generated by the main encoder to be trained for the i-th sample; represents the model embedding vector generated by the main encoder to be trained for the i-th sample; represents the model embedding vector generated by the main encoder to be trained for the i-th sample; represents the model embedding vector generated by the main encoder to be trained for the i-th sample; represents the temperature coefficient; dummy variable, as an index, traversing all samples in the data batch; index of the anchor sample.

[0028] In one specific embodiment, the model optimization module comprises:

[0029] a loss calculation unit, configured to calculate a hybrid training target as the loss, the hybrid training target is a weighted combination of the cross-entropy loss between the target prior distribution and the model predicted distribution and the standard lnfoNCE loss ;

[0030] a parameter updating unit, configured to update the model parameters of the main encoder by a back propagation algorithm according to the hybrid training target .

[0031] The second aspect of the present application provides a semantic structure alignment method of a batch prior contrast learning framework, which comprises the following steps:

[0032] S1, input a data batch to the prior encoder, and process it by the prior encoder to generate the prior embedding as a semantic structure reference;

[0033] S2, based on the generated prior embedding, generate the data-dependent target prior distribution for each anchor sample in the data batch by calculating the similarity between the prior embeddings;

[0034] S3, perform a data augmentation operation on the input data batch, and provide the data batch after data augmentation to the main encoder for processing to generate the model embedding;​

[0035] S4. Based on the generated model embedding, the model prediction distribution is generated for each anchor sample in the data batch by calculating the similarity between the model embeddings.

[0036] S5. According to the generated target prior distribution and the generated model prediction distribution, the difference between the target prior distribution and the model prediction distribution is calculated to obtain a loss value, and the model parameters of the main encoder are updated according to the loss value.

[0037] In one embodiment, in step S2, the generated target prior distribution of data dependency specifically includes:

[0038] The cosine similarity between the prior embedding of the anchor sample and the prior embedding of other samples in the batch is calculated.

[0039] According to the preset similarity threshold T, the cosine similarity is filtered, the similarity lower than the similarity threshold T is excluded in the subsequent normalization calculation, and the similarity not lower than the similarity threshold T is temperature coefficient scaling processing;

[0040] The Softmax function is applied to the filtered and scaled values to generate the target prior distribution.

[0041] In one embodiment, in step S4, the generated model prediction distribution specifically includes:

[0042] The cosine similarity between the model embedding of the anchor sample and the model embedding of other samples in the batch is calculated.

[0043] A uniform temperature coefficient is applied to all calculated cosine similarity values Scaling processing;

[0044] The Softmax function is applied to the scaled values to generate the model prediction distribution.

[0045] In one embodiment, in step S5, the difference between the target prior distribution and the model prediction distribution is calculated to obtain a loss value, specifically including:

[0046] The cross-entropy loss between the target prior distribution and the model prediction distribution is calculated to obtain a semantic structure alignment loss component;

[0047] The InfoNCE loss based on the model embedding calculation standard is obtained to obtain an instance discrimination loss component;

[0048] The semantic structure alignment loss component and the instance discrimination loss component are combined by weighting according to a preset weight factor to obtain a final loss value.

[0049] The application provides a batch prior contrastive learning framework and a semantic structure alignment method.

[0050] 1、The application generates prior embedding by setting a prior encoder, and generates a data-dependent target prior distribution for samples in a batch by using a target prior distribution generation module. The target distribution is not a traditional one-hot distribution, but allows multiple samples within the batch that are similar in semantics to the anchor sample to have a non-zero target probability. Therefore, during model optimization, the problem of false negative examples in contrastive learning is effectively alleviated, and the semantic consistency of the learned representation is improved.

[0051] 2、The application generates a sparse and data-dependent soft target prior distribution by using a pre-trained prior encoder and combining a threshold processing unit. Compared with traditional contrastive learning, which only relies on the supervision signal of a single positive sample pair, the target distribution provided by the application contains multi-dimensional and key semantic relationships between samples in the batch. This provides more rich and refined supervision information for the training of the main encoder, guiding the model to learn a representation space that can reflect complex semantic structures, rather than just distinguishing different instances.

[0052] 3、The hybrid training target adopted by the application combines the semantic structure alignment loss component and the instance discrimination loss component by weighting, achieving the collaborative optimization of the two learning goals. The instance discrimination loss component ensures that the representation learned by the model has sufficient discriminability, while the semantic structure alignment loss component guides the model to organize the samples in the embedding space in accordance with semantic logic. This combination enables the final learned feature representation to have both high discriminability and good semantic structure. BRIEF DESCRIPTION OF DRAWINGS

[0053] Figure 1 The learning framework architecture of the application;

[0054] Figure 2 The alignment method flowchart of the application.

[0055] 10, a prior embedding generation module; 20, a target prior distribution generation module; 30, a model embedding generation module; 40, a model prediction distribution generation module; 50, a model optimization module. DETAILED DESCRIPTION

[0056] With reference to the drawings, the technical solutions in the embodiments of the present application will be clearly and completely described in the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without creative work fall within the protection scope of the present application.

[0057] With reference to Figure 1 , Figure 1 is a structural block diagram of a batch prior contrast learning framework according to an embodiment of the present application. The present application provides a batch prior contrast learning framework, which comprises a prior embedding generation module 10, a target prior distribution generation module 20, a model embedding generation module 30, a model predicted distribution generation module 40 and a model optimization module 50.

[0058] In a specific embodiment of the present application, the overall working principle of the framework is as follows: the framework receives an input data batch and inputs it into two parallel processing paths. The first path uses the prior embedding generation module 10 and the target prior distribution generation module 20 to generate a target prior distribution for the data batch as the basis for subsequent model parameter updates. The second path uses the model embedding generation module 30 and the model predicted distribution generation module 40 to generate a model predicted distribution reflecting the current model state for the same data batch.

[0059] The outputs of the two paths, i.e. the target prior distribution and the model predicted distribution, are transmitted to the model optimization module 50. The model optimization module 50 calculates a loss value according to the difference between the two distributions, and updates the parameters of the main encoder in the model embedding generation module 30 according to the loss value. The process is iteratively performed on multiple data batches to complete the training of the main encoder.

[0060] Specifically, in the first processing path, the prior embedding generation module 10 receives the input data batch and generates a corresponding prior embedding for each data sample in the batch through its internally pre-trained and weight-frozen prior encoder. Subsequently, the generated prior embeddings are output to the target prior distribution generation module 20. The target prior distribution generation module 20 processes the received prior embeddings, calculates the similarity between the samples, and generates a data-dependent target prior distribution for each anchor sample in the batch.

[0061] In the second processing path, the model embedding generation module 30 first performs data augmentation operation on the input data batch, and then sends the data-augmented data batch to the main encoder inside it to be trained to generate corresponding model embeddings. These generated model embeddings are output to the model prediction distribution generation module 40. The model prediction distribution generation module 40 processes the received model embeddings, calculates the similarity between samples, and generates a model prediction distribution for each anchor sample in the batch.

[0062] Finally, the input end of the model optimization module 50 is connected with the output end of the target prior distribution generation module 20 and the output end of the model prediction distribution generation module 40, respectively, for receiving the target prior distribution and the model prediction distribution. The output of the model optimization module 50 acts on the main encoder in the model embedding generation module 30 to realize the update of the model parameters.

[0063] Referring to Figure 1 The function of the prior embedding generation module 10 is to receive an input data batch and generate a corresponding prior embedding vector in a high-dimensional feature space for each data sample in the data batch. The output of the prior embedding generation module 10, i.e., the generated prior embedding set, will be the direct basis for the subsequent module to generate a target prior distribution.

[0064] The core component of the prior embedding generation module 10 is the prior encoder. In a specific embodiment, the prior encoder is a pre-trained neural network model. The pre-training means that the prior encoder has completed training on a large-scale, general data set (for example, the ImageNet data set in the field of computer vision), so that it has the ability to extract universal features. The specific network structure of the prior encoder is not particularly limited and can be various deep neural network structures known to those skilled in the art, such as the ResNet series model based on convolutional neural network (CNN) or the Vision Transformer (ViT) model based on Transformer structure.

[0065] In the embodiment of the present application, the model parameters of the prior encoder are kept weight frozen during the entire training period of the main encoder. The specific technical implementation of weight freezing is that during the training process, the gradient of the model parameters of the prior encoder is not calculated or is set to zero, so the optimizer will not update these parameters. This setting ensures that for the same input sample, the prior embedding vector output by the prior embedding generation module 10 is determined and constant at any stage of training, thereby providing a stable reference benchmark for subsequent semantic structure alignment.

[0066] In specific operation, when a data batch containing input data batch of data samples When fed into the prior embedding generation module 10, the prior encoder will independently perform forward propagation computation on each sample in the batch (where ) and the process can be represented as:

[0067] ;

[0068] wherein denotes the -th input sample in the data batch; denotes the mapping function represented by the prior encoder; denotes the prior embedding vector generated for sample The prior embedding generation module 10 finally outputs a set of prior embedding vectors .

[0069] In an optional implementation, the prior embedding vectors output from the prior encoder are normalized, e.g. L2 normalization, before being passed to the target prior distribution generation module 20. This operation scales the length of each embedding vector to 1, making the subsequent cosine similarity based computation more stable.

[0070] The output of the prior embedding generation module 10 is connected to the input of the target prior distribution generation module 20, for passing the generated set of prior embedding vectors to the next processing stage.

[0071] Referring to Figure 1 , the input of the target prior distribution generation module 20 is connected to the output of the prior embedding generation module 10, and the function is to receive the set of prior embedding vectors output by the prior embedding generation module 10, and based on the set, generate for each anchor sample in the data batch a target prior distribution that is data-dependent and serves as a supervisory signal. The output of the target prior distribution generation module 20, i.e. the target prior distribution, will be passed to the model optimization module 50.

[0072] In a specific embodiment, the internal processing flow of the target prior distribution generation module 20 can be decomposed into three units: a similarity calculation unit, a thresholding processing unit and a distribution generation unit.

[0073] The similarity calculation unit processes the input set of prior embedding vectors . For any sample in the data batch that is selected as an anchor sample, the unit will calculate the cosine similarity between its prior embedding vector and all other samples wherein the cosine similarity between the prior embedding vectors of .

[0074] Next, the thresholding unit receives all the similarity values outputted by the similarity calculation unit. The function of the thresholding unit is to filter and process the similarity values according to a pre-set similarity threshold to generate a set of prior logits. The purpose of this process is to retain the information of sample pairs with high semantic similarity to the anchor sample while excluding the interference introduced by low similarity sample pairs. The specific calculation process is shown in the following formula:

[0075] ;

[0076] wherein, is the prior temperature coefficient; represents the prior logits between sample and sample after thresholding processing; represents the cosine similarity function used to calculate the similarity between two vectors; represents the prior embedding vector generated by the prior encoder for the th sample; represents the prior embedding vector generated by the prior encoder for the th sample; represents the pre-set similarity threshold; represents negative infinity; represents the conditional judgment used to define the different calculation paths of the piecewise function.

[0077] The distribution generation unit receives the set of prior logits generated by the thresholding unit (wherein is not involved in the calculation). The distribution generation unit applies the Softmax function to this set of logits for normalization, thereby generating the final target prior distribution for the anchor sample . The calculation process is shown in the following formula:

[0078] ;

[0079] wherein, represents the target probability of sample given the anchor sample ; is the exponential function; and are the prior logits calculated by the thresholding unit; the index ​Traversing all samples in the data batch except the anchor samples The target prior distribution generation module 20 outputs a corresponding target prior distribution for each anchor sample in the batch.

[0080] Referring to Figure 1 , the function of the model embedding generation module 30 is to receive an input data batch and generate a corresponding model embedding vector for each data sample in the data batch through its internal main encoder to be trained. The model embedding generation module 30 works in parallel with the prior embedding generation module 10, and the model embedding set output by the model embedding generation module 30 will be used to generate a model prediction distribution reflecting the current model learning state.

[0081] In a specific embodiment, the internal processing flow of the model embedding generation module 30 can be divided into two units: a data augmentation unit and an embedding generation unit.

[0082] The data augmentation unit processes the original input data batch received. Data augmentation is a technique that transforms data without changing the semantic label of the data sample itself to increase data diversity. In this embodiment, the data augmentation unit applies one or more pre-set data augmentation operations to each sample in the batch. Non-limiting examples of the data augmentation operations include random cropping, random horizontal or vertical flipping, color jittering (including random adjustment of brightness, contrast, and saturation), Gaussian blur, and random grayscale, etc. After processing, the data augmentation unit outputs a data-augmented data batch.

[0083] Next, the embedding generation unit receives the data-augmented data batch output by the data augmentation unit. The core component of this unit is a main encoder to be trained, denoted as Unlike the prior encoder, the main encoder Model parameters are initialized at the beginning of training and continuously updated throughout the training process by the model optimization module 50. The network structure of the main encoder may be various deep neural networks known to those skilled in the art, such as ResNet or Vision Transformer, etc., which can be the same as or different from the prior encoder.

[0084] In specific operations, when a data-augmented data batch is sent into the embedding generation unit, the main encoder performs forward propagation calculation on each augmented sample in the batch, the process of which can be represented as:

[0085] ;

[0086] wherein, denotes the -th data-augmented sample in the batch; denotes the mapping function represented by the main encoder, whose parameter is ; denotes the model embedding vector generated for sample . The final output of this module is a set consisting of model embedding vectors of all samples in the batch.

[0087] In an optional implementation, the model embedding vectors output from the main encoder are also subject to an L2 normalization before being passed to the model prediction distribution generation module 40, to ensure that the lengths of all embedding vectors are unified to 1.

[0088] The output of the model embedding generation module 30 is connected to the input of the model prediction distribution generation module 40, for passing the generated model embedding set to the next processing stage. Meanwhile, the main encoder in this module is also connected to the model optimization module 50, for receiving the update instruction of model parameters.

[0089] Referring to Figure 1 , the input of the model prediction distribution generation module 40 is connected to the output of the model embedding generation module 30, and the function of the model prediction distribution generation module 40 is to receive the model embedding set output by the model embedding generation module 30, and generate a model prediction distribution for each anchor sample in the data batch based on the set. This distribution reflects the judgment of the main encoder to be trained on the semantic similarity between samples in the batch at the current training stage. The output of the model prediction distribution generation module 40, i.e., the model prediction distribution, will be transmitted to the model optimization module 50.

[0090] In a specific embodiment, the internal processing flow of the model prediction distribution generation module 40 can be divided into two units: a calculation unit and a generation unit.

[0091] The calculation unit processes the input model embedding set . For any sample selected as an anchor sample in the data batch , this unit calculates the cosine similarity between its model embedding vector and the model embedding vectors of all other samples in the batch.

[0092] Next, the generation unit receives all the similarity values output by the similarity calculation unit. The function of this unit is to convert these unfiltered similarity values into a probability distribution by temperature scaling and Softmax normalization. The specific calculation process is as follows:

[0093] ;

[0094] where, is the model prediction distribution; is the exponential function in mathematics, i.e., the exponential operation with the natural constant as the base; is the function representing the cosine similarity calculation; represents the model embedding vector generated by the main encoder to be trained for the th sample; represents the model embedding vector generated by the main encoder to be trained for the th sample; represents the model embedding vector generated by the main encoder to be trained for the th sample; represents the temperature coefficient; is a dummy variable that serves as an index to iterate through all samples in the data batch; is the index of the anchor sample.

[0095] The model prediction distribution generation module 40 performs the above operation for each anchor sample within the batch, generates a corresponding model prediction distribution for each anchor sample, and outputs it.

[0096] Referring to Figure 1 , the input end of the model optimization module 50 is connected to the output end of the target prior distribution generation module 20 and the output end of the model prediction distribution generation module 40, respectively. The function of the model optimization module 50 is to receive the target prior distribution and the model prediction distribution output by the target prior distribution generation module 20 and the model prediction distribution generation module 40, respectively, to calculate a hybrid training target as the total loss according to the difference between the two, and to perform updates on the model parameters of the main encoder in the model embedding generation module 30 according to the total loss.

[0097] In a specific embodiment, the internal processing flow of the model optimization module 50 can be divided into two units: a loss calculation unit and a parameter update unit.

[0098] The function of the loss calculation unit is to calculate a hybrid training target . This hybrid training target is composed of two loss components combined by weighting: one is the cross-entropy loss , another is the standard InfoNCE loss for instance discrimination

[0099] for each anchor sample in the data batch , its semantic structure alignment loss component is obtained by computing the cross-entropy between the target prior distribution and the model predicted distribution . The computation process is shown in the following formula:

[0100] ;

[0101] wherein, is the output of the target prior distribution generation module 20; is the output of the model predicted distribution generation module 40; is the natural logarithm function; denotes an integral, indicating the semantic structure alignment loss component computed for a single anchor sample is the lower index of summation, defining the value range of the summation variable .

[0102] the instance discrimination loss component is computed based on the model embedding output by the model embedding generation module 30. For an anchor sample and its positive sample pair within the same batch (usually different data augmentation views of the same original sample), the computation process of the loss is shown in the following formula:

[0103] ;

[0104] wherein, denotes the model embedding vector generated by the main encoder to be trained for the th sample; denotes the model embedding vector generated by the main encoder to be trained for the th sample; denotes the model embedding vector generated by the main encoder to be trained for the th sample; is a function denoting the cosine similarity calculation; denotes the temperature coefficient; is the natural logarithm function; is the instance discrimination loss value computed for a single anchor sample ; is the exponential function in mathematics, i.e., the exponential operation with the natural constant as the base; is the index of summation, defining the summation variable .

[0105] The loss calculation unit combines the two loss components by a preset weight factor to obtain the final mixed training target . The final loss of the entire data batch is the average of all anchor sample losses. The calculation process is as follows:

[0106] ;

[0107] In the formula, is the batch size; is a weight factor with a value range between 0 and 1, used to balance the contribution of the two loss components; is the index of the anchor sample; represents a whole, referring to the final calculated mixed training total loss used to update the model parameters; is the instance discrimination loss value calculated for a single anchor sample ; represents a whole, referring to the semantic structure alignment loss component calculated for a single anchor sample .

[0108] The parameter updating unit receives the mixed training target calculated by the loss calculation unit . According to the loss value, the parameter updating unit calculates the gradient of the loss with respect to the main encoder model parameters by the backpropagation algorithm. Then, an optimizer (such as the stochastic gradient descent (SGD) or Adam optimizer) is used to update the model parameters of the main encoder according to the calculated gradient. In this updating process, the model parameters of the prior encoder in the prior embedding generation module 10 remain fixed.

[0109] Referring to Figure 2 , Figure 2 is the flowchart of the semantic structure alignment method according to an embodiment of the present application. The present application provides a semantic structure alignment method of a batch prior contrastive learning framework. The method is executed as follows for an input data batch in a training cycle:

[0110] S1, generating prior embeddings, step S1 corresponds to the work of the prior embedding generation module 10. An input data batch containing a plurality of data samples is provided. The data batch is input to a pre-trained and model parameter fixed prior encoder. The prior encoder performs independent forward propagation calculation on each sample in the batch, maps each sample to a high-dimensional feature space, thereby generating a set of prior embeddings corresponding to the input samples. This set of prior embeddings provides a stable semantic structure reference for the subsequent steps.

[0111] S2, generating target prior distribution, step S2 corresponds to the work of the target prior distribution generation module 20. Step S2 receives the set of prior embeddings generated by step S1. For each anchor sample in the batch, first calculate the cosine similarity between its prior embedding and the prior embedding of all other samples in the batch. Then, according to a pre-set similarity threshold , filter and process the calculated similarity values: for the similarity not lower than the threshold , use the prior temperature coefficient to scale it; for the similarity lower than the threshold , set its corresponding logit value to negative infinity in the subsequent calculation to exclude its probability contribution after normalization. Finally, apply the Softmax function to the set of filtered and scaled values to generate a data-dependent target prior distribution for the anchor sample.

[0112] S3, generating model embeddings, step S3 corresponds to the work of the model embedding generation module 30, which is executed in parallel with steps S1 and S2. Data augmentation is performed on the same input data batch as in step S1 to generate an augmented data batch. Subsequently, the augmented data batch is input to a to-be-trained main encoder. The main encoder performs forward propagation calculation on each sample in the augmented batch to generate a corresponding set of model embeddings. This set of model embeddings reflects the feature extraction capability of the main encoder in the current training state.

[0113] S4, generating model prediction distribution, step S4 corresponds to the work of the model prediction distribution generation module 40. Step S4 receives the set of model embeddings generated by step S3. For each anchor sample in the batch, first calculate the cosine similarity between its model embedding and the model embedding of all other samples in the batch. Then, apply a uniform temperature coefficient to scale the calculated cosine similarity values, and apply the Softmax function to the scaled values to normalize them, thereby generating a model prediction distribution for the anchor sample.

[0114] S5, calculating loss and optimizing model. Step S5 corresponds to the work of the model optimization module 50. This step S5 receives the target prior distribution generated by step S2 and the model prediction distribution generated by step S4. A hybrid training target is calculated as the final loss value. The loss value is specifically composed of two parts: the first part is to calculate the cross entropy between the target prior distribution and the model prediction distribution, and the semantic structure alignment loss component is obtained; the second part is to calculate the standard InfoNCE loss based on the model embedding, and the instance discrimination loss component is obtained. Then, according to the final loss value, the gradient of the loss with respect to the main encoder model parameters is calculated by the back propagation algorithm, and the optimizer is used to perform an optimization update on the model parameters of the main encoder.

Claims

1. A batch prior contrastive learning framework, characterized in that, The learning framework comprises: A priori embedding generation module, configured to process an input data batch by using a pre-trained and weight-frozen priori encoder to generate corresponding priori embedding; A target priori distribution generation module, configured to receive the priori embedding and generate a data-dependent target priori distribution for each anchor sample in the data batch based on the similarity between the priori embedding; A model embedding generation module, configured to process the data batch after data augmentation by using a main encoder to be trained to generate corresponding model embedding; A model prediction distribution generation module, configured to receive the model embedding and generate a model prediction distribution for each anchor sample in the data batch based on the similarity between the model embedding; A model optimization module, connected to the target priori distribution generation module and the model prediction distribution generation module respectively to receive the target priori distribution and the model prediction distribution, and to calculate a loss based on the difference between the target priori distribution and the model prediction distribution, and to update the model parameters of the main encoder according to the loss.

2. The batch prior contrastive learning framework of claim 1, wherein, In the priori embedding generation module, the priori encoder is a pre-trained neural network model, and the parameters of the neural network model remain fixed during the entire training process of the main encoder.

3. The batch prior contrastive learning framework of claim 1, wherein, The target priori distribution generation module comprises: a similarity computation unit configured to compute a cosine similarity between the prior embedding of the anchor sample and the prior embedding of the other sample with the prior embedding of the other sample within the batch ;​ a thresholding processing unit configured to process the computed similarity according to a preset similarity threshold the computed similarity is processed to generate a priori The computation process is as follows: ； In the formula, The a priori temperature coefficient; This indicates that after thresholding, the sample... With sample Prior logit between; This represents the cosine similarity function, used to calculate the similarity between two vectors; This indicates that the prior encoder is for the first... Prior embedding vectors generated from each sample; This indicates that the prior encoder is for the first... Prior embedding vectors generated from each sample; This indicates the preset similarity threshold; Indicates negative infinity; This indicates a conditional statement used to define different computation paths for a piecewise function. Distribution generation unit, used for the prior The Softmax function is applied to generate the target prior distribution. .

4. The batch prior contrastive learning framework of claim 1, wherein, The model embedding generation module comprises: A data augmentation unit, configured to perform a preset data augmentation operation on the input data batch to generate the data batch after data augmentation; An embedding generation unit, configured to input the data batch after data augmentation to the main encoder to be trained to generate the model embedding.

5. The batch prior contrastive learning framework of claim 1, wherein, The model prediction distribution generation module comprises: a computing unit for computing a model embedding of the anchor sample with the model embeddings of the other samples of the batch a cosine similarity between ; A generating unit is configured to scale the cosine similarity by a temperature coefficient and apply a Softmax function, thereby generating the model prediction distribution The calculation process is as follows: ； wherein, is the model prediction distribution; is the exponential function in mathematics, i.e. the exponential operation with the natural constant as the base; is the function, representing the cosine similarity calculation; represents the model embedding vector generated by the main encoder to be trained for the th sample; represents the model embedding vector generated by the main encoder to be trained for the th sample; represents the model embedding vector generated by the main encoder to be trained for the th sample; represents the temperature coefficient; is a dummy variable, as an index, to iterate over all samples in a data batch; is the index of the anchor sample.

6. The batch prior contrastive learning framework of claim 1, wherein, The model optimization module comprises: a loss computation unit configured to compute a hybrid training objective as the loss, the hybrid training objective is a weighted combination of a cross-entropy loss between the target prior distribution and the model predicted distribution and a standard lnfoNCE loss ​ a parameter updating unit configured to update model parameters of the main encoder according to the mixed training target by a back propagation algorithm.

7. A method for aligning semantic structures of a batch prior contrastive learning framework according to any one of claims 1-6, wherein, The method comprises the following steps: S1, inputting a data batch to the priori encoder and processing the data batch by the priori encoder to generate the priori embedding as a semantic structure reference; S2, generating the data-dependent target priori distribution for each anchor sample in the data batch based on the generated priori embedding by calculating the similarity between the priori embedding; S3, performing a data augmentation operation on the input data batch and providing the data batch after data augmentation to the main encoder to process the data batch by the main encoder to generate the model embedding; S4, generating the model prediction distribution for each anchor sample in the data batch based on the generated model embedding by calculating the similarity between the model embedding; S5, calculating the difference between the target priori distribution and the model prediction distribution to obtain a loss value according to the generated target priori distribution and the generated model prediction distribution, and performing optimization update on the model parameters of the main encoder according to the loss value.

8. The method of Claim 7, wherein, In step S2, the target priori distribution generation module comprises: calculating the cosine similarity between the priori embedding of the anchor sample and the priori embedding of other samples in the batch; The cosine similarities are filtered according to a preset similarity threshold T, similarities lower than the similarity threshold T are excluded in subsequent normalization calculation, and similarities not lower than the similarity threshold T are subjected to temperature coefficient scaling processing; The Softmax function is applied to the filtered and scaled values to generate the target prior distribution.

9. The method of claim 7, wherein, In step S4, the generating of the model prediction distribution specifically includes: calculating the cosine similarity between the model embedding of the anchor point sample and the model embedding of other samples in the batch; applying a uniform temperature coefficient to all computed cosine similarity values scaling; The Softmax function is applied to the scaled values for normalization to generate the model prediction distribution.

10. The method of claim 7, wherein, In step S5, the calculating of the difference between the target prior distribution and the model prediction distribution to obtain a loss value specifically includes: calculating the cross-entropy loss between the target prior distribution and the model prediction distribution to obtain a semantic structure alignment loss component; calculating the InfoNCE loss based on the model embedding standard to obtain an instance discrimination loss component; According to a preset weight factor, the semantic structure alignment loss component and the instance discrimination loss component are weighted and combined to obtain the final loss value.

Images