A contrastive self-supervised learning training method and system

Abstract

The embodiment of the application discloses a contrast self-supervised learning training method and system, the method comprises the following steps: a sample data set is calculated with a first data enhancement operator and a data enhancement operator to obtain a data enhancement sample data set; the data enhancement sample data set is input into a first model and a second model to obtain a positive sample feature set and a negative sample feature set; a positive-negative sample combined feature set is obtained according to the original image corresponding to the negative sample feature set and the positive sample feature set; a loss function is calculated according to the positive-negative sample combined feature set; the gradient is calculated based on the loss function and the parameters of the first model are updated, the parameters of the second model are updated based on the updated parameters of the first model, and the cycle is repeated until the loss function meets the set condition. The designed loss function and data enhancement operator are suitable for various contrast self-supervised learning methods and have better generalization and robustness.

Description

TECHNICAL FIELD

[0001] The embodiment of the present application relates to the technical field of computer, in particular to a contrastive self-supervised learning training method and system. BACKGROUND

[0002] Supervised learning has been very successful in the past, but it is heavily dependent on expensive and rare human labels, and is prone to generalization errors, pseudo-relevance and adversarial attacks, so the related research of supervised learning has entered a bottleneck period. Recently, the pre-training mode based on self-supervised learning has made very great progress in computer vision with its good data utilization efficiency and generalization ability, and has formed a set of mainstream training paradigm, called "self-supervised pre-training-task-based fine-tuning".

[0003] The current mainstream model training method mainly uses supervised learning, the core idea of which is to use labeled data to provide strong supervision feedback information for the model. During training, the model can be continuously adjusted by calculating the loss function (i.e. strong supervision feedback information) and using gradient back propagation technology, and slowly fit the target output distribution. As can be seen, supervised learning is a very intuitive and efficient training method, but its shortcomings are also very obvious. It is heavily dependent on expensive and rare human labels, and most of the massive data in specific scenarios in actual situations can only be labeled, leaving a large amount of unlabeled data that cannot be utilized. In addition, training with a small amount of data will make the model have poor generalization and robustness, and be prone to noise attacks and other problems. SUMMARY

[0004] Therefore, the embodiment of the present application provides a contrastive self-supervised learning training method and system, and the designed loss function and data augmentation operator set are suitable for various contrastive self-supervised learning methods and have better generalization and robustness.

[0005] In order to achieve the above purpose, the embodiment of the present application provides the following technical scheme:

[0006] According to the first aspect of the embodiment of the present application, a contrastive self-supervised learning training method is provided, which comprises:

[0007] sampling a sample data set from an unlabeled data set, and selecting a first data augmentation operator and a second data augmentation operator from a data augmentation operator set;

[0008] calculating data augmentation sample data sets from the sample data set and the first data augmentation operator and the second data augmentation operator, respectively;

[0009] inputting the data augmentation sample data sets into a first model and a second model, respectively, to obtain a positive sample feature set and a negative sample feature set;

[0010] According to the original image corresponding to the negative sample feature set and the positive sample feature set, a positive-negative sample combined feature set is obtained;

[0011] According to the positive-negative sample combined feature set, a loss function is calculated;

[0012] Based on the loss function, the gradient is calculated and the parameters of the first model are updated, and the parameters of the second model are updated based on the updated parameters of the first model, and the cycle is repeated until the loss function meets the set condition.

[0013] Optionally, according to the original image corresponding to the negative sample feature set and the positive sample feature set, a positive-negative sample combined feature set is obtained, comprising:

[0014] Randomly sampling from the negative sample feature set and and the positive sample feature set and respectively form a positive-negative sample combined feature set and x p·≠k· represents the original image corresponding to the negative sample feature set, and p·≠k· represents that all images in the sample data set x k are not the same image.

[0015] Optionally, according to the positive-negative sample combined feature set, a loss function is calculated, comprising:

[0016] According to the positive combination feature set, a target function is calculated;

[0017] The target function is taken as a constraint term, and the loss function is calculated based on the InfoNCE loss function.

[0018] Optionally, the target function is taken as a constraint term according to the following formula, and the loss function is calculated based on the InfoNCE loss function:

[0019]

[0020] Wherein, a i· and a i·· respectively represent the first data enhancement operator and the second data enhancement operator, a i and a j are respectively the first data enhancement sample data set and the second data enhancement sample data set in the data enhancement sample data set; F and F' are respectively the first model and the second model with the same structure; X is an unlabeled data set, I is an invariant semantic data set in the unlabeled data set, Y S is a proxy task set for contrastive self-supervised learning; PD is a method for measuring distribution difference distance; σ represents a weight factor.

[0021] Optionally, the conditional distribution of the contrastive self-supervised learning is represented as follows:

[0022]

[0023] wherein P(Y S |I,a i ) and P(Y S t |I,a j ) represent the conditional distribution of the self-supervised learning agent task, and A represents a set of data augmentation operators.

[0024] Optionally, after obtaining the positive sample feature set and the negative sample feature set, the method further comprises:

[0025] storing the negative sample feature set to a negative sample cache area for random sampling.

[0026] According to a second aspect of the embodiments of the present application, a contrastive self-supervised learning training system is provided, the system comprising:

[0027] a data sampling module configured to sample a sample data set from an unlabeled data set, and select a first data augmentation operator and a second data augmentation operator from a set of data augmentation operators;

[0028] an augmented sample module configured to calculate data augmentation sample data sets from the sample data set and the first data augmentation operator and the second data augmentation operator, respectively;

[0029] a positive and negative sample feature module configured to input the data augmentation sample data sets into a first model and a second model, respectively, to obtain a positive sample feature set and a negative sample feature set;

[0030] a combined feature module configured to obtain a positive and negative sample combined feature set according to the original image corresponding to the negative sample feature set and the positive sample feature set;

[0031] a loss function module configured to calculate a loss function according to the positive and negative sample combined feature set;

[0032] an updating module configured to calculate a gradient based on the loss function and update the parameters of the first model, update the parameters of the second model based on the updated parameters of the first model, and repeat until the loss function meets a set condition.

[0033] Optionally, the loss function module is configured to:

[0034] calculate a target function according to the positive and negative combined feature set;

[0035] use the target function as a constraint term, and calculate the loss function based on an InfoNCE loss function.

[0036] According to a third aspect of the embodiments of the present application, an electronic device is provided, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the method of the first aspect.

[0037] According to a fourth aspect of the embodiments of the present application, a computer readable storage medium is provided, having stored thereon computer readable instructions executable by a processor to implement the method of the first aspect.

[0038] To sum up, the embodiments of the present application provide a contrastive self-supervised learning training method and system, sample a sample data set from an unlabeled data set, select a first data augmentation operator and a second data augmentation operator from a data augmentation operator set; the sample data set is calculated to obtain a data augmentation sample data set with the first data augmentation operator and the second data augmentation operator; the data augmentation sample data set is input into a first model and a second model respectively to obtain a positive sample feature set and a negative sample feature set; a positive-negative sample combined feature set is obtained according to the original image corresponding to the negative sample feature set and the positive sample feature set; a loss function is calculated according to the positive-negative sample combined feature set; the gradient is calculated based on the loss function and the parameters of the first model are updated, the parameters of the second model are updated based on the updated parameters of the first model, and the cycle is repeated until the loss function meets the set condition. The designed loss function and data augmentation operator set are suitable for various contrastive self-supervised learning methods, and have better generalization and robustness. BRIEF DESCRIPTION OF DRAWINGS

[0039] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings needed in the following embodiment or prior art description will be briefly introduced. Obviously, the drawings in the following description are only exemplary, and for those skilled in the art, other implementation drawings can be obtained without creative labor on the basis of the provided drawings.

[0040] The structures, proportions, sizes, etc. shown in the specification are only used to cooperate with the content disclosed in the specification, to be understood and read by those skilled in the art, and do not define the limiting conditions for the implementation of the present application, so they do not have technical significance. Any modification of structure, change of proportion relationship or adjustment of size, without affecting the effect and purpose that can be achieved by the present application, should still fall within the scope of the technical content disclosed by the present application.

[0041] Figure 1 A contrastive self-supervised learning training method flowchart is provided for the embodiments of the present application;

[0042] Figure 2 A causal model schematic diagram provided for an embodiment of the present application;

[0043] Figure 3 A training flow schematic diagram provided for an embodiment of the present application;

[0044] Figure 4 A contrastive self-supervised learning training system block diagram provided for an embodiment of the present application;

[0045] Figure 5 A structural schematic diagram of an electronic device provided by an embodiment of the present application is shown;

[0046] Figure 6 A schematic diagram of a computer readable storage medium provided by an embodiment of the present application is shown. DETAILED DESCRIPTION

[0047] The embodiments of the present application will be described in detail by specific embodiments, and those skilled in the art can easily understand other advantages and effects of the present application from the content disclosed in the specification. Obviously, the described embodiments are part of the embodiments of the present application, not all. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of protection of the present application.

[0048] Self-supervised learning is a model training method, which mainly uses auxiliary tasks (pretext) to mine its own supervised information from large-scale unsupervised data, and trains the network through the constructed supervised information, so as to learn the representation valuable for downstream tasks. In order to solve various problems existing in the supervised learning process, the current mainstream solution is mainly based on the training method of self-supervised learning.

[0049] The current self-supervised learning can be divided into three technical directions:

[0050] First, contrastive learning. First, a batch of data is constructed by data augmentation (such as horizontal flip, vertical flip, contrast adjustment, cropping, etc.), that is, the original sample in the batch and the sample based on the enhanced sample are regarded as positive examples, and other samples and samples based on other samples are regarded as negative examples; Then input the positive and negative samples into two neural network models with the same structure, and the two neural network models with the same structure output the embedding representation of the sample at the same time. Finally, through a specific loss function (such as calculating the distance between the two embedding representations), the distance between the positive samples is shortened, and the distance between the negative samples is pushed away, so that the neural network model learns the ability to extract sample representation without labels, replacing the previous pre-training phase which requires large-scale labeled data;

[0051] Second, generative learning. Generally, an "encoder-decoder" structure is adopted, and the main practice is: using an encoder to encode an image into a explicit representation vector, then reconstructing the image through a decoder from the explicit representation vector, and finally minimizing the reconstruction error to enable the model to learn the explicit representation vector containing the latent information under the condition of no label;

[0052] Third, generative contrastive learning (also known as adversarial learning). This method combines the previous two methods and adopts a "generator-discriminator" structure, wherein the generator generally adopts an "encoder-decoder" structure, and the discriminator is generally a classifier structure. The main practice is: first, generating a fake sample through the generator, and then inputting the true and fake samples into the discriminator to distinguish the true and fake samples.

[0053] The existing mainstream self-supervised technology mainly adopts contrastive learning or generative learning. Among them, the mainstream algorithms in the direction of contrastive learning include MoCo series, SimCLR, MOBY, etc. Such algorithms have the following problems: 1. The current mainstream contrastive learning does not have explicit constraints on the representation learning of the model, so it cannot confirm that the representation learned by the model can effectively express the actual effective high-level semantic information in the image, thereby increasing the difficulty of transfer learning of the downstream task; 2. As can be seen from the technical background introduction, contrastive learning relies heavily on data augmentation technology, and in order to ensure the performance of the model, different data augmentation technologies need to be carefully designed under different tasks. Therefore, the performance of the model is limited by the design of the data augmentation technology; 3. The current mainstream contrastive learning lacks actual theoretical support and cannot explicitly explain its rationality and effectiveness.

[0054] In view of the above problems, the embodiment of the present application proposes a contrastive self-supervised learning training method based on an invariant causal mechanism. An interpretable invariant causal mechanism is introduced to explicitly constrain self-supervised learning from two aspects of data augmentation and loss function design, which can not only enable the model to truly learn the actual effective high-level semantic information in the image, but also ensure the "source domain to target domain transfer problem" of the "self-supervised pre-training-task-based fine-tuning" training paradigm.

[0055] Figure 1 The contrastive self-supervised learning training method provided by the embodiment of the present application is shown, and the method comprises:

[0056] Step 101: sampling a sample data set from an unlabeled data set, and selecting a first data augmentation algorithm and a second data augmentation algorithm from a data augmentation algorithm set;

[0057] Step 102: calculating data augmentation sample data sets from the sample data set and the first data augmentation algorithm and the second data augmentation algorithm, respectively;

[0058] Step 103: inputting the data enhanced sample dataset into the first model and the second model respectively to obtain a positive sample feature set and a negative sample feature set;

[0059] Step 104: obtaining a positive-negative sample combined feature set according to the original image corresponding to the negative sample feature set and the positive sample feature set;

[0060] Step 105: calculating a loss function according to the positive-negative sample combined feature set;

[0061] Step 106: calculating a gradient based on the loss function and updating the parameters of the first model, updating the parameters of the second model based on the updated parameters of the first model, and repeating until the loss function meets a set condition.

[0062] In a possible implementation, in step 103, obtaining a positive-negative sample combined feature set according to the original image corresponding to the negative sample feature set and the positive sample feature set includes:

[0063] randomly sampling and from the negative sample feature set and respectively to form a positive-negative sample combined feature set and x p·≠k· represents the original image corresponding to the negative sample feature set, and p·≠k· represents that all images in the sample dataset x k are not the same image.

[0064] In a possible implementation, after obtaining the positive sample feature set and the negative sample feature set in step 103, the method further includes:

[0065] storing the negative sample feature set to a negative sample cache area for random sampling.

[0066] In a possible implementation, in step 105, calculating a loss function according to the positive-negative sample combined feature set includes:

[0067] calculating a target function according to the positive-negative combined feature set; taking the target function as a constraint term, and calculating the loss function based on an InfoNCE loss function.

[0068] In a possible implementation, taking the target function as a constraint term, and calculating the loss function based on an InfoNCE loss function according to the following formula:

[0069]

[0070] wherein a i· and a i··respectively denote a first data augmentation operator and a second data augmentation operator, a i and a j are respectively a first data augmentation sample data set and a second data augmentation sample data set in the data augmentation sample data set; F and F' are respectively a first model and a second model with the same structure; X is an unlabeled data set, I is an invariant semantic data set in the unlabeled data set, Y S is a proxy task set of contrastive self-supervised learning; PD is a method for measuring distribution difference distance; σ represents a weight factor.

[0071] In a possible implementation, the conditional distribution of the contrastive self-supervised learning is expressed as follows:

[0072]

[0073] wherein P(Y S |I,a i ) and P(Y S t |I,a j ) each represent a conditional distribution of a self-supervised learning proxy task, and A represents a set of data augmentation operators.

[0074] The contrastive self-supervised learning training method provided by the embodiments of the present application is further described below with reference to the accompanying drawings.

[0075] First, the core problem solved by self-supervised learning is defined and described: let X be an unlabeled data set, be a set of unknown downstream tasks, wherein Y t represents the tthdownstream task. The core goal of self-supervised learning is to pre-train a model F by utilizing X, and the representation learned by the pre-trained model F* can provide effective information to solve the downstream task set Y.

[0076] In order to more reasonably and effectively solve this core problem, the two constituent elements of "invariant semantics" and "perturbation semantics" in the image are explicitly separated, and then the two elements and the self-supervised proxy task Y S and Y are causally modeled through a causal graph, and the obtained causal graph is as shown in Figure 2 The solid directed line in the causal graph represents "G affects H", and the dashed directed line represents "W is composed of U"; based on the above causal graph and the invariant causal mechanism, the following settings can be obtained:

[0077] (1) all image data X can be regarded as being composed of "invariant semantics" (denoted by I) and "perturbation semantics" (denoted by N). The "invariant semantics" can refer to important content information in the image, such as the attributes of objects in the image; the "perturbation semantics" can refer to factors such as brightness, color temperature, and visual angle of the image;

[0078] (2) For downstream task set Y, it only has relevance with "invariant semantics" and has no association with "perturbation semantics", i.e. "invariant semantics" can be regarded as independent variable, and downstream task set Y result is dependent variable;

[0079] (3) "Invariant semantics" and "perturbation semantics" are independent events, i.e. no matter how "perturbation semantics" changes, the important content information in the image will not change. For example, given an image of "sheepdog", no matter what perturbation is done to the image (such as changing the color temperature of the image), the "sheepdog" information of the image should not be changed.

[0080] For setting (3), it can be represented by the following conditional probability equation:

[0081]

[0082] Wherein, P(Y t |I,n i ) and P(Y t |I,n j ) represent the conditional distribution of downstream tasks (i.e. target domain), n i and n j represent different perturbations.

[0083] This conditional probability equation can also be applied to the proxy task set Y S of self-supervised learning, and the data enhancement operator set A used in self-supervised learning is regarded as "perturbation semantics", i.e.:

[0084]

[0085] Wherein, P(YS|I,a i ) and P(Y S t |I,a j ) represent the conditional distribution of self-supervised learning proxy tasks (i.e. source domain), a i and a j represent different data enhancement operator subsets.

[0086] Based on this formula, a new objective function L new of contrastive self-supervised learning can be obtained:

[0087]

[0088] Wherein, E represents expectation, refers to the expectation of data enhancement under a certain operator subset {a i ,a jThe objective function under} is PD, which refers to any method for measuring the distance of distribution differences, such as KL divergence, JS divergence, Wasserstein distance, etc., and ρ represents the threshold of the distribution difference distance.

[0089] This formula represents a new objective function, where st PD(P(Y)) is the objective function. s |I,a i ), P(Y s |I,a j )) represents the explicit constraint expectation formula

[0090] In other words, the expected value derived from the expected value formula must meet the constraints.

[0091] In machine learning, the objective function is the function that needs to be optimized. Simply put, it provides an optimization direction when solving a problem; once optimization is performed based on the objective function, the solution to the problem can be obtained. The loss function is the function that needs to be minimized in machine learning. Generally, the loss function is the negative value of the objective function, or the negative value of the objective function plus a constraint term (also called a regularization term). The advantage of converting the objective function into a loss function is that the traditional "gradient descent" optimization method can be used to optimize the model parameters (also called training), thus simplifying the optimization problem.

[0092] In this embodiment, the above-mentioned new objective function is used as a constraint term and applied to the InfoNCE loss function of mainstream contrastive learning to obtain a new loss function. This new loss function is then applied to the contrastive self-supervised learning training stage.

[0093]

[0094] Among them, a i· and a i·· They refer to a i The data augmentation operator subset contains different data augmentation operators, where σ is a weighting factor used to adjust the importance of the constraint terms.

[0095] This loss function is obtained by substituting the InfoNCE loss function into the L mentioned above. new In the objective function framework, it is the negative value of the objective function plus constraint terms.

[0096] Finally, the training process is as follows: Figure 3 As shown: x k This represents the k-th batch of data randomly sampled from the unlabeled dataset X. and This represents two data augmentation operators in A*; Indicates sample x k go through the result of the calculation; for example represents the flipping image operation, then represents the image sample x k the flipped image.

[0097] x p·≠k· represents the original image corresponding to all features obtained by negative sample sampling, and the subscript p≠k represents that it is not the same as x k All images in x

[0098] F refers to the model, and similarly is for the feature obtained after calculation by the F model. represents a set of features obtained after calculation by the models F and F.

[0099] L total and Figure 3 The loss function used in and is the same, which is for the entire data set X and the set of data augmentation operators (which can be understood as a general expression for the whole). The expression of the loss function in Figure 3 is for a sample x k in the data set X and two data augmentation operators. It can be understood as an expression for a specific sample and a specific augmentation operator.

[0100] Figure 3 The training process of includes the following steps:

[0101] Step 1: randomly sample a sample x k from the unlabeled data set X;

[0102] Step 2: randomly select two data augmentation operators and

[0103] Step 3: use k and and to obtain data augmented samples and

[0104] Step 4: input and to the model F and the model F', respectively, to obtain and

[0105] Step 5: input and Stored in negative sample buffer (in computer memory, not shown in the figure)

[0106] Step 6: Random sampling from the negative sample buffer and and respectively constitute

[0107] Step 7: Based on and Calculate L total ;

[0108] Step 8: Calculate the gradient based on L total , and then update the parameters of the model F through the back propagation mechanism; F' uses the momentum update method to update the parameters of the updated model F to the parameters of F';

[0109] Step 9: Loop steps 1 to 8 until L total tends to be stable and does not decrease, i.e. end.

[0110] Since the contrastive learning used in the embodiments of the present application uses a twin structure, the models F' and F are of the same structure, but different parameter update methods are used during training, in which F uses the normal gradient back propagation update method, and F' uses the momentum update method in the MoCo series. Through the new loss function L total , the model can be fed back with an explicit supervision signal of "invariant semantics" during training, encouraging the model to actively learn more significant "invariant semantics", thereby narrowing the representation distance of data with the same "invariant semantics".

[0111] In addition to improving the loss function design, the embodiments of the present application also design a corresponding data augmentation operator set A* based on causal graphs and invariant causal mechanisms. The data augmentation operator set only selects actions that enhance the "perturbed semantics" level, including image flipping, image rotation, image translation, image contrast adjustment, image brightness adjustment, image multi-scale random cropping, and image saliency background removal. Through this data augmentation operator set A*, the model performance is greatly improved and can reach the performance of supervised learning.

[0112] In order to solve the problems of poor generalization, poor robustness and strong dependence on labeled data in current supervised learning, the key is how to effectively utilize a large amount of unlabeled data through a self-supervised method and mine truly effective information from a large amount of unlabeled data set to improve generalization and robustness. The embodiment of the present application proposes a contrastive self-supervised learning training method based on an invariant causal mechanism, introduces an invariant causal mechanism that plays a dominant constraint role, designs a new loss function and a data augmentation operator set for the "invariant semantics" to optimize the current contrastive self-supervised learning method, and the loss function and the data augmentation operator set of the embodiment of the present application are applicable to a plurality of contrastive self-supervised learning methods. The model trained by the embodiment of the present application performs well in downstream tasks and reaches or even exceeds the performance of the model trained by supervised learning in a plurality of downstream tasks, and at the same time, has better generalization and robustness.

[0113] In summary, the embodiment of the present application provides a contrastive self-supervised learning training method, which samples a sample data set from an unlabeled data set, selects a first data augmentation operator and a second data augmentation operator from a data augmentation operator set; the sample data set is calculated to obtain a data augmentation sample data set by the first data augmentation operator and the second data augmentation operator; the data augmentation sample data set is input into a first model and a second model to obtain a positive sample feature set and a negative sample feature set; a positive-negative sample combined feature set is obtained according to the original image corresponding to the negative sample feature set and the positive sample feature set; a loss function is calculated according to the positive-negative sample combined feature set; the parameters of the first model are updated based on the gradient calculated based on the loss function, and the parameters of the second model are updated based on the updated parameters of the first model, and the cycle is repeated until the loss function meets the set condition. The designed loss function and data augmentation operator set are applicable to a plurality of contrastive self-supervised learning methods and have better generalization and robustness.

[0114] Based on the same technical concept, the embodiment of the present application also provides a contrastive self-supervised learning training system, as shown in Figure 4 The system comprises:

[0115] The data sampling module 401 is configured to sample a sample data set from an unlabeled data set and select a first data augmentation operator and a second data augmentation operator from a data augmentation operator set;

[0116] The enhanced sample module 402 is configured to calculate a data augmentation sample data set from the sample data set and the first data augmentation operator and the second data augmentation operator;

[0117] The positive-negative sample feature module 403 is configured to input the data augmentation sample data set into a first model and a second model to obtain a positive sample feature set and a negative sample feature set;

[0118] The combination feature module 404 is configured to obtain a positive-negative sample combination feature set according to the original image corresponding to the negative sample feature set and the positive sample feature set.

[0119] The loss function module 405 is configured to calculate a loss function according to the positive-negative sample combination feature set.

[0120] The update module 406 is configured to calculate a gradient based on the loss function and update the parameters of the first model, update the parameters of the second model based on the updated parameters of the first model, and repeat until the loss function meets a set condition.

[0121] In a possible implementation, the loss function module 405 is configured to:

[0122] calculate a target function according to the positive-negative combination feature set; take the target function as a constraint term, and calculate the loss function based on an InfoNCE loss function.

[0123] The embodiments of the present application also provide an electronic device corresponding to the method provided in the foregoing embodiments. Please refer to Figure 5 which shows a schematic diagram of an electronic device provided by some embodiments of the present application. The electronic device 20 can include a processor 200, a memory 201, a bus 202 and a communication interface 203, the processor 200, the communication interface 203 and the memory 201 are connected through the bus 202; the memory 201 stores a computer program which can run on the processor 200, and the processor 200 runs the computer program to execute the method provided in any of the foregoing embodiments of the present application.

[0124] The memory 201 can include a high-speed random access memory (RAM) and can also include a non-volatile memory such as at least one disk memory. The communication connection between the system network element and at least one other network element is realized through at least one physical port 203 (which can be wired or wireless), and the Internet, a wide area network, a local network, a metropolitan area network, etc. can be used.

[0125] The bus 202 can be an ISA bus, a PCI bus or an EISA bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. The memory 201 is used to store a program, and the processor 200 executes the program after receiving an execution instruction. The method disclosed in any of the foregoing embodiments of the present application can be applied to the processor 200 or realized by the processor 200.

[0126] The processor 200 can be an integrated circuit chip having a signal processing capability. In the implementation process, each step of the above method can be completed by the integrated logic circuit or the instruction in the form of software in the processor 200. The processor 200 described above can be a general processor, including a central processing unit (CPU), a network processor (NP), etc.; can also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a ready programmable gate array (FPGA) or other programmable logic device, a discrete gate or transistor logic device, a discrete hardware component. Each method, step and logic block diagram disclosed in the embodiments of the present application can be implemented or executed. The general processor can be a microprocessor or the processor can also be any conventional processor. The steps of the method disclosed in combination with the embodiments of the present application can be directly embodied as a hardware code processor for execution, or a combination of hardware and software modules in the code processor for execution. The software module can be located in a random access memory, a flash memory, a read only memory, a programmable read only memory or an electrically erasable programmable memory, a register, etc. The storage medium in the art. The storage medium is located in the memory 201, and the processor 200 reads the information in the memory 201, and combines the hardware to complete the steps of the above method.

[0127] The electronic device provided by the embodiments of the present application and the method provided by the embodiments of the present application have the same beneficial effects as the method they adopt, run or implement.

[0128] The embodiments of the present application also provide a computer readable storage medium corresponding to the method provided by the preceding embodiments. Please refer to Figure 6 The computer readable storage medium shown is an optical disc 30, and a computer program (i.e. program product) is stored on the optical disc 30. When the computer program is run by a processor, the method provided by any of the preceding embodiments is executed.

[0129] It should be noted that examples of the computer readable storage medium can also include, but are not limited to, a phase change memory (PRAM), a static random access memory (SRAM), a dynamic random access memory (DRAM), other types of random access memory (RAM), a read only memory (ROM), an electrically erasable programmable read only memory (EEPROM), a flash memory or other optical, magnetic storage medium, which will not be described one by one here.

[0130] The computer readable storage medium provided by the above embodiments of the present application and the method provided by the embodiments of the present application have the same beneficial effects as the method adopted, run or implemented by the application program stored therein.

[0131] It is noted that

[0132] The algorithms and displays presented herein are not inherently related to any particular computer, virtual apparatus, or other apparatus. Various general purpose systems can be used with programs in accordance with the teachings herein, or it can prove convenient to construct more specialized apparatus to perform the required method steps. The required structure for a variety of these systems will be apparent from the description above. In addition, the present application is not described with reference to any particular programming language. It will be appreciated that a variety of programming languages can be used to implement the teachings of the application as described herein, and any references below to specific languages are provided for disclosure of enablement only.

[0133] In the description provided herein, numerous specific details are set forth. However, it is understood that embodiments of the application can be practiced without these specific details. In some instances, well-known methods, structures and techniques have not been described in detail in order to avoid obscuring the understanding of this description.

[0134] Similarly, it is to be understood that the mechanical details of the application that have been set forth above in the context of a few illustrative embodiments are for purposes of example only and that various modifications, changes and adaptations will be apparent to those skilled in the art. Number of specific details given herein are meant to be illustrative only and therefore specific embodiments of the application have been shown by way of example in the drawings and will be described in detail in the text which follows. However, it is to be understood that the application is not limited in its application to the details of construction and the arrangement of components set forth in the description or illustrated in the drawings. Methods and apparatuses have been described above by way of example only. It is to be appreciated that those skilled in the art can modify the preferred embodiments as described above and still be within the scope of the application. Accordingly, the scope of the application is defined only by the claims which follow.

[0135] Those skilled in the art will appreciate that the modules in the apparatuses in the embodiments can be adapted and arranged in one or more apparatuses other than the embodiments. The modules or units or components in the embodiments can be combined into one module or unit or component, and further can be divided into more sub-modules or sub-units or sub-components. Any combination of all the features disclosed in the specification (including the accompanying claims, abstract and drawings), and any method or apparatus so disclosed, can be made unless the contrary is explicitly stated by the context. Each feature disclosed in the description (including the accompanying claims, abstract and drawings), can be replaced by alternative features serving the same, equivalent or similar purpose, unless the contrary is explicitly stated by the context.

[0136] Furthermore, those skilled in the art will understand that although some embodiments described herein include certain features but not others included in other embodiments, combinations of features from different embodiments are intended to be within the scope of this application and form different embodiments. For example, in the following claims, any of the claimed embodiments can be used in any combination.

[0137] The various component embodiments of this application can be implemented in hardware, or as software modules running on one or more processors, or a combination thereof. Those skilled in the art will understand that microprocessors or digital signal processors (DSPs) can be used in practice to implement some or all of the functions of some or all of the components in the virtual machine creation apparatus according to embodiments of this application. This application can also be implemented as a device or apparatus program (e.g., a computer program and computer program product) for performing part or all of the methods described herein. Such an implementation of this application can be stored on a computer-readable medium, or can be in the form of one or more signals. Such signals can be downloaded from an Internet website, provided on a carrier signal, or provided in any other form.

[0138] It should be noted that the above embodiments are illustrative of this application and not restrictive, and that those skilled in the art can devise alternative embodiments without departing from the scope of the appended claims. In the claims, any reference signs placed between parentheses should not be construed as limiting the claims. The word "comprising" does not exclude the presence of elements or steps not listed in the claims. The word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. This application can be implemented by means of hardware comprising several different elements and by means of a suitably programmed computer. In the unit claims enumerating several means, several of these means may be embodied by the same item of hardware. The use of the words first, second, and third, etc., does not indicate any order. These words can be interpreted as names.

[0139] The above description is merely a preferred embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A contrastive self-supervised learning training method, characterized in that, The method includes: A sample dataset is obtained by sampling from an unlabeled dataset, and the first and second data augmentation operators are selected from the set of data augmentation operators; The sample dataset is used to calculate the data augmentation sample dataset by the first data augmentation operator and the second data augmentation operator, respectively; The data augmentation sample dataset is input into the first model and the second model respectively to obtain the positive sample feature set and the negative sample feature set; A combined positive and negative sample feature set is obtained based on the original image corresponding to the negative sample feature set and the positive sample feature set; Calculate the objective function based on the combined feature set of positive and negative samples; The objective function is used as a constraint term, and the loss function is calculated based on the InfoNCE loss function. The gradient is calculated based on the loss function and the parameters of the first model are updated. The parameters of the second model are then updated based on the updated parameters of the first model. This process is repeated until the loss function meets the set conditions. The objective function is used as a constraint term, and the loss function is calculated based on the InfoNCE loss function according to the following formula: Among them, a i∙ and a i∙∙ These represent the first data augmentation operator and the second data augmentation operator, respectively. and These are the first and second data augmentation sample datasets in the data augmentation sample dataset; F and F' are the first and second models with the same structure, respectively; X is the unlabeled dataset, I is the invariant semantic dataset in the unlabeled dataset, and Y... S To compare the agent task set for self-supervised learning; PD is a method for measuring the distance of distribution differences; σ represents the weighting factor.

2. The method as described in claim 1, characterized in that, Based on the original image corresponding to the negative sample feature set and the positive sample feature set, a combined positive and negative sample feature set is obtained, including: Randomly sample from the negative sample feature set and and the positive sample feature set and Form positive and negative sample combination feature sets respectively and ;x p∙≠k∙ p represents the original image corresponding to the negative sample feature set, and p∙≠k∙ represents the sample dataset x. k All images in the document are not the same image.

3. The method as described in claim 1, characterized in that, The conditional distribution of contrast-supervised learning is expressed by the following formula: in, and Both represent the conditional distribution of the self-supervised learning agent task, and A represents the set of data augmentation operators.

4. The method as described in claim 1, characterized in that, After obtaining the positive sample feature set and the negative sample feature set, the method further includes: The negative sample feature set is stored in the negative sample buffer for random sampling.

5. A contrastive self-supervised learning training system, characterized in that, The system includes: The data sampling module is used to sample the sample dataset from the unlabeled dataset and select the first and second data augmentation operators from the set of data augmentation operators. An augmented sample module is used to calculate an augmented sample dataset by combining the sample dataset with the first data augmentation operator and the second data augmentation operator, respectively. The positive and negative sample feature module is used to input the data augmentation sample dataset into the first model and the second model respectively to obtain the positive sample feature set and the negative sample feature set; The combined feature module is used to obtain a combined feature set of positive and negative samples based on the original image corresponding to the negative sample feature set and the positive sample feature set; The loss function module is used to calculate the objective function based on the combined positive and negative sample feature set; and to calculate the loss function based on the InfoNCE loss function, using the objective function as a constraint term. The update module is used to calculate the gradient based on the loss function and update the parameters of the first model, update the parameters of the second model based on the updated parameters of the first model, and repeat until the loss function meets the set conditions. The loss function module is used to calculate the loss function based on the InfoNCE loss function, taking the objective function as a constraint term according to the following formula: Among them, a i∙ and a i∙∙ These represent the first data augmentation operator and the second data augmentation operator, respectively. and These are the first and second data augmentation sample datasets in the data augmentation sample dataset; F and F' are the first and second models with the same structure, respectively; X is the unlabeled dataset, I is the invariant semantic dataset in the unlabeled dataset, and Y... S To compare the agent task set for self-supervised learning; PD is a method for measuring the distance of distribution differences; σ represents the weighting factor.

6. An electronic device, comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that the processor executes the computer program to implement the method as claimed in any one of claims 1-4.

7. A computer-readable storage medium, characterized in that, It stores computer-readable instructions that can be executed by a processor to implement the method as described in any one of claims 1-4.

Images