Training method and device of pre-trained classification model, equipment and medium

Abstract

The application relates to the technical fields of machine learning, artificial intelligence and medical health, and proposes a pre-training classification model training method, device, equipment and medium, which comprises the following steps: acquiring a batch training sample set; determining a target label category contained in the batch training sample set; expanding the batch training sample set according to the target label category and an initial label embedding layer corresponding to the target label category, to obtain an expanded batch training sample set; and training a pre-training classification model and the label embedding layer by using a contrast learning loss function according to the expanded batch training sample set, to obtain a trained target classification model and a target label embedding layer. Through the technical scheme, the model training process is simplified, overfitting is avoided, and the generalization effect is improved.

Description

[Technical Field]

[0001] This application relates to the fields of machine learning, artificial intelligence and healthcare technology, and in particular to a training method, apparatus, device and medium for a pre-trained classification model. [Background Technology]

[0002] In the healthcare field, during smart healthcare implementation, users can input voice commands. The backend categorizes the user's intent based on the voice and then provides corresponding feedback. This requires using a pre-trained classification model to classify user intents. Similarly, in the medical insurance field, when reviewing various medical documents submitted by users, it is necessary to classify the types of medical documents, which also requires the use of a pre-trained classification model.

[0003] Existing pre-trained classification models typically employ the cross-entropy loss function for fine-tuning classification tasks. This loss function is a sample-level learning objective; during training, it only focuses on the match between the predicted result and the true label for each sample, without considering the commonalities between samples, especially those with the same label. While the cross-entropy loss function performs well on some tasks, it suffers from unstable performance and is prone to overfitting on more difficult tasks.

[0004] Existing literature has proposed supervised contrastive learning, which aims to make the representations of samples with the same label converge and increase the distance between the representations of samples with different labels, so that the model can learn the semantic relationships between samples. However, this supervised contrastive learning has the following two problems: (1) it will make the representations of samples with the same label too concentrated, which is called representation collapse. This makes the decision boundary of the model too narrow and it is easy to misclassify samples during testing; (2) this type of method still requires the model to train a classification head using the cross-entropy loss function. [Summary of the Invention]

[0005] This application provides a training method, apparatus, device, and medium for a pre-trained classification model, aiming to solve the technical problems in related technologies where the cross-entropy loss function of classification models is prone to overfitting, and existing contrastive learning is prone to misclassifying samples.

[0006] In a first aspect, embodiments of this application provide a training method for a pre-trained classification model, comprising:

[0007] Obtain a batch of training sample sets;

[0008] Determine the target label categories contained in the batch training sample set;

[0009] Based on the target label category and its corresponding initial label embedding layer, the batch training sample set is expanded to obtain the expanded batch training sample set.

[0010] Based on the expanded batch training sample set, the pre-trained classification model and the label embedding layer are trained using the contrastive learning loss function to obtain the trained target classification model and target label embedding layer.

[0011] Secondly, embodiments of this application provide a training apparatus for a pre-trained classification model, comprising:

[0012] The acquisition module is used to acquire batch training sample sets;

[0013] The determination module is used to determine the target label category contained in the batch training sample set;

[0014] The expansion module expands the batch training sample set according to the target label category and its corresponding initial label embedding layer to obtain the expanded batch training sample set.

[0015] The training module is used to train the pre-trained classification model and the label embedding layer using the contrastive learning loss function based on the expanded batch training sample set, so as to obtain the trained target classification model and target label embedding layer.

[0016] Thirdly, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the above-described infectious disease spatial prediction method.

[0017] Fourthly, a computer-readable storage medium is provided, which stores a computer program that, when executed by a processor, implements the steps of the above-described infectious disease spatial prediction method.

[0018] The above-described scheme, implemented using a pre-trained classification model training method, apparatus, device, and medium, allows for the acquisition of a batch training sample set; determination of the target label categories contained in the batch training sample set; expansion of the batch training sample set based on the target label categories and their corresponding initial label embedding layers to obtain an expanded batch training sample set; and training of the pre-trained classification model and the label embedding layers using a contrastive learning loss function based on the expanded batch training sample set to obtain the trained target classification model and target label embedding layer. In this invention, the proposed pre-trained classification model training method involves comparative learning between samples based on labels, and the input of samples and label embeddings into a unified semantic space. This allows the learning of sample representations to directly align with the classification task, facilitating the formation of high-quality decision boundaries. Furthermore, it eliminates the need for the model to learn a classification head using a cross-entropy loss function, simplifying the model training process. Based on the expanded batch training sample set, a contrastive learning loss function is used to train the pre-trained classification model and label embedding layer. This ensures that the label embeddings are evenly distributed across the semantic space, thus widening the distance between samples and resulting in a more uniform distribution. This avoids overfitting and improves generalization performance. This invention is applicable to data classification in the field of medical and health technology. For example, in the field of medical consultation, patients can input medical questions, and the pre-trained classification model classifies the patient's intent based on these questions. Similarly, in the field of medical insurance, when reviewing various medical documents submitted by users, it is necessary to classify the types of these documents. In this case, a pre-trained classification model can be used to classify the medical documents by type, such as into reports, discharge summaries, etc. When the above technical solution of this invention is applied to the field of medical consultation, the batch training sample set can be multiple medical questions. The model is trained based on these multiple medical questions to obtain the trained target classification model and target label embedding layer, which are then used to identify and classify the intent of the medical questions. Thus, the pre-trained classification model obtained can achieve better classification results when applied to fields such as medical consultation and medical insurance. [Attached Image Description]

[0019] To more clearly illustrate the technical solutions of the embodiments of this application, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0020] Figure 1 A schematic flowchart of a training method for a pre-trained classification model according to an embodiment of this application is shown.

[0021] Figure 2A schematic flowchart of a training method for a pre-trained classification model according to another embodiment of this application is shown.

[0022] Figure 3 A block diagram of a training apparatus for a pre-trained classification model according to an embodiment of this application is shown.

[0023] Figure 4 A block diagram of a computer device according to one embodiment of this application is shown.

Detailed Implementation Methods

[0024] To better understand the technical solution of this application, the embodiments of this application will be described in detail below with reference to the accompanying drawings.

[0025] It should be understood that the described embodiments are merely some, not all, of the embodiments in this application. All other embodiments obtained by those skilled in the art based on the embodiments in this application without inventive effort are within the scope of protection of this application.

[0026] The terminology used in the embodiments of this application is for the purpose of describing particular embodiments only and is not intended to be limiting of this application. The singular forms “a,” “the,” and “the” used in the embodiments of this application and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise.

[0027] To simplify the model training process, avoid overfitting, and improve generalization performance, this application proposes a training method, apparatus, device, and medium for a pre-trained classification model.

[0028] The following detailed description of some embodiments of this application is provided in conjunction with the accompanying drawings. Unless otherwise specified, the following embodiments and features can be combined with each other.

[0029] It should be noted that the embodiments of this application can acquire and process relevant data based on artificial intelligence technology. Artificial intelligence is the theory, method, technology, and application system that uses digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to obtain optimal results.

[0030] Foundational technologies for artificial intelligence generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing, operating / interactive systems, and mechatronics. AI software technologies mainly encompass computer vision, robotics, biometrics, speech processing, natural language processing, and machine learning / deep learning.

[0031] Please see Figure 1 , Figure 1 A schematic flowchart illustrating a training method for a pre-trained classification model according to an embodiment of this application is shown. This pre-trained classification model training method simplifies the model training process, avoids overfitting, and improves generalization performance. Specifically, this pre-trained classification model training method utilizes and improves upon supervised contrastive learning.

[0032] like Figure 1 As shown, the process of training a pre-trained classification model according to an embodiment of this application includes:

[0033] Step S101: Obtain the batch training sample set;

[0034] In this context, "batch" refers to the process of transmitting samples to the model one batch at a time during model training. Each batch contains multiple samples, forming a batch training sample set. Specifically, the training samples in the batch training sample set are all labeled samples.

[0035] The above-described technical solution of the present invention is applicable to data classification in the field of medical and health technology.

[0036] When the above-described technical solution of this invention is applied to the field of medical consultation, the batch training sample set can be medical problem data. When the above-described technical solution of this invention is applied to the field of medical insurance, the batch training sample set can be medical document data, etc. Depending on the purpose of the pre-trained classification model, corresponding medical data can be selected as the batch training sample set.

[0037] In one embodiment, preferably, obtaining a batch training sample set includes:

[0038] The pre-trained classification model is used to transform each training sample in each batch to determine the second vector representation corresponding to each training sample.

[0039] Based on the second vector representation and label corresponding to each training sample, determine the batch training sample set corresponding to that batch.

[0040] For each sample x in a batch, the pre-trained classification model assigns a vector representation hx to this sample; hx is then passed through a linear layer and an activation function, meaning hx undergoes a transformation. Thus, for each batch, each sample has a vector representation h and a label y. Therefore, the batch training sample set H corresponds to each batch. b Represented as:

[0041] H b ={(h j y j )|0≤j<Nb}

[0042] Among them, h j Let y represent the vector representation of the j-th training sample. j N represents the label of the j-th training sample. b This represents the total number of training samples in the batch training sample set.

[0043] Step S102: Determine the target label category contained in the batch training sample set;

[0044] Based on the classification label of each training sample, the target label category contained in the batch training sample set can be determined.

[0045] When the batch training sample set consists of medical problem data, the corresponding label categories can be health-related, disease-related, general, etc. When the batch training sample set consists of medical document data, the corresponding label categories can be lab reports, reports, hospitalization summaries, expense lists, etc.

[0046] Step S103: Expand the batch training sample set according to the target label category and its corresponding initial label embedding layer to obtain the expanded batch training sample set;

[0047] In one embodiment, preferably, step S103 includes:

[0048] Using an initial label embedding layer corresponding to each target label category, the corresponding target label category vector is determined; the target label category vector and the target label are added as training samples to the batch training sample set to obtain the expanded batch training sample set.

[0049] Suppose the pre-trained classification model has K classes. Initialize a label embedding layer, meaning each class corresponds to a randomly initialized label embedding. These K class label embeddings will be used for parameter learning during subsequent training.

[0050] Assume there are K samples in the batch training sample set. b If there are 10 tag categories, then these K b Each category's label embedding is also considered a labeled sample, so the batch training sample set can be expanded to:

[0051] H b A ={(h j y j )|0≤j<N b +K b}

[0052] Among them, H b A h represents the expanded batch training sample set. j Let y represent the vector representation of the j-th training sample. j N represents the label of the j-th training sample. b K represents the total number of training samples in the batch training sample set. b Indicates the number of target tag categories.

[0053] Step S104: Based on the expanded batch training sample set, the pre-trained classification model and the label embedding layer are trained using the contrastive learning loss function to obtain the trained target classification model and target label embedding layer.

[0054] In one embodiment, preferably, the method further includes:

[0055] Identify the set of positive samples in each batch that have the same label as each training sample;

[0056] Identify the set of negative samples with different sample labels for each batch and each training sample.

[0057] In one embodiment, preferably, training the pre-trained classification model and the label embedding layer using a contrastive learning loss function based on the expanded batch training sample set to obtain the trained target classification model and target label embedding layer includes: calculating the contrastive learning loss function based on the expanded batch training sample set, the positive sample set, and the negative sample set; and performing backpropagation based on the contrastive learning loss function to update the parameters of the pre-trained classification model and the initial label embedding layer to obtain the trained target classification model and target label embedding layer.

[0058] Specifically, for this expanded batch training sample set, a contrastive learning loss function is used to bring samples with the same label closer together and widen the distance between samples with different labels. The contrastive loss function is expressed as:

[0059]

[0060] Where, N b K represents the total number of training samples in the batch training sample set. b s(h) represents the number of target label categories. j h s ) represents h j and h s cosine similarity, s(h j h s’ ) represents h jand h s’ The cosine similarity, Ip(j) represents the positive sample set, I N (j) represents the negative sample set, and τ represents the temperature coefficient.

[0061] In this invention, the proposed training method for a pre-trained classification model involves comparative learning between samples based on labels. The samples and label embeddings are then integrated into a unified semantic space, allowing the learning of sample representations to directly align with the classification task. This facilitates the formation of high-quality decision boundaries and eliminates the need for the model to learn a classification head using a cross-entropy loss function, simplifying the model training process. Furthermore, based on an expanded batch training sample set, a comparative learning loss function is used to train the pre-trained classification model and the label embedding layer. This ensures that the label embeddings are evenly distributed across the semantic space, creating greater spacing between samples and resulting in a more uniform distribution. This avoids overfitting and improves generalization performance.

[0062] Figure 2 A schematic flowchart of a training method for a pre-trained classification model according to an embodiment of this application is shown.

[0063] like Figure 2 As shown, in one embodiment, preferably, the method further includes:

[0064] Step S201: Obtain test samples;

[0065] When the above-mentioned technical solution of the present invention is applied to the field of medical consultation, the batch training sample set can be multiple medical questions, and the test samples also correspond to the medical questions raised by users, thereby classifying the medical questions raised by users and determining the corresponding question categories.

[0066] When the above-mentioned technical solution of the present invention is applied to the field of medical insurance, the batch sample set can be various medical documents, such as test reports, discharge summaries, expense lists, etc., and the test samples also correspond to medical documents. In this way, the types of medical documents are classified and the corresponding document categories are determined.

[0067] Step S202: Determine the first vector representation corresponding to each test sample according to the target classification model;

[0068] Step S203: Determine the tag category vector corresponding to each tag category based on the target tag embedding layer;

[0069] Step S204: Calculate the similarity between the first vector representation and the label category vector corresponding to each test sample;

[0070] Specifically, the cosine similarity between the first vector representation and the label category vector can be calculated.

[0071] Step S205: Determine the target label category corresponding to each test sample based on the similarity.

[0072] The label category with the highest similarity is determined as the target label category for each test sample.

[0073] In this way, the trained target classification model and target label embedding layer are used to classify and identify samples. The samples and label embeddings are placed into a unified semantic space, allowing the learning of sample representations to directly align with the classification task. This facilitates the formation of high-quality decision boundaries and eliminates the need for the model to learn a classifier head through the cross-entropy loss function, simplifying the model training process. Furthermore, based on the expanded batch training sample set, the pre-trained classification model and label embedding layer are trained using a contrastive learning loss function. This ensures that the label embeddings are evenly distributed throughout the semantic space, creating greater spacing between samples and resulting in a more uniform distribution. This avoids overfitting and improves generalization performance.

[0074] The technical solutions described above are applicable to data classification in the field of medical and health technology. For example, in the field of medical consultation, patients can input medical questions, and a pre-trained classification model can classify the patient's intent based on the input medical questions. As another example, in the field of medical insurance, when reviewing various medical documents submitted by users, it is necessary to classify the types of medical documents. In this case, a pre-trained classification model can be used to classify the medical documents by type, such as into categories like reports and discharge summaries.

[0075] When the above-described technical solution of this invention is applied to the field of medical consultation, the batch training sample set can be multiple medical questions. The model is trained based on these multiple medical questions to obtain a trained target classification model and a target label embedding layer, thereby enabling intent recognition and classification of medical questions. The test samples also correspond to the medical questions raised by the user, thus classifying the user-raised medical questions and determining the corresponding question categories.

[0076] When the above-described technical solution of this invention is applied to the field of medical insurance, the batch sample set can be various medical documents, such as laboratory reports, discharge summaries, and expense lists. A classification model is trained based on these medical documents to obtain a trained target classification model and a target label embedding layer, which is then used to classify the medical documents to be classified. The test samples also correspond to medical documents, thus classifying the types of medical documents and determining the corresponding document categories.

[0077] Of course, this case can also be applied to other medical and health fields that require classification, which will not be listed here.

[0078] Figure 3A block diagram of a training apparatus for a pre-trained classification model according to an embodiment of this application is shown.

[0079] like Figure 3 As shown, in a second aspect, embodiments of this application provide a training apparatus 30 for a pre-trained classification model, comprising:

[0080] Module 31 is used to acquire batch training sample sets;

[0081] Module 32 is used to determine the target label category contained in the batch training sample set;

[0082] The expansion module 33 expands the batch training sample set according to the target label category and its corresponding initial label embedding layer to obtain the expanded batch training sample set.

[0083] Training module 34 is used to train the pre-trained classification model and the label embedding layer using the contrastive learning loss function based on the expanded batch training sample set, so as to obtain the trained target classification model and target label embedding layer.

[0084] In one embodiment, preferably

[0085] The acquisition module is also used to: acquire test samples;

[0086] The determining module is further configured to: determine the first vector representation corresponding to each test sample based on the target classification model;

[0087] The vector determination module is used to determine the tag category vector corresponding to each tag category based on the target tag embedding layer;

[0088] The calculation module is used to calculate the similarity between the first vector representation and the label category vector corresponding to each test sample;

[0089] The classification module is used to determine the target label category corresponding to each test sample based on the similarity.

[0090] In one embodiment, preferably, each tag category corresponds to an initial tag embedding layer.

[0091] In one embodiment, preferably, the acquisition module is used for:

[0092] The pre-trained classification model is used to transform each training sample in each batch to determine the second vector representation corresponding to each training sample.

[0093] Based on the second vector representation and label corresponding to each training sample, determine the batch training sample set corresponding to that batch.

[0094] In one embodiment, preferably, the expansion module is used for:

[0095] Use the initial tag embedding layer corresponding to each target tag category to determine the corresponding target tag category vector;

[0096] The target label category vector and the target label are added as training samples to the batch training sample set to obtain the expanded batch training sample set.

[0097] In one embodiment, preferably, the determining module is further configured to:

[0098] Identify the set of positive samples in each batch that have the same label as each training sample;

[0099] Identify the set of negative samples with different sample labels for each batch and each training sample.

[0100] In one embodiment, preferably, the training module is used for:

[0101] Based on the expanded batch training sample set, the positive sample set, and the negative sample set, calculate the contrastive learning loss function;

[0102] Based on the contrastive learning loss function, backpropagation is performed to update the parameters of the pre-trained classification model and the parameters of the initial label embedding layer, so as to obtain the trained target classification model and target label embedding layer.

[0103] The above-described scheme, implemented using a pre-trained classification model training method, apparatus, device, and medium, allows for the acquisition of a batch training sample set; determination of the target label categories contained in the batch training sample set; expansion of the batch training sample set based on the target label categories and their corresponding initial label embedding layers to obtain an expanded batch training sample set; and training of the pre-trained classification model and the label embedding layers using a contrastive learning loss function based on the expanded batch training sample set to obtain the trained target classification model and target label embedding layer. In this invention, the proposed pre-trained classification model training method involves comparative learning between samples based on labels, and the input of samples and label embeddings into a unified semantic space. This allows the learning of sample representations to directly align with the classification task, facilitating the formation of high-quality decision boundaries. Furthermore, it eliminates the need for the model to learn a classification head using a cross-entropy loss function, simplifying the model training process. Based on the expanded batch training sample set, a contrastive learning loss function is used to train the pre-trained classification model and label embedding layer. This ensures that the label embeddings are evenly distributed across the semantic space, thus widening the distance between samples and resulting in a more uniform distribution. This avoids overfitting and improves generalization performance. This invention is applicable to data classification in the field of medical and health technology. For example, in the field of medical consultation, patients can input medical questions, and the pre-trained classification model classifies the patient's intent based on these questions. Similarly, in the field of medical insurance, when reviewing various medical documents submitted by users, it is necessary to classify the types of these documents. In this case, a pre-trained classification model can be used to classify the medical documents by type, such as into reports, discharge summaries, etc. When the above technical solution of this invention is applied to the field of medical consultation, the batch training sample set can be multiple medical questions. The model is trained based on these multiple medical questions to obtain the trained target classification model and target label embedding layer, which are then used to identify and classify the intent of the medical questions. Thus, the pre-trained classification model obtained can achieve better classification results when applied to fields such as medical consultation and medical insurance.

[0104] Thirdly, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the above-described infectious disease spatial prediction method.

[0105] Fourthly, a computer-readable storage medium is provided, which stores a computer program that, when executed by a processor, implements the steps of the above-described infectious disease spatial prediction method.

[0106] It should be noted that those skilled in the art will understand that, for the sake of convenience and brevity, the specific working process of the training device and each module of the pre-trained classification model described above can be referred to the corresponding process in the aforementioned training method embodiment of the pre-trained classification model, and will not be repeated here.

[0107] It should be noted that those skilled in the art will understand that, for the sake of convenience and brevity, the specific working process of the model training device and each module described above can be referred to the corresponding process in the aforementioned training method embodiment of the pre-trained classification model, and will not be repeated here.

[0108] The training device for the aforementioned pre-trained classification model can be implemented as a computer program, which can be used in, for example... Figure 4 It runs on the computer device shown.

[0109] Figure 4 A block diagram of a computer device according to one embodiment of this application is shown.

[0110] See Figure 4 The computer device includes a processor, memory, and network interface connected via a system bus, wherein the memory may include storage media and internal memory.

[0111] The storage medium may store an operating system and a computer program. The computer program includes program instructions that, when executed, cause the processor to perform any of the multi-source data infectious disease spatial prediction methods provided in the embodiments of this application.

[0112] The processor provides computing and control capabilities, supporting the operation of the entire computer device.

[0113] Internal memory provides an environment for the execution of computer programs stored in the storage medium. When executed by a processor, these programs can enable the processor to perform methods for analyzing the transmission paths of any infectious disease or training predictive neural networks. The storage medium can be non-volatile or volatile.

[0114] This network interface is used for network communication, such as sending assigned tasks. Those skilled in the art will understand that... Figure 4 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0115] It should be understood that the processor can be a Central Processing Unit (CPU), but it can also be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Among these, a general-purpose processor can be a microprocessor or any conventional processor.

[0116] The computer device in this application embodiment exists in various forms, including but not limited to:

[0117] (1) Mobile communication devices: These devices are characterized by their mobile communication capabilities and primarily aim to provide voice and data communication. These terminals include: smartphones (e.g., iPhones), multimedia phones, feature phones, and low-end phones, etc.

[0118] (2) Ultra-mobile personal computer devices: These devices fall under the category of personal computers, possessing computing and processing capabilities, and generally also have mobile internet access features. These terminals include PDAs, MIDs, and UMPCs, such as the iPad.

[0119] (3) Portable entertainment devices: These devices can display and play multimedia content. This category includes audio and video players (such as iPods), handheld game consoles, e-book readers, as well as smart toys and portable car navigation devices.

[0120] (4) Server: A device that provides computing services. The components of a server include a processor, hard disk, memory, system bus, etc. Servers are similar to general computer architectures, but because they need to provide highly reliable services, they have higher requirements in terms of processing power, stability, reliability, security, scalability, and manageability.

[0121] (5) Other electronic devices with data interaction functions.

[0122] Additionally, embodiments of this application provide a computer-readable storage medium storing computer-executable instructions, which are used to perform the following steps:

[0123] Obtain a batch of training sample sets;

[0124] Determine the target label categories contained in the batch training sample set;

[0125] Based on the target label category and its corresponding initial label embedding layer, the batch training sample set is expanded to obtain the expanded batch training sample set.

[0126] Based on the expanded batch training sample set, the pre-trained classification model and the label embedding layer are trained using the contrastive learning loss function to obtain the trained target classification model and target label embedding layer.

[0127] In one embodiment, preferably, it further includes:

[0128] Obtain test samples;

[0129] The first vector representation corresponding to each test sample is determined based on the target classification model;

[0130] The tag category vector corresponding to each tag category is determined based on the target tag embedding layer;

[0131] Calculate the similarity between the first vector representation and the label category vector corresponding to each test sample;

[0132] The target label category corresponding to each test sample is determined based on the similarity.

[0133] In one embodiment, preferably, each tag category corresponds to an initial tag embedding layer.

[0134] In one embodiment, preferably, obtaining a batch training sample set includes:

[0135] The pre-trained classification model is used to transform each training sample in each batch to determine the second vector representation corresponding to each training sample.

[0136] Based on the second vector representation and label corresponding to each training sample, determine the batch training sample set corresponding to that batch.

[0137] In one embodiment, preferably, the batch training sample set is expanded according to the target label category and its corresponding initial label embedding layer to obtain an expanded batch training sample set, including:

[0138] Use the initial tag embedding layer corresponding to each target tag category to determine the corresponding target tag category vector;

[0139] The target label category vector and the target label are added as training samples to the batch training sample set to obtain the expanded batch training sample set.

[0140] In one embodiment, preferably, the method further includes:

[0141] Identify the set of positive samples in each batch that have the same label as each training sample;

[0142] Identify the set of negative samples with different sample labels for each batch and each training sample.

[0143] In one embodiment, preferably, the pre-trained classification model and the label embedding layer are trained using a contrastive learning loss function based on the expanded batch training sample set to obtain the trained target classification model and target label embedding layer, including:

[0144] Based on the expanded batch training sample set, the positive sample set, and the negative sample set, calculate the contrastive learning loss function;

[0145] Based on the contrastive learning loss function, backpropagation is performed to update the parameters of the pre-trained classification model and the parameters of the initial label embedding layer, so as to obtain the trained target classification model and target label embedding layer.

[0146] It should be noted that the functions or steps that can be implemented by the computer-readable storage medium or electronic device described above can be referred to the relevant descriptions in the foregoing method embodiments. To avoid repetition, they will not be described one by one here.

[0147] The technical solution of this application has been described in detail above with reference to the accompanying drawings. Through the technical solution of this application, the relevant operations of canary release can be integrated into the release system. Developers only need to make integration settings in the release system to enable the release system to call the deployment system to perform the corresponding deployment for canary release, thereby reducing the complexity of canary release deployment and improving the efficiency and reliability of canary release.

[0148] It should be understood that the term "and / or" used in this article is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. Additionally, the character " / " in this article generally indicates that the preceding and following related objects have an "or" relationship.

[0149] It should be understood that although the terms "first," "second," etc., may be used to describe the setting units in the embodiments of this application, these setting units should not be limited to these terms. These terms are only used to distinguish the setting units from each other. For example, without departing from the scope of the embodiments of this application, the first setting unit may also be referred to as the second setting unit, and similarly, the second setting unit may also be referred to as the first setting unit.

[0150] Depending on the context, the word "if" as used here can be interpreted as "when," "when," "in response to determination," or "in response to detection." Similarly, depending on the context, the phrase "if determination" or "if detection (of the stated condition or event)" can be interpreted as "when determination," "in response to determination," "when detection (of the stated condition or event)," or "in response to detection (of the stated condition or event)."

[0151] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0152] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or in a combination of hardware and software functional units.

[0153] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), Rambus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0154] The above-described embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included within the protection scope of the present invention.

Claims

1. A training method for a pre-trained classification model, characterized in that, include: Obtain a batch of training sample sets; Determine the target label categories contained in the batch training sample set; Based on the target label category and its corresponding initial label embedding layer, the batch training sample set is expanded to obtain the expanded batch training sample set. Based on the expanded batch training sample set, the pre-trained classification model and the label embedding layer are trained using the contrastive learning loss function to obtain the trained target classification model and target label embedding layer. Specifically, the batch training sample set is expanded according to the target label category and its corresponding initial label embedding layer to obtain an expanded batch training sample set, including: Use the initial tag embedding layer corresponding to each target tag category to determine the corresponding target tag category vector; The target label category vector and the target label are added as training samples to the batch training sample set to obtain the expanded batch training sample set.

2. The training method for the pre-trained classification model according to claim 1, characterized in that, The method further includes: Obtain test samples; The first vector representation corresponding to each test sample is determined based on the target classification model; The tag category vector corresponding to each tag category is determined based on the target tag embedding layer; Calculate the similarity between the first vector representation and the label category vector corresponding to each test sample; The target label category corresponding to each test sample is determined based on the similarity.

3. The training method for the pre-trained classification model according to claim 1, characterized in that, Each tag category corresponds to an initial tag embedding layer.

4. The training method for the pre-trained classification model according to claim 1, characterized in that, Obtaining a batch of training sample sets includes: The pre-trained classification model is used to transform each training sample in each batch to determine the second vector representation corresponding to each training sample. Based on the second vector representation and label corresponding to each training sample, determine the batch training sample set corresponding to that batch.

5. The training method for the pre-trained classification model according to claim 1, characterized in that, The method further includes: Identify the set of positive samples in each batch that have the same label as each training sample; Identify the set of negative samples with different sample labels for each batch and each training sample.

6. The training method for the pre-trained classification model according to claim 5, characterized in that, Based on the expanded batch training sample set, the pre-trained classification model and the label embedding layer are trained using a contrastive learning loss function to obtain the trained target classification model and target label embedding layer, including: Based on the expanded batch training sample set, the positive sample set, and the negative sample set, calculate the contrastive learning loss function; Based on the contrastive learning loss function, backpropagation is performed to update the parameters of the pre-trained classification model and the parameters of the initial label embedding layer, so as to obtain the trained target classification model and target label embedding layer.

7. A training device for a pre-trained classification model, characterized in that, include: The acquisition module is used to acquire batch training sample sets; The determination module is used to determine the target label category contained in the batch training sample set; The expansion module expands the batch training sample set according to the target label category and its corresponding initial label embedding layer to obtain the expanded batch training sample set. The training module is used to train the pre-trained classification model and the label embedding layer using the contrastive learning loss function based on the expanded batch training sample set, so as to obtain the trained target classification model and target label embedding layer. Specifically, the batch training sample set is expanded according to the target label category and its corresponding initial label embedding layer to obtain an expanded batch training sample set, including: Use the initial tag embedding layer corresponding to each target tag category to determine the corresponding target tag category vector; The target label category vector and the target label are added as training samples to the batch training sample set to obtain the expanded batch training sample set.

8. A computer device, characterized in that, include: At least one processor; And, a memory communicatively connected to the at least one processor; The memory stores instructions executable by the at least one processor, the instructions being configured to perform the method according to any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that, The device stores computer-executable instructions for performing the method flow as described in any one of claims 1 to 6.

Images