Contrast-based learning training methods, devices, equipment, and media based on sentence vectors

Abstract

This invention belongs to the field of text recognition technology and can be applied to the medical field. Specifically, it relates to a contrastive learning training method, apparatus, device, and medium based on sentence vectors. The method includes: acquiring training samples; inputting the training samples into a contrastive learning model to be trained; constructing text and extracting vector features from the training text according to a first preset rule to obtain a first sentence vector; performing text transformation and extracting vector features from the training text according to a second preset rule to obtain a second sentence vector; training the contrastive learning model based on the first and second sentence vectors until the parameters of the contrastive learning model converge, thus obtaining the contrastive learning model. This invention trains the contrastive learning model based on the differing sentence vectors to obtain a contrastive learning model with efficient and accurate recognition performance for simple masked text and complex text with replaced or deleted characters / words, thereby optimizing text recognition technology.

Description

Technical Field

[0001] This invention relates to the field of text recognition technology, and in particular to a contrastive learning training method, apparatus, device, and medium based on sentence vectors. Background Technology

[0002] In the medical field, with the development of technology, many publicly available paper-based medical documents are uploaded to online platforms for sharing by medical professionals. These documents can be uploaded via scanning or other non-scanning methods. However, due to poor storage conditions or prolonged storage time, some paper documents have missing text, resulting in incomplete content and affecting user comprehension. With the rapid development of text recognition technology, users can now identify and predict missing text in documents, providing assistance in reading. This technology primarily involves constructing positive sample pairs using multiple random data augmentation techniques on a single sample, then training the model with these pairs to enhance its sentence representation and predict missing text. However, this method is only suitable for predicting missing medical text. For medical text where characters or words have been replaced or deleted, the recognition and prediction results are poor, impacting user experience.

[0003] Therefore, optimizing text recognition technology so that it can not only accurately identify medical document text with simple masks, but also efficiently and accurately identify complex medical document text with replaced or deleted characters / words is a problem that needs to be solved. Summary of the Invention

[0004] This invention provides a contrastive learning training method, apparatus, device, and medium based on sentence vectors, aiming to obtain a contrastive learning model with efficient, accurate, and effective recognition performance for both simple masked medical literature texts and complex medical literature texts with character / word replacements / deletions, thereby optimizing text recognition technology.

[0005] To achieve the aforementioned objectives, the first aspect of this invention proposes a contrastive learning training method based on sentence vectors, the method comprising:

[0006] Obtain training samples, which include multiple training texts;

[0007] The training samples are input into the contrastive learning model to be trained;

[0008] The training text is constructed and vector features are extracted according to the first preset rule to obtain the first sentence vector;

[0009] The training text is converted and vector features are extracted according to the second preset rule to obtain the second sentence vector;

[0010] The contrastive learning model to be trained is trained based on the first sentence vector and the second sentence vector until the parameters of the contrastive learning model to be trained converge, thus obtaining the contrastive learning model.

[0011] Further, the step of constructing text and extracting vector features from the training text according to the first preset rule to obtain the first sentence vector includes:

[0012] The training text is constructed using the dropout mechanism to obtain the constructed text.

[0013] The BERT encoder is used to extract vector features from the constructed text to obtain the first sentence vector.

[0014] Further, the step of performing text conversion and vector feature extraction on the training text according to the second preset rule to obtain the second sentence vector includes:

[0015] The characters and text in the training text are converted to obtain the converted text;

[0016] The BERT encoder is used to extract vector features from the converted text to obtain the second sentence vector.

[0017] Further, the step of training the contrastive learning model to be trained based on the first sentence vector and the second sentence vector until the parameters of the contrastive learning model to be trained converge to obtain the contrastive learning model includes:

[0018] Using the first sentence vector as a positive sample, the contrastive learning model to be trained is trained to focus on the second sentence vector, and the loss value of the contrastive learning model to be trained is calculated according to the preset loss function.

[0019] The loss value is compared with the loss threshold to determine whether the loss value is less than the loss threshold.

[0020] If the loss value is less than the loss threshold, the parameters of the contrastive learning model to be trained are adjusted according to the training parameters corresponding to the loss value until the parameters in the contrastive learning model to be trained converge, thereby obtaining the contrastive learning model.

[0021] Further, the step of training the contrastive learning model to focus on the second sentence vector using the first sentence vector as a positive sample, and calculating the loss value of the contrastive learning model to be trained according to a preset loss function, includes:

[0022] Using the first sentence vector as a positive sample, the contrastive learning model to be trained is trained to focus on the second sentence vector, and the first similarity loss between the first sentence vector and the second sentence vector is calculated according to the preset first loss function;

[0023] Using the first sentence vector as a positive sample, text prediction is performed on the second sentence vector to obtain the third sentence vector;

[0024] The second similarity loss between the first sentence vector and the third sentence vector is calculated based on a preset second loss function;

[0025] The loss value of the contrastive learning model to be trained is calculated based on the preset loss function, the first similarity loss, and the second similarity loss.

[0026] Furthermore, the preset loss function is: Among them, L contrast This represents the first similarity loss;

[0027] This represents the second similarity loss.

[0028] This application also provides a contrastive learning training device based on sentence vectors, characterized in that the device comprises:

[0029] The acquisition module is used to acquire training samples, which include multiple training texts.

[0030] The input module is used to input the training samples into the contrastive learning model to be trained;

[0031] The construction module is used to construct text and extract vector features from the training text according to a first preset rule to obtain a first sentence vector.

[0032] The conversion module is used to perform text conversion and vector feature extraction on the training text according to a second preset rule to obtain a second sentence vector.

[0033] The training module is used to train the contrastive learning model to be trained based on the first sentence vector and the second sentence vector until the parameters of the contrastive learning model to be trained converge, thereby obtaining the contrastive learning model.

[0034] Furthermore, the training module includes:

[0035] The training unit is used to train the contrastive learning model to focus on the second sentence vector using the first sentence vector as a positive sample, and to calculate the loss value of the contrastive learning model to be trained according to a preset loss function.

[0036] A comparison unit is used to compare the loss value with a loss threshold and determine whether the loss value is less than the loss threshold.

[0037] An adjustment unit is used to adjust the parameters of the contrastive learning model to be trained according to the training parameters corresponding to the loss value if the loss value is less than the loss threshold, until the parameters in the contrastive learning model to be trained converge to obtain the contrastive learning model.

[0038] This application also provides a computer device, including a memory and a processor, wherein the memory stores a computer program, characterized in that the processor executes the computer program to implement the steps of the sentence vector-based contrastive learning training method described in any of the above claims.

[0039] This application also provides a computer-readable storage medium storing a computer program thereon, characterized in that, when the computer program is executed by a processor, it implements the steps of the sentence vector-based contrastive learning training method described in any of the preceding claims.

[0040] Beneficial Effects: This application obtains training samples, inputs these samples into a contrastive learning model to be trained, and performs text construction and vector feature extraction on the training text in the training samples according to a first preset rule to obtain a first sentence vector. This first sentence vector will serve as the training sample for the training model to perform insensitive learning transformation. Then, according to a second preset rule, the training text is transformed and vector feature extracted to obtain a second sentence vector. This second sentence vector will serve as the training sample for the training model to perform sensitive learning transformation. Based on the first and second sentence vectors, the contrastive learning model to be trained is trained until its parameters converge, resulting in a contrastive learning model. This contrastive learning model possesses the performance to efficiently, accurately, and effectively recognize simple masked text and complex text with replaced or deleted characters / words. It can be used to identify and predict uncertain text in shared medical literature in the medical field to obtain high-confidence text recognition results, thereby optimizing text recognition technology. Attached Figure Description

[0041] Figure 1 This is a schematic flowchart of an embodiment of the contrastive learning training method based on sentence vectors in this application;

[0042] Figure 2 This is a schematic diagram of another embodiment of the contrastive learning training method based on sentence vectors in this application;

[0043] Figure 3 This is a schematic diagram of another embodiment of the contrastive learning training method based on sentence vectors in this application;

[0044] Figure 4 This is a schematic diagram of another embodiment of the contrastive learning training method based on sentence vectors in this application;

[0045] Figure 5 This is a schematic diagram of another embodiment of the contrastive learning training method based on sentence vectors in this application;

[0046] Figure 6 This is a schematic diagram of an embodiment of the contrastive learning training device based on sentence vectors in this application;

[0047] Figure 7 This is a schematic block diagram of an embodiment of the computer device of this application.

[0048] The realization of the objective, functional features and advantages of the present invention will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0049] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0050] Those skilled in the art will understand that, unless specifically stated otherwise, the singular forms “a,” “an,” “the,” and “the” used herein may also include the plural forms. It should be further understood that the term “comprising” as used in this specification means the presence of features, integers, steps, operations, elements, modules, and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, modules, components, and / or groups thereof. It should be understood that when we say an element is “connected” or “coupled” to another element, it can be directly connected or coupled to the other element, or there may be intermediate elements. Furthermore, “connected” or “coupled” as used herein can include wireless connections or wireless coupling. The term “and / or” as used herein includes all or any modules and all combinations of one or more associated listed items.

[0051] It will be understood by those skilled in the art that, unless otherwise defined, all terms used herein (including technical and scientific terms) have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. It should also be understood that terms such as those defined in general dictionaries should be understood to have the same meaning as in the context of the prior art, and should not be interpreted in an idealized or overly formal sense unless specifically defined as herein.

[0052] Reference Figure 1 This invention provides a contrastive learning training method based on sentence vectors, comprising the following steps S1-S5:

[0053] S1: Obtain training samples, which include multiple training texts.

[0054] This embodiment applies to text recognition in the medical field. Before model training, training samples are prepared, which are texts in the medical field, such as publicly available medical literature. Since model training involves multiple rounds, the training samples include multiple training texts. These training samples serve as the basis for extracting training data for subsequent model training.

[0055] S2: Input the training samples into the contrastive learning model to be trained.

[0056] After obtaining the training samples, when there are missing texts in publicly available medical literature, these missing texts may be semantically unchanged or semantically altered. Therefore, when the training samples are input into the contrastive learning model to be trained, the model includes two training mechanisms: a training mechanism for simple samples and a training mechanism for complex samples. The training mechanism for simple samples refers to the training mechanism for missing texts with unchanged semantics, and the training mechanism for complex samples refers to the training mechanism for missing texts with altered semantics. Subsequently, the model will be trained based on these two training mechanisms to achieve performance that can accurately identify both simple and complex texts.

[0057] S3: Based on the first preset rule, the training text is constructed and vector features are extracted to obtain the first sentence vector.

[0058] After inputting the training samples into the contrastive learning model to be trained, the training text is constructed and vector features are extracted according to the first preset rule to obtain the first sentence vector. The first preset rule is a processing rule that does not change the semantics of the sentence itself. Specifically, the training text is constructed based on the dropout mechanism, where the dropout mechanism only deactivates some characters / words in the training text, and the semantics of the sentence itself do not change, resulting in a simple constructed text. The BERT encoder is used to extract vector features from the constructed text to obtain the first sentence vector. The acquisition of the first sentence vector provides training samples for subsequent training of the contrastive learning model to be trained on the insensitive transformation learning performance.

[0059] S4: Perform text conversion and vector feature extraction on the training text according to the second preset rule to obtain the second sentence vector.

[0060] After constructing text and extracting vector features from the training text according to the first preset rule to obtain the first sentence vector, the training text is further transformed and extracted according to the second preset rule to obtain the second sentence vector. The second preset rule is a processing rule that changes the semantics of the sentence itself. This processing rule increases the complexity of the training samples and provides training samples for subsequent training of the contrastive learning model's performance on sensitive transformation learning. The processing of the second preset rule includes: transforming the characters in the training text, wherein the transformation includes character / word replacement and deletion, thereby changing the semantics of the sentence itself to obtain transformed text; and using the BERT encoder to extract vector features from the transformed text to obtain the second sentence vector.

[0061] S5: Train the contrastive learning model to be trained based on the first sentence vector and the second sentence vector until the parameters of the contrastive learning model to be trained converge, and obtain the contrastive learning model.

[0062] According to the second preset rule, the training text is converted and vector features are extracted to obtain the second sentence vector. Then, the contrastive learning model to be trained is trained based on the first sentence vector and the second sentence vector until the parameters of the contrastive learning model to be trained converge, thus obtaining the contrastive learning model. Specifically, using the first sentence vector as a positive sample, the contrastive learning model to be trained is trained to focus on the second sentence vector, and the first similarity loss between the first sentence vector and the second sentence vector is calculated according to the preset first loss function; and using the first sentence vector as a positive sample, text prediction is performed on the second sentence vector to obtain the third sentence vector; the second similarity loss between the first sentence vector and the third sentence vector is calculated according to the preset second loss function; the loss value of the contrastive learning model to be trained is calculated according to the preset loss function combined with the first similarity loss and the second similarity loss, and the loss value is compared with the loss threshold to determine whether the loss value is less than the loss threshold; if the loss value is less than the loss threshold, the parameters of the contrastive learning model to be trained are adjusted according to the training parameters corresponding to the loss value until the parameters in the contrastive learning model to be trained converge, thus obtaining the contrastive learning model. The contrastive learning model can be applied to text recognition in the medical field. By using the contrastive learning model to identify medical text with masked, replaced, or deleted characters / text, it can identify and predict the corresponding text of the masked, replaced, or deleted characters / text, helping users to interpret the text with masked, replaced, or deleted characters / text, thereby improving the user experience.

[0063] This embodiment provides a contrastive learning training method based on sentence vectors. Training samples are obtained and input into the contrastive learning model to be trained. According to a first preset rule, the training text in the training samples is used for text construction and vector feature extraction to obtain a first sentence vector, which serves as the training sample for the training model to perform insensitive learning transformations. According to a second preset rule, the training text is used for text transformation and vector feature extraction to obtain a second sentence vector, which serves as the training sample for the training model to perform sensitive learning transformations. The contrastive learning model is trained based on the first and second sentence vectors until the parameters of the model converge, resulting in a contrastive learning model. This model possesses the performance to efficiently, accurately, and effectively recognize simple masked text and complex text with replaced or deleted characters / words. It can be used to identify and predict uncertain text in the medical field to obtain high-confidence text recognition results, thereby optimizing text recognition technology.

[0064] Reference Figure 2 In one embodiment, the above-mentioned text construction and vector feature extraction of the training text according to the first preset rule to obtain the first sentence vector includes:

[0065] S21: Based on the dropout mechanism, construct the training text to obtain the constructed text.

[0066] Dropout is a regularization method proposed to alleviate overfitting in Convolutional Neural Networks (CNNs). It can indeed effectively alleviate the overfitting phenomenon. Dropout, also known as random deactivation, simply means that during the forward propagation of the model training phase, the activation values ​​of certain neurons are deactivated with a certain probability, making the model more generalizable.

[0067] S21: Use the BERT encoder to extract vector features from the constructed text to obtain the first sentence vector.

[0068] BERT (Bidirectional Encoder Representation from Transformers) is a pre-trained language representation model. It emphasizes that instead of using traditional unidirectional language models or shallow concatenation of two unidirectional language models for pre-training, it adopts a new masked language model (MLM) to generate deep bidirectional language representations.

[0069] As described above, the dropout mechanism randomly deactivates characters / words in the training text, causing the activation values ​​of certain character / word neurons to stop working with a certain probability, thus forming new training text, denoted as the constructed text. This constructed text does not change the semantics of the sentence itself. Therefore, the constructed text obtained by the dropout mechanism can serve as training samples for training the contrastive learning model to perform insensitive transformation learning, enabling the model to efficiently and accurately recognize text with simple masks (character / word deactivation). The BERT encoder is used to extract vector features from the constructed text, converting it into a mathematical representation suitable for the model to process and recognize, thus obtaining the first sentence vector. The dropout mechanism is used to construct text from the training text to obtain training text that does not change the semantics of the sentence itself, providing training samples for subsequent training of the contrastive learning model's performance in recognizing insensitive transformation learning text.

[0070] Reference Figure 3 In one embodiment, the above-mentioned text conversion and vector feature extraction of the training text according to the second preset rule to obtain the second sentence vector includes:

[0071] S31: Convert the characters and text in the training text to obtain the converted text.

[0072] When the characters and words in the training text are transformed, the transformation includes the replacement and deletion of characters / words, thereby obtaining the transformed text. The transformed text changes the semantics of the sentence itself. Therefore, the transformed text can be used as a training sample for training the contrastive learning model to be trained for sensitive transformation learning, so that the model can efficiently and accurately identify text in which characters / words have been replaced or deleted.

[0073] S32: Use the BERT encoder to extract vector features from the converted text to obtain the second sentence vector.

[0074] As described above, when converting the characters and words in the training text, the conversion includes character / word replacement and deletion, resulting in converted text. This converted text alters the semantics of the sentence itself. The converted text can serve as training samples for training the contrastive learning model's sensitive transformation learning, enabling the model to efficiently and accurately recognize text where characters / words have been replaced or deleted. A BERT encoder is used to extract vector features from the converted text, transforming it into a mathematical representation suitable for the model's processing and recognition, thus obtaining the second sentence vector. Converting the characters and words in the training text to obtain training text with altered sentence semantics provides training samples for subsequent training of the contrastive learning model's performance in recognizing sensitive transformation learning text.

[0075] Reference Figure 4 In one embodiment, the training of the contrastive learning model to be trained based on the first sentence vector and the second sentence vector until the parameters of the contrastive learning model to be trained converge to obtain the contrastive learning model includes:

[0076] S51: Using the first sentence vector as a positive sample, train the contrastive learning model to focus on the second sentence vector, and calculate the loss value of the contrastive learning model to be trained according to the preset loss function.

[0077] Since the first sentence vector is the training text that does not change the semantics of the sentence itself, the first sentence vector is used as a positive sample to train the contrastive learning model to pay attention to the second sentence vector. That is, the contrastive learning model is trained to pay attention to the training text that changes the semantics of the sentence itself. This enables the contrastive learning model to accurately identify simple samples (text that does not change the semantics of the sentence itself) and also to accurately identify complex samples (text that changes the semantics of the sentence itself), thereby obtaining a high-confidence recognition result.

[0078] S52: Compare the loss value with the loss threshold to determine whether the loss value is less than the loss threshold.

[0079] S53: If the loss value is less than the loss threshold, then adjust the parameters of the contrastive learning model to be trained according to the training parameters corresponding to the loss value until the parameters in the contrastive learning model to be trained converge to obtain the contrastive learning model.

[0080] As mentioned above, uncertain texts in the medical field take various forms, including defective texts with unchanged semantics and defective texts with altered semantics. Therefore, using the first sentence vector as a positive sample, the contrastive learning model is trained to focus on training texts with altered sentence semantics (the second sentence vector). This allows the contrastive learning model to accurately identify both simple samples (texts with unchanged sentence semantics, i.e., the first sentence vector) and complex samples (texts with altered sentence semantics, i.e., the second sentence vector), thereby obtaining high-confidence recognition results. By calculating the loss value and comparing it with a loss threshold, the training process is judged to meet the requirements. That is, the loss value of the contrastive learning model is calculated according to a preset loss function, where the preset loss function calculates the similarity loss between the recognition results of the training texts. The higher the similarity, the smaller the loss value, and the higher the recognition confidence. The higher the confidence level of the result, the closer similar data are to dissimilar data, and the more effectively the data representation is learned. If the loss value is less than the loss threshold, the training process is determined to meet the training requirements. The training parameters that meet the training requirements are retained and used as the initial training parameters for the next training stage. This training process is repeated until the parameters in the contrastive learning model to be trained converge. The expected parameters corresponding to the expected training results in the final training stage are used as the fixed parameters of the contrastive learning model to be trained, thus obtaining the contrastive learning model. The contrastive learning model has the performance of efficiently, accurately, and effectively recognizing simple masked text and complex text with replaced characters / words. The contrastive learning model can be used to recognize and predict uncertain text in the medical field to obtain high-confidence text recognition results, so that users can interpret the uncertain text based on the text recognition results, thereby improving the user experience.

[0081] Reference Figure 5 In one embodiment, the above-mentioned method of using the first sentence vector as a positive sample to train the contrastive learning model to focus on the second sentence vector, and calculating the loss value of the contrastive learning model to be trained according to a preset loss function, includes:

[0082] S511: Using the first sentence vector as a positive sample, train the contrastive learning model to focus on the second sentence vector, and calculate the first similarity loss between the first sentence vector and the second sentence vector according to the preset first loss function.

[0083] Using the first sentence vector as a positive sample, the contrastive learning model to be trained is trained to focus on the second sentence vector. The purpose is to mine the training samples to find difficult samples within the training samples, and then force the model to pay more attention to these difficult samples by increasing the weight of these samples or their occurrence ratio.

[0084] S512: Using the first sentence vector as a positive sample, perform text prediction on the second sentence vector to obtain the third sentence vector.

[0085] Using the first sentence vector as a positive sample, text prediction is performed on the second sentence vector. The purpose is to use the first sentence vector (text without changing the semantics of the sentence itself) as prior text to predict the characters / words that have been replaced or deleted in the second sentence vector, thereby obtaining the predicted text. Vector features are extracted from the predicted text to obtain the third sentence vector.

[0086] S513: Calculate the second similarity loss between the first sentence vector and the third sentence vector according to the preset second loss function.

[0087] S514: Calculate the loss value of the contrastive learning model to be trained based on the preset loss function, the first similarity loss, and the second similarity loss.

[0088] As described above, using the first sentence vector as a positive sample, the contrastive learning model to be trained is trained to focus on the second sentence vector, and a first similarity loss between the first sentence vector and the second sentence vector is calculated according to a preset first loss function; using the first sentence vector (text without changing the semantics of the sentence itself) as prior text, the characters / words that are replaced or deleted in the second sentence vector are predicted, thereby obtaining the predicted text, and vector features are extracted from the predicted text to obtain the third sentence vector; a second similarity loss between the first sentence vector and the third sentence vector is calculated according to a preset second loss function; and the loss value of the contrastive learning model to be trained is calculated according to the preset loss function combined with the first similarity loss and the second similarity loss.

[0089] In one embodiment, the aforementioned preset loss function is: Among them, L contrast This represents the first similarity loss;

[0090] This represents the second similarity loss.

[0091] As mentioned above, the accuracy of contrastive learning models in text recognition is related to their own model structure, but depends more on model training. To accurately determine whether the model training meets the requirements, the loss value can be calculated as an auxiliary judgment. The formula for calculating the loss value is as follows: Among them, L contrast This represents the first similarity loss; This represents the second similarity loss.

[0092] Reference Figure 6 This invention also provides a contrastive learning training device based on sentence vectors, comprising:

[0093] The acquisition module 10 is used to acquire training samples, which include multiple training texts;

[0094] Input module 20 is used to input the training samples into the contrastive learning model to be trained;

[0095] Construction module 30 is used to construct text and extract vector features from the training text according to the first preset rule to obtain the first sentence vector;

[0096] The conversion module 40 is used to perform text conversion and vector feature extraction on the training text according to the second preset rule to obtain the second sentence vector;

[0097] The training module 50 is used to train the contrastive learning model to be trained based on the first sentence vector and the second sentence vector until the parameters of the contrastive learning model to be trained converge, thereby obtaining the contrastive learning model.

[0098] As mentioned above, the sentence vector-based contrastive learning training device can realize the sentence vector-based contrastive learning training method.

[0099] In one embodiment, the above-mentioned construction module 30 further includes:

[0100] A construction unit is used to construct text from the training text based on the dropout mechanism to obtain constructed text.

[0101] The first extraction unit is used to extract vector features from the constructed text using a BERT encoder to obtain the first sentence vector.

[0102] In one embodiment, the conversion module 40 further includes:

[0103] The conversion unit is used to convert the characters and text in the training text to obtain the converted text;

[0104] The second extraction unit is used to extract vector features from the converted text using the BERT encoder to obtain the second sentence vector.

[0105] In one embodiment, the training module 50 further includes:

[0106] The training unit is used to train the contrastive learning model to focus on the second sentence vector using the first sentence vector as a positive sample, and to calculate the loss value of the contrastive learning model to be trained according to a preset loss function.

[0107] A comparison unit is used to compare the loss value with a loss threshold and determine whether the loss value is less than the loss threshold.

[0108] An adjustment unit is used to adjust the parameters of the contrastive learning model to be trained according to the training parameters corresponding to the loss value if the loss value is less than the loss threshold, until the parameters in the contrastive learning model to be trained converge to obtain the contrastive learning model.

[0109] In one embodiment, the training module 50 further includes:

[0110] The first computing unit is used to train the contrastive learning model to focus on the second sentence vector using the first sentence vector as a positive sample, and to calculate the first similarity loss between the first sentence vector and the second sentence vector according to a preset first loss function.

[0111] The prediction unit is used to perform text prediction on the second sentence vector using the first sentence vector as a positive sample, to obtain the third sentence vector;

[0112] The second calculation unit is used to calculate the second similarity loss between the first sentence vector and the third sentence vector according to a preset second loss function;

[0113] The third calculation unit is used to calculate the loss value of the contrastive learning model to be trained based on the preset loss function, the first similarity loss, and the second similarity loss.

[0114] In one embodiment, the aforementioned preset loss function is: Among them, L contrast This represents the first similarity loss;

[0115] This represents the second similarity loss.

[0116] Reference Figure 7 The present invention also provides a computer device, the internal structure of which can be as follows: Figure 7As shown, the computer device includes a processor, memory, network interface, and database connected via a system bus. The processor is designed to provide computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores operating devices, computer programs, and the database. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The database stores relevant data for sentence vector-based contrastive learning training methods. The network interface is used for communication with external terminals via a network connection. Furthermore, the computer device may also include input devices and a display screen. When the aforementioned computer program is executed by a processor, it implements a contrastive learning training method based on sentence vectors, comprising the following steps: acquiring training samples, the training samples including multiple training texts; inputting the training samples into a contrastive learning model to be trained; constructing text and extracting vector features from the training texts according to a first preset rule to obtain a first sentence vector; performing text transformation and extracting vector features from the training texts according to a second preset rule to obtain a second sentence vector; training the contrastive learning model to be trained based on the first sentence vector and the second sentence vector until the parameters of the contrastive learning model to be trained converge, thereby obtaining a contrastive learning model. Those skilled in the art will understand that... Figure 7 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer equipment on which the present application is applied.

[0117] One embodiment of this application also provides a computer-readable storage medium storing a computer program. When executed by a processor, the computer program implements a contrastive learning training method based on sentence vectors, comprising the following steps: acquiring training samples, the training samples including multiple training texts; inputting the training samples into a contrastive learning model to be trained; constructing text and extracting vector features from the training texts according to a first preset rule to obtain a first sentence vector; performing text conversion and extracting vector features from the training texts according to a second preset rule to obtain a second sentence vector; training the contrastive learning model to be trained based on the first sentence vector and the second sentence vector until the parameters of the contrastive learning model to be trained converge, thereby obtaining a contrastive learning model. It is understood that the computer-readable storage medium in this embodiment can be a volatile readable storage medium or a non-volatile readable storage medium.

[0118] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media provided in this application and in the embodiments may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual-speed SDRAM (SSRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0119] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, apparatus, article, or method that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, apparatus, article, or method. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, apparatus, article, or method that includes that element.

[0120] The above description is merely a preferred embodiment of the present invention and does not limit the patent scope of the present invention. Any equivalent structural or procedural transformations made based on the content of the present invention's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of the present invention.

Claims

1. A contrastive learning training method based on sentence vectors, characterized in that, The method includes: Obtain training samples, which include multiple training texts; The training samples are input into the contrastive learning model to be trained; The training text is constructed and vector features are extracted according to the first preset rule to obtain the first sentence vector; The training text is converted and vector features are extracted according to the second preset rule to obtain the second sentence vector; The contrastive learning model to be trained is trained based on the first sentence vector and the second sentence vector until the parameters of the contrastive learning model to be trained converge, thus obtaining the contrastive learning model; The step of training the contrastive learning model based on the first sentence vector and the second sentence vector until the parameters of the contrastive learning model converge to obtain the contrastive learning model includes: Using the first sentence vector as a positive sample, the contrastive learning model to be trained is trained to focus on the second sentence vector, and the first similarity loss between the first sentence vector and the second sentence vector is calculated according to the preset first loss function; Using the first sentence vector as a positive sample, text prediction is performed on the second sentence vector to obtain the third sentence vector; The second similarity loss between the first sentence vector and the third sentence vector is calculated based on a preset second loss function; The loss value of the contrastive learning model to be trained is calculated based on the preset loss function, the first similarity loss, and the second similarity loss; The loss value is compared with the loss threshold to determine whether the loss value is less than the loss threshold. If the loss value is less than the loss threshold, the parameters of the contrastive learning model to be trained are adjusted according to the training parameters corresponding to the loss value until the parameters in the contrastive learning model to be trained converge, thereby obtaining the contrastive learning model.

2. The contrastive learning training method based on sentence vectors according to claim 1, characterized in that, The step of constructing text and extracting vector features from the training text according to a first preset rule to obtain a first sentence vector includes: The training text is constructed using the dropout mechanism to obtain the constructed text. The BERT encoder is used to extract vector features from the constructed text to obtain the first sentence vector.

3. The contrastive learning training method based on sentence vectors according to claim 1, characterized in that, The step of performing text conversion and vector feature extraction on the training text according to the second preset rule to obtain the second sentence vector includes: The characters and text in the training text are converted to obtain the converted text; The BERT encoder is used to extract vector features from the converted text to obtain the second sentence vector.

4. The contrastive learning training method based on sentence vectors according to claim 1, characterized in that, The preset loss function is: ;in, This represents the first similarity loss; This represents the second similarity loss.

5. A contrastive learning training device based on sentence vectors, characterized in that, The apparatus for implementing the method according to any one of claims 1-4, the apparatus comprising: The acquisition module is used to acquire training samples, which include multiple training texts. The input module is used to input the training samples into the contrastive learning model to be trained; The construction module is used to construct text and extract vector features from the training text according to a first preset rule to obtain a first sentence vector. The conversion module is used to perform text conversion and vector feature extraction on the training text according to a second preset rule to obtain a second sentence vector. The training module is used to train the contrastive learning model to be trained based on the first sentence vector and the second sentence vector until the parameters of the contrastive learning model to be trained converge, thereby obtaining the contrastive learning model.

6. The contrastive learning training device based on sentence vectors according to claim 5, characterized in that, The training module includes: The training unit is used to train the contrastive learning model to focus on the second sentence vector using the first sentence vector as a positive sample, and to calculate the loss value of the contrastive learning model to be trained according to a preset loss function. A comparison unit is used to compare the loss value with a loss threshold and determine whether the loss value is less than the loss threshold. An adjustment unit is used to adjust the parameters of the contrastive learning model to be trained according to the training parameters corresponding to the loss value if the loss value is less than the loss threshold, until the parameters in the contrastive learning model to be trained converge to obtain the contrastive learning model.

7. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the sentence vector-based contrastive learning training method as described in any one of claims 1 to 4.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the steps of the contrastive learning training method based on sentence vectors as described in any one of claims 1 to 4.

Images