Model training method and apparatus

Abstract

This invention discloses a model training method and apparatus. The method involves encoding a first sample, a second sample, and a third sample using an original encoding model to obtain first, second, and third encoding results. The second encoding result is then randomly shuffled in both the character position and character encoding dimensions to obtain a fourth encoding result. The first and third encoding results are used to construct negative sample pairs, and the first and fourth encoding results are used to construct positive sample pairs. All positive and negative sample pairs are obtained by applying the above method to all sentences in the text to be trained. Based on these positive and negative sample pairs, a contrastive learning method is used to calculate the loss to optimize the original encoding model. This invention can optimize the original encoding model to achieve the technical effect of recommending similar texts with high accuracy even when content specifications are inconsistent across multiple platforms or when text tags are missing.

Description

Technical Field

[0001] This application relates to the field of computer technology, and more specifically, to a model training method, apparatus, computer device, and storage medium. Background Technology

[0002] With the rapid expansion of the marketing field and the shift in business strategies of emerging media, more and more merchants in the FMCG industry are moving their advertising and marketing efforts from traditional television broadcasts to content distribution on public domains such as Douyin / Xiaohongshu and private domains such as WeChat Moments. By relying on influencers and studios to create, manage, and operate content across both public and private domains, they aim to attract consumers and convert product sales. In this process, a large amount of content advertising and copywriting materials accumulate. The challenge now is to effectively connect and manage this copywriting content to facilitate downstream search and secondary creation.

[0003] When users use a content management platform, they collect various marketing copy from different marketing platforms and add it to a content pool. During secondary creation, they search this content pool, selecting useful and satisfactory copy for their work. This process involves multiple searches to gather a batch of marketing copy that meets the requirements. If these copy entries in the content pool could be correlated, when a user finds the first matching article, we could push more similar copy to them, significantly reducing search time and improving the efficiency of their content creation, thus facilitating content flow and distribution across different platforms. However, in real-world scenarios, the existing raw encoding model BERT suffers from difficulties in recommending similar text and inaccurate recommendations due to inconsistencies in content standards across multiple platforms and missing copy tags.

[0004] There is currently no effective solution to the problem that the original coding model in related technologies is inaccurate in similar text recommendation results due to inconsistencies in content specifications across multiple platforms and the lack of text tags. Summary of the Invention

[0005] This invention provides a model training method, apparatus, computer device, and storage medium to solve the problem in the related art where the original encoded model results in inaccurate similar text recommendation results due to inconsistencies in content specifications across multiple platforms and missing text tags.

[0006] To achieve the above objectives, a first aspect of the present invention provides a model training method, comprising:

[0007] Step 1: Obtain the normalized training text information;

[0008] Step 2: Randomly select a sentence from the text information to be trained as the first sample, perform data augmentation on the first sample to obtain the second sample, and take another sentence from the text information to be trained, excluding the first sample, as the third sample.

[0009] Step 3: Encode the first sample, the second sample, and the third sample respectively using the original encoding model to obtain the first encoding result, the second encoding result, and the third encoding result. Randomly shuffle the second encoding result in both the character position dimension and the character encoding dimension to obtain the fourth encoding result.

[0010] Step 4: Construct a negative sample pair from the first and third encoding results, and construct a positive sample pair from the first and fourth encoding results;

[0011] Step 5: Perform steps 2-4 on all statements in the text information to be trained to obtain all positive and negative sample pairs;

[0012] Step 6: Calculate the loss using contrastive learning based on the positive and negative sample pairs to optimize the original coding model.

[0013] Optionally, in one possible implementation of the first aspect, obtaining the normalized training text information includes:

[0014] Obtain marketing copy information from different channels;

[0015] The marketing copy information is cleaned by data cleaning, which includes: filtering out meaningless special symbols, replacing meaningful special symbols with text commas, and replacing all non-comma Chinese and English symbols with text commas.

[0016] Information entropy is calculated on the marketing copy information after data cleaning, and meaningless repetitive long text phrases are filtered out based on word frequency and word length; the normalized training text information is obtained.

[0017] Optionally, in one possible implementation of the first aspect, the calculation of information entropy on the data-cleaned marketing copy information, and the filtering out meaningless, repetitive long text phrases based on word frequency and word length, includes:

[0018] By setting word frequency and word length, meaningless repetitive long text phrases are filtered out by calculating information entropy. The formula for calculating information entropy includes:

[0019]

[0020] Among them, w represents the phrase for which information entropy is to be calculated, len(w) < m indicates that the length of this word is restricted within m, freq(w) > k indicates that the occurrence frequency of this word is greater than k, and P(i | W ) represents the probability situation of all words surrounding the word w.

[0021] Optionally, in a possible implementation manner of the first aspect, the method of using contrastive learning to calculate the loss and optimize the encoding model includes:

[0022] The loss is calculated using the following formula:

[0023]

[0024] Among them, ri and rj represent positive samples, ri and rk represent negative sample pairs, sim(ri, rj) represents calculating the cosine similarity of vectors, and t is the temperature normalization factor;

[0025] By reducing the distance between positive sample pairs and increasing the distance between negative sample pairs, the loss is reduced and the original encoding model is optimized.

[0026] In the second aspect of the embodiments of the present invention, a model training device is provided, including:

[0027] An acquisition module, configured to acquire the text information to be trained after normalization processing;

[0028] A sample selection module, configured to randomly select a sentence from the text information to be trained as the first sample, perform data augmentation processing on the first sample to obtain the second sample, and use another sentence in the text information to be trained except the first sample as the third sample;

[0029] An encoding module, configured to encode the first sample, the second sample, and the third sample through the original encoding model respectively to obtain the first encoding result, the second encoding result, and the third encoding result, and randomly shuffle the second encoding result in the word position dimension and the word encoding dimension to obtain the fourth encoding result;

[0030] A sample pair construction module, configured to construct the first encoding result and the third encoding result into a negative sample pair, and construct the first encoding result and the fourth encoding result into a positive sample pair;

[0031] A repeated execution module, configured to perform the processing of the sample selection module, the encoding module, and the sample pair construction module on all sentences in the text information to be trained to obtain all positive and negative sample pairs;

[0032] A loss calculation module, configured to calculate the loss according to the positive and negative sample pairs by using the method of contrastive learning to optimize the original encoding model.

[0033] Optionally, in a possible implementation of the second aspect, the obtaining module includes:

[0034] An obtaining unit, configured to obtain marketing copy information from different channels;

[0035] A data cleaning unit, configured to perform data cleaning on the marketing copy information, where the data cleaning includes: filtering out meaningless special symbols, uniformly replacing meaningful special symbols with text commas, and replacing all non-comma Chinese and English symbols with text commas;

[0036] A data filtering unit, configured to calculate the information entropy of the marketing copy information after data cleaning, and filter out meaningless and repeatedly occurring long text phrases according to word frequency and word length; to obtain the normalized text information to be trained.

[0037] Optionally, in a possible implementation of the second aspect, the data filtering unit includes:

[0038] Set the word frequency and word length, and filter out meaningless and repeatedly occurring long text phrases by calculating the information entropy, where the information entropy calculation formula includes:

[0039]

[0040] where, w represents the phrase for which the information entropy is to be calculated, len(w)<m indicates that the length of this word is to be limited within m, freq(w)>k indicates that the occurrence frequency of this word is to be greater than k, and P(i | W ) represents the probability situation of all words surrounding the word w.

[0041] Optionally, in a possible implementation of the second aspect, the loss calculation module is further configured to perform the following steps:

[0042] Calculate the loss using the following formula:

[0043]

[0044] where, ri, rj represent positive samples, ri, rk represent negative sample pairs, sim(ri, rj) represents the calculation of the cosine similarity of vectors, and t is the temperature normalization factor;

[0045] By shortening the distance between positive sample pairs and lengthening the distance between negative sample pairs, the loss is reduced, and the original coding model is optimized.

[0046] In a third aspect of the embodiments of the present invention, a computer device is provided, including a memory and a processor, where the memory stores a computer program that can run on the processor, and when the processor executes the computer program, the steps in the above-mentioned method embodiments are implemented.

[0047] A fourth aspect of the present invention provides a readable storage medium storing a computer program, which, when executed by a processor, is used to implement the steps of the method described in the first aspect of the present invention and various possible designs of the first aspect.

[0048] The model training method, apparatus, computer equipment, and storage medium provided by this invention involve: randomly selecting a sentence from the text information to be trained as a first sample; performing data augmentation on the first sample to obtain a second sample; and selecting another sentence from the text information to be trained, excluding the first sample, as a third sample. The first, second, and third samples are then encoded using the original encoding model to obtain first, second, and third encoding results, respectively. The second encoding result is then randomly shuffled in both the character position and character encoding dimensions to obtain a fourth encoding result. The first and third encoding results are used to construct a negative sample pair, and the first and fourth encoding results are used to construct a positive sample pair. All positive and negative sample pairs are obtained by applying the above method to all sentences in the text information to be trained. Based on the positive and negative sample pairs, a contrastive learning method is used to calculate the loss to optimize the original encoding model. This invention can optimize the original encoding model to achieve the technical effect of recommending similar texts with high accuracy in situations such as inconsistent content specifications across multiple platforms and missing text tags. Attached Figure Description

[0049] Figure 1 , 2 This is a flowchart illustrating a model training method according to an embodiment of this application;

[0050] Figure 3 This is a flowchart illustrating the process of filtering out useless interference information using different cleaning rules.

[0051] Figure 4 This is a schematic diagram of the structure of a model training device according to an embodiment of this application. Detailed Implementation

[0052] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0053] The terms "first," "second," "third," "fourth," etc. (if present) in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein.

[0054] It should be understood that in the various embodiments of the present invention, the sequence number of each process does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0055] It should be understood that in this invention, "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion, for example, a process, method, system, product, or device that includes a series of steps or units is not necessarily limited to those steps or units that are explicitly listed, but may include other steps or units that are not explicitly listed or that are inherent to such process, method, product, or device.

[0056] It should be understood that in this invention, "multiple" refers to two or more. "And / or" is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, "and / or B" can represent: A existing alone, A and B existing simultaneously, and B existing alone. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "Contains A, B, and C", "Contains A, B, and C" means that all three A, B, and C are contained; "Contains A, B, or C" means that one of A, B, and C is contained; "Contains A, B, and / or C" means that any one, two, or three of A, B, and C are contained.

[0057] It should be understood that in this invention, "B corresponding to A", "B corresponding to A", "A and B correspond", or "B and A correspond" means that B is associated with A, and B can be determined based on A. Determining B based on A does not mean determining B solely based on A; B can also be determined based on A and / or other information. Matching A and B is defined as a similarity between A and B that is greater than or equal to a preset threshold.

[0058] Depending on the context, "if" as used here can be interpreted as "when," "when," "in response to determination," or "in response to detection."

[0059] The existing technology has the following problems:

[0060] 1. Content guidelines differ across platforms; Xiaohongshu posts are long articles with fixed wording and contain a large number of emojis, affecting the encoding model's ability to encode the text; WeChat Moments posts are short and contain a lot of meaningless text.

[0061] 2. Unlabeled content data; after text is encoded by the model, similar texts can be obtained through cosine similarity. However, this requires the model to be trained in advance on some similar text pairs or similar sentence pairs, which requires labeled data. However, in practice, labeling text requires a lot of manual labor. Currently, most of the text information obtained is unlabeled corpus.

[0062] The technical solution of the present invention will be described in detail below with reference to specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments.

[0063] This invention provides a model training method, such as... Figure 1 , 2 The flowchart shown includes:

[0064] Step 1: Obtain the normalized training text information.

[0065] In this step, the training text information includes training copy from different channels, such as collecting marketing copy from clients (e.g., a FMCG brand) across various platforms, including Xiaohongshu, Douyin, and WeChat Moments. Normalization involves segmenting the marketing copy information from different channels (due to inconsistent content standards across platforms) and using different cleaning rules to filter out useless and interfering information. Xiaohongshu marketing copy is characterized by long articles, fixed wording, and a large number of emojis, affecting text encoding; WeChat Moments copy is characterized by brevity and contains a large amount of meaningless text, such as "Add WeChat to get discount codes and the latest product launch information."

[0066] Step 2: Randomly select a sentence from the text information to be trained as the first sample, perform data augmentation on the first sample to obtain the second sample, and take another sentence from the text information to be trained, excluding the first sample, as the third sample.

[0067] In step 2,

[0068] First, randomly select one sentence from the normalized training text information as the first sample a, and another sentence as the third sample b;

[0069] Secondly, perform data augmentation on the first sample a, that is, randomly replace the adjectives in the first sample a with synonyms / near-synonyms (50% randomly replaced, 50% not replaced) to obtain the second sample a'.

[0070] Step 3: Encode the first sample, the second sample, and the third sample respectively through the original encoding model to obtain the first encoding result, the second encoding result, and the third encoding result. Randomly shuffle the second encoding result in the word position dimension and the word encoding dimension to obtain the fourth encoding result.

[0071] In Step 3, encode the samples a, a', and b through the original bert encoding model to obtain the encoding results a1, a1', and b1. Randomly shuffle the encoding result of a1' in the token dimension (word position dimension) and the embedding dimension (word encoding dimension) to obtain a2.

[0072] Specifically, an example is given for the above "randomly shuffle in the word position dimension and the self-encoding dimension":

[0073] The test sentence "anti-dandruff and itching-relieving shampoo" is a sentence of 7 characters, which is encoded into a three-dimensional vector of (1, 7, 768). The first dimension represents the number of sentences here, which is 1 sentence; the second dimension represents the number of characters in the corresponding sentence, which is 7 characters; the third dimension represents how many numbers are used to represent each character, and here 768 digital features are used to represent each character.

[0074] Randomly shuffling in the word position dimension means randomly shuffling in the second dimension. For example, after shuffling, this sentence may become "shampoo for itching and anti-dandruff". Randomly shuffling in the word encoding dimension means randomly shuffling in the third dimension. For example, we originally used 768 numbers to represent the character "去" [0.32, 0.98, 0.12, 0.55, 0.86, 0.77........0.32,0.65]. There are 768 numbers in total here. After shuffling, the encoding of the 768 numbers representing the character "去" may become [0.98, 0.32, 0.12,0.65,0.86, 0.77........0.32,0.55].

[0075] Step 4: Construct the first encoding result and the third encoding result into a negative sample pair, and construct the first encoding result and the fourth encoding result into a positive sample pair.

[0076] Step 5: Perform Steps 2-4 on all sentences in the text information to be trained to obtain all positive and negative sample pairs.

[0077] In steps 4-5, a1 and b1 are constructed as negative sample pairs, and a1 and a2 are constructed as positive sample pairs. The above operation is performed on N text pairs in the text information to be trained to obtain all sample pairs.

[0078] Step 6: Calculate the loss using contrastive learning based on the positive and negative sample pairs to optimize the original coding model.

[0079] In step 6, the contrastive learning loss is calculated using the following formula:

[0080]

[0081] Where ri and rj represent positive samples, ri and rk represent negative sample pairs, sim(ri, rj) represents the calculation of vector cosine similarity, and t is the temperature normalization factor;

[0082] By narrowing the distance between positive sample pairs and widening the distance between negative sample pairs, the loss is reduced, thus optimizing the original encoding model. The loss calculation employs contrastive learning loss, thereby fine-tuning the BERT model.

[0083] In one embodiment, obtaining the normalized training text information includes:

[0084] Obtain marketing copy information from different channels;

[0085] The marketing copy information is cleaned by data cleaning, which includes: filtering out meaningless special symbols, replacing meaningful special symbols with text commas, and replacing all non-comma Chinese and English symbols with text commas.

[0086] Information entropy is calculated on the marketing copy information after data cleaning, and meaningless repetitive long text phrases are filtered out based on word frequency and word length; the normalized training text information is obtained.

[0087] In this step, such as Figure 3 As shown, marketing copy information from different channels is categorized (content standards vary across platforms), and different cleaning rules are used to filter out useless and distracting information.

[0088] First, considering that Xiaohongshu's marketing copy is characterized by long articles, fixed wording, and a large number of emojis, it is necessary to filter out meaningless emojis. Meanwhile, meaningful emojis, such as checkmarks and crosses describing effects, and exclamation marks and question marks indicating tone, should be uniformly replaced with text commas that provide a pause. Additionally, all non-comma Chinese and English symbols should be replaced with text commas.

[0089] Secondly, the marketing copy in WeChat Moments is characterized by being short and containing a large amount of meaningless text information. Therefore, it is necessary to calculate the information entropy of such marketing copy information, and filter out meaningless and repeatedly appearing long text phrases according to word frequency and word length. The information entropy calculation formula includes:

[0090]

[0091] Among them, w represents the phrase for which information entropy is to be calculated, len(w)<m means the length of this word is limited within m, freq(w)>k means the occurrence frequency of this word is greater than k, and P(i | W ) represents the probability situation of all words surrounding the word w.

[0092] Specifically, the greater the information entropy, the more chaotic the surrounding words are, indicating that there are many surrounding words for the word w itself, and this word is probably considered as an information that can be fixed because it can be combined with multiple words. The less the information entropy, the purer the surrounding words are, indicating that there are few adjacent words around the word w itself. Then this word is very likely to be combined with the surrounding words to form a new phrase, and then it will be judged whether a phrase can be constructed under the condition of adding 1 to the ngram.

[0093] When m is set to 25 and k is set to 10, some frequently occurring meaningless long texts can be obtained through the information entropy method. For example, in the WeChat Moments materials, we can obtain meaningless long texts such as "Add WeChat to get the discount code and get the latest product launch information". Since these long texts appear in a large number of copies, they are not helpful for us to distinguish texts, so they need to be removed.

[0094] The model training method provided by the present invention randomly selects a sentence from the text information to be trained as the first sample, performs data augmentation processing on the first sample to obtain the second sample, and uses another sentence in the text information to be trained except the first sample as the third sample; respectively encodes the first sample, the second sample, and the third sample through the original encoding model to obtain the first encoding result, the second encoding result, and the third encoding result, and randomly shuffle the second encoding result in the word position dimension and the word encoding dimension to obtain the fourth encoding result; constructs the first encoding result and the third encoding result into a negative sample pair, and constructs the first encoding result and the fourth encoding result into a positive sample pair; performs the above method on all sentences in the text information to be trained to obtain all positive and negative sample pairs; calculates the loss according to the positive and negative sample pairs by using the contrast learning method to optimize the original encoding model. The present invention can optimize the original encoding model to achieve the technical effect of recommending highly accurate similar texts in cases where the content specifications are inconsistent on multiple platforms and the copy tags are missing.

[0095] Technical effects:

[0096] To address the issue of inconsistent content standards across multiple platforms, this invention employs different data cleaning methods, sets specific emoji cleaning rules, calculates word information entropy to filter out useless phrases, and normalizes the content information to obtain plain text that the encoding model can encode.

[0097] To address the issue of unlabeled data, this invention employs a contrastive learning approach, constructing positive and negative sample pairs based on the text itself, and using contrastive learning loss for loss calculation to fine-tune the BERT model.

[0098] Embodiments of the present invention also provide a model training apparatus, such as... Figure 4 As shown, it includes:

[0099] The acquisition module is used to acquire the normalized training text information.

[0100] The sample selection module is used to randomly select a sentence from the text information to be trained as a first sample, perform data augmentation processing on the first sample to obtain a second sample, and take another sentence from the text information to be trained, excluding the first sample, as a third sample.

[0101] The encoding module is used to encode the first sample, the second sample, and the third sample respectively using the original encoding model to obtain the first encoding result, the second encoding result, and the third encoding result. The second encoding result is randomly shuffled in the character position dimension and the character encoding dimension to obtain the fourth encoding result.

[0102] The sample pair construction module is used to construct a negative sample pair from the first encoding result and the third encoding result, and to construct a positive sample pair from the first encoding result and the fourth encoding result;

[0103] The repeated execution module is used to process all statements in the text information to be trained by the sample selection module, encoding module and sample pair construction module to obtain all positive and negative sample pairs.

[0104] The loss calculation module is used to calculate the loss based on the positive and negative sample pairs using a contrastive learning method, in order to optimize the original coding model.

[0105] The readable storage medium can be a computer storage medium or a communication medium. A communication medium includes any medium that facilitates the transfer of computer programs from one location to another. A computer storage medium can be any available medium accessible to a general-purpose or special-purpose computer. For example, a readable storage medium is coupled to a processor, enabling the processor to read information from and write information to the readable storage medium. Of course, the readable storage medium can also be a component of the processor. The processor and the readable storage medium can reside in an Application-Specific Integrated Circuit (ASIC). Alternatively, the ASIC can be located in a user equipment. Of course, the processor and the readable storage medium can also exist as discrete components in a communication device. The readable storage medium can be a read-only memory (ROM), random access memory (RAM), CD-ROM, magnetic tape, floppy disk, and optical data storage device, etc.

[0106] The present invention also provides a program product including executable instructions stored in a readable storage medium. At least one processor of the device can read the executable instructions from the readable storage medium, and the at least one processor executes the executable instructions to cause the device to implement the methods provided in the various embodiments described above.

[0107] In the above-described terminal or server embodiments, it should be understood that the processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), etc. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in this invention can be directly manifested as being executed by a hardware processor, or executed by a combination of hardware and software modules within the processor.

[0108] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A model training method, characterized in that, include: Step 1: Obtain the normalized training text information; Step 2: Randomly select a sentence from the text information to be trained as the first sample, perform data augmentation on the first sample to obtain the second sample, and take the other sentence from the text information to be trained besides the first sample as the third sample. Step 3: Encode the first sample, the second sample, and the third sample respectively using the original encoding model to obtain the first encoding result, the second encoding result, and the third encoding result. Randomly shuffle the second encoding result in both the character position dimension and the character encoding dimension to obtain the fourth encoding result. Step 4: Construct a negative sample pair from the first and third encoding results, and construct a positive sample pair from the first and fourth encoding results; Step 5: Perform steps 2-4 on all statements in the text information to be trained to obtain all positive and negative sample pairs; Step 6: Based on the positive and negative sample pairs, calculate the loss using contrastive learning to optimize the original coding model; The process of obtaining the normalized training text information includes: Obtain marketing copy information from different channels; The marketing copy information is cleaned by data cleaning, which includes: filtering out meaningless special symbols, replacing meaningful special symbols with text commas, and replacing all non-comma Chinese and English symbols with text commas. Information entropy is calculated on the marketing copy information after data cleaning. Meaningless repeated long text phrases are filtered out based on word frequency and word length to obtain normalized training text information.

2. The model training method according to claim 1, characterized in that, The process of calculating information entropy from cleaned marketing copy and filtering out meaningless, repetitive long phrases based on word frequency and length includes: By setting word frequency and word length, meaningless repetitive long text phrases are filtered out by calculating information entropy. The formula for calculating information entropy includes: in, w Words indicating that information entropy calculation is to be performed. len(w) <m This indicates that the length of the word must be limited to 1000 words. m within, freq(w)>k This indicates that the frequency of this word is greater than [a certain value]. k , P(i | w) Indicates that it appears w The probability of all words surrounding this word.

3. The model training method according to claim 1, characterized in that, The method of calculating loss and optimizing the encoding model using contrastive learning includes: The loss is calculated using the following formula: Where ri and rj represent positive samples, ri and rk represent negative sample pairs, sim(ri, rj) represents the calculation of vector cosine similarity, and t is the temperature normalization factor; By narrowing the distance between positive sample pairs and widening the distance between negative sample pairs, the loss is reduced and the original coding model is optimized.

4. A model training device, characterized in that, include: The acquisition module is used to acquire the normalized training text information. The sample selection module is used to randomly select a sentence from the text information to be trained as a first sample, perform data augmentation on the first sample to obtain a second sample, and take another sentence from the text information to be trained, excluding the first sample, as a third sample. The encoding module is used to encode the first sample, the second sample, and the third sample respectively using the original encoding model to obtain the first encoding result, the second encoding result, and the third encoding result. The second encoding result is randomly shuffled in the character position dimension and the character encoding dimension to obtain the fourth encoding result. The sample pair construction module is used to construct a negative sample pair from the first encoding result and the third encoding result, and to construct a positive sample pair from the first encoding result and the fourth encoding result; The repeated execution module is used to process all statements in the text information to be trained by the sample selection module, encoding module and sample pair construction module to obtain all positive and negative sample pairs. The loss calculation module is used to calculate the loss based on the positive and negative sample pairs using a contrastive learning method, so as to optimize the original coding model; The acquisition module includes: The acquisition unit is used to acquire marketing copy information from different channels; The data cleaning unit is used to clean the marketing copy information. The data cleaning includes: filtering out meaningless special symbols, replacing meaningful special symbols with text commas, and replacing all non-comma Chinese and English symbols with text commas. The data filtering unit is used to calculate the information entropy of the marketing copy information after data cleaning, and to filter out meaningless repeated long text phrases based on word frequency and word length; thus obtaining normalized training text information.

5. The model training apparatus according to claim 4, characterized in that, The data filtering unit is also configured to perform the following steps: By setting word frequency and word length, meaningless repetitive long text phrases are filtered out by calculating information entropy. The formula for calculating information entropy includes: in, w Words indicating that information entropy calculation is to be performed. len(w) <m This indicates that the length of the word must be limited to 1000 words. m within, freq(w)>k This indicates that the frequency of this word is greater than [a certain value]. k , P(i | w) Indicates that it appears w The probability of all words surrounding this word.

6. The model training apparatus according to claim 4, characterized in that, The loss calculation module is also used to perform the following steps: The contrastive learning loss is calculated using the following formula: Where ri and rj represent positive samples, ri and rk represent negative sample pairs, sim(ri, rj) represents the calculation of vector cosine similarity, and t is the temperature normalization factor; By narrowing the distance between positive sample pairs and widening the distance between negative sample pairs, the loss is reduced and the original coding model is optimized.

7. A computer device comprising a memory and a processor, the memory storing a computer program executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the model training method according to any one of claims 1 to 3.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the steps of the model training method according to any one of claims 1 to 3.

Images