A text processing method, device and storage medium based on text repetition

Abstract

The present invention relates to the field of electronic digital data processing technology, and in particular to a text processing method, device and storage medium based on text repetition. The method comprises: obtaining a text a of the i-th category to be sampled i , a i The number of texts to be sampled is q i , a i Included q i The labels of the texts to be sampled are the same; get the number of samples s of the i-th category i ; if q i >s i , then from a i Randomly obtain s i The texts to be sampled constitute the discriminant sample text B of the i-th category. i ; for B i Perform grouping and obtain the grouping result R i , R i Including v groups; according to R i The text similarity between the included groups is obtained by B i The repetition between Chinese texts is i ; If sim i ≤sim0, then B i The target sampled text of the i-th category is determined. The present invention improves the efficiency of obtaining sampled text and improves the reasoning ability of a semantic analysis model trained based on the sampled text.

Description

Technical Field

[0001] The present invention relates to the technical field of electronic digital data processing, and in particular to a text processing method, device and storage medium based on text repetitiveness. Background Art

[0002] When sampling a text set corresponding to a certain label for training a semantic analysis model, the following problem may arise: the texts in the text set corresponding to the label have a high degree of repetition. If random sampling is used to sample from the text set corresponding to the label, the sampled texts may have high similarities, affecting the reasoning ability of the semantic analysis model trained based on the sampled texts about the label.

[0003] The prior art has disclosed a method for clustering texts based on text similarity. For example, a Chinese patent application with publication number CN16401565A discloses a method for training a text clustering model and a text clustering method based thereon. The patent obtains a text clustering model with stronger semantic analysis capability based on a pre-trained BERT model, and clustering of the texts to be clustered can be achieved based on the text clustering model. Although the method of clustering a text set corresponding to a certain label is conducive to improving the text differences between the obtained sampled texts, the efficiency of obtaining sampled texts by clustering a text set corresponding to a certain label is lower than the efficiency of obtaining sampled texts by random sampling. How to balance the differences between the sampled texts and the efficiency of obtaining the sampled texts is an urgent problem to be solved. Summary of the invention

[0004] The purpose of the present invention is to provide a text processing method, device and storage medium based on text repetition, so as to take into account the differences between sampled texts and the efficiency of obtaining sampled texts, improve the efficiency of obtaining sampled texts and improve the reasoning ability of semantic analysis models trained based on sampled texts.

[0005] According to a first aspect of the present invention, a text processing method based on text repetition comprises the following steps:

[0006] P100, get the i-th type of text to be sampled a i , a i The number of texts to be sampled is q i , a i Included q i The labels of the texts to be sampled are the same, but different i The labels of the texts to be sampled are different, and the value of i ranges from 1 to n, where n is the number of categories of the texts to be sampled.

[0007] P200, get the number of samples s of the i-th category i .

[0008] P300, if q i >s i , then from a i Randomly obtain s i The texts to be sampled constitute the discriminant sample text B of the i-th category. i .

[0009] P400, for B i Perform grouping and obtain the grouping result R i , R i It includes v groups, where v is the preset number of groups.

[0010] P500, according to R i The text similarity between the included groups is obtained by B i The repetition between Chinese texts is i .

[0011] P600, if sim i ≤sim0, then enter P700; sim0 is the preset similarity threshold.

[0012] P700, B i Determine the target sample text for the i-th category.

[0013] According to a second aspect of the present invention, an electronic device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the above-mentioned text processing method based on text repetition when executing the computer program.

[0014] According to a third aspect of the present invention, a computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the text processing method based on text repetition is implemented.

[0015] The present invention has at least the following beneficial effects:

[0016] The present invention is for the i-th type of sampled text a i , if a i The number of texts to be sampled is q i Greater than the number of samples s of the i-th class i , then start from a i Randomly obtain s i The texts to be sampled constitute the discriminant sample text B of the i-th category. i ; Through B i By grouping and judging the text similarity between groups, the present invention obtains Bi The repetition between Chinese texts is i , and further sim i Compare with the preset similarity threshold sim0; if sim i ≤sim0, that is, from a i Randomly obtain s i The discriminant sample text B of the i-th category is composed of the texts to be sampled i Only when the repetition of Chinese text is low, B i Determine the target sample text for the i-th category to solve the problem in B i If the Chinese text has a high degree of repetition, directly change B i The trained semantic analysis model is determined to be a target sample text of the i-th category i The problem of low reasoning ability of the corresponding labels; moreover, the B of the present invention i It is obtained by random sampling method, and the sampling efficiency of random sampling method is relatively high. i ≤sim0 can be avoided by i The clustering method is used to obtain sample text, thereby achieving the purpose of taking into account the efficiency of obtaining sample text. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0018] Figure 1 This is a flowchart of a text processing method based on text repetition provided in Example 1 of the present invention. DETAILED DESCRIPTION

[0019] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0020] Embodiment 1:

[0021] According to this embodiment, a text processing method based on text repetition is provided, such as Figure 1 As shown, including:

[0022] P100, get the i-th type of text to be sampled a i , a i The number of texts to be sampled is q i , a i Included q i The labels of the texts to be sampled are the same, but different i The labels of the texts to be sampled are different, and the value of i ranges from 1 to n, where n is the number of categories of the texts to be sampled.

[0023] In this embodiment, the text to be sampled is A, A=(a1, a2, ..., a i ,…,a n ), a i is the i-th type of text to be sampled.

[0024] P200, get the number of samples s of the i-th category i .

[0025] In this embodiment, s i For experience value.

[0026] P300, if q i >s i , then from a i Randomly obtain s i The texts to be sampled constitute the discriminant sample text B of the i-th category. i .

[0027] In this embodiment, P300 further includes: if q i ≤s i , then a i Determine the target sample text for the i-th category.

[0028] P400, for B i Perform grouping and obtain the grouping result R i , R i It includes v groups, where v is the preset number of groups.

[0029] Optional, for B i The included texts are randomly grouped so that each R i,t The number of included texts is greater than or equal to floor(s i / v) and less than or equal to ceil(s) i / v), floor() is used to round down, and ceil() is used to round up.

[0030] P500, according to R i The text similarity between the included groups is obtained by B i The repetition between Chinese texts is i .

[0031] Optionally, the P500 includes:

[0032] P510, get the spliced ​​text CR i , CR i Including v concatenated texts, CR i Include the tth concatenated text CR i,t To R i,t The text obtained by concatenating the included texts; R i,t For B i The tth group is obtained by grouping; the value range of t is 1 to v.

[0033] In this embodiment, CR i ={CR i,1 ,CR i,2 ,…,CR i,t ,…,CR i,v}.

[0034] Preferably, R i,t The included text is concatenated from beginning to end to get CR i,t .

[0035] P520, Get CR i The text similarity of any two texts in SH is added to the preset similarity set SH. The obtained SH includes D text similarities. SH includes the d-th text similarity sh d CR i The text similarity between the d-th pair of texts in , where d ranges from 1 to D, and D is CR i The corresponding number of text pairs, D=v×(v-1) / 2; SH is initialized to an empty set.

[0036] In this embodiment, SH={sh1,sh2,…,sh d ,…,sh D}.

[0037] In this embodiment, from CR i Choose any two texts to form a pair of texts, CR i The amount of text included is v, so CR i The corresponding number of text pairs is v×(v-1) / 2.

[0038] Those skilled in the art are aware that any method for obtaining text similarity in the prior art falls within the protection scope of the present invention.

[0039] P530, get sim i ,sim i =sum(SH) / D, sum( ) is the sum.

[0040] Based on P510-P530, this embodiment can obtain B i The repetition between Chinese texts is i ,sim i The larger the value, the higher the B i The higher the repetition between Chinese texts.

[0041] P600, if sim i ≤sim0, then enter P700; sim0 is the preset similarity threshold.

[0042] P700, B i Determine the target sample text for the i-th category.

[0043] In this embodiment, for the i-th type of sampled text a i , if a i The number of texts to be sampled is q i Greater than the number of samples s of the i-th class i , then start from a i Randomly obtain s i The texts to be sampled constitute the discriminant sample text B of the i-th category. i ; Through B i To group and determine the text similarity between groups, this embodiment obtains B i The repetition between Chinese texts is i , and further sim i Compare with the preset similarity threshold sim0; if sim i ≤sim0, that is, from a i Randomly obtain s i The discriminant sample text B of the i-th category is composed of the texts to be sampled i Only when the repetition of Chinese text is low, B i Determine the target sample text for the i-th category to solve the problem in B i If the Chinese text has a high degree of repetition, directly change B i The trained semantic analysis model is determined to be a target sample text of the i-th category i The corresponding label has a low reasoning ability; moreover, B in this embodiment i It is obtained by random sampling, which has a relatively high sampling efficiency. i ≤sim0 can be avoided by i The clustering method is used to obtain sample text, thereby achieving the purpose of taking into account the efficiency of obtaining sample text.

[0044] In this embodiment, P600 also includes: if simi >sim0, then enter P800.

[0045] P800, according to a i The text similarity between the texts to be sampled includes a i Perform clustering and obtain the clustering result G i , G i =(g i,1 ,g i,2 ,…,g i,j ,…,g i,m ), g i,j For a i The jth cluster obtained by clustering, the value of j ranges from 1 to m, and m is the value of a i The number of clusters obtained by clustering.

[0046] Those skilled in the art are aware that any method for obtaining text similarity in the prior art falls within the protection scope of the present invention.

[0047] Those skilled in the art know that any clustering method in the prior art falls within the protection scope of the present invention. Optionally, a density-based clustering method (such as DBSCAN) is used to cluster a i Clustering is performed, where a i Each of the texts to be sampled is taken as a point, and the distance between the two texts to be sampled is used to realize a i The distance between two to-be-sampled texts is negatively correlated with the text similarity between the two to-be-sampled texts. Therefore, a i The sampled texts with higher text similarity are clustered into one cluster, a i The sampled texts with lower text similarity are clustered into different clusters.

[0048] P810, Get G i The number of texts to be sampled in the smallest cluster s i,min .

[0049] In this embodiment, G i The smallest cluster in G i The cluster with the least amount of text is included.

[0050] P820, if s i,min ≥f(s i / m), then go to P830; f( ) is the rounding function.

[0051] Optionally, f( ) is a floor function or a round-to-nearest function.

[0052] P830, traverse G i , from g i,j Get f(si / m) texts to be sampled constitute the initial sampling text of the i-th category.

[0053] Optional, randomly selected from g i,j Get f(s i / m) texts to be sampled constitute the initial sampling text of the i-th category.

[0054] P840, if m×f(s i / m)=s i , then the initial sampled text of the i-th category is determined as the target sampled text of the i-th category; if m×f(s i / m)>s i , then randomly delete m×f(s i / m)-s i text, the initial sampled text of the i-th category after deletion is determined as the target sampled text of the i-th category, and the deleted m×f(s i / m)-s i The text belongs to a i m×f(s i / m)-s i clusters; if m×f(s i / m) i , then from a i Randomly get s from the remaining text i -m×f(s i / m) to-be-sampled texts are added to the initial sampled text of the i-th category, and the added initial sampled text of the i-th category is determined as the target sampled text of the i-th category; a i The remaining text is a i The text except the initial sampled text of the i-th category.

[0055] Based on P800-P840, this embodiment is in sim i >sim0, first according to a i The text similarity between the texts to be sampled includes a i Perform clustering, and for the m clusters obtained, obtain the number of texts s included in the cluster with the least number of texts (i.e., the smallest cluster) among these m clusters i,min , if s i,min ≥f(s i / m), then from each g i,j Get f(s i / m) samples of texts to be sampled constitute the initial sampled text of the i-th category. If m×f(s i / m)=s i , then the initial sampled text of the i-th category is determined as the target sampled text of the i-th category; if m×f(s i ​ / m)≠s i , then a small adjustment is made based on the initial sampled text of the i-th category, minimizing the i The difference in the number of texts sampled in different clusters. i The m clusters represent a i m kinds of text in a, so this embodiment realizes i The diverse sampling improves the subsequent trained semantic analysis model i The reasoning ability of the corresponding label.

[0056] In this embodiment, P820 further includes: if s i,min <f(s i / m), then enter P821.

[0057] P821, traverse G i , from each g i,j Get i,min The texts to be sampled constitute the first sampled text of the i-th category.

[0058] Optional, randomly selected from each g i,j Get i,min The texts to be sampled constitute the first sampled text of the i-th category.

[0059] P822, obtain the first remaining sampling quantity Δε1, Δε1=s i -m×s i,min .

[0060] P823, get the clustering result G after the first sampling 1 i , G 1 i =(g 1 i,1 ,g 1 i,2 ,…,g 1 i,j ,…,g 1 i,m ), g 1 i,j From g i,j Get i,min The clusters are composed of the remaining texts after the texts to be sampled.

[0061] In this embodiment, if g i,j The amount of text included is greater than s i,min , then g 1 i,j is a non-empty cluster; if g i,j Includes an amount of text equal to s i,min , then g1 i,j An empty cluster.

[0062] P824, Get G 1 i The number of non-empty clusters in δ.

[0063] P825, if Δε1=δ, then from G 1 i Obtain one to-be-sampled text from each non-empty cluster and append it to the first sampled text of the i-th category, and determine the first sampled text of the i-th category after the appending as the target sampled text of the i-th category.

[0064] Based on P821-P825, this embodiment is in G i The number of texts to be sampled in the smallest cluster s i,min Less than f(s i / m) in the case of i,j Get s i,min The texts to be sampled constitute the first sampled text of the i-th category, and then obtain the first remaining sample number Δε1 and the number of non-empty clusters δ. If δ=Δε1, then obtain 1 text to be sampled from each non-empty cluster and append it to the first sampled text of the i-th category. Therefore, this embodiment minimizes the number of samples from a i The difference in the number of texts sampled in different clusters due to a i The m clusters represent a i m kinds of text in a, so this embodiment realizes i The diverse sampling improves the subsequent trained semantic analysis model i The reasoning ability of the corresponding label.

[0065] In this embodiment, P825 further includes: if Δε1<δ, then from G 1 i Obtain one to-be-sampled text from each target cluster of G and append it to the first sampled text of the i-th category, and determine the first sampled text of the i-th category after appending as the target sampled text of the i-th category; the target cluster is 1 i Δε1 non-empty clusters are randomly selected from the δ non-empty clusters.

[0066] In this embodiment, G i The number of texts to be sampled in the smallest cluster s i,min Less than f(s i / m) in the case of i,j Get i,minThe texts to be sampled constitute the first sampled text of the i-th category, and then obtain the first remaining sample quantity Δε1 and the number of non-empty clusters δ. If Δε1<δ, a small adjustment is made to the first sampled text of the i-th category to minimize the number of samples from a i The difference in the number of texts sampled in different clusters due to a i The m clusters represent a i m kinds of text in a, so this embodiment realizes i The diverse sampling improves the subsequent trained semantic analysis model i The reasoning ability of the corresponding label.

[0067] Preferably, P825 also includes: if Δε1>δ, enter P826.

[0068] P826, Get G 1 i The number of texts to be sampled in the smallest non-empty cluster s 1 i,min .

[0069] P827, if s 1 i,min ≥f(Δε1 / δ), then enter P828.

[0070] P828, from G 1 i Get f(Δε1 / δ) texts to be sampled from each non-empty cluster and append them to the first sampled text of the i-th category.

[0071] P829, if f(Δε1 / δ)×δ=Δε1, the first sampled text of the i-th category after appending is determined as the target sampled text of the i-th category.

[0072] Based on P826-P829, this embodiment is in G i The number of texts to be sampled in the smallest cluster s i,min Less than f(s i / m) in the case of i,j Get i,min The texts to be sampled constitute the first sampled text of the i-th category, and then obtain the first remaining sample quantity Δε1 and the number of non-empty clusters δ. If δ<Δε1, obtain G 1 i The number of texts to be sampled in the smallest non-empty cluster s 1 i,min , if s 1 i,min≥ f(Δε1 / δ), then f(Δε1 / δ) text samples to be sampled are obtained from each non-empty cluster and appended to the first sampled text of the i-th class. If f(Δε1 / δ) × δ = Δε1, the appended first sampled text of the i-th class is determined as the target sampled text of the i-th class. Thus, in this embodiment, the difference in the number of texts sampled from different clusters of a is minimized. Since the m clusters of a i represent m types of texts in a, this embodiment realizes diversified sampling of a i , and improves the inference ability of the subsequent trained semantic analysis model for the labels corresponding to a i . i i

[0073] In this embodiment, P829 further includes: if f(Δε1 / δ) × δ ≠ Δε1, the target sampled text of the i-th class is obtained in a manner similar to P840, which will not be elaborated here.

[0074] In this embodiment, P827 further includes: if s 1 i,min < f(Δε1 / δ), the target sampled text of the i-th class is obtained in a manner similar to P821 - P825, which will not be elaborated here.

[0075] Embodiment 2:

[0076] Considering that when sampling a text set including different labels for semantic analysis model training, if the difference in the number of texts corresponding to different labels is large, when randomly sampling the text set or sampling according to the proportion of the number of texts corresponding to different labels, the texts corresponding to the labels with a smaller number of corresponding texts in the sampled texts account for a smaller proportion, resulting in a poor inference ability of the trained semantic analysis model for the labels corresponding to the smaller number of corresponding texts; and if the difference in the number of texts corresponding to different labels is large, if a quantity threshold is directly set according to experience, when the number of texts corresponding to a certain label is greater than the quantity threshold, a small proportion of texts are sampled from the texts corresponding to that label; when the number of texts corresponding to a certain label is less than the quantity threshold, a large proportion of texts are sampled from the texts corresponding to that label; this sampling method is relatively rough, making the number distribution of texts corresponding to different labels in the sampled texts quite different from the number distribution of texts corresponding to different labels in the original text set. Therefore, on the basis of Embodiment 1, this embodiment further optimizes the process of obtaining the sampling quantity s i of the i-th class. Specifically, s i = f((q i ) 0.5 / ((q i / (∑ n i=1 qi )) 1 / x )), x is the preset first value, x is a positive integer, and f( ) is the rounding function.

[0077] Based on the formula s of this embodiment i =f((q i ) 0.5 / ((q i / (∑ n i=1 q i )) 1 / x )) can extract as much text as possible from a text set including different labels in a class with a small number of texts. When the sampled text is subsequently used as training text for training a semantic analysis model, the reasoning ability of the trained semantic analysis model for labels corresponding to classes with a small number of texts can be improved. Moreover, based on the formula s of this embodiment, i =f((q i ) 0.5 / ((q i / (∑ n i=1 q i )) 1 / x )) can also achieve the purpose of sampling more classes that include a larger number of texts than classes that include a smaller number of texts, and can improve the consistency between the distribution of the number of texts corresponding to different labels in the sampled text and the distribution of the number of texts corresponding to different labels in the text to be sampled.

[0078] Optionally, x is an empirical value, for example, x=3.

[0079] Preferably, x=f( l g((max(Q)) 0.5 )) or x=f( l g((∑ n i=1 q i ) 0.5 )), Q is the sampling number sequence of A, Q=(q1,q2,…,q i ,…,q n ), max() is the maximum value function, l g is the logarithm to base 10. Small-scale experiments show that when x=f( l g((max(Q)) 0.5 )) or x=f( l g((∑ n i=1 q i ) 0.5 )), the quantity distribution of texts corresponding to different labels in the sampled text is basically consistent with the quantity distribution of texts corresponding to different labels in A.

[0080] Example 3:

[0081] Considering that in Example 1, based on the repetition degree sim between texts in Chinese i to judge whether to cluster and sample a i is a relatively complex process, which affects the efficiency of obtaining the target sampling text of the i-th category. Based on Example 1, after P200 and before P300, the text processing method based on text repetition degree further includes: i

[0082]

[0083] i .

[0084] In this example, A = {a1, a2,..., a i ,..., a n}.

[0085] P202, obtain the sum w of the number of texts to be sampled included in the top u categories with the largest number of texts to be sampled in A, where u is a preset category number threshold.

[0086] Optionally, u is an empirical value or u = f(r'×n), where r' is a preset label ratio threshold. Optionally, r' is 5%.

[0086] P203, if w / (∑ n i=1 q i ) < η, then enter P300; η is a preset ratio threshold; otherwise, enter P204.

[0087] Optionally, η is an empirical value. Optionally, η is 50%.

[0088] P204, obtain the sampling ratio r i of the i-th category, r i = s i / q i .

[0089] P205, if r i < r0, then enter P800; r0 is a preset sampling ratio threshold; otherwise, randomly obtain s i texts to be sampled from a i to form the target sampling text of the i-th category.

[0090] In this example, r0 is an empirical value. Optionally, r0 is 5% or 10%.

[0091] Based on P202 - P203, when w / (∑ ni=1 q i )≥η, that is, the number of texts corresponding to different labels included in A is quite different, that is, the number of texts corresponding to different labels is relatively unbalanced, execute P204-P205 to divide the n types of texts to be sampled included in A into 2 categories based on the preset sampling ratio threshold. The first category is the class with a sampling ratio less than the preset sampling ratio threshold (corresponding to the class with a large number of texts in A), and the other category is the class with a sampling ratio greater than or equal to the preset sampling ratio threshold (corresponding to the class with a small number of texts in A); the method of judging whether to perform cluster sampling based on the sampling ratio is more efficient, and for the class with a sampling ratio greater than or equal to the preset sampling ratio threshold, since the sampling ratio is relatively large, even if the class is directly randomly sampled, it will not have much impact on the inference effect of the label corresponding to the class by the later trained semantic analysis model. And in w / (∑ n i=1 q i )<η, that is, the number of texts corresponding to different labels included in A is relatively small, that is, when the number of texts corresponding to different labels is relatively balanced, the difference in sampling ratio of each category is relatively small, and P300-P700 is executed without obtaining the preset sampling ratio threshold, but i If the repetition is high, cluster sampling is performed on the class to improve the inference effect of the later trained semantic analysis model on the label corresponding to the class.

[0092] Embodiment 4:

[0093] This embodiment provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the following steps are implemented:

[0094] P100, get the i-th type of text to be sampled a i , a i The number of texts to be sampled is q i , a i Included q i The labels of the texts to be sampled are the same, the value range of i is from 1 to n, and n is the number of categories of the texts to be sampled.

[0095] P200, get the number of samples s of the i-th category i .

[0096] P300, if q i >s i , then from a i Randomly obtain s i The texts to be sampled constitute the discriminant sample text B of the i-th category. i .

[0097] P400, for B i Perform grouping and obtain the grouping result R i , R i It includes v groups, where v is the preset number of groups.

[0098] P500, according to R i The text similarity between the included groups is obtained by B i The repetition between Chinese texts is i .

[0099] P600, if sim i ≤sim0, then enter P700; sim0 is the preset similarity threshold.

[0100] P700, B i Determine the target sample text for the i-th category.

[0101] Embodiment five:

[0102] This embodiment provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the following steps are implemented:

[0103] P100, get the i-th type of text to be sampled a i , a i The number of texts to be sampled is q i , a i Included q i The labels of the texts to be sampled are the same, the value range of i is from 1 to n, and n is the number of categories of the texts to be sampled.

[0104] P200, get the number of samples s of the i-th category i .

[0105] P300, if q i >s i , then from a i Randomly obtain s i The texts to be sampled constitute the discriminant sample text B of the i-th category. i .

[0106] P400, for B i Perform grouping and obtain the grouping result R i , R i It includes v groups, where v is the preset number of groups.

[0107] P500, according to R i The text similarity between the included groups is obtained by B i The repetition between Chinese texts is i .

[0108] P600, if sim i ≤sim0, then enter P700; sim0 is the preset similarity threshold.

[0109] P700, B i Determine the target sample text for the i-th category.

[0110] Embodiment six:

[0111] Each text in the target sample text obtained in Example 1 has a corresponding labeled label, but the labeled label may be wrong, that is, the label annotated on the text does not match the text. In order to clean the target sample text, the text processing method based on text repetition in this embodiment also includes the following steps:

[0112] L100, get the text to be cleaned cle, cle=(cle1,cle2,…,cle ra ,…,cle to ),cle ra is the rath text to be cleaned, the value range of ra is 1 to to, and to is the number of texts to be cleaned.

[0113] In this embodiment, the target sample text is determined as tex, and to is the number of the target sample text.

[0114] L200, traverse cle, get cle ra Corresponding prompt word pro ra , pro ra Includes thought chain example text mod ra and the text to be inferred ans ra , mod ra =(mod ra,1 ,mod ra,2 ,…,mod ra,b ,…,mod ra,num ), mod ra,b For pro ra Includes the b-th thought chain example text, where b ranges from 1 to num, and num is cle ra The number of corresponding thought chain example texts; each mod ra,b Includes a corresponding target text tar ra,b , a corresponding label l ab ra,b and a corresponding judgment text tho ra,b , tho ra,b Including a pair l ab ra,b Is it tar? ra,bThe label's judgment mark sig ra,b and sig ra,b Corresponding reasoning text; ans ra Including cle ra andcle ra Tags l ab ra .

[0115] In this embodiment, if the value of num is small, that is, the number of thought chain example texts included in the prompt words input into the trained large language model is small, it is not conducive to the trained large language model to comprehensively learn the thinking of judging whether the label is correct, affecting the accuracy of the judgment result; if the value of num is large, that is, the number of thought chain example texts included in the prompt words input into the trained large language model is large, the trained large language model learns the thinking of judging whether the label is correct. It takes a long time, which is not conducive to quickly giving a judgment result; as an optional specific implementation method, num is an empirical value, optional, 2≤num≤10.

[0116] As a preferred embodiment, each l ab ra,b and l ab ra is a sub-tag under the same parent tag, so pro ra When used as a prompt word for a trained large language model, it is helpful for the trained large language model to learn the thinking that is more relevant to the text to be inferred, and further improve the accuracy of the output results of the trained large language model. For example, num=4, l ab ra For marriage and family disputes, l ab ra,1 For contract disputes, l ab ra,2 Disputes over compensation for property damage, l ab ra,3 For maintenance disputes, l ab ra,4 For marriage and family disputes, l ab ra , l ab ra,1 , l ab ra,2 , l ab ra,3 and l ab ra,4 They are all secondary tags. l ab ra , l ab ra,1 , l ab ra,2 ,l ab ra,3 and l ab ra,4 The parent tags (i.e. first-level tags) of are all disputes. l ab ra , l ab ra,1 , l ab ra,2 , l ab ra,3 and l ab ra,4 All are sub-tags of the dispute tag.

[0117] As a specific implementation, tar ra,1 For boyfriends and girlfriends who have disputes over emotional issues, l ab ra,1 In pursuit of unsuccessful entanglement, sig ra,1 For error, sig ra,b The corresponding reasoning text is: The label belongs to the dispute problem caused by the continuous entanglement when one of the men and women pursues the other party unsuccessfully, but the text mentions boyfriend and girlfriend, which conflicts with the unsuccessful pursuit. Therefore, the label is wrong.

[0118] L300, pro ra Input the trained large language model to be judged by the trained large language model l ab ra Is it cle ra .

[0119] Those skilled in the art are aware that any large language model in the prior art falls within the protection scope of the present invention.

[0120] In this embodiment, ra It is a prompt word based on the thought chain. In this embodiment, pro ra As the input of the trained large language model, it is helpful for the trained large language model to learn the thinking of judging whether the annotated label is the label of the text, improve the accuracy of the output result of the trained large language model, and then improve the accuracy of cleaning.

[0121] L400, based on the trained large language model l ab ra Is it cle ra The result of the label judgment determines whether to cle ra Wash it off.

[0122] In this embodiment, for each text to be cleaned, the corresponding prompt word is first obtained, and the prompt word includes num thought chain example texts and the text to be inferred. The num thought chain example texts are used to enable the trained large language model to better learn the thinking of judging whether a label is a label of the text, so that the trained large language model can give a more accurate judgment result; the text to be inferred includes the text to be judged itself and the label corresponding to the text to be judged; thus, by inputting the prompt word corresponding to the text to be judged into the trained large language model, it is possible to automatically judge whether the label of the text is correct, thereby solving the problem of manual judgment of whether the label of the text is correct in the prior art, which occupies a lot of manual resources.

[0123] As a preferred specific implementation, L400 includes: if the trained large language model is l ab ra Is it cle ra If the judgment result of the tag is yes, enter L410.

[0124] L410, obtain the accuracy rate tpr of the trained large language model for the judgment result of yes.

[0125] In this embodiment, if the trained large language model l ab ra Is it cle ra The result of the label is yes, indicating that the trained large language model judges l ab ra for cle ra .

[0126] Optionally, the accuracy of the trained large language model for judging the result as yes is obtained based on the test text. When the accuracy tpr of the trained large language model for judging the result as yes is 90% and the trained large language model l ab ra Is it cle ra When the result of the label is yes, it means l ab ra for cle ra The probability of the label is 90%; when the trained large language model judges that the accuracy rate tpr is 78% and the trained large language model is l ab ra Is it cle ra When the result of the label is yes, it means l ab ra for cle ra The probability of the label is 78%.

[0127] L420, if tpr ≥ tf, then it is determined not to clean cle ra ; tf is a preset accuracy threshold.

[0128] In this embodiment, tf is an empirical value. Optionally, tf is 90% or 95%.

[0129] Based on L410 - L420, in this embodiment, when the accuracy of the trained large language model for the judgment result of "yes" is relatively high, the judgment result of the trained large language model is directly used as the standard. Thus, the text corresponding to the judgment result of "yes" output by the trained large language model in cle can be retained, realizing the rapid cleaning of these texts.

[0130] In this embodiment, L420 further includes: if tpr < tf, then enter L421.

[0131] L421, obtain the qua - time judgment results of the trained large language model on l ab ra whether it is the label of cle ra , where qua is a preset number of repeated inferences, qua is an odd number and qua ≥ 3.

[0132] L422, if there are more than or equal to (qua + 1) / 2 times of judgment results of "no" among the qua - time judgment results, then it is determined to clean cle ra ; otherwise, it is determined not to clean cle ra .

[0133] Based on L421 - L422, in this embodiment, when the accuracy of the trained large language model for the judgment result of "yes" is relatively low, the judgment result of the trained large language model is not directly used as the standard. Instead, the trained large language model is used to make multiple judgments on the prompt words corresponding to the text, and the judgment result with the most occurrences among the multiple judgments is used as the final judgment result. Thus, this embodiment can reduce the judgment deviation triggered by factors such as hallucinations in the trained large language model, and further improve the accuracy of cleaning.

[0134] As a preferred specific implementation manner, L400 further includes: if the judgment result of the trained large language model on l ab ra whether it is the label of cle ra is "no", then enter L430.

[0135] L430, obtain the false positive rate fpr of the trained large language model for the judgment result of "no".

[0136] In this embodiment, if the trained large language model on l ab raWhether it is cle ra The judgment result of the label is no, indicating that the trained large language model judges l ab ra is not cle ra of the label.

[0137] Optionally, the accuracy rate of the trained large language model for the judgment result of no is obtained based on the test text. When the false positive rate fpr of the trained large language model for the judgment result of no is 68% and the trained large language model for l ab ra Whether it is cle ra The judgment result of the label is no, indicating l ab ra is not cle ra the probability of the label is 68%; when the false positive rate fpr of the trained large language model for the judgment result of yes is 92% and the trained large language model for l ab ra Whether it is cle ra The judgment result of the label is no, indicating l ab ra is not cle ra the probability of the label is 92%.

[0138] L440, if fpr < tf, then enter L450; otherwise, clean up cle ra Clean it up.

[0139] In this embodiment, tf is an empirical value. Optionally, tf is 90% or 95%.

[0140] L450, obtain the qua - th judgment result of the trained large language model for l ab ra Whether it is cle ra of the label, where qua is the preset number of repeated inferences, qua is an odd number and qua ≥ 3.

[0141] In this embodiment, qua is an empirical value. Optionally, qua is 3 or 5.

[0142] L460, if there are more than or equal to (qua + 1) / 2 times of judgment results of no in the qua - th judgment results, then judge to clean up cle ra ; otherwise, judge not to clean up cle ra Clean it up.

[0143] In this embodiment, if there are greater than or equal to (qua+1) / 2 negative judgment results among qua judgment results, it means that most of the judgment results among the qua judgment results are negative; if there are less than (qua+1) / 2 negative judgment results among the qua judgment results, it means that most of the judgment results among the qua judgment results are positive.

[0144] Based on L430-L460, when the accuracy of the trained large language model in judging the result of "no" is low, this embodiment does not directly take the judgment result of the trained large language model as the standard, but reuses the trained large language model to make multiple judgments on the prompt words corresponding to the text, and takes the judgment result with the largest number of occurrences in the multiple judgments as the final judgment result. Therefore, this embodiment can also recall the text from the text whose corresponding preliminary judgment result is "no", reduce the judgment deviation of the trained large language model triggered by the illusion light factor, and improve the accuracy of cleaning.

[0145] Embodiment seven:

[0146] If the prompt words used to input into the trained large language model in the sixth embodiment are constructed manually, there is still a problem of occupying a lot of human resources. In order to further reduce the occupation of human resources, this embodiment, based on the sixth embodiment, also includes automatically obtaining pro ra process; specific, pro ra The acquisition process includes:

[0147] L210, will cle ra Input to the first trained model to obtain cle ra First-level label l ab 1 ra ; The trained first model is used to obtain the primary label of the input text, and the accuracy of the trained first model in obtaining the primary label of the input text is greater than or equal to tf, where tf is a preset accuracy threshold.

[0148] In this embodiment, the accuracy of the first-level label of the text inferred by the trained first model is relatively high. ra The output of the first model trained with ra First-level label l ab 1 ra .

[0149] In this embodiment, the trained first model is used to obtain the first-level label of the input text. The first-level label is a rough label. The first-level label also includes several second-level labels. The second-level label is a refined label. In this embodiment, the accuracy of the second-level label of the text inferred by the trained first model is low (i.e., less than tf). Therefore, this embodiment only uses the trained first model to obtain the first-level label of the text, and does not use the trained first model to obtain the second-level label of the text.

[0150] In this embodiment, the first model is a neural network model, and a supervised training method is used when training the first model. Optionally, the process of training the first model includes: obtaining a training text set, the training text set includes a plurality of training texts; obtaining a training label set, the training label set includes a primary label corresponding to each training text; training the first model according to the training text set and the training label set to obtain a trained first model.

[0151] L220, according to l ab 1 ra Match in the preset thinking chain sample text library bas to obtain l ab 1 ra Matching num thought chain example text mod ra ; bas=(bas1,bas2,…,bas ce ,…,bas he ),bas ce The sub-library of example texts of the thinking chain corresponding to the ceth first-level label corresponding to cle, bas ce =(bas ce,1 ,bas ce,2 ,…,bas ce,me ,…,bas ce,te ),bas ce,me is the example text of the thinking chain corresponding to the meth second-level label included in the ceth first-level label corresponding to cle, the value range of me is 1 to te, te is the number of second-level labels included in the ceth first-level label corresponding to cle, the value range of ce is 1 to he, he is the number of first-level labels corresponding to cle; num is the value of cle ra The number of corresponding thought chain example texts.

[0152] In this embodiment, bas ce,me Includes several thought chain sample texts, bas ce,me The corresponding labels in each thought chain example text are the meth second-level labels included in the ceth first-level labels of cle, but bas ce,meThe judgment identifiers corresponding to the included different thinking chain example texts may be the same or different, bas ce,me The judgment identifier corresponding to the included thinking chain example text is used to indicate whether the label corresponding to the thinking chain example text is the label of the target text corresponding to the thinking chain example text. For example, if bas ce,me the judgment identifier corresponding to the included thinking chain example text is correct or yes, it indicates that the label corresponding to the thinking chain example text is the label of the target text corresponding to the thinking chain example text; if bas ce,me the judgment identifier corresponding to the included thinking chain example text is wrong or no, it indicates that the label corresponding to the thinking chain example text is not the label of the target text corresponding to the thinking chain example text.

[0153] Optionally, L220 includes:

[0154] L221, if bas ce and l ab 1 ra match, then enter L222.

[0155] In this embodiment, if bas ce the corresponding first-level label and l ab 1 ra are the same, then judge bas ce and l ab 1 ra match; otherwise, judge bas ce and l ab 1 ra do not match.

[0156] L222, if te = num, then obtain 1 thinking chain example text from each bas ce,me to construct mod ra ; if te > num, then enter L223; if te < num, then enter L225.

[0157] L223, obtain the priority of each bas ce,me .

[0158] In this embodiment, the priority of each bas ce,me is known. Optionally, the more the number of thinking chain example texts included in bas ce,me , the higher the priority of bas ce,me .

[0159] L224, obtain 1 thinking chain example text from each target sub-library to construct modra , the target sub-library is bas ce The top num thought chain example text sub-libraries with the highest priority.

[0160] L225, from every bas ce,me Get 1 thought chain example text to build the initial text.

[0161] L226, obtain the text deviation quantity Δq, Δq=num-te;

[0162] L227, from bas ce The remaining thought chain example texts are obtained from Δq thought chain example texts and appended to the initial text, and the appended initial text is determined as mod ra bas ce The remaining thought chain example text is bas ce Example texts of thought chains other than the initial text.

[0163] Based on L221-L227, this embodiment is from l ab 1 ra Matching bas ce num thought chain example texts were obtained from the code and mod ra Automatic construction, eliminating the need to manually build mods ra Moreover, this embodiment takes as much as possible from l ab 1 ra Matching bas ce Get the thought chain sample text of different secondary tags in order to make the mod ra Includes example text of thought chains corresponding to various secondary tags, which is helpful for including mod later ra Prompt words included ra When input into the trained large language model, the large language model can more comprehensively learn the thinking of judging whether the secondary label is the secondary label of the text, so that the trained large language model can give a more accurate judgment result and improve the accuracy of cleaning.

[0164] L230, according to mod ra cle ra andcle ra Secondary tags l ab 2 ra Build cle ra Corresponding prompt word pro ra .

[0165] Based on L210-L230, this embodiment can automatically build prora , which eliminates the process of manually constructing prompt words, improves the efficiency of constructing prompt words, and reduces the manual resources occupied in the text cleaning process.

[0166] As a preferred specific implementation, the process of obtaining num includes:

[0167] L201, get the preset thinking chain sample text library bas l ab 1 ra Matching bas ce Number of thought chain example texts included pn ra .

[0168] L202, get the number of initial thought chain example texts cn ra ,cn ra =f(min(num max ,max(num min ,num avg ×pn ra / pn avg ))), num max The maximum number of preset thought chain sample texts, num max =tok max / (min(tok avg,1 ,tok avg,2 ,…,tok avg,ce ,…,tok avg,he )), tok max is the total number of characters in the preset thought chain sample text, tok avg,ce For bas ce The average number of characters in the example text of the thought chain included; num min The minimum number of preset thought chain sample texts, num min =tok max / (max(tok avg,1 ,tok avg,2 ,…,tok avg,ce ,…,tok avg,he )); num avg is the average number of preset thought chain sample texts, num avg =tok max / (avg(tok avg,1 ,tok avg,2 ,…,tok avg,ce ,…,tok avg,he )); pn avgis the preset threshold value of the number of thought chain example texts; max() is the maximum value, min() is the minimum value, avg() is the average value, and f() is the integer.

[0169] In this embodiment, the average number of example thinking chain texts included in each example thinking chain text sub-library in bas is determined as the preset number threshold of example thinking chain texts.

[0170] L203, if cn ra Greater than or equal to l ab 1 ra The number of secondary tags included, then cn ra Determined as num; if cn ra Less than l ab 1 ra The number of secondary tags included will be l ab 1 ra The number of secondary tags included is identified as num.

[0171] In this embodiment, if cn ra Less than l ab 1 ra The number of secondary tags included is prioritized from l ab 1 ra Matching bas ce Select or reconstruct a thought chain example text composition mod containing a small number of words ra To enable the mod ra The number of characters included does not exceed tok max That's it.

[0172] Based on L201-L203, this embodiment can achieve the purpose of setting different thinking chain example texts according to the text to be cleaned. Specifically, considering the character number limit corresponding to the prompt word, the maximum number, minimum number and average number of thinking chain example texts are used as reference factors. It can also set a larger number of thinking chain example texts when the thinking chain example texts corresponding to the first-level label of the text to be cleaned include a large number of thinking chain example texts, so as to improve the diversity of the set thinking chain example texts; considering the requirements of comprehensive learning and reasoning of the trained large language model, the number of set thinking chain example texts meets the requirement of being greater than or equal to the number of second-level labels included in the first-level label of the text to be cleaned, so that the trained large language model can learn comprehensive label judgment thinking, thereby improving the accuracy of judgment.

[0173] Embodiment eight:

[0174] In the first embodiment, the label of the target sample text may be wrong. In order to obtain a correct and more comprehensive label for each text in the target sample text, the text processing method based on text repetition in this embodiment further includes:

[0175] F100, get the text tex of the label to be obtained, tex=(tex1,tex2,…,tex ac ,…,tex ae ),tex ac is the text of the acth label to be obtained, the value range of ac is 1 to ae, ae is the number of texts of the label to be obtained; each tex ac Corresponding to several original labels.

[0176] In this embodiment, the target sample text is determined as tex, and ae is the number of the target sample text.

[0177] In this embodiment, each tex ac The number of corresponding original labels is greater than or equal to 1, and the original label may be tex ac The corresponding correct label may also be tex ac The corresponding wrong label, and the proportion of correct labels in the original labels of all texts corresponding to tex is greater than the proportion of wrong labels in the original labels; as a specific implementation method, each tex ac The corresponding original labels are manually annotated labels.

[0178] F200, according to the text of the first label to be obtained in tex 1 Get the trained first label inference model mode1, mode1=(mode 1,1 ,mode 1,2 ,…,mode 1,av ,…,mode 1,ax ), mode 1,av According to tex 1 Get the avth trained first label inference model, where av ranges from 1 to ax, and ax is the value of tex 1 The number of trained first-label inference models obtained; ax ≥ 2.

[0179] In this embodiment, the text to be obtained in the tex tag is divided into two parts, one of which is tex 1 , the other part is tex 2 ; optional, tex 1 The number of texts to be labeled in is 1 / 2 of the number of texts to be labeled in tex. 2The number of texts with tags to be obtained in [text] is also 1 / 2 of the number of texts with tags to be obtained in tex.

[0180] In this embodiment, each mode 1,av of the training samples comes from tex 1 , and the training samples corresponding to different modes 1,av are different; as a specific implementation, F200 includes:

[0181] F210, which divides tex 1 to obtain the divided text subsets zc1, zc1 = (zc 1,1 , zc 1,2 , …, zc 1,av , …, zc 1,ax ), where zc 1,av is the ax-th text subset obtained by dividing tex 1 .

[0182] Optionally, tex 1 is evenly divided into ax text subsets.

[0183] F220, which initializes the model variable ha to 1.

[0184] F230, which selects ax - 1 text subsets other than zc 1,ha from zc1 as the ha-th training sample.

[0185] F240, which uses the ha-th training sample and the corresponding original tags to train the ha-th first label inference model, obtaining the ha-th trained first label inference model mode 1,ha .

[0186] F250, if ha < ax, then ha = ha + 1, and repeat F230 - F240 until ha = ax.

[0187] Based on F210 - F250, when ha is 1, executing F230 - F240 can obtain the 1st trained first label inference model mode 1,1 ; when ha is 2, executing F230 - F240 can obtain the 2nd trained first label inference model mode 1,2 , and so on. When ha is ax, executing F230 - F240 can obtain the ax-th trained first label inference model mode 1,ax ; thus, ax trained first label inference models can be obtained after executing F210 - F250.

[0188] F300, which obtains the text tex of the second tags to be obtained in tex according to mode12 The inference label qa of the text to be labeled in 2 ,qa 2 =(qa 2 1,qa 2 2,…,qa 2 er ,…,qa 2 zn ), qa 2 er For tex 2 The inference label of the erth text to be labeled, er ranges from 1 to zn, zn is tex 2 The number of texts to be labeled, qa 2 er =(qa 2 er,1 ,qa 2 er,2 ,…,qa 2 er,av ,…,qa 2 er,ax ), qa 2 er,av According to mode 1,av Get tex 2 The inference label of the text to be labeled in erth; tex 2 Remove tex from tex 1 The text of the tag to be obtained outside the qa 2 er,av Includes several inference tags.

[0189] In this embodiment, tex 2 The text of the erth label to be obtained in is used as the input of the first trained first label inference model, and mode 1,1 Inferred tex 2 The inference label qa of the erth text to be labeled 2 er,1 ; 2 The text of the erth label to be obtained is used as the input of the second trained first label inference model, and mode 1,2 Inferred tex 2 The inference label qa of the erth text to be labeled 2 er,2 ; And so on, tex 2 The text of the erth label to be obtained is used as the input of the axth trained first label inference model, and mode can be obtained. 1,ax Inferred tex 2The inference label qa of the erth text to be labeled 2 er,ax .

[0190] F400, traverse qa 2 , for qa 2 er Filter and get the filtered inference label nqa 2 er ;nqa 2 er =(nqa 2 er,1 ,nqa 2 er,2 ,…,nqa 2 er,bv ,…,nqa 2 er,bx ), nqa 2 er,bv For qa 2 er The bvth inference label remaining after screening, the value range of bv is 1 to bx, bx is the value of qa 2 er The number of inference labels remaining after filtering.

[0191] In this embodiment, qa 2 er The purpose of screening is to 2 er The wrong labels are filtered out and the correct labels are retained.

[0192] As a first preferred specific implementation, F400 includes:

[0193] F410, get qa 2 Each inference label that appears in qa 2 The number of times it appears in .

[0194] F420, if qa 2 A certain inference label appears in qa 2 If the number of occurrences in is greater than or equal to the preset threshold, qa 2 The inference label that appears in is determined as the inference label retained after screening; otherwise, qa 2 The inference label that appears in is determined to be the inference label that is filtered out after filtering.

[0195] In this embodiment, qa 2 The inference tags whose occurrence times in the list are greater than or equal to the preset threshold are judged as tex 2 The correct label of the text to be labeled in er is qa2 The inference tags whose occurrence times are less than the preset threshold are judged as tex 2 The wrong label of the text of the erth label to be obtained in . Optionally, the preset number of times threshold is ax / 2.

[0196] Based on F410-F420, this embodiment realizes the 2 er Screening, since this embodiment will qa 2 The inference labels that appear more often in qa are judged as correct labels, while 2 The inference labels that appear more often in the above are consistent inference results of a large number of trained first label inference models. Therefore, the nqa obtained based on F410-F420 in this embodiment is 2 er The label is tex 2 The probability of the correct label of the erth text to be labeled is higher, which improves the accuracy of the label of the text finally obtained.

[0197] As a second preferred specific implementation, F400 includes:

[0198] F401, get qa 2 er The inference tag wqa appears in 2 er , wqa 2 er =(wqa 2 er,1 ,wqa 2 er,2 ,…,wqa 2 er,cv ,…,wqa 2 er,cx ), wqa 2 er,cv For QA 2 er The cvth inference label appears in , the value range of cv is 1 to cx, and cx is qa 2 er The number of inference labels that appear in .

[0199] In this embodiment, different wqa 2 er,cv corresponds to different inference labels; since different trained first-label inference models may infer the same or different inference labels, cx is less than or equal to qa 2 er Each qa in 2 er,av The sum of the number of inference labels included.

[0200] F402, traverse wqa 2 er , using the trained large language model to judge wqa 2 er,cv Is it tex 2 The label of the text of the erth label to be obtained.

[0201] Those skilled in the art know that any large language model in the prior art falls within the protection scope of the present invention.

[0202] Preferably, the cue words based on the thought chain make the trained large language model judge wqa 2 er,cv Is it tex 2 The label of the erth text to be labeled is included in the prompt word, and several example texts of the thought chain related to the content to be judged are included. Therefore, the trained large language model can be used to better learn the thinking of judging whether the label is correct and improve the accuracy of the judgment result.

[0203] F403, if yes, then wqa 2 er,cv Append to the preset erth tag sequence; otherwise, wqa is not appended. 2 er,cv Append to the preset erth label sequence; the preset erth label sequence is initialized to an empty sequence.

[0204] F404, determine the preset erth label sequence as nqa 2 er .

[0205] Based on F401-F404, this embodiment implements qa based on the trained large language model 2 er The screening of NQA is improved through the trained large language model. 2 er The accuracy of the inference labels in .

[0206] F500, nqa 2 er Determine as tex 2 The label of the erth text to be judged.

[0207] In this embodiment, each text in the text tex to be labeled corresponds to an original label, and a part of the text tex in tex is used. 1 A plurality of trained first label inference models can be obtained from the corresponding original labels, and another part of the text tex in tex can be obtained according to the plurality of trained first label inference models.2 inference labels; since the present embodiment uses multiple trained first label inference models to obtain inference labels for texts, different trained first label inference models may obtain different inference labels for texts, and therefore the inference labels for texts obtained by the present embodiment are relatively comprehensive; moreover, the present embodiment also screens the inference labels obtained according to multiple trained first label inference models to select the correct labels from the relatively comprehensive inference labels; thus, the present embodiment achieves the purpose of obtaining correct and relatively comprehensive labels for texts.

[0208] As an optional specific implementation, ax is an empirical value, for example, 3≤ax≤5.

[0209] As a preferred specific implementation, the method for obtaining ax includes:

[0210] F201, get tex 1 The number of texts to be included in the tags to be obtained 1 .

[0211] F202, get the initial number of divisions hua0, hua0=floor(shu 1 / shu0), shu0 is the preset text quantity threshold, and floor() is rounding down.

[0212] In this embodiment, shu0 is an empirical value, which corresponds to the minimum number of training samples required to train a first label inference model into a model with an accuracy greater than or equal to a preset accuracy threshold.

[0213] F203, obtain the first division quantity hua1, hua1=min(max(hua0,hua min ),hua max ), min() is the minimum value, max() is the maximum value, min is the preset minimum number of divisions, max The preset maximum number of divisions.

[0214] In this embodiment, hua min and hua max All are experience values.

[0215] F204, from tex 1 The text of the to-be-acquired label with a ratio of (hua1-1) / hua1 and the corresponding original label are used to train the test label inference model to obtain a trained test label inference model.

[0216] F205, if the accuracy of the trained test label inference model is greater than or equal to a preset accuracy threshold, hua1 is determined as ax.

[0217] In this embodiment, if the accuracy of the trained test label inference model is less than the preset accuracy threshold, 1 The text of the to-be-acquired label with the proportion of hua1 / (hua1+1) and the corresponding original label are used to train the test label inference model to obtain a trained test label inference model. If the accuracy of the trained test label inference model is greater than or equal to the preset accuracy threshold, hua1+1 is determined as ax; otherwise, tex 1 The text of the to-be-acquired label with a proportion of (hua1+1) / (hua1+2) and the corresponding original label are used to train the test label inference model to obtain a trained test label inference model. If the accuracy of the trained test label inference model is greater than or equal to the preset accuracy threshold, hua1+2 is determined as ax; otherwise, tex 1 The text of the to-be-obtained label with a proportion of (hua1+2) / (hua1+3) and the corresponding original label are used to train the test label inference model to obtain a trained test label inference model; and so on, until the accuracy of the trained test label inference model is greater than or equal to the preset accuracy threshold, the corresponding number of divisions is determined as ax.

[0218] Based on the ax obtained from F201-F205, tex 1 Divide, you can make tex 1 The number of texts corresponding to the proportion of (hua1-1) / hua1 is relatively large, which improves the subsequent tex-based 1 The accuracy of the first label inference model trained on text with a ratio of (hua1-1) / hua1 is conducive to improving the accuracy of the final text label.

[0219] As an optional specific implementation, after F500, the method for obtaining the text label further includes:

[0220] F600, traverse tex 2 The original label corresponding to the text of the erth label to be obtained in tex 2 The original label corresponding to the text of the erth label to be obtained in nqa 2 er If it exists in 2 The original label corresponding to the text of the erth label to be obtained is correct; otherwise, judge tex 2 The original label corresponding to the text of the erth label to be obtained is wrong.

[0221] Therefore, this embodiment is based on tex 2 The label nqa corresponding to the text of the erth label to be obtained 2 er Implemented the tex 2 Determine the original label corresponding to the text of the erth label to be obtained.

[0222] In this embodiment, tex is obtained 1 The method of obtaining the label of each text to be labeled is the same as the above method of obtaining the text 2 The method of obtaining the label of each text to be labeled is similar, except that: 1 In the process of labeling each text to be labeled, according to the tex 2 Get the trained second label inference model mode2, which includes several 2 Obtain the trained second label inference model; obtain tex according to mode2 1 The inference label qa of the text to be labeled in 1 ,qa 1 Includes all tex obtained by the trained second label inference model 1 The inference label of each text to be labeled; traverse qa 1 , for qa 1 er Filter and get the filtered inference label nqa 1 er ; will nqa 1 er Determine as tex 1 The label of the erth text to be judged in . Get tex 1 The specific process of obtaining the label of each text to be labeled is the same as the above-mentioned process of obtaining the text label of the tex 2 The specific process of labeling each text to be labeled is similar and will not be repeated here.

[0223] Embodiment nine:

[0224] In order to obtain a named entity recognition model capable of performing entity annotation of multiple preset types on the target sample text, the text processing method based on text repetition in this embodiment further includes the following steps:

[0225] D100, obtain the training text set data of the trained named entity recognition model, data={data1,data2,…,data se ,…,data st}, data seis the se-th training text set of the trained named entity recognition model, data se The invention comprises a plurality of training texts, wherein the value range of se is 1 to st, and st is the number of training text sets of the trained named entity recognition model; the trained named entity recognition model is a model obtained by training using data, and the trained named entity recognition model is used to recognize entities of a preset type ty in the input text and mark the corresponding labels; ty=(ty1, ty2, ..., ty mt ,…,ty ht ),ty mt It is the mtth preset type, the value range of mt is 1 to ht, and ht is the number of preset types.

[0226] In this embodiment, the trained named entity recognition model is obtained by training using the st training text sets included in the data. The purpose of training the named entity recognition model using the st training text sets included in the data is to enable the trained named entity recognition model to have the function of recognizing ht preset types of entities.

[0227] In this embodiment, when the named entity recognition model is trained using the st training text sets included in data, at least one entity of a preset type is annotated when each training text set is annotated, and entities other than ht entities of the preset types are not annotated, and each entity of the ht preset types has been annotated by at least one training text set. Thus, the trained named entity recognition model has and only has the function of recognizing entities of ht preset types.

[0228] D200, traverse data, if using data se When training the named entity recognition model, se The entity of the mtth preset type is labeled, then the data se The proportion of positive samples of the mt-th preset type of entity in data is determined as data se The mtth weight qz se,mt ; Otherwise, set qz se,mt is 0.

[0229] In this embodiment, when the named entity recognition model is trained using the st training text sets included in data, not all entities of ht preset types are annotated when each training text set is annotated. For example, st=2, ht=4, when the named entity recognition model is trained using the first training text set and the second training text set, only entities of the first preset type and the second preset type are annotated when the first training text set is annotated, and only entities of the second preset type, the third preset type, and the fourth preset type are annotated when the second training text set is annotated.

[0230] In this embodiment, data se The proportion of positive samples of the mt-th preset type of entity in data is se The number of entities marked as entities of the mtth preset type in data se The ratio of the number of entities in se When training the named entity recognition model, se The entity of the mtth preset type is labeled, then the data se The value of the positive sample ratio of the mt-th preset type of entity in is assigned to qz se,mt ; If you use data se When training the named entity recognition model, there is no data se To label the entity of the mtth preset type, then qz se,mt Set to 0.

[0231] D300, obtain the positive sample proportion cb of ht preset types of entities in the test text set, cb=(cb1,cb2,…,cb mt ,…,cb ht ), cb mt is the proportion of positive samples of the mt-th preset type of entity in the test text set.

[0232] In this embodiment, the positive sample ratio of the mt-th preset type of entity in the test text set is the ratio of the number of entities in the test text set that are marked as entities of the mt-th preset type to the number of entities in the test text set.

[0233] In this embodiment, the proportion of positive samples of the mt-th preset type of entity in the test text set can represent the proportion of positive samples of the mt-th preset type of entity in the text set to be subsequently recognized by the trained named entity recognition (for example, the target sampling text). Optionally, the test text set is a text extracted from the text set to be subsequently recognized by the trained named entity recognition, and the ht preset types of entities in the test text set are annotated by manual annotation methods, and the positive sample proportions of the ht preset types of entities are obtained.

[0234] D400, obtain the positive sample ratio zb of ht preset types of entities in the newly added text set, zb=(zb1,zb2,…,zb mt ,…,zb ht ), zb mt It is the proportion of positive samples of the mt-th preset type of entity in the newly added text set.

[0235] In this embodiment, the newly added text set is a text obtained in order to enable the trained text set to have the function of identifying entities of the newly added preset type, and the positive sample ratio of the mt-th preset type entity in the newly added text set is the ratio of the number of entities marked as entities of the mt-th preset type in the newly added text set to the number of entities in the newly added text set. As an optional specific implementation, D400 includes:

[0236] D410, obtaining a preset proportion of text from the newly added text set to form a first extracted text set cq.

[0237] Optionally, a random extraction method is used to obtain a preset proportion of texts from the newly added text set to form a first extracted text set cq, where the preset proportion is an empirical value.

[0238] D420, using the trained named entity recognition model to perform ht preset types of entity recognition on each text in cq, and obtain labeling results of ht preset types of entities corresponding to each text in cq.

[0239] In this embodiment, each text in cq is used as the input of the trained named entity recognition model to obtain the corresponding labeling results of ht preset types of entities.

[0240] D430, according to the annotation result of each text in cq corresponding to the entity of the mt-th preset type, it is determined as zb mt .

[0241] In this embodiment, zb mt =∑ fs fe=1 en fe,mt / (∑ fs fe=1 q fe ),zh fe,mt is the number of entities annotated as entities of the mtth preset type in the feth text in cq, qh fe It is the number of entities included in the fe-th text in cq. The value range of fe is 1 to fs, and fs is the number of texts included in cq.

[0242] D500, according to qz se,mt、cb and zb determine the optimal amount of text to be obtained from the newly added text set for training the trained named entity recognition model.

[0243] The trained named entity recognition model in this embodiment is obtained by training based on the training text set data, which has the function of identifying ht preset types of entities; the newly added text set is the text obtained in this embodiment in order to enable the trained text set to have the function of identifying the newly added preset type of entities. This embodiment determines the optimal number of texts to be obtained from the newly added text set for training the trained named entity recognition model based on the positive sample ratio of ht preset types of entities in the test text set, the positive sample ratio zb of ht preset types of entities in the newly added text set, and the weight of each training text set corresponding to each preset type in data, wherein the newly added text set has the optimal number of texts to be obtained from the newly added text set for training the trained named entity recognition model. The proportion of positive samples of ht preset types of entities determines the extent of the impact of using the newly added text set to train the trained named entity recognition model on the recognition accuracy of the original ht preset types of entities. The proportion of positive samples of ht preset types of entities in the test text set and the weight of each training text set corresponding to each preset type in the data reflect the recognition accuracy of the trained named entity recognition model for the original ht preset types of entities. Therefore, this embodiment can achieve the purpose of taking into account the recognition accuracy of the re-trained named entity recognition model for the original ht preset types of entities and the recognition accuracy of the newly added preset types of entities.

[0244] As a preferred embodiment, D500 includes:

[0245] D510, get the loss dif of the newly added text, dif=(∑ ht mt=1 |cb mt -((∑ st se=1 qz se,mt ×mu se )+zb mt ×zq) / (su se + zq)|)+ke / zq,mu se For data se The amount of text included, su se is the corresponding qz in data se,mt The sum of the number of texts included in the training text set that is not 0; ke is a preset adjustment coefficient, ke>0, and zq is the number of texts to be obtained from the newly added text set for training the trained named entity recognition model.

[0246] In this embodiment, ke is an empirical value, optional, 0 <ke<1。

[0247] D520, determining the value of zq when dif takes the minimum value as the optimal number of texts obtained from the newly added text set for training the trained named entity recognition model.

[0248] Based on the above preferred specific implementation manner, the loss of newly added texts corresponding to the optimal number of texts obtained is minimal. Obtaining texts from the newly added text set according to the optimal number of texts as training samples for the trained named entity recognition model can make the proportion of positive samples of entities of each preset type used for the named entity recognition model closer to the proportion of positive samples of entities of the corresponding preset type in the test sample, so that the named entity recognition model trained with the texts of the optimal number of texts in the newly added text set has a higher recognition accuracy rate for entities of the original preset type; obtaining texts from the newly added text set according to the optimal number of texts as training samples for the trained named entity recognition model can also make the number of texts obtained from the newly added text set larger, so that the named entity recognition model trained with the texts of the optimal number of texts in the newly added text set has a higher recognition accuracy rate for entities of the newly added preset type; therefore, the optimal number of texts obtained based on this preferred specific implementation manner can achieve the purpose of taking into account the recognition accuracy rate of the named entity recognition model for both entities of the original preset type and entities of the newly added preset type.

[0249] As an optional specific implementation, after D500, when the computer readable instructions are executed by the processor, the following steps are further implemented:

[0250] D600, fk texts are obtained from the newly added text set to form a text set to be annotated, where fk is the optimal number of texts.

[0251] D700, obtaining the annotation result res corresponding to the text set to be annotated, res including the annotation result ty corresponding to the text set to be annotated and the annotation result of the newly added preset type corresponding to the text set to be annotated; the newly added preset type and any ty mt Are different.

[0252] In this embodiment, the annotation result of ty corresponding to the to-be-annotated text set includes each ty corresponding to the to-be-annotated text set. mt The labeling results.

[0253] D800: train the trained named entity recognition model using the to-be-annotated text set and res.

[0254] Based on D600-D800, the obtained trained named entity recognition model not only has the function of identifying entities of newly added preset types, but also enables the trained named entity recognition model to reduce or improve the recognition accuracy of entities of the original preset types, thereby improving the recognition function of the trained named entity recognition model as a whole.

[0255] Although some specific embodiments of the present invention have been described in detail by way of example, it will be appreciated by those skilled in the art that the above examples are for illustration only and are not intended to limit the scope of the present invention. It will also be appreciated by those skilled in the art that various modifications may be made to the embodiments without departing from the scope and spirit of the present invention. The scope of the present invention is defined by the appended claims.

Claims

1. A text processing method based on text repetition, characterized in that: The following steps are involved: P100, get the i-th type of text to be sampled a i , a i The number of texts to be sampled is q i , a i Included q i The labels of the texts to be sampled are the same, but different i The labels of the texts to be sampled are different, and the value range of i is 1 to n, where n is the number of categories of the texts to be sampled; P200, get the number of samples s of the i-th category i ; P300, if q i >s i , then from a i Randomly obtain s i The texts to be sampled constitute the discriminant sample text B of the i-th category. i ; P400, for B i Perform grouping and obtain the grouping result R i , R i Includes v groups, where v is the preset number of groups; P500, according to R i The text similarity between the included groups is obtained by B i The repetition between Chinese texts is i ; P600, if sim i ≤sim0, then enter P700; sim0 is the preset similarity threshold; P700, B i Determine the target sample text of the i-th category; After P200 and before P300, the method further includes: P201, obtain the text set A to be sampled, A includes n types of texts to be sampled, and the i-th type of text to be sampled included in A is a i ; P202, obtaining the sum w of the number of texts to be sampled included in the first u categories with the largest number of texts to be sampled in A, where u is a preset category number threshold; P203, if w / (∑ n i=1 q i )<η, then enter P300; η is the preset ratio threshold; P500 includes: P510, get the spliced ​​text CR i , CR i Including v concatenated texts, CR i Include the tth concatenated text CR i,t To R i,t The text obtained by concatenating the included texts; R i,t For B i The tth group is obtained by grouping; the value range of t is 1 to v; P520, Get CR i The text similarity of any two texts in SH is added to the preset similarity set SH. The obtained SH includes D text similarities. SH includes the d-th text similarity sh d CR i The text similarity between the d-th pair of texts in , where d ranges from 1 to D, and D is CR i The number of corresponding text pairs, D = v × (v-1) / 2; SH is initialized to an empty set; P530, get sim i ,sim i =sum(SH) / D, sum( ) is the sum.

2. The text processing method based on text repetition according to claim 1, characterized in that: Each R i,t The number of included texts is greater than or equal to floor(s i / v) and less than or equal to ceil(s) i / v), floor() is used to round down, and ceil() is used to round up.

3. The text processing method based on text repetition according to claim 1, characterized in that: P300 also includes: If q i ≤s i , then a i Determine the target sample text for the i-th category.

4. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the text processing method based on text repetition as described in any one of claims 1 to 3 is implemented.

5. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the text processing method based on text repetition as claimed in any one of claims 1 to 3 is implemented.

Images