Methods and apparatus for calculating label accuracy

Abstract

The present invention discloses a method and device for calculating the accuracy of a label. The method comprises: obtaining a text to be processed and a first label corresponding to the text to be processed; determining a similar text corresponding to the text to be processed and a second label corresponding to the similar text; determining a first weighted distance according to the text to be processed and the similar text; determining a second weighted distance according to the first label and the second label; determining the accuracy of the first label according to the first weighted distance and the second weighted distance, wherein the accuracy characterizes the similarity between the first label and the second label. The present invention solves the technical problem of low efficiency in determining the accuracy between text and labels in the prior art.

Description

Technical Field

[0001] The present invention relates to the field of data matching, and in particular to a method and device for calculating label accuracy. Background Art

[0002] The question bank is an indispensable resource in the Internet + education scenario, and in the process of building the question bank, there will inevitably be data with incorrect relationships between some questions and the associated chapters or knowledge points. In the actual business of educational tablets, teachers select questions based on chapters or knowledge points, but there is a problem of mismatch between questions and labels. Some questions may even be beyond the scope of the curriculum, which reduces the experience of teachers using the question bank. In addition, when students use the personalized question-solving function, if there is an error in the relationship between questions and labels, it is easy to reduce the accuracy of the questions pushed to students and reduce the effect of students' consolidation exercises on knowledge points or chapters.

[0003] Currently, the correctness of questions and chapters or knowledge points can be verified by manual inspection, but the amount of data in the question bank usually reaches millions or even tens of millions, and the relationship data between questions, chapters and knowledge points is even more. If we rely solely on manual calibration of the relationship data between questions, chapters and knowledge points, it will consume a lot of manpower costs. In addition, in the actual question bank construction process, the error rate of the relationship between questions and chapters or knowledge points labels is not high, so for each question, it takes a long time to find errors in most question labels.

[0004] To address the above-mentioned problems, no effective solution has been proposed yet. Summary of the invention

[0005] The embodiments of the present invention provide a method and device for calculating label accuracy, so as to at least solve the technical problem of low efficiency in determining the accuracy between text and label in the prior art.

[0006] According to one aspect of an embodiment of the present invention, a method for calculating label accuracy is provided, comprising: obtaining a text to be processed and a first label corresponding to the text to be processed; determining similar text corresponding to the text to be processed and a second label corresponding to the similar text; determining a first weighted distance based on the text to be processed and the similar text; determining a second weighted distance based on the first label and the second label; determining the accuracy of the first label based on the first weighted distance and the second weighted distance, wherein the accuracy characterizes the similarity between the first label and the second label.

[0007] According to another aspect of an embodiment of the present invention, a device for calculating label accuracy is also provided, including: an acquisition module for acquiring a text to be processed and a first label corresponding to the text to be processed; a first determination module for determining similar text corresponding to the text to be processed and a second label corresponding to the similar text; a second determination module for determining a first weighted distance based on the text to be processed and the similar text; a third determination module for determining a second weighted distance based on the first label and the second label; and a fourth determination module for determining the accuracy of the first label based on the first weighted distance and the second weighted distance, wherein the accuracy characterizes the similarity between the first label and the second label.

[0008] According to another aspect of an embodiment of the present invention, a non-volatile storage medium is provided, in which a computer program is stored, wherein the computer program is configured to execute the above-mentioned label accuracy calculation method when running.

[0009] According to another aspect of an embodiment of the present invention, a processor is further provided, the processor being used to run a program, wherein the program is configured to execute the above-mentioned label accuracy calculation method when running.

[0010] In an embodiment of the present invention, a method of calculating the accuracy between the text to be processed and the label by using the label of a text similar to the text to be processed is adopted. After obtaining the text to be processed and a first label corresponding to the text to be processed, similar text corresponding to the text to be processed and a second label corresponding to the similar text are determined, and a first weighted distance is determined based on the text to be processed and the similar text, and a second weighted distance is determined based on the first label and the second label. Finally, the accuracy of the first label is determined based on the first weighted distance and the second weighted distance, wherein the accuracy characterizes the similarity between the first label and the second label.

[0011] In the above process, by using the label of the similar text corresponding to the text to be processed as a pseudo label to guide the calculation of the accuracy between the text to be processed and the first label, the process does not need to manually pre-label the relationship between the text to be processed and the first label, thereby reducing the manpower cost consumed by manual calibration and improving the efficiency of calculating the accuracy between the text and the label. In addition, the present application calculates the accuracy between the text and the label in two dimensions, ensuring the reliability of the calculation accuracy.

[0012] It can be seen that the solution provided in the present application achieves the purpose of calculating the accuracy between text and labels, thereby realizing the technical effect of improving the efficiency of calculating the accuracy between text and labels, and further solving the technical problem of low efficiency in determining the accuracy between text and labels in the prior art. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] The drawings described herein are used to provide a further understanding of the present invention and constitute a part of this application. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings:

[0014] Figure 1 is a flow chart of a method for calculating label accuracy according to an embodiment of the present invention;

[0015] Figure 2 is an optional accuracy prediction flow chart of the relationship between a topic and a label according to an embodiment of the present invention;

[0016] Figure 3 4 is a schematic diagram of a device for calculating label accuracy according to an embodiment of the present invention. DETAILED DESCRIPTION

[0017] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0018] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0019] Example 1

[0020] According to an embodiment of the present invention, an embodiment of a method for calculating label accuracy is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0021] Figure 1 is a flow chart of a method for calculating label accuracy according to an embodiment of the present invention. Figure 1 As shown, the method comprises the following steps:

[0022] Step S102, obtaining the text to be processed and the first label corresponding to the text to be processed.

[0023] In step S102, the text to be processed may be a title, and the first label corresponding to the text to be processed may be the label corresponding to the title. For example, the text to be processed is "Among the following slogans, the one that best reflects the nature of the May Fourth Movement is", and the corresponding label is "May Fourth Movement".

[0024] In an optional embodiment, the user imports the marked topics and tags in batches into the terminal device, the terminal device calculates the accuracy between the marked topics and tags, and outputs topics and tags with accuracy lower than the preset accuracy for user confirmation.

[0025] Step S104, determining similar text corresponding to the text to be processed and a second label corresponding to the similar text.

[0026] Optionally, in step S104, multiple texts whose similarity with the feature vector corresponding to the text to be processed is greater than a preset similarity can be obtained from a preset database, and these multiple texts are similar texts. After obtaining the similar texts, the similar texts can be processed with the Forbidden City word vector model to obtain the second label corresponding to the similar text.

[0027] Step S106: determining a first weighted distance according to the text to be processed and the similar text.

[0028] Optionally, the accuracy of the relationship between the text to be processed and the first label can be evaluated by using the weighted distance (ie, the first weighted distance) between features corresponding to the text to be processed and the first label and features corresponding to similar text and the second label.

[0029] Step S108: determining a second weighted distance according to the first label and the second label.

[0030] Optionally, the distance may be characterized by weighting the cluster centers of the clusters of the first label and the second label corresponding to similar texts, and the accuracy of the text to be processed and the first label may be measured by the distance between the first label and the second label.

[0031] Step S110: determining the accuracy of the first tag according to the first weighted distance and the second weighted distance, wherein the accuracy represents the similarity between the first tag and the second tag.

[0032] It should be noted that step S110 uses two dimensions to calculate the accuracy between the text and the label (ie, the first weighted distance and the second weighted distance mentioned above), thereby ensuring the reliability of the calculation accuracy.

[0033] Based on the scheme defined by the above steps S102 to S110, it can be known that in an embodiment of the present invention, the label of the text similar to the text to be processed is used to calculate the accuracy between the text to be processed and the label. After obtaining the text to be processed and the first label corresponding to the text to be processed, the similar text corresponding to the text to be processed and the second label corresponding to the similar text are determined, and a first weighted distance is determined according to the text to be processed and the similar text, and a second weighted distance is determined according to the first label and the second label. Finally, the accuracy of the first label is determined according to the first weighted distance and the second weighted distance, wherein the accuracy characterizes the similarity between the first label and the second label.

[0034] It is easy to notice that in the above process, by using the label of the similar text corresponding to the text to be processed as a pseudo label to guide the calculation of the accuracy between the text to be processed and the first label, the process does not need to manually pre-label the relationship between the text to be processed and the first label, thereby reducing the manpower cost consumed by manual calibration and improving the efficiency of calculating the accuracy between the text and the label. In addition, the present application calculates the accuracy between the text and the label in two dimensions, ensuring the reliability of the calculation accuracy.

[0035] It can be seen that the solution provided in the present application achieves the purpose of calculating the accuracy between text and labels, thereby realizing the technical effect of improving the efficiency of calculating the accuracy between text and labels, and further solving the technical problem of low efficiency in determining the accuracy between text and labels in the prior art.

[0036] In an optional embodiment, after obtaining the text to be processed and the first label, similar text corresponding to the text to be processed and the second label corresponding to the similar text are determined. Specifically, first obtain the first text vector of the text to be processed and the first label vector of the first label, and obtain the preset feature vectors corresponding to multiple preset texts stored in the preset database, then determine the first similarity between each preset text and the text to be processed based on the preset feature vector and the first text vector, and obtain a first number of first target preset texts from the preset database in order of the first similarity, and determine the first preset label corresponding to the first target preset text, and finally, determine that the first target preset text is a similar text, and determine that the first preset label is the second label.

[0037] Optional, such as Figure 2 The accuracy prediction flowchart of the relationship between the title and the label is shown in Figure 2In the method, the title Q (i.e., the text to be processed) and the label T (i.e., the first label) are respectively input into the pre-trained model to extract the vector Fq (i.e., the first text vector) corresponding to the title Q and the vector Ft (i.e., the first label vector) corresponding to the label T, and then the most similar K (i.e., the first number) titles are searched from the preset database according to the first text vector Fq to obtain the first target preset text.

[0038] In an optional embodiment, after obtaining the text to be processed and the first label, it is necessary to first obtain the first text vector of the text to be processed and the first label vector of the first label. Specifically, first, remove the invalid information in the text to be processed and the first label to obtain the processed text to be processed and the processed first label, and perform feature extraction on the processed text to be processed based on a preset word vector model to obtain the first text vector, and perform feature extraction on the processed first label based on a preset word vector model to obtain the first label vector.

[0039] Optionally, first, clean the content of the title Q and the label T, retain the valid text information in the title, remove invalid content such as Html tags and pictures in the title, and use all the title texts in the preset database to train the Word2Vec model, extract the text feature vectors Fq and Ft of the title Q and label T, so as to obtain the first text vector and the first label vector.

[0040] Optional, by Figure 2 It can be seen that after obtaining the first target preset text, the weighted distance is calculated from two dimensions. Among them, in the first dimension, the first weighted distance is determined according to the text to be processed and the similar text. Specifically, firstly, the second similarity between the first target preset text and the text to be processed is calculated, and the second target preset text with a second similarity greater than the preset similarity is obtained from the first number of first target preset texts, and the second text vector corresponding to the second target preset text is determined. Then, the second preset label corresponding to the second target preset text is obtained from the preset database, and the second label vector corresponding to the second preset label is determined, and the first text vector and the first label vector are subjected to a first set-link operation to obtain a first set-link feature vector, and the second text vector and the second label vector are subjected to a second set-link operation to obtain a second set-link feature vector, and finally, the first weighted distance is determined according to the first set-link feature vector and the second set-link feature vector.

[0041] It should be noted that the above-mentioned first set-association operation includes: performing a concatenation operation on the first text vector and the first label vector in a first preset order, and the dimension of the first set-association feature vector is the sum of the dimension of the first text vector and the dimension of the first label vector. For example, the first text vector includes three elements abc, and the first label vector includes three elements cde. The first set-association operation is performed on the first text vector and the first label vector in the order of the first text vector and the first label vector, and the obtained first set-association feature vector includes six elements abccde, wherein the dimension of the first text vector is 1×3, the dimension of the first label vector is 1×3, and the dimension of the first label vector is 1×3, then the dimension of the first set-association feature vector is 1×6.

[0042] In addition, the second set-association operation includes: performing a concatenation operation on the second text vector and the second label vector according to a second preset order, and the dimension of the second set-association feature vector is the sum of the dimension of the second text vector and the dimension of the second label vector. The second set-association operation is the same as the first set-association operation, except that the text vector and the label vector contain different elements, which will not be described here.

[0043] Optional, such as Figure 2 As shown, in the first dimension, first, M questions (i.e., the second target preset text) whose similarity with question Q is greater than the threshold Alpha (i.e., the preset similarity) are extracted from the K questions, and Tm tags associated with the M questions (i.e., the second preset tags) are extracted from the preset database. Then, the vectors Fq and Ft corresponding to question Q and tag T are clustered to obtain the clustered feature vector Fqt (i.e., the first clustered feature vector). Then, the text representations of the M questions and all their associated tags are extracted and clustered to obtain the feature set of similar questions and tags (i.e., the second clustered feature vector) Fm = {Fij}, where i∈(1,2,...,M), j∈(Tm1,...,Tmi), and each question may be associated with multiple tags.

[0044] Furthermore, after obtaining the first set-linked feature vector and the second set-linked feature vector, a first weighted distance is determined according to the first set-linked feature vector and the second set-linked feature vector. Specifically, first, the cosine distance between the first set-linked feature vector and the second set-linked feature vector is calculated to obtain a first result, and then the similarity between the text to be processed and the second target preset text is calculated to obtain a third similarity, and finally, the first result is weighted and summed according to the third similarity to obtain a first weighted distance.

[0045] Optionally, the first weighted distance D1 may satisfy the following formula:

[0046]

[0047] Among them, the first weighted distance represents the similarity between each feature in the Fqt and Fm sets, Sqi represents the similarity between question Q and the i-th similar question, and S(·) represents the cosine distance between two feature vectors.

[0048] In an optional embodiment, if Figure 2 As shown, in the second dimension, the second weighted distance is determined according to the first label and the second label. Specifically, the second label is first clustered to obtain multiple clusters, and the cluster feature vector corresponding to the cluster center of each cluster is obtained, and then the cosine distance between the cluster feature vector and the first label vector corresponding to the first label is calculated to obtain the second result, and finally, the second result is weighted and summed to obtain the second weighted distance.

[0049] Optional, such as Figure 2 As shown, the associated labels Tk (i.e., the second labels) of the most similar K questions are extracted from the preset database, and the feature vectors (i.e., the second label vectors) Fk={Fij'} of all labels in Tk are extracted using the Word2Vec model, where i∈(1,2,...,K), j∈(Tk1,...,Tki). Then, K-means clustering is used to cluster all labels of similar questions (i.e., the second labels) to obtain the corresponding C clusters and the cluster feature vectors Fc={Fi} of the C cluster centers, where i∈(1,2,...,C). Finally, the cosine distance S(F qt , F i ).

[0050] Furthermore, after obtaining the first set-linked feature vector and the second set-linked feature vector, a weighted sum is performed on the second result to obtain a second weighted distance. Specifically, first, the first label quantity of the labels contained in each cluster is obtained, and the second label quantity of the labels corresponding to the first target preset text is determined. Then, the ratio of the first label quantity to the second label quantity is calculated to obtain a weight value, and the second result is weighted summed according to the weight value to obtain a second weighted distance.

[0051] Optionally, the total number of labels N (i.e., the number of second labels) in the Tk set and the total number of labels Ni (i.e., the number of first labels) in each cluster are counted, and the second weighted distance D2 is calculated by the following formula:

[0052]

[0053] The second weighted distance represents the feature similarity between Fqt and each cluster center, where S(·) represents the cosine distance between two feature vectors.

[0054] Further, such as Figure 2As shown, after obtaining the first weighted distance and the second weighted distance from two dimensions respectively, the accuracy of the first label is determined according to the first weighted distance and the second weighted distance. Specifically, first, the product of the first weight value and the first weighted distance is calculated to obtain the third result, and the product of the second weight value and the second weighted distance is calculated to obtain the fourth result. Finally, the sum of the third result and the fourth result is calculated to obtain the accuracy. Among them, the first weight value represents the influence of the first weighted distance on the accuracy, and the second weight value represents the influence of the second weighted distance on the accuracy.

[0055] Optionally, the accuracy D may satisfy the following formula:

[0056] D=μ·D1+(1-μ)·D2

[0057] In the above formula, μ∈[0, 1] is a parameter used to adjust the influence of the two weighted distances on the accuracy result. The final accuracy D∈[-1, 1] is the value of D that is closer to 1 when the title and label accuracy is high, and closer to -1 when the accuracy is low.

[0058] From the above content, it can be seen that the solution provided by this application uses similar question labels of questions as pseudo labels to guide the calculation of the accuracy of the relationship between questions and labels, without the need to manually pre-label a large amount of question and label relationship data. In addition, this solution uses two dimensions to evaluate the accuracy of the relationship between questions and labels, where the first dimension is to use the weighted distance between the input question and label features and the most similar question and label features in the similar question set to evaluate the accuracy of the relationship between questions and labels; the other dimension also uses the weighted distance between the input label and the cluster center of each label cluster of similar exercises to measure the accuracy of the question and label by the distance between the label and the labels of most similar questions.

[0059] Example 2

[0060] According to an embodiment of the present invention, there is also provided an embodiment of a device for calculating label accuracy, wherein: Figure 3 is a schematic diagram of a device for calculating label accuracy according to an embodiment of the present invention, such as Figure 3 As shown, the device includes: an acquisition module 301 , a first determination module 303 , a second determination module 305 , a third determination module 307 and a fourth determination module 309 .

[0061] Among them, the acquisition module 301 is used to acquire the text to be processed and the first label corresponding to the text to be processed; the first determination module 303 is used to determine the similar text corresponding to the text to be processed and the second label corresponding to the similar text; the second determination module 305 is used to determine the first weighted distance according to the text to be processed and the similar text; the third determination module 307 is used to determine the second weighted distance according to the first label and the second label; the fourth determination module 309 is used to determine the accuracy of the first label according to the first weighted distance and the second weighted distance, wherein the accuracy represents the similarity between the first label and the second label.

[0062] It should be noted that the above-mentioned acquisition module 301, first determination module 303, second determination module 305, third determination module 307 and fourth determination module 309 correspond to steps S102 to S110 in the above-mentioned embodiment, and the examples and application scenarios implemented by the five modules and the corresponding steps are the same, but are not limited to the contents disclosed in the above-mentioned embodiment 1.

[0063] Optionally, the first determination module includes: a first acquisition module, a second acquisition module, a fifth determination module, a sixth determination module and a seventh determination module. Among them, the first acquisition module is used to obtain the first text vector of the text to be processed and the first label vector of the first label; the second acquisition module is used to obtain the preset feature vectors corresponding to multiple preset texts stored in the preset database; the fifth determination module is used to determine the first similarity between each preset text and the text to be processed according to the preset feature vector and the first text vector; the sixth determination module is used to obtain the first number of first target preset texts from the preset database in the order of the first similarity, and determine the first preset label corresponding to the first target preset text; the seventh determination module is used to determine that the first target preset text is a similar text, and determine that the first preset label is the second label.

[0064] Optionally, the first acquisition module includes: a removal module, a first extraction module and a second extraction module. The removal module is used to remove invalid information in the text to be processed and the first label to obtain the processed text to be processed and the processed first label; the first extraction module is used to perform feature extraction on the processed text to be processed based on a preset word vector model to obtain a first text vector; the second extraction module is used to perform feature extraction on the processed first label based on a preset word vector model to obtain a first label vector.

[0065] Optionally, the second determination module includes: a first calculation module, an eighth determination module, a ninth determination module, a first cluster association module, a second cluster association module and a tenth determination module. Among them, the first calculation module is used to calculate the second similarity between the first target preset text and the text to be processed; the eighth determination module is used to obtain the second target preset text whose second similarity is greater than the preset similarity from the first number of first target preset texts, and determine the second text vector corresponding to the second target preset text; the ninth determination module is used to obtain the second preset label corresponding to the second target preset text from the preset database, and determine the second label vector corresponding to the second preset label; the first cluster association module is used to perform a first cluster association operation on the first text vector and the first label vector to obtain a first cluster association feature vector, wherein the first cluster association operation includes: according to A concatenation operation is performed on the first text vector and the first label vector according to a first preset order, and the dimension of the first set-linked feature vector is the sum of the dimension of the first text vector and the dimension of the first label vector; a second set-linked module is used to perform a second set-linked operation on the second text vector and the second label vector to obtain a second set-linked feature vector, wherein the second set-linked operation includes: a concatenation operation is performed on the second text vector and the second label vector according to a second preset order, and the dimension of the second set-linked feature vector is the sum of the dimension of the second text vector and the dimension of the second label vector; a tenth determination module is used to determine the first weighted distance according to the first set-linked feature vector and the second set-linked feature vector.

[0066] Optionally, the tenth determination module includes: a second calculation module, a third calculation module and a fourth calculation module. The second calculation module is used to calculate the cosine distance between the first set-linked feature vector and the second set-linked feature vector to obtain a first result; the third calculation module is used to calculate the similarity between the text to be processed and the second target preset text to obtain a third similarity; the fourth calculation module is used to perform weighted summation on the first result according to the third similarity to obtain a first weighted distance.

[0067] Optionally, the third determination module includes: a clustering module, a third acquisition module, a fifth calculation module and a sixth calculation module. The clustering module is used to perform clustering processing on the second label to obtain multiple clusters; the third acquisition module is used to obtain the cluster feature vector corresponding to the cluster center of each cluster; the fifth calculation module is used to calculate the cosine distance between the cluster feature vector and the first label vector corresponding to the first label to obtain the second result; the sixth calculation module is used to perform weighted summation on the second result to obtain the second weighted distance.

[0068] Optionally, the sixth calculation module includes: a fourth acquisition module, an eleventh determination module, a seventh calculation module and an eighth calculation module. Among them, the fourth acquisition module is used to obtain the first label quantity of the labels contained in each cluster; the eleventh determination module is used to determine the second label quantity of the labels corresponding to the first target preset text; the seventh calculation module is used to calculate the ratio of the first label quantity to the second label quantity to obtain a weight value; the eighth calculation module is used to perform weighted summation on the second result according to the weight value to obtain a second weighted distance.

[0069] Optionally, the fourth determination module includes: a ninth calculation module, a tenth calculation module and an eleventh calculation module. The ninth calculation module is used to calculate the product of the first weight value and the first weighted distance to obtain a third result, wherein the first weight value represents the influence of the first weighted distance on the accuracy; the tenth calculation module is used to calculate the product of the second weight value and the second weighted distance to obtain a fourth result, wherein the second weight value represents the influence of the second weighted distance on the accuracy; the eleventh calculation module is used to calculate the sum of the third result and the fourth result to obtain the accuracy.

[0070] Example 3

[0071] According to another aspect of an embodiment of the present invention, a non-volatile storage medium is provided, in which a computer program is stored, wherein the computer program is configured to execute the method for calculating the label accuracy in the above-mentioned embodiment 1 when running.

[0072] Example 4

[0073] According to another aspect of an embodiment of the present invention, a processor is further provided, the processor being used to run a program, wherein the program is configured to execute the method for calculating the label accuracy in the above-mentioned embodiment 1 when running.

[0074] The serial numbers of the above embodiments of the present invention are only for description and do not represent the advantages or disadvantages of the embodiments.

[0075] In the above embodiments of the present invention, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0076] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only schematic. For example, the division of the units can be a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0077] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0078] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0079] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, magnetic disk or optical disk and other media that can store program codes.

[0080] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principle of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A method for calculating label accuracy, characterized in that: include: Acquire a text to be processed and a first tag corresponding to the text to be processed; Determine similar text corresponding to the text to be processed and a second label corresponding to the similar text; Determining a first weighted distance according to the text to be processed and the similar text; Determining a second weighted distance according to the first label and the second label; The accuracy of the first tag is determined according to the first weighted distance and the second weighted distance, wherein the accuracy represents the similarity between the first tag and the second tag.

2. The method according to claim 1, characterized in that Determining similar text corresponding to the text to be processed and a second label corresponding to the similar text includes: Obtaining a first text vector of the text to be processed and a first label vector of the first label; Obtaining preset feature vectors corresponding to a plurality of preset texts stored in a preset database; Determining a first similarity between each preset text and the text to be processed according to the preset feature vector and the first text vector; Acquire a first number of first target preset texts from the preset database in the order of the first similarities, and determine first preset tags corresponding to the first target preset texts; The first target preset text is determined to be the similar text, and the first preset tag is determined to be the second tag.

3. The method according to claim 2, characterized in that Acquiring a first text vector of the text to be processed and a first label vector of the first label includes: Removing invalid information from the text to be processed and the first label to obtain a processed text to be processed and a processed first label; Performing feature extraction on the processed text to be processed based on a preset word vector model to obtain the first text vector; Based on the preset word vector model, feature extraction is performed on the processed first label to obtain the first label vector.

4. The method according to claim 2, characterized in that: Determining a first weighted distance according to the to-be-processed text and the similar text includes: Calculating a second similarity between the first target preset text and the text to be processed; Acquire second target preset texts whose second similarity is greater than a preset similarity from the first number of first target preset texts, and determine a second text vector corresponding to the second target preset text; Acquire a second preset label corresponding to the second target preset text from the preset database, and determine a second label vector corresponding to the second preset label; Performing a first set-associative operation on the first text vector and the first label vector to obtain a first set-associative feature vector, wherein the first set-associative operation includes: performing a concatenation operation on the first text vector and the first label vector in a first preset order, and the dimension of the first set-associative feature vector is the sum of the dimension of the first text vector and the dimension of the first label vector; Performing a second set-associative operation on the second text vector and the second label vector to obtain a second set-associative feature vector, wherein the second set-associative operation includes: performing a concatenation operation on the second text vector and the second label vector according to a second preset order, and the dimension of the second set-associative feature vector is the sum of the dimension of the second text vector and the dimension of the second label vector; The first weighted distance is determined according to the first set-associative feature vector and the second set-associative feature vector.

5. The method according to claim 4, characterized in that Determining the first weighted distance according to the first set-associative feature vector and the second set-associative feature vector includes: Calculating a cosine distance between the first set-associative eigenvector and the second set-associative eigenvector to obtain a first result; Calculating the similarity between the text to be processed and the second target preset text to obtain a third similarity; The first results are weighted and summed according to the third similarity to obtain the first weighted distance.

6. The method according to claim 2, characterized in that Determining a second weighted distance according to the first label and the second label includes: Performing clustering processing on the second label to obtain multiple clusters; Get the cluster feature vector corresponding to the cluster center of each cluster; Calculating a cosine distance between the cluster feature vector and a first label vector corresponding to the first label to obtain a second result; Perform a weighted sum on the second results to obtain the second weighted distance.

7. The method according to claim 6, characterized in that Performing weighted summation on the second results to obtain the second weighted distance includes: Obtaining the first label quantity of the labels contained in each cluster; Determine the second label quantity of the label corresponding to the first target preset text; Calculate the ratio of the number of the first tags to the number of the second tags to obtain a weight value; The second results are weighted and summed according to the weight value to obtain the second weighted distance.

8. The method according to claim 1, characterized in that Determining the accuracy of the first tag according to the first weighted distance and the second weighted distance includes: Calculating a product of a first weight value and the first weighted distance to obtain a third result, wherein the first weight value represents the degree of influence of the first weighted distance on the accuracy; Calculating a product of a second weight value and the second weighted distance to obtain a fourth result, wherein the second weight value represents the degree of influence of the second weighted distance on the accuracy; The sum of the third result and the fourth result is calculated to obtain the accuracy.

9. A device for calculating label accuracy, characterized in that: include: An acquisition module, used for acquiring a text to be processed and a first label corresponding to the text to be processed; A first determination module, used to determine similar text corresponding to the text to be processed and a second label corresponding to the similar text; A second determination module, used for determining a first weighted distance according to the text to be processed and the similar text; a third determining module, configured to determine a second weighted distance according to the first label and the second label; A fourth determination module is used to determine the accuracy of the first tag according to the first weighted distance and the second weighted distance, wherein the accuracy represents the similarity between the first tag and the second tag.

10. A non-volatile storage medium, characterized in that: The non-volatile storage medium stores a computer program, wherein the computer program is configured to execute the method for calculating the tag accuracy described in any one of claims 1 to 8 when running.

11. A processor, characterized in that: The processor is used to run a program, wherein the program is configured to execute the method for calculating the tag accuracy described in any one of claims 1 to 8 when running.

Images