A data processing system for updating a lexicon

Abstract

The application relates to a data processing system for updating a word library, which comprises a preset word library, a processor and a memory storing a computer program, and when the computer program is executed by the processor, the following steps are realized: a candidate text set is acquired, an intermediate text set is acquired, a specific word list corresponding to the candidate text set is acquired according to the candidate text set and the intermediate text set, a target text set corresponding to the specific word is acquired, a target label list is acquired, a candidate label set is acquired according to the target text set and the target label list, a key label set corresponding to the specific word list is acquired, each key label list in the key label set is subjected to a deduplication treatment, and a label corresponding to each specific word in the specific word list is acquired to update the preset word library. The application is not limited to a same text database in terms of a text source, can be compared with other text databases, the accuracy of the acquired specific word is improved, and the accuracy of the acquired updated word library is high.

Description

Technical Field

[0001] This invention relates to the field of text processing technology, and in particular to a data processing system for updating a lexicon. Background Technology

[0002] With the development of internet technology, the number of citizens using the internet in my country has increased dramatically, and the amount of text information on the internet has exploded. Along with this explosive growth of text information on the internet, more and more new words have emerged. At the same time, in the process of the emergence of new words, specific words, that is, words with specific meanings, have also emerged. How to accurately identify specific words on the internet and obtain specific word tags based on specific words to update the word database has become a popular research direction.

[0003] Currently, the existing technology for updating the thesaurus involves: clustering the text to obtain its labels, extracting new words from the text, and obtaining the corresponding labels for the new words based on the new words and the existing labels, thereby updating the thesaurus.

[0004] In summary, the problem with the method of updating the thesaurus is that the text source is limited to the same text database, and it fails to compare with other text databases, which reduces the accuracy of the specific words obtained, and thus makes the accuracy of the updated thesaurus relatively low. Summary of the Invention

[0005] To address the aforementioned technical problems, the present invention adopts the following technical solution: a data processing system for updating a lexicon, the system comprising a preset lexicon, a processor, and a memory storing a computer program, wherein when the computer program is executed by the processor, the following steps are implemented:

[0006] S100, Obtain a candidate text set, which includes several candidate texts, and the candidate texts are texts obtained from the first data providing platform set within a preset first time period.

[0007] S200, Obtain the intermediate text set R = {R1, ..., R2} a , ..., R z}, R a Let a be the a-th intermediate text, a = 1...z, where z is the number of intermediate texts. The intermediate text is the text obtained from the second data providing platform during a preset second time period.

[0008] S300, Based on the candidate text set and R, obtain the specific word list J = {J1, ..., J...} corresponding to the candidate text set. ε , ..., J β}, J ε Let ε be the ε-th specific word, where ε = 1...β, and β is the number of specific words.

[0009] S400, Based on R and J, obtain the target text set J corresponding to J. 0 ={J 0 1, ..., J 0 ε , ..., J 0 β}, J 0 ε ={J 0 ε1 , ..., J 0 ελ , ..., J 0 εγ(ε)}, J 0 ελ For J ε The corresponding target text in the target text list is the λ-th target text, where λ = 1...γ(ε), and γ(ε) is J. ε The number of target texts in the corresponding target text list, where J ε ∈R a At that time, R a Insert into J 0 ε middle.

[0010] S500, Based on R and the preset vocabulary, obtain the target tag list G = {G1, ..., G...} corresponding to R. a , ..., G z}, G a For R a The corresponding target label.

[0011] S600, according to J 0 And G, get J 0 The corresponding candidate label set GJ 0 ={GJ 0 1, ..., GJ 0 ε ..., GJ 0 β}, GJ 0 ε ={GJ 0 ε1 ..., GJ 0 ελ ..., GJ 0 εγ(ε)}, GJ 0 ελ For J 0 ελ The corresponding candidate labels, where J 0 ελ With R a When they match, obtain GJ. 0ελ =G a .

[0012] S700, according to J and GJ 0 Obtain the key tag set J' = {J'1, ..., J'} corresponding to J. ε , ..., J' β}, the list of the ε-th key tags in J' ε ={J' ε1 , ..., J' ελ , ..., J' εγ(ε)}, where J' ελ =GJ 0 ελ .

[0013] S800, for each J' in J' ε Perform deduplication to obtain each J in J. ε The corresponding tags are used to update the preset thesaurus.

[0014] Compared with existing technologies, this invention has significant advantages. Through the above technical solution, the data processing system for updating a thesaurus provided by this invention achieves considerable technological advancement and practicality, and has broad industrial application value. It has at least the following advantages: This invention relates to a data processing system for updating a thesaurus. The system includes a preset thesaurus, a processor, and a memory storing a computer program. When the computer program is executed by the processor, the following steps are implemented: obtaining a candidate text set, wherein the candidate texts are texts obtained from a first data providing platform within a preset first time period; obtaining an intermediate text set, wherein the intermediate texts are texts obtained from a second data providing platform... The system obtains text from a preset second time period. Based on a candidate text set and an intermediate text set, it obtains a list of specific words corresponding to the candidate text set. Based on the intermediate text set and the list of specific words, it obtains a target text set corresponding to the specific words. Based on the intermediate text set and a preset thesaurus, it obtains a list of target tags corresponding to the intermediate text set. Based on the target text set and the list of target tags, it obtains a set of candidate tags corresponding to the target text set. Based on the list of specific words and the candidate tag set, it obtains a set of key tags corresponding to the specific words. It then performs deduplication on each key tag list in the key tag set and obtains the tags corresponding to each specific word in the specific word list to update the preset thesaurus. This invention does not limit the text source to a single text database; it allows comparison with other text databases, improving the accuracy of the obtained specific words and thus resulting in a higher accuracy of the updated thesaurus.

[0015] The above description is merely an overview of the technical solution of the present invention. In order to better understand the technical means of the present invention and to implement it in accordance with the contents of the specification, and to make the above and other objects, features and advantages of the present invention more apparent and understandable, preferred embodiments are described below in detail with reference to the accompanying drawings. Attached Figure Description

[0016] Figure 1 A flowchart of the execution computer program for a data processing system for updating a thesaurus, provided as an embodiment of the present invention;

[0017] Figure 2 This is a flowchart of step S300 in an embodiment of the present invention. Detailed Implementation

[0018] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of them. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0019] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or server that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or devices.

[0020] This invention provides a data processing system for updating a lexicon. The system includes a preset lexicon, a processor, and a memory storing a computer program. When the computer program is executed by the processor, it performs the following steps, such as... Figure 1 As shown:

[0021] S100, Obtain a candidate text set, which includes several candidate texts, and the candidate texts are texts obtained from the first data providing platform within a preset first time period.

[0022] Specifically, the first data providing platform set includes several first data providing platforms. The first data providing platform is a data providing platform for publishing real-time news information. Those skilled in the art know that any data providing platform for publishing real-time news information in the prior art falls within the protection scope of this invention, and will not be elaborated further here. Among them, the first data providing platform is, for example, a social media platform.

[0023] Specifically, the start time of the preset first time period is 1:00 AM on the same day, and the time span of the preset first time period is 20 to 22 hours, for example, the preset first time period is from 1:00 AM to 10:00 PM on the same day.

[0024] S200, Obtain the intermediate text set R = {R1, ..., R2} a , ..., R z}, R a Let a be the a-th intermediate text, a = 1...z, where z is the number of intermediate texts. The intermediate text is the text obtained from the second data providing platform during a preset second time period.

[0025] Specifically, the second data providing platform is an app platform for user social interaction. As those skilled in the art know, any existing app platform for user social interaction falls within the protection scope of this invention, and will not be elaborated further here. The second data providing platform is, for example, instant messaging software.

[0026] Specifically, each intermediate text corresponds to a region. This can be understood as follows: each intermediate text is obtained from the same region, and different intermediate texts correspond to different regions. For example, when the second data providing platform is instant messaging software, the region corresponding to the intermediate text is an instant messaging software group, and each intermediate text corresponds to an instant messaging software group.

[0027] Specifically, the data sources for the intermediate text include text, voice, and images.

[0028] Furthermore, when the data provided by the second data providing platform is voice, the voice is converted into text. Those skilled in the art know that any method of converting voice into text in the prior art falls within the protection scope of this invention, and will not be elaborated further here.

[0029] Furthermore, when the data provided by the second data providing platform is an image, the image is processed using a text recognition model to obtain text. As those skilled in the art know, any text recognition model in the prior art falls within the protection scope of this invention, and will not be elaborated further here, such as the OCR text recognition model.

[0030] Specifically, the start time of the preset second time period is 72 hours before 12:00 noon on the same day, and the time span of the preset second time period is 72 to 84 hours. For example, the preset second time period is from 3 days before 12:00 noon on the same day to 12:00 noon on the same day.

[0031] As mentioned above, the text data in the second data provider platform is newer than the text data provided by the first data provider platform. That is, the same text data appears later in time in the first data provider platform than in the second data provider platform. By obtaining text from both the first and second data provider platforms, the accuracy of obtaining specific words in the future is improved.

[0032] S300, Based on the candidate text set and R, obtain the specific word list J = {J1, ..., J...} corresponding to the candidate text set. ε , ..., J β}, J ε Let ε be the ε-th specific word, where ε = 1...β, and β is the number of specific words.

[0033] Specifically, J is obtained in S300 through the following steps, such as... Figure 2 As shown:

[0034] S301, Based on the candidate text set, obtain the keyword set A' = {A'2, ..., A'}. x , ..., A' p}, A' x ={A' x1 , ..., A' xe , ..., A' xt(x)}, A' xe Let t(x) be the e-th keyword in the x-th keyword list, where e = 1...t(x), and t(x) is A'. x The number of keywords, x = 2...p, where p is the number of keyword types.

[0035] Specifically, A' is obtained in S301 through the following steps. x :

[0036] S3011, Based on the candidate text set, obtain the candidate statement list A = {A1, ..., A...} i , ..., A n}, A i Let i be the i-th candidate statement, i = 1...n, where n is the number of candidate statements.

[0037] Specifically, the candidate statement list includes several candidate statements. Among them, the candidate statements are statements obtained by splitting a candidate text set according to identifiers. Those skilled in the art know that any method of splitting statements according to punctuation marks in the prior art falls within the protection scope of the present invention and will not be elaborated herein.

[0038] S3013. According to A, obtain the corresponding candidate word set QA = {QA1,..., QA i ,..., QA n}, QA i = {QA i2 ,..., QA ix ,..., QA ip}, QA ix = {QA 1 ix ,..., QA y ix ,..., QA q(x) ix}, QA y ix is the y-th candidate word in the x-th type of candidate word list corresponding to A, y = 1... q(x), q(x) is the number of candidate words in the x-th type of candidate word list corresponding to A, x = 2... p, p is the number of candidate word types. Among them, q(x) meets the following conditions:

[0039] q(x) = f(i) - x + 1, f(i) is the number of literal characters corresponding to A i .

[0040] Specifically, the number of literal characters corresponding to the candidate words in the x-th type of candidate word list is x. It can be understood that: for the candidate words in QA i2 , the number of literal characters corresponding to them is 2, such as "Beijing"; for the candidate words in QA i3 , the number of literal characters corresponding to them is 3, such as "clairvoyance".

[0041] Specifically, the value range of p is 4 ≤ p ≤ 10.

[0042] Specifically, the candidate words are words obtained by successively splitting the literal characters corresponding to the candidate statements according to the candidate word types. It can be understood that: when the candidate statement is "The sky is so blue today", the corresponding candidate word set is "today", "sky's", "of the sky", "sky is", "is so blue", "today's", "sky's sky", "of the sky is", "sky is so blue", "today's sky", "sky's sky is", "of the sky is so blue", etc.

[0043] ​​​​S3015, Based on the initial text set, obtain the candidate priority set FA = {FA1, ..., FA2} corresponding to QA. i , ..., FA n}, FA i ={FA i2 , ..., FA ix , ..., FA ip}, FA ix ={FA 1 ix , ..., FA y ix , ..., FA q(x) ix}, where FA y ix Meets the following conditions:

[0044] K y ix For QA y ix The frequency of occurrence in the initial text set, K yg ix For QA y ix The frequency of the g-th character in the corresponding list of text strings in the initial text set.

[0045] Specifically, the initial text set includes several initial texts, which are texts obtained from the first data providing platform during a preset third time period.

[0046] Furthermore, the starting time of the preset third time period is in the range of 10 days before 12 noon to 20 days before 12 noon, wherein the time span of the preset third time period is 10 to 20 days, for example, the preset third time period is from 10 days before 12 noon to 12 noon.

[0047] S3017, Based on FA, obtain A' x , where, when FA y ix When ≥F', QA y ix Insert into A' x In this context, F' represents the preset first priority threshold.

[0048] Specifically, the value of F' ranges from 0.8 to 0.9.

[0049] S302, Based on the initial text set, obtain the initial word set D = {D2, ..., D...} x , ..., D p}, D x={D x1 , ..., D xv , ..., D xb(x)}, D xv Let v be the v-th initial word in the initial word list of the x-th class, where v = 1...b(x), and b(x) is the number of initial words in the initial word list of the x-th class.

[0050] Specifically, D x The method of obtaining A' x The acquisition method is the same, and can be referred to S3011 to S3017.

[0051] S303, Based on A' and D, obtain the target word set C = {C2, ..., C} corresponding to the candidate text set. x , ..., C p}, C x ={C x1 , ..., C xθ , ..., C xφ(x)}, where C xθ Let θ be the θ-th target word in the x-th target word list, where θ = 1...φ(x), and φ(x) is C. x The number of target words.

[0052] Specifically, C is obtained in S303 through the following steps. x :

[0053] S3031, according to A' x and D x , get A' x The corresponding first similarity set DA' x ={DA' x1 , ...,DA' xe , ...,DA' xt(x)},DA' xe ={DA' 1 xe , ...,DA' v xe , ...,DA' b(x) xt(x)}, where DA' v xe For A' xe With D xv The first similarity between them.

[0054] Specifically, the method for obtaining the first similarity is known to those skilled in the art, and any text similarity calculation method can be selected according to actual needs, all of which fall within the protection scope of this invention, and will not be elaborated here.

[0055] S3033, when DA' vxe <F 0 When, A' xe Insert into C x In the middle, among which, F 0 This is the preset first similarity threshold.

[0056] Specifically, F 0 The value range is 0.3 to 0.5.

[0057] S304, Based on R, obtain the intermediate word set R' = {R'1, ..., R'} corresponding to R. a , ..., R' z}, R' a ={R' a1 , ..., R' ad , ..., R' ah(a)}, R' ad Let h(a) be the d-th intermediate word corresponding to the r-th intermediate text, where d = 1...h(a), and h(a) is the number of intermediate words corresponding to the a-th intermediate text.

[0058] Specifically, the intermediate words are words obtained by segmenting the intermediate text and then deduplicating it. Those skilled in the art know that any segmentation method in the prior art is within the protection scope of this invention, and will not be elaborated here.

[0059] The preferred method for word segmentation is the Jieba word segmentation method.

[0060] Furthermore, those skilled in the art will recognize that any deduplication method in the prior art falls within the protection scope of this invention, and will not be elaborated further here.

[0061] S305, according to C x and R' a , get C xθ The corresponding second similarity set CR' xe ={CR' 1 xe , ..., CR' a xe , ..., CR' z xe}, CR' a xe ={CR' a1 xe , ..., CR' ad xe , ..., CR' ah(a) xe}, CR' ad xe C xθ With R' ad The second similarity between them.

[0062] Specifically, the method for obtaining the second similarity is the same as the method for obtaining the first similarity.

[0063] S306, when CR' ad xe >F 1 At that time, determine C xθ For a specific word and C xθ Insert into J, where F 1 This is the preset second similarity threshold.

[0064] Specifically, F 1 The value range is 0.8 to 0.9.

[0065] As described above, target words are obtained from the candidate text set provided by the first data provider platform, and intermediate words are obtained from the intermediate text provided by the second data provider platform. By comparing the obtained target words with the intermediate words, a list of specific words is obtained. This method is not limited to the same text data provider platform. Compared with the text data provided by the first data provider platform, the text data in the second data provider platform is newer. That is, for the same text data, the time when it appears on the first data provider platform is lagging behind the time when it appears on the second data provider platform, which makes the accuracy of the obtained specific words higher.

[0066] S400, Based on R and J, obtain the target text set J corresponding to J. 0 ={J 0 1, ..., J 0 ε , ..., J 0 β}, J 0 ε ={J 0 ε1 , ..., J 0 ελ , ..., J 0 εγ(ε)}, J 0 ελ For J ε The corresponding target text in the target text list is the λ-th target text, where λ = 1...γ(ε), and γ(ε) is J. ε The number of target texts in the corresponding target text list, where J ε ∈R a At that time, R a Insert into J 0 ε middle.

[0067] Specifically, the target text is intermediate text containing specific words obtained from the intermediate text set.

[0068] S500, Based on R and the preset vocabulary, obtain the target tag list G = {G1, ..., G...} corresponding to R. a , ..., G z}, G a For R a The corresponding target label.

[0069] Specifically, G is obtained in S500 through the following steps. a :

[0070] S501, Based on R', obtain the intermediate word frequency set U' = {U'1, ..., U'} corresponding to R'. a , ……, U' z}, U' a ={U' a1 , ……, U' ad , ……, U' ah(a)}, U' ad ={U' 1 ad , ……, U' a ad , ……, U' z ad}, where U' a ad For R' ad In R a The frequency of occurrence.

[0071] S502, Based on U', obtain the target priority set Y' = {Y'1, ..., Y'} corresponding to R'. a , ..., Y' z}, Y' a ={Y' a1 , ..., Y' ad , ..., Y' ah(a)}, Y' ad For R' ad The corresponding target priority, where Y' ad Meets the following conditions:

[0072] μ' ad For U' ad middleU' a ad The quantity that is not zero.

[0073] S503, Based on Y', obtain the specified word set TR' = {TR'1, ..., TR'} corresponding to R'. a , ..., TR' z}, TR' a ={TR' a1, ..., TR' aζ , ..., TR' aη(a)}, TR' aζ For R' a The ζ-th specified word in the corresponding specified word list, wherein the specified word is R' obtained from R'. ad Satisfy its corresponding Y' ad Middle words that are greater than the preset second priority threshold.

[0074] Specifically, the preset second priority threshold ranges from 0.75 to 0.95.

[0075] S504, Based on U' and TR', obtain the specified word frequency set WR' = {WR'1, ..., WR'} corresponding to TR'. a , ...,WR' z}, WR' a ={WR' a1 , ...,WR' aζ , ...,WR' aη(a)}, WR' aζ For TR' aζ The corresponding specified word frequency, wherein the specified word frequency is the frequency of the specified word obtained from the intermediate word frequency set.

[0076] S505, Based on WR', obtain the target keyword list R corresponding to R. 0 ={R 0 1, ..., R 0 a , ..., R 0 z}, R 0 a For R a The corresponding target keywords, where WR' aζ For WR' a When R reaches its maximum value, 0 a =TR' aζ .

[0077] S506, Obtain the preset vocabulary B = {B1, ..., B} j , ..., B m}, B j ={B j1 , ..., B jr , ..., B js(j)}, where B jr Let s(j) be the r-th preset word in the j-th preset word tag, where r = 1...s(j), s(j) is the number of preset words in the j-th preset word tag, j = 1...m, and m is the number of preset word tags.

[0078] Specifically, the preset word tags are word tags corresponding to preset words, such as word tags for fraud.

[0079] S507, according to R 0 And B, obtain the target label list G = {G1, ..., G...} corresponding to R. a , ..., G z}, where G a For R a The corresponding target label, where, when R 0 a With B jr When they are consistent, obtain G. a =B j .

[0080] As described above, based on the intermediate text set and the preset lexicon, the intermediate text set is processed to obtain the target tags corresponding to the intermediate text set. By improving the accuracy of obtaining the target tags corresponding to the intermediate text set, the accuracy of obtaining specific word tags in the subsequent process is higher. By combining the attributes of the text data provided by the second data platform, the accuracy of the updated lexicon obtained is improved.

[0081] S600, according to J 0 And G, get J 0 The corresponding candidate label set GJ 0 ={GJ 0 1, ..., GJ 0 ε ..., GJ 0 β}, GJ 0 ε ={GJ 0 ε1 ..., GJ 0 ελ ..., GJ 0 εγ(ε)}, GJ 0 ελ For J 0 ελ The corresponding candidate labels, where J 0 ελ With R a When they match, obtain GJ. 0 ελ =G a .

[0082] Specifically, the candidate tags are the target tags corresponding to the target text obtained from the target tag list.

[0083] S700, according to J and GJ 0Obtain the key tag set J' = {J'1, ..., J'} corresponding to J. ε , ..., J' β}, the list of the ε-th key tags in J' ε ={J' ε1 , ..., J' ελ , ..., J' εγ(ε)}, where J' ελ =GJ 0 ελ .

[0084] Specifically, the key tags are candidate tags corresponding to specific words obtained from the candidate tag set.

[0085] S800, for each J' in J' ε Perform deduplication to obtain each J in J. ε The corresponding tags are used to update the preset thesaurus.

[0086] Specifically, the process of updating the preset lexicon is to insert the acquired specific words and their corresponding key tags into the preset lexicon. As those skilled in the art know, any method in the prior art that inserts words and tags into a database falls within the protection scope of this invention, and will not be elaborated further here.

[0087] This invention provides a data processing system for updating a thesaurus. The system includes a preset thesaurus, a processor, and a memory storing a computer program. When the computer program is executed by the processor, the following steps are performed: obtaining a candidate text set, wherein the candidate text is text obtained from a first data providing platform within a preset first time period; obtaining an intermediate text set, wherein the intermediate text is text obtained from a second data providing platform within a preset second time period; obtaining a list of specific words corresponding to the candidate text set based on the candidate text set and the intermediate text set; obtaining a target text set corresponding to the specific word list based on the intermediate text set and the specific word list; obtaining a list of target tags corresponding to the intermediate text set based on the intermediate text set and the preset thesaurus; obtaining a candidate tag set corresponding to the target text set based on the target text set and the target tag list; obtaining a set of key tags corresponding to the specific word list based on the specific word list and the candidate tag set; performing deduplication processing on each key tag list in the key tag set; and obtaining the tag corresponding to each specific word in the specific word list to update the preset thesaurus. This invention does not limit the text source to a single text database, but allows for comparison with other text databases, improving the accuracy of the obtained specific words and thus resulting in a more accurate updated thesaurus.

[0088] While specific embodiments of the invention have been described in detail by way of examples, it should be understood by those skilled in the art that the above examples are for illustrative purposes only and are not intended to limit the scope of the invention. It should also be understood by those skilled in the art that various modifications can be made to the embodiments without departing from the scope and spirit of the invention. The scope of the invention is defined by the appended claims.

Claims

1. A data processing system for updating a thesaurus, characterized in that, The system includes a preset dictionary, a processor, and a memory storing a computer program. When the computer program is executed by the processor, the following steps are performed: S100, Obtain a candidate text set, the candidate text set including a plurality of candidate texts, the candidate texts being texts obtained from the first data providing platform set within a preset first time period; S200, Obtain the intermediate text set R = {R1, ..., R2} a , ..., R z }, R a Let a be the a-th intermediate text, a=1……z, z is the number of intermediate texts, wherein the intermediate text is the text obtained from the second data providing platform during a preset second time period; S300, Based on the candidate text set and R, obtain the specific word list J={J1, ..., J2} corresponding to the candidate text set. ε , ..., J β }, J ε For the ε-th specific word, ε = 1...β, where β is the number of specific words; J is obtained in S300 through the following steps: S301, Based on the candidate text set, obtain the keyword set A' = {A'2, ..., A'}. x , ..., A' p }, A' x ={A' x1 , ..., A' xe , ..., A' xt（x） }, A' xe Let t(x) be the e-th keyword in the x-th keyword list, where e = 1...t(x), and t(x) is A' x The number of keywords in the text, x=2……p, where p is the number of keyword types; S302, Based on the initial text set, obtain the initial word set D = {D2, ..., D...} x , ..., D p }, D x ={D x1 , ..., D xv , ..., D xb（x） }, D xv Let v be the v-th initial word in the initial word list of the x-th class, where v = 1...b(x), and b(x) is the number of initial words in the initial word list of the x-th class. S303, Based on A' and D, obtain the target word set C={C2, ..., C...} corresponding to the candidate text set. x , ..., C p }, C x ={C x1 , ..., C xθ , ..., C xφ（x） }, where C xθ Let θ be the θ-th target word in the x-th target word list, where θ = 1...φ(x), and φ(x) is C. x The number of target words; S304, Based on R, obtain the intermediate word set R'={R'1, ..., R'} corresponding to R. a , ..., R' z }, R' a ={R' a1 , ..., R' ad , ..., R' ah（a） }, R' ad Let d be the d-th intermediate word corresponding to the r-th intermediate text, where d = 1...h(a), and h(a) is the number of intermediate words corresponding to the a-th intermediate text; S305, according to C x and R' a , get C xθ The corresponding second similarity set CR' xe ={CR' 1 xe , ..., CR' a xe , ..., CR' z xe }, CR' a xe ={CR' a1 xe , ..., CR' ad xe , ..., CR' ah（a） xe }, CR' ad xe C xθ With R' ad The second similarity between them; S306, when CR' ad xe >F 1 At that time, determine C xθ For a specific word and C xθ Insert into J, where F 1 The second similarity threshold is preset. S400, Based on R and J, obtain the target text set J corresponding to J. 0 ={J 0 1, ..., J 0 ε , ..., J 0 β }, J 0 ε ={J 0 ε1 , ..., J 0 ελ , ..., J 0 εγ（ε） }, J 0 ελ For J ε The corresponding target text in the target text list is the λ-th target text, λ=1……γ(ε), where γ(ε) is J. ε The number of target texts in the corresponding target text list, where J ε ∈R a At that time, R a Insert into J 0 ε middle; S500, based on R and the preset vocabulary, obtain the target tag list G={G1, ..., G...} corresponding to R. a , ..., G z }, G a For R a The corresponding target label; S600, according to J 0 And G, get J 0 The corresponding candidate label set GJ 0 ={GJ 0 1, ..., GJ 0 ε ..., GJ 0 β }, GJ 0 ε ={GJ 0 ε1 ..., GJ 0 ελ ..., GJ 0 εγ（ε） }, GJ 0 ελ For J 0 ελ The corresponding candidate labels, where J 0 ελ With R a When they match, obtain GJ. 0 ελ =G a ; S700, according to J and GJ 0 Obtain the key tag set J'={J'1, ..., J'} corresponding to J. ε , ..., J' β }, the list of the ε-th key tags in J' ε ={J' ε1 , ..., J' ελ , ..., J' εγ（ε） }, where J' ελ =GJ 0 ελ ; S800, for each J' in J' ε Perform deduplication to obtain each J in J. ε The corresponding tags are used to update the preset thesaurus.

2. The data processing system for updating the lexicon according to claim 1, characterized in that, The first data providing platform set includes several first data providing platforms, and the first data providing platforms are data providing platforms for publishing real-time news information.

3. The data processing system for updating the lexicon according to claim 1, characterized in that, The second data providing platform is an app platform used for user social networking.

4. The data processing system for updating the lexicon according to claim 1, characterized in that, The preset first time period starts at 1:00 AM on the same day, and the time span of the preset first time period is 20 to 22 hours.

5. The data processing system for updating the lexicon according to claim 1, characterized in that, The database also includes an initial text set.

6. The data processing system for updating the lexicon according to claim 1, characterized in that, The intermediate words are words obtained by segmenting the intermediate text and then deduplicating it.

7. The data processing system for updating the lexicon according to claim 1, characterized in that, F 1 The value range is 0.8 to 0.

9.

8. The data processing system for updating the lexicon according to claim 1, characterized in that, In S500, G is obtained through the following steps. a : S501, Based on R', obtain the intermediate word frequency set U'={U'1, ..., U'} corresponding to R'. a , ..., U' z }, U' a ={U' a1 , ……, U' ad , ..., U' ah（a） }, U' ad ={U' 1 ad , ……, U' a ad , ……, U' z ad }, where U' a ad For R' ad In R a Frequency of occurrence in; S502, Based on U', obtain the target priority set Y'={Y'1, ..., Y'} corresponding to R'. a , ..., Y' z }, Y' a ={Y' a1 , ..., Y' ad , ..., Y' ah（a） }, Y' ad For R' ad The corresponding target priority, where Y' ad Meets the following conditions: ,μ' ad For U' ad middleU' a ad The number of non-zero values; where h(a) is the number of intermediate words corresponding to the a-th intermediate text, and R is the set of intermediate words R'. S503, Based on Y', obtain the specified word set TR'={TR'1, ..., TR'} corresponding to R'. a , ..., TR' z }, TR' a ={TR' a1 , ..., TR' aζ , ..., TR' aη（a） }, TR' aζ For R' a The ζ-th specified word in the corresponding specified word list, wherein the specified word is R' obtained from R'. ad Satisfy its corresponding Y' ad Middle words that exceed the preset second priority threshold; S504, Based on U' and TR', obtain the specified word frequency set WR'={WR'1, ..., WR'} corresponding to TR'. a , ...,WR' z }, WR' a ={WR' a1 , ...,WR' aζ , ...,WR' aη（a） }, WR' aζ For TR' aζ The corresponding specified word frequency, wherein the specified word frequency is the frequency of the specified word obtained from the intermediate word frequency set; S505, Based on WR', obtain the target keyword list R corresponding to R. 0 ={R 0 1, ..., R 0 a , ..., R 0 z }, R 0 a For R a The corresponding target keywords, where WR' aζ For WR' a When R reaches its maximum value, 0 a =TR' aζ ; S506, Obtain the preset vocabulary B={B1, ..., B...} j , ..., B m }, B j ={B j1 , ..., B jr , ..., B js（j） }, where B jr Let s(j) be the r-th preset word in the j-th preset word tag, where r = 1...s(j), s(j) is the number of preset words in the j-th preset word tag, j = 1...m, and m is the number of preset word tags; S507, according to R 0 And B, obtain the target label list G={G1, ..., G...} corresponding to R. a , ..., G z }, where G a For R a The corresponding target label, where, when R 0 a With B jr When they are consistent, obtain G. a =B j .

9. The data processing system for updating the lexicon according to claim 8, characterized in that, The preset second priority threshold value ranges from 0.75 to 0.95.

Images