Word vector building method for manuscript processing and manuscript processing method and device

Abstract

The present disclosure relates to a word vector building method for manuscript processing and a manuscript processing method and device, comprising: obtaining a label column required for building a word vector model from an information database; and extracting a preset number of segmented words and vector values of each segmented word from each manuscript stored in the information database. For each label column, the segmented words and vector values are matched to form a data set containing the label column, the segmented words and the vector values. Based on the formed data set, a word vector model is built. The word vector model is a matrix formed by multiple data sets. The present disclosure is designed based on a model based on the support vector machine principle, and the actual business requirements are met through the parameter adjustment processing of the mapping feature process. Using the eigenvalues of a single article and the eigenvalues of the overall model, some specific algorithms can be used to assist in realizing functions such as signature column recommendation.

Description

Technical Field

[0001] This disclosure relates to the field of manuscript issuance, and in particular to a word vector construction method, manuscript processing method and apparatus for manuscript processing. Background Technology

[0002] Financial articles require high real-time performance and accuracy, which necessitates that the article production system provide rapid and error-free article issuance technical support. Editors also need to handle business processes such as recommending articles for publication when issuing them.

[0003] Currently, most articles are processed using natural language models based on neural networks. However, these models have a large number of parameters, resulting in high deployment costs in actual production environments, high resource consumption under high concurrency, and significant computational and time requirements for model updates. Therefore, how to use support vector machine-based natural language models for business processing of financial articles has become a pressing issue for those skilled in the art. Summary of the Invention

[0004] In view of this, this disclosure proposes a word vector construction method, a manuscript processing method and apparatus for manuscript processing, which uses a natural language model based on support vector machine to perform business processing on manuscripts in the financial field, thereby reducing deployment costs.

[0005] According to one aspect of this disclosure, a method for building a word vector model for manuscript processing is provided, including:

[0006] The necessary tag columns for building a word vector model are obtained from the information database, and a preset number of word segments and the vector values ​​of each word segment are extracted from each article stored in the information database.

[0007] For each of the aforementioned tag categories, the corresponding word segmentation and vector value are matched to form a data set containing the tag category, the word segmentation, and the vector value;

[0008] Based on the constructed data set, the word vector model is constructed.

[0009] The word vector model is a matrix formed by multiple datasets.

[0010] In one possible implementation, when extracting a preset number of word segments and the vector values ​​of each word segment from the manuscripts stored in the information database, the results are obtained based on an open-source lexicon.

[0011] In one possible implementation, when extracting a preset number of words from each manuscript based on an open-source thesaurus, the process includes:

[0012] The manuscript is segmented using the word segmentation toolkit in the lexicon, and a first word segmentation set and a second word segmentation set are extracted from the manuscript; wherein the first word segmentation set and the second word segmentation set are obtained by different word segmentation extraction algorithms.

[0013] Based on the information database, the IDF values ​​of each word in the first word segmentation set and the IDF values ​​of each word in the second word segmentation set are calculated respectively.

[0014] The first word segmentation sequence is obtained by sorting the words in the first word segmentation set according to the size of their IDF values, and the first n words in the first word segmentation sequence are selected as the first words to be extracted.

[0015] The words in the second word segmentation set are sorted according to their IDF values ​​to obtain a second word segmentation sequence. The first n words in the second word segmentation sequence are selected as the second words to be extracted.

[0016] Where n > N, and N is the preset number;

[0017] The first N words are selected from the word segmentation set consisting of n first words to be extracted and n second words to be extracted as the preset number of words.

[0018] In one possible implementation, when each of the aforementioned tag categories is matched with the corresponding word segment and the vector value to form a data set containing the tag category, the word segment, and the vector value, each tag category corresponds one-to-one with each of the aforementioned word segments.

[0019] In one possible implementation, the step of updating the word vector model is also included;

[0020] Specifically, when updating the word vector model, it is based on newly added articles in the information database.

[0021] According to another aspect of this disclosure, a method for manuscript processing based on a word vector model is provided, comprising:

[0022] Get the currently pending manuscripts;

[0023] The pre-built word vector model is invoked, and the parameter information required for processing the current manuscript is extracted from the word vector model.

[0024] The manuscript to be processed is processed accordingly using the parameter information.

[0025] The word vector model is constructed using the construction method described in any one of claims 1 to 5.

[0026] In one possible implementation, the processing of the manuscript to be processed includes:

[0027] The manuscripts awaiting processing will be recommended for signing off in the relevant section.

[0028] In one possible implementation, when issuing a column recommendation for the manuscript to be processed, the following steps are included:

[0029] In the word vector model, the keywords and word vectors under each issuance category constitute the matrix of the current issuance category, and the feature value of each issuance category is calculated.

[0030] Construct a word matrix for the manuscript to be processed based on the manuscript to be processed;

[0031] Calculate the similarity between the feature value of each issued column and the word matrix of the manuscript to be processed;

[0032] The category for issuing documents with a similarity greater than a preset value is the recommended category for the manuscript to be processed.

[0033] According to another aspect of this disclosure, a manuscript processing apparatus is provided, comprising: a word segmentation and extraction module, a word vector model building module, and a manuscript processing module;

[0034] The word segmentation extraction module is configured to obtain the tag columns required for building the word vector model from the information database, and extract a preset number of word segments and the vector values ​​of each word segment from each article stored in the information database.

[0035] The word vector model building module is configured to build the word vector model based on the data set consisting of the tag column, the word segmentation, and the vector values.

[0036] The manuscript processing module is configured to extract the parameter information required for current manuscript processing from the word vector model, and use the parameter information to process the manuscript accordingly.

[0037] According to another aspect of this disclosure, a manuscript processing device is provided, characterized in that it includes: a processor;

[0038] Memory used to store processor-executable instructions;

[0039] The processor is configured to implement the above method when executing the executable instructions.

[0040] This disclosure applies to the processing of manuscripts using a constructed word vector model. The word vector model is built upon an information database, where all manuscripts are signed off, and each signed off manuscript includes its corresponding tag category. Each manuscript stored in the information database is segmented into words, and a predetermined number of words with higher weights and their word vectors are extracted. The tag categories, word segments, and word vectors constitute a dataset. The resulting matrix, formed by arranging and combining all the obtained datasets, constitutes the word vector model. This disclosure uses a model designed based on the principles of support vector machines, and through parameter tuning in the feature mapping process, it meets actual business needs. By utilizing the feature values ​​of individual articles and the feature values ​​of the overall model, and with the assistance of specific algorithms, functions such as plagiarism detection, image recommendation, tag recommendation, and signed off category recommendation can be achieved.

[0041] Other features and aspects of this disclosure will become clear from the following detailed description of exemplary embodiments with reference to the accompanying drawings. Attached Figure Description

[0042] The accompanying drawings, which are included in and form part of this specification, illustrate exemplary embodiments, features, and aspects of this disclosure together with the specification and serve to explain the principles of this disclosure.

[0043] Figure 1 A flowchart illustrating a method for building a word vector model for manuscript processing according to an embodiment of this application is shown.

[0044] Figure 2 This diagram shows the main structure of the manuscript processing apparatus according to an embodiment of this application;

[0045] Figure 3 This diagram shows the main structure of the manuscript processing device according to an embodiment of this application;

[0046] Figure 4 This illustrates the word segmentation of the manuscript and its weight values ​​obtained in an embodiment of this application. Detailed Implementation

[0047] Various exemplary embodiments, features, and aspects of this disclosure will now be described in detail with reference to the accompanying drawings. The same reference numerals in the drawings denote elements that have the same or similar functions. Although various aspects of the embodiments are shown in the drawings, they are not necessarily drawn to scale unless specifically indicated otherwise.

[0048] The term “exemplary” as used herein means “serving as an example, embodiment, or illustration.” Any embodiment illustrated herein as “exemplary” is not necessarily to be construed as superior to or better than other embodiments.

[0049] Furthermore, to better illustrate this disclosure, numerous specific details are set forth in the following detailed description. Those skilled in the art will understand that this disclosure can be practiced without certain specific details. In some instances, methods, means, components, and circuits well known to those skilled in the art have not been described in detail in order to highlight the main points of this disclosure.

[0050] Figure 1 A flowchart illustrating a method for building a word vector model for manuscript processing according to an embodiment of this application is shown. Figure 2 This diagram shows the main structure of the manuscript processing apparatus according to an embodiment of this application. Figure 3 This diagram shows the main structure of the manuscript processing device according to an embodiment of this application. Figure 4 This illustrates the word segmentation of the manuscript and its weight values ​​obtained in an embodiment of this application. For example... Figure 1 As shown, the method for building a word vector model for manuscript processing includes: Step S100: Obtaining the tag categories required for building the word vector model from the information database. Step S200: Extracting a preset number of word segments and their vector values ​​from each manuscript stored in the information database. Step S300: Matching word segments and vector values ​​for each tag category to form a data set containing the tag category, word segments, and vector values. Step S400: Building the word vector model based on the constructed data set. The word vector model is a matrix formed by multiple data sets.

[0051] This disclosure applies to the processing of manuscripts using a constructed word vector model. The word vector model is built upon an information database, where all manuscripts are signed off, and each signed off manuscript includes its corresponding tag category. Each manuscript stored in the information database is segmented into words, and a predetermined number of words with higher weights and their word vectors are extracted. The tag categories, word segments, and word vectors constitute a dataset. The resulting matrix, formed by arranging and combining all the obtained datasets, constitutes the word vector model. This disclosure uses a model designed based on the principles of support vector machines, and through parameter tuning in the feature mapping process, it meets actual business needs. By utilizing the feature values ​​of individual articles and the feature values ​​of the overall model, and with the assistance of specific algorithms, functions such as plagiarism detection, image recommendation, tag recommendation, and signed off category recommendation can be achieved.

[0052] It should be noted here that the reference... Figure 4 The preset number of word segments is 5 to 15, preferably 10. The top 10 word segments with the highest weight values ​​in a manuscript are selected as the preset word segments.

[0053] In one possible implementation, when extracting a preset number of word segments and their vector values ​​from the articles stored in the information database, the results are obtained based on an open-source thesaurus. The vector values ​​used in the open-source thesaurus employ commonly used techniques in this field, which will not be elaborated upon here.

[0054] In one possible implementation, when extracting a preset number of words from each manuscript based on an open-source thesaurus, word segmentation tools within the open-source thesaurus can be used to segment each manuscript. Specifically, when using word segmentation tools from the open-source thesaurus to segment the manuscript, the open-source jieba word segmentation toolkit can be used: github.com / semua / jiebago.

[0055] Simultaneously, after segmenting each manuscript using the word segmentation toolkit from the aforementioned open-source thesaurus, a first segmentation set and a second segmentation set are extracted from each manuscript. It should be noted that both the first and second segmentation sets are datasets obtained by segmenting the same manuscript, and they are derived using different word segmentation extraction algorithms employed during the manuscript segmentation process using the open-source thesaurus's word segmentation toolkit. By using different word segmentation extraction algorithms to segment the same manuscript, the segmentation results are more comprehensive, avoiding omissions and effectively ensuring the accuracy of the subsequently built word vector model.

[0056] For example, the first word segmentation set can be obtained by segmenting the manuscript using the Tag algorithm, while the second word segmentation set can be obtained by segmenting the manuscript using the TextRank algorithm. When using the Tag algorithm and the TextRank algorithm to segment each manuscript according to an open-source thesaurus to obtain the first and second word segmentation sets, combining the advantages of both algorithms makes the calculation results more in line with expectations.

[0057] After obtaining the first and second word segmentation sets of the manuscript, word segmentation can be filtered from the first and second word segmentation sets to obtain the final word segmentation data. In one possible implementation, the filtering of word segmentation from the first and second word segmentation sets can be based on the IDF value of each word in the word segmentation set.

[0058] That is, based on the information database, the IDF values ​​of each word in the first word segmentation set and the second word segmentation set are calculated respectively. Then, the words in the first word segmentation set are sorted according to their IDF values ​​to obtain the first word segmentation sequence, and the first n words are selected as the first words to be extracted. Simultaneously, the words in the second word segmentation set are sorted according to their IDF values ​​to obtain the second word segmentation sequence, and the first n words are selected as the second words to be extracted. It should be noted that when sorting the words according to their IDF values, they can be sorted in descending order of their IDF values.

[0059] It should also be noted that n > N, where N is the preset number.

[0060] After selecting n first-to-extract words from the first word segmentation set and n second-to-extract words from the second word segmentation set using the above method, the operation of taking the first N words from the word segmentation set consisting of the n first-to-extract words and the n second-to-extract words as the preset number of words can be performed, thereby obtaining the final word segmentation result of the manuscript.

[0061] When selecting the first N segments from the segmentation set consisting of n first words to be extracted and n second words to be extracted as the preset number of segments, the following method can be used.

[0062] First, the n first words to be extracted and the n second words to be extracted are merged to obtain a set of words to be extracted. Then, the weight of each word segment in the set of words to be extracted is calculated to obtain the weight of each word segment. Then, according to the weight of each word segment in the sequence of words to be extracted, the words in the sequence of words to be extracted are sorted, and the top N words in the sorting result are extracted as the final words to be extracted.

[0063] When calculating the weight value of each word segment in the word set to be extracted, the TF-IDF value of each word segment can be calculated, and the calculated TF-IDF value of each word segment can be used as the weight value of each word segment.

[0064] In one possible implementation, when using the TF-IDF value of each word segment as the weight value of each word segment to extract words, it also includes weighting the words in the word set to be extracted, and then calculating the weight value of the word segment based on the weight coefficient obtained after weighting.

[0065] It should be noted here that the weighting of word segments in the extracted word set can be achieved in the following way.

[0066] That is, firstly, word segments that meet the weighting requirements are selected from the set of words to be extracted. Then, the selected word segments that meet the weighting requirements are subjected to the corresponding weighting process to obtain the corresponding weight coefficients. Finally, the weight value of the word segment is calculated based on the obtained weight coefficients and the TF-IDF value of the word segment.

[0067] Among them, word segments that meet the requirements for preprocessing include at least one of the following: repeated words, entity words, and long word segments. Repeated words refer to word segments that are repeated in the set of words to be extracted. Furthermore, different types of word segments that meet the requirements for preprocessing will have different preprocessing methods.

[0068] Specifically, for repeated words, weighting coefficients can be set customwise. For entity words, weighting can be increased by multiplying the weight by x for place names (ns), y for organization names (nt), and z for Xinhua Finance custom terms (nk). Here, x, y, and z are all weighting coefficients. For long word segments, the weighting coefficient can be determined by multiplying the number of characters by p. In one possible implementation, x can be set to 6–10, preferably 10; y can be set to 3–5, preferably 5; z can be set to 5–8, preferably 6; and p can be set to 1–1.5, preferably 1.2.

[0069] It should also be noted that for word segments selected from the word set to be extracted that meet the weighting requirements, after determining the corresponding weight coefficients using any of the methods mentioned above, the weight value of the word segment can be directly calculated using multiplication based on the determined weight coefficients and the TF_IDF value of the word segment. That is, weight coefficient * TF_IDF. For word segments in the word set to be extracted that do not meet the weighting requirements, the TF_IDF value can be directly used as the weight value.

[0070] Furthermore, when sorting the segments in the word sequence to be extracted according to their weights, they can be arranged in ascending order of weight value from largest to smallest. Then, the top N segments are selected as preset segments for word vector construction. The value of N can be set to 5-10, preferably 10.

[0071] Furthermore, based on the IDF values ​​of each word in the first word segmentation set and the IDF values ​​of each word in the second word segmentation set, the words in the first word segmentation set and the second word segmentation set are sorted respectively. After selecting the top n words in the sorting as the first word to be extracted and the second word to be extracted respectively, before merging the n first words to be extracted and the n second words to be extracted into the word to be extracted set, a preprocessing operation is also included for the first words to be extracted.

[0072] The preprocessing of the first set of words to be extracted includes: deleting non-allowpos words (allowpos words are nouns, verbs, proper nouns, etc., that can express the semantic meaning of the content) from the first set of words to be extracted. That is, the non-allowpos words are deleted from the n first set of words to be extracted obtained by word segmentation of the manuscript using the Tag algorithm, and then the remaining first set of words to be extracted are combined with the n second set of words to be extracted to obtain the set of words to be extracted. This can effectively reduce the amount of computation while ensuring the accuracy of the word vector model.

[0073] Additionally, it's important to note that after weighting the words in the word set to be extracted to obtain weight coefficients, and then calculating the weight values ​​of the extracted words based on these weight coefficients, a smoothing process is performed on the resulting weight values ​​using an activation function. The activation function is defined as "math.pow(0.9,index)*value". Here, index is the index of the word's position after sorting, the purpose of which is to exponentially reduce the weight of words ranked lower. value is a fixed value of 0.001.

[0074] Furthermore, when performing weighting processing on word segments in the word set that meet the weighting requirements to obtain weight coefficients, the process also includes: directly multiplying the custom word weights (custom weight * TF-IDF value). Simultaneously, for word segments in the word set that have agreed-upon synonym replacements, the word segments in the word set are directly replaced with their synonyms.

[0075] After obtaining the word segments used to build the word vector model through any of the above methods, the vector values ​​of each word segment can be directly obtained from the open-source lexicon, and then the word vector model can be built based on the obtained tags, word segments and vector values.

[0076] When building the word vector model, for each tag category, a corresponding word segment and its corresponding vector value are matched. In other words, for each tag category, a corresponding word segment and vector value are matched, forming a dataset containing tag categories, word segments, and vector values, with each tag category corresponding to each word segment in a one-to-one manner. The articles in the news database are all signed-off articles, each with its corresponding tag category. A dataset is constructed based on the article's tag category, the word segments obtained from the article, and the vector values ​​of those word segments.

[0077] For example, an information database contains N' tag categories and a large number of issued articles, each corresponding to at least one tag category. Article A in the information database is segmented into 10 words: Top0Word, Top1Word, Top2Word...Top9Word, with vector values ​​Value0, Value1, Value2...Value9. Article A belongs to the tag categories Category0, Category2, and Category6. The dataset constructed from article A is as follows:

[0078] {[[Category0,Top0Word,Value0],……[Category0,Top9Word,Value9]]

[0079] [[Category2,Top0Word,Value0],……[Category2,Top9Word,Value9]]

[0080] [[Category6,Top0Word,Value0],……[Category6,Top9Word,Value9]]}

[0081] Based on the above method, the remaining articles in the information database are processed in the same way. Then, the data sets of each article are arranged and combined to obtain the word vector model. For the same word segmentation in different articles that belong to the same tag category, only one is retained, and its corresponding vector value is the sum of the vector values ​​of the two identical word segments.

[0082] For example, the 10 word segments of manuscript B are Top1Word, Top2Word, Top10Word, Top11Word...Top18Word, and their vector values ​​are Value1, Value2, Value10, Value11...Value18, respectively. Manuscript B belongs to the following tag categories: Category0, Category1, and Category2. The dataset constructed from manuscript B is as follows:

[0083] {[[Category0,Top1Word,Value1],……[Category0,Top18Word,Value18]]

[0084] [[Category1,Top1Word,Value1],……[Category1,Top18Word,Value18]]

[0085] [[Category2,Top1Word,Value1],……[Category2,Top18Word,Value18]]}

[0086] By merging the datasets of manuscript A and manuscript B, we obtain a word vector model based on manuscript A and manuscript B. This word vector model can be represented as:

[0087] {[[Category0,Top0Word,Value0],[Category0,Top1Word,Value1+Value1],[Category0,Top2Word,Value2+Value2]……[Category0,Top18Word,Value18]]

[0088] [[Category1,Top1Word,Value1],……[Category1,Top18Word,Value18]]

[0089] [[Category2,Top0Word,Value0],[Category2,Top1Word,Value1+Value1],[Category2,Top2Word,Value2+Value2]……[Category2,Top18Word,Value18]]

[0090] [[Category6,Top0Word,Value0],……[Category6,Top9Word,Value9]]}

[0091] Based on the above method, the data sets of all articles in the information database are merged to obtain the final word vector model.

[0092] For the completed word vector model, this disclosure also includes a step of updating the word vector model; wherein, when updating the word vector model, it is based on newly added articles in the information database. The newly added articles in the information database are processed to build a dataset, and the resulting dataset is merged into the word vector model to complete the update of the word vector model.

[0093] This disclosure also provides a method for manuscript processing based on a word vector model, including: obtaining the current manuscript to be processed; calling a pre-built word vector model to extract the parameter information required for the current manuscript processing from the word vector model; and using the parameter information to process the manuscript accordingly; wherein the word vector model is built using the above-mentioned construction method.

[0094] In one possible implementation, the processing of manuscripts includes: assigning manuscripts to recommended categories. As the number of new manuscripts increases, the number of tagged categories that need to be maintained also grows. Editors, as content producers, need to push manuscripts to appropriate categories for readers. At this point, setting up the categories to be assigned becomes a time-consuming and labor-intensive task. Using the word vector model disclosed herein, and with the assistance of certain rules, editors can be quickly assisted in recommending tagged categories.

[0095] In one possible implementation, when recommending a category for a manuscript to be processed, the process includes: constructing a matrix of word segments and word vectors for each tag category in the word vector model, and calculating the feature value of each tag category. Using the methods described in steps S100 and S200 above, a word matrix for the manuscript to be processed is constructed. It will be understood by those skilled in the art that the constructed word matrix is ​​a matrix composed of word segments and their vector values ​​extracted from the manuscript to be processed. The number of extracted word segments is preset, such as 10.

[0096] Then, the cosine similarity algorithm is used to calculate the similarity between the feature value of each issued category and the word matrix of the manuscript to be processed; issued categories with a similarity greater than a preset value are recommended categories for the manuscript to be processed. Specifically, all data sets containing the same tag category are extracted from the word vector matrix, and these extracted data sets constitute the matrix of the current tag category. The similarity between the feature vectors of the word matrix of the manuscript to be processed and the feature values ​​of the matrix of each tag category is calculated, and the top n most similar tag categories are selected as recommended categories for the manuscript to be processed. The value of n can be set according to the actual applicable situation.

[0097] Furthermore, according to another aspect of this disclosure, a manuscript processing apparatus 100 is also provided, see [reference]. Figure 2 The manuscript processing apparatus 100 of this embodiment includes: a word segmentation extraction module 110, a word vector model building module 120, and a manuscript processing module 130. The word segmentation extraction module 110 is configured to obtain the tag categories required for building the word vector model from an information database, and extract a preset number of word segments and vector values ​​of each word segment from each manuscript stored in the information database. The word vector model building module 120 is configured to build a word vector model based on a data set composed of tag categories, word segments, and vector values. The manuscript processing module 130 is configured to extract the parameter information required for processing the current manuscript from the word vector model, and use the parameter information to perform corresponding processing on the manuscript to be processed.

[0098] Furthermore, according to another aspect of this disclosure, a manuscript processing device 200 is also provided. See also... Figure 3The manuscript processing device 200 of this disclosure includes a processor 210 and a memory 220 for storing executable instructions of the processor 210. The processor 210 is configured to implement any of the aforementioned methods for manuscript processing based on a word vector model when executing the executable instructions.

[0099] It should be noted here that the number of processors 210 can be one or more. Furthermore, the document processing device 200 of this embodiment may also include an input device 230 and an output device 240. The processors 210, memory 220, input device 230, and output device 240 can be connected via a bus or other means, without specific limitations here.

[0100] The memory 220, as a computer-readable storage medium, can be used to store software programs, computer-executable programs, and various modules, such as the program or module corresponding to the manuscript processing method of this embodiment. The processor 210 executes various functional applications and data processing of the manuscript processing device 200 by running the software program or module stored in the memory 220.

[0101] Input device 230 can be used to receive input digital numbers or signals. These signals may include key signals related to user settings and function control of the device / terminal / server. Output device 240 may include a display device such as a screen.

[0102] According to another aspect of this disclosure, a non-volatile computer-readable storage medium is also provided, on which computer program instructions are stored, which, when executed by processor 210, implement any of the preceding manuscript processing methods.

[0103] The various embodiments of this disclosure have been described above. These descriptions are exemplary and not exhaustive, nor are they limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is chosen to best explain the principles, practical application, or technical improvements to the embodiments in the market, or to enable others skilled in the art to understand the embodiments disclosed herein.

Claims

1. A method for constructing a word vector model for manuscript processing, characterized in that, include: The necessary tag columns are obtained from the information database to build the word vector model, and a preset number of word segments and the vector values ​​of each word segment are extracted from each article stored in the information database. For each of the aforementioned tag categories, the corresponding word segmentation and vector value are matched to form a data set containing the tag category, the word segmentation, and the vector value; Based on the constructed data set, the word vector model is constructed. The word vector model is a matrix formed by multiple datasets; Specifically, when constructing the word vector model based on the constructed data set, it is achieved by merging the data sets of all articles in the information database; When extracting a preset number of word segments and the vector values ​​of each word segment from the manuscripts stored in the information database, the results are obtained based on an open-source lexicon. When extracting a preset number of words from each manuscript based on an open-source thesaurus, the following steps are included: The manuscript is segmented using the word segmentation toolkit in the lexicon, and a first word segmentation set and a second word segmentation set are extracted from the manuscript; wherein the first word segmentation set and the second word segmentation set are obtained using different word segmentation extraction algorithms. Based on the information database, the IDF values ​​of each word in the first word segmentation set and the IDF values ​​of each word in the second word segmentation set are calculated respectively. The first word segmentation sequence is obtained by sorting the words in the first word segmentation set according to the size of their IDF values, and the first n words in the first word segmentation sequence are selected as the first words to be extracted. The words in the second word segmentation set are sorted according to their IDF values ​​to obtain a second word segmentation sequence. The first n words in the second word segmentation sequence are selected as the second words to be extracted. Where n > N, and N is the preset number; The first N words are selected from the word segmentation set consisting of n first words to be extracted and n second words to be extracted as a preset number of word segments. When selecting the first N words from the word segmentation set consisting of n first words to be extracted and n second words to be extracted as the preset number of word segments, the weight value of each word in the word segmentation set is calculated to obtain the weight of each word segment. Then, according to the weight of each word segment in the word segmentation sequence, the word segments in the word segmentation sequence are sorted, and the word segments that rank in the top N of the sorted order are extracted from the word segmentation sequence as the final required word segments.

2. The method for building a word vector model for manuscript processing according to claim 1, characterized in that, For each of the aforementioned tag categories, each tag category corresponds to a matching word segment and a matching vector value. When a data set containing the tag category, the word segment, and the vector value is formed, each tag category corresponds one-to-one with each of the aforementioned word segments.

3. The method for building a word vector model for manuscript processing according to any one of claims 1 to 2, characterized in that, It also includes the step of updating the word vector model; Specifically, when updating the word vector model, it is based on newly added articles in the information database.

4. A method for manuscript processing based on a word vector model, characterized in that, The word vector model is constructed using the construction method described in any one of claims 1 to 3, including: Get the currently pending manuscripts; The pre-built word vector model is invoked, and the parameter information required for processing the current manuscript is extracted from the word vector model. The manuscript to be processed is processed accordingly using the parameter information.

5. The method according to claim 4, characterized in that, When processing the manuscript to be processed, the following steps are included: The manuscripts awaiting processing will be recommended for signing off in the relevant section.

6. The method according to claim 5, characterized in that, When recommending manuscripts for publication in designated sections, the following steps are included: In the word vector model, the keywords and word vectors under each issuance category constitute the matrix of the current issuance category, and the feature value of each issuance category is calculated. Construct a word matrix for the manuscript to be processed based on the manuscript to be processed; Calculate the similarity between the feature value of each issued column and the word matrix of the manuscript to be processed; The category for issuing documents with a similarity greater than a preset value is the recommended category for the manuscript to be processed.

7. A manuscript processing device, characterized in that, include: The module comprises a word segmentation and extraction module, a word vector model building module, and a manuscript processing module; the word vector model is built using the building method described in any one of claims 1 to 3. The word segmentation extraction module is configured to obtain the tag columns required for building the word vector model from the information database, and extract a preset number of word segments and the vector values ​​of each word segment from each article stored in the information database. The word vector model building module is configured to build the word vector model based on the data set consisting of the tag column, the word segmentation, and the vector values. The manuscript processing module is configured to extract the parameter information required for the current manuscript processing from the word vector model, and use the parameter information to process the manuscript accordingly.

8. A manuscript processing device, characterized in that, include: processor; Memory used to store processor-executable instructions; The processor is configured to implement the method of any one of claims 1 to 6 when executing the executable instructions.

Images