Functional label classification method and system based on text content analysis

Abstract

The invention discloses a function label classification method and system based on text content analysis. The method comprises the steps that text content information is collected; the text content is disassembled and analyzed, and a label is inserted into each disassembled part according to an analysis result; and carrying out statistics on the labels, and analyzing a statistical result to obtain a classification result of the text content. The text classification precision is improved, effective processing of text complex semantics and similarity between sentences is achieved, and it is ensured that the classification result is stable and reliable. And multi-layer and complex-structure texts can be processed, so that the classification result better conforms to the actual content. And calculating the importance coefficient of each label in the full text by utilizing a macroscopic statistical method, comprehensively considering the importance of each label in different levels of texts, and accurately determining a full text classification result. And a detailed retrieval tag system is established, so that quick retrieval and positioning of subsequent information are facilitated, and the efficiency and accuracy of information retrieval are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of text content analysis, and in particular to a method and system for classifying job labels based on text content analysis. Background Art

[0002] With the rapid development and widespread application of information technology, the amount and complexity of text data are increasing. How to effectively extract valuable information from massive text data has become an important research direction in the field of information processing. Among the many text analysis technologies, the functional label classification method based on text content has important application value, such as document management, information retrieval, public opinion analysis, automatic summarization and text classification.

[0003] Traditional text classification methods mainly rely on keyword matching and statistical learning models. Although these methods can achieve text classification to a certain extent, they often have problems such as low accuracy, poor robustness, and difficulty in dealing with multi-level text structures when processing complex texts. Therefore, proposing a functional label classification method based on text content analysis to improve the accuracy and effectiveness of text classification is a hot topic in current research. Summary of the invention

[0004] In view of the above-mentioned problems, the present invention is proposed.

[0005] Therefore, the technical problem solved by the present invention is that the existing function label classification method has the problems of poor flexibility and low accuracy.

[0006] In order to solve the above technical problems, the present invention provides the following technical solutions: a method for classifying job labels based on text content analysis, comprising:

[0007] Collect text content information;

[0008] Disassemble and analyze the text content, and insert labels into each disassembled part according to the analysis results;

[0009] The tags are counted and the statistical results are analyzed to obtain the classification results of the text content.

[0010] As a preferred solution of the method for classifying job labels based on text content analysis described in the present invention, the text content includes: importing the text into the system, the system identifying the text information, and preprocessing the text;

[0011] The preprocessing includes using a pre-trained BERT model to identify each level of title and text in the text, and stratify according to the title level;

[0012] During the stratification process, the titles are arranged from high-level titles to low-level titles according to their levels; the main text is given an attention weight of 1, and the lowest-level title is given an attention weight of 2. According to the order of arrangement, the attention weight increases by 1 for each higher title level. The attention weight of the highest-level title is set to L.

[0013] After the stratification is completed, the text content of the title and body content are disassembled separately.

[0014] As a preferred solution of the function label classification method based on text content analysis described in the present invention, the text content decomposition includes, for the content of the same paragraph, dividing the text content into sentences to obtain a sentence set {S 1 ,S 2 ,…,S n};

[0015] Using the Word2Vec algorithm, each sentence S i Convert to vector representation V(S i );

[0016]

[0017] Among them, W(S i ) represents sentence S i The word set in, v(w) represents the word vector of word w;

[0018] Select sentence j in the text that is adjacent to sentence i and calculate the cosine similarity θ(S i ,S j ):

[0019]

[0020] Construct sentence similarity matrix M, M[i][j] = θ(S i ,S j );

[0021] Where M[i][j] represents the element in the i-th row and j-th column of the matrix M;

[0022] Set the merge threshold like Then sentence i and sentence j are merged into the same sentence set; if Then sentence i and sentence j are separated so that sentence i and sentence j are in two different sentence sets respectively;

[0023] After the decomposition is completed, content analysis is performed on each set of decomposition sentences.

[0024] As a preferred solution of the method for classifying job labels based on text content analysis described in the present invention, wherein: the content analysis includes preprocessing the sentences in each sentence set, including removing stop words and punctuation marks;

[0025] Use the Word2Vec algorithm to convert each sentence in the sentence set into a vector representation, and take the average of the sentences in each sentence set to obtain the vector representation of the sentence set;

[0026]

[0027] Among them, C k represents the k-th sentence set, V(S i ) represents sentence S i A vector of

[0028] Vectorize the preset labels and calculate the cosine similarity between each sentence set and the classification label;

[0029]

[0030] Among them, L m Represents the vector representation of the mth preset label, similarity(C k ,L m ) represents the k-th sentence set and label L m similarity; take similarity(C k ,L m ) is used as the label selection basis to generate the classification label of the sentence set.

[0031] As a preferred solution of the method for classifying job labels based on text content analysis of the present invention, the content analysis further includes: counting the number of characters in the text under each title, recorded as D; counting the number of characters in each sentence set, recorded as J;

[0032] Calculate the proportion of each sentence set under its respective title:

[0033]

[0034] Calculate the proportion of the title of each sentence set in the full text:

[0035]

[0036] Where T represents the number of characters in the full text;

[0037] Insert the proportion P of the sentence set under the title into the classification label corresponding to each sentence set, and complete the final classification through macro statistics of the full text.

[0038] As a preferred solution of the method for classifying job labels based on text content analysis described in the present invention, wherein: the macro statistics of the entire text includes counting all labels in the entire text to obtain N types of labels;

[0039] Calculate the importance coefficient of each tag in the full text:

[0040]

[0041] Among them, P q Indicates the proportion of the sentence set containing the qth tag in the current tag category under the corresponding title; F q Indicates the attention weight of the title of the qth tag in the current tag category; Z q Indicates the proportion of the content matched by the qth tag in the current tag type in the full text; Q indicates the number of tags in the current tag type;

[0042] Cluster all the tags in the text to get M clustering results, and calculate the importance coefficient of all tags in each clustering result in the full text:

[0043]

[0044] Among them, P u Indicates the proportion of the sentence set containing the u-th label in the corresponding title in the current clustering result; F u Indicates the attention weight of the title of the u-th tag in the current tag category; Z u It indicates the proportion of the content matched by the u-th tag in the current tag type in the whole text; U indicates the number of tags in the current clustering result.

[0045] As a preferred solution of the method for classifying job labels based on text content analysis described in the present invention, the final classification includes a preset threshold value δ as a standard for judging whether the clustering effect is available; if Y>δ, the classification label of the cluster center of the clustering result is judged as the final classification result;

[0046] If Y≤δ, the clustering effect is judged to be unavailable. The text is classified according to the importance coefficient ZY of each label in the full text, and the classification label corresponding to the maximum value of ZY is selected as the final classification result;

[0047] The final classification result is output as the total classification result of the text content; at the same time, ZY≥ All tags of are used as searchable tags for text content;

[0048] When searching, you can locate the current text by entering the tag content.

[0049] A job label classification system based on text content analysis using the method of the present invention is characterized by:

[0050] A collection unit, collecting text content information;

[0051] The label unit disassembles and analyzes the text content and inserts labels into each disassembled part according to the analysis results;

[0052] The classification unit counts the tags and analyzes the statistical results to obtain the classification results of the text content.

[0053] A computer device comprises: a memory and a processor; the memory stores a computer program, wherein: the processor implements the steps of any one of the methods of the present invention when executing the computer program.

[0054] A computer-readable storage medium stores a computer program, wherein: when the computer program is executed by a processor, the steps of any one of the methods of the present invention are implemented.

[0055] Beneficial effects of the present invention: The functional label classification method based on text content analysis provided by the present invention uses the pre-trained BERT model to perform text preprocessing and stratification, accurately identifies and distinguishes titles and texts at all levels in the text, and improves the accuracy of text classification. Combined with the Word2Vec algorithm and cosine similarity calculation, it realizes effective processing of complex semantics of text and similarity between sentences, ensuring that the classification results are stable and reliable. By assigning different attention weights to titles and texts, it is possible to process multi-level and complex structured texts, making the classification results more consistent with the actual content. Using macro-statistical methods, the importance coefficient of each label in the full text is calculated, and the importance of each label in different levels of text is comprehensively considered to accurately determine the full-text classification results. Establishing a detailed retrieval label system facilitates the rapid retrieval and positioning of subsequent information, and improves the efficiency and accuracy of information retrieval. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative work.

[0057] Figure 1 An overall flow chart of a method for classifying functional labels based on text content analysis provided in the first embodiment of the present invention. DETAILED DESCRIPTION

[0058] In order to make the above-mentioned purposes, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are described in detail below in conjunction with the drawings of the specification. Obviously, the described embodiments are part of the embodiments of the present invention, but not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary persons in the art without creative work should fall within the scope of protection of the present invention.

[0059] Example 1, reference Figure 1 , as an embodiment of the present invention, provides a method for classifying job labels based on text content analysis, comprising:

[0060] S1: Collect text content information.

[0061] The text content includes importing the text into the system, the system identifying the text information, and preprocessing the text.

[0062] The preprocessing includes using a pre-trained BERT model to identify each level of title and text in the text, and stratify them according to the title level.

[0063] The specific BERT model processing process is to load the pre-trained BERT model and word segmenter. Segment the text into sentences or paragraphs. Use the BERT model to encode each sentence and obtain the embedded vector representation of each sentence.

[0064] During the stratification process, the titles are arranged from high-level titles to low-level titles according to their levels; the attention weight is given to the main text as 1, and the attention weight is given to the lowest-level title as 2. According to the order of arrangement, the attention weight increases by 1 for each higher title level, and the attention weight of the highest-level title is set to L. After the stratification is completed, the text content of the title and the main text is disassembled separately.

[0065] S2: Disassemble and analyze the text content, and insert labels into each disassembled part according to the analysis results.

[0066] The text content decomposition includes, for the content of the same paragraph, dividing the text content into sentences to obtain a sentence set {S 1 ,S 2 ,…,S n}.

[0067] Using the Word2Vec algorithm, each sentence S i Convert to vector representation V(S i ).

[0068]

[0069] Among them, W(S i ) represents sentence S i The word set in , v(w) represents the word vector of word w.

[0070] Select sentence j in the text that is adjacent to sentence i and calculate the cosine similarity θ(S i ,S j ):

[0071]

[0072] Construct sentence similarity matrix M, M[i][j] = θ(S i ,S j ).

[0073] Among them, M[i][j] represents the element in the i-th row and j-th column of the matrix M.

[0074] Set the merge threshold like Then sentence i and sentence j are merged into the same sentence set; if Then, sentence i and sentence j are decomposed so that sentence i and sentence j are in two different sentence sets. After the decomposition is completed, content analysis is performed on each decomposed sentence set.

[0075] It should be said that by segmenting the text into sentences and converting them into vector representations, the semantic relationship between sentences can be captured more accurately. Using the Word2Vec algorithm to convert each sentence into a vector representation, high-dimensional text data can be mapped to a low-dimensional vector space, which facilitates the subsequent calculation of the cosine similarity between sentences. By constructing a sentence similarity matrix, the similarity between sentences can be intuitively represented. Set a merging threshold, merge similar sentences into a sentence set based on similarity, and process dissimilar sentences separately. This method can effectively aggregate sentences with similar semantics, avoid information loss and redundancy, and improve the accuracy and efficiency of text content analysis. Finally, an in-depth content analysis is performed on the disassembled sentence set to lay the foundation for subsequent label classification and information extraction. This processing method can not only better understand the details of the text, but also improve the accuracy and robustness of text classification.

[0076] The content analysis includes preprocessing the sentences in each sentence set, including removing stop words and punctuation marks, converting each sentence in the sentence set into a vector representation using the Word2Vec algorithm, and averaging the sentences in each sentence set to obtain the vector representation of the sentence set.

[0077]

[0078] Among them, C krepresents the k-th sentence set, V(S i ) represents sentence S i Vector.

[0079] Vectorize the preset labels and calculate the cosine similarity between each sentence set and the classification label.

[0080]

[0081] Among them, L m Represents the vector representation of the mth preset label, similarity(C k ,L m ) represents the k-th sentence set and label L m similarity; take similarity(C k ,L m ) is used as the label selection basis to generate the classification label of the sentence set.

[0082] The number of characters in the text under each title is counted and recorded as D; the number of characters in each sentence set is counted and recorded as J.

[0083] Calculate the proportion of each sentence set under its respective title:

[0084]

[0085] Calculate the proportion of the title of each sentence set in the full text:

[0086]

[0087] Where T represents the number of characters in the full text. The proportion P of the sentence set under the title is inserted into the classification label corresponding to each sentence set, and the final classification is completed through macro statistics of the full text.

[0088] It should be said that by preprocessing each sentence set, removing stop words and punctuation, and converting the sentences into vector representations using the Word2Vec algorithm, the semantic relationship between sentences can be better captured. Taking the average of the sentence vectors in each sentence set to obtain the vector representation of the sentence set helps to simplify and aggregate information, and facilitates the subsequent calculation of the cosine similarity with the classification label. Vectorizing the preset labels and calculating the similarity between each sentence set and the classification label can determine the most matching classification label through the maximum value of the similarity, thereby improving the accuracy and reliability of the classification. By counting the number of characters in the text and the number of characters in the sentence set under each title and calculating their proportions, the importance of each sentence set in the overall text can be more comprehensively reflected. These proportions are inserted into the classification label, and the full text is finally classified through macro-statistical methods to ensure the comprehensiveness and accuracy of the classification results, thereby effectively improving the accuracy and efficiency of text content analysis and classification.

[0089] S3: Count the tags and analyze the statistical results to obtain classification results of the text content.

[0090] Count all the tags in the entire text and get N tag types.

[0091] Calculate the importance coefficient of each tag in the full text.

[0092]

[0093] Among them, P q Indicates the proportion of the sentence set containing the qth tag in the current tag category under the corresponding title; F q Indicates the attention weight of the title of the qth tag in the current tag category; Z q It indicates the proportion of the content matched by the qth tag in the current tag type in the full text; Q indicates the number of tags in the current tag type.

[0094] Cluster all the tags in the text to obtain M clustering results, and calculate the importance coefficients of all tags in each clustering result in the full text:

[0095]

[0096] Among them, P u Indicates the proportion of the sentence set containing the u-th label in the corresponding title in the current clustering result; F u Indicates the attention weight of the title of the u-th tag in the current tag category; Z u It indicates the proportion of the content matched by the u-th tag in the current tag type in the whole text; U indicates the number of tags in the current clustering result.

[0097] A threshold value δ is preset as the standard for judging whether the clustering effect is available; if Y>δ, the classification label of the cluster center of the clustering result is judged as the final classification result. If Y≤δ, the clustering effect is judged to be unavailable, and the text is classified according to the importance coefficient ZY of each label in the full text, and the classification label corresponding to the maximum value of ZY is selected as the final classification result.

[0098] The final classification result is output as the total classification result of the text content; at the same time, ZY≥ All tags are used as searchable tags for text content. When searching, you can locate the current text by entering the tag content.

[0099] It is important to say that at the macro level, a comprehensive analysis of all the tags in the full text is conducted to finally determine the most representative classification tags and provide efficient text retrieval functions. By counting all the tags in the entire text and obtaining N tag types, we can fully understand the tag distribution of the text. The importance coefficient of each tag in the full text is calculated, and the importance of each tag can be quantified by combining the proportion of the tag in the sentence set, the attention weight of the title, and the proportion of the tag matching content in the full text. All tags are clustered, the comprehensive importance coefficient of all tags in each clustering result is calculated, and a threshold δ is set to judge the clustering effect to ensure the accuracy and reliability of the classification results. If the central tag of the clustering result meets the threshold condition, it is used as the final classification result; otherwise, the importance coefficient of each tag in the full text is reclassified to ensure the objectivity of the result. The final classification result not only reflects the main content of the text, but also improves the efficiency of subsequent information retrieval through the generated retrieval tags, so that users can quickly locate relevant texts. This comprehensive analysis and classification method improves the comprehensiveness and accuracy of text content analysis, while optimizing information retrieval functions.

[0100] On the other hand, this embodiment also provides a job label classification system based on text content analysis, which includes:

[0101] A collection unit collects text content information.

[0102] The label unit disassembles and analyzes the text content, and inserts labels into each disassembled part according to the analysis results.

[0103] The classification unit counts the tags and analyzes the statistical results to obtain the classification results of the text content.

[0104] If the above functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.

[0105] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by an instruction execution system, device or apparatus (such as a computer-based system, a system including a processor, or other system that can fetch instructions from an instruction execution system, device or apparatus and execute instructions), or in conjunction with such instruction execution systems, devices or apparatuses. For the purposes of this specification, "computer-readable medium" can be any device that can contain, store, communicate, propagate or transmit a program for use by an instruction execution system, device or apparatus, or in conjunction with such instruction execution systems, devices or apparatuses.

[0106] More specific examples of computer-readable media (a non-exhaustive list) include the following: an electrical connection with one or more wires (electronic device), a portable computer disk case (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disk read-only memory (CDROM). In addition, the computer-readable medium may even be a paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, deciphering or, if necessary, processing in another suitable manner, and then stored in a computer memory.

[0107] It should be understood that the various parts of the present invention can be implemented by hardware, software, firmware or a combination thereof. In the above-mentioned embodiments, a plurality of steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0108] Example 2 is an embodiment of the present invention, which provides a method for classifying functional labels based on text content analysis. In order to verify the beneficial effects of the present invention, scientific demonstration is carried out through economic benefit calculation and simulation experiments.

[0109] Hardware environment:

[0110] CPU: Intel Core i7-10700K

[0111] Memory: 32GB RAM

[0112] Storage: 1TB SSD

[0113] GPU: NVIDIA GeForce RTX 3080

[0114] Software environment:

[0115] Operating system: Windows 10

[0116] Programming language: Python 3.8

[0117] Main libraries and tools: Transformers (BERT model), Gensim (Word2Vec), scikit-learn

[0118] Dataset: Contains different types of documents, such as technical documents, legal documents, scientific papers, and news reports, with 10 documents of each type, for a total of 40 documents.

[0119] The specific experimental steps are as follows:

[0120] Text import: Import all documents into the system. The system automatically recognizes and reads the text content.

[0121] Preprocessing: Use the pre-trained BERT model to preprocess each document, identify each level of title and text, and stratify them according to the title level. In this process, the system assigns an attention weight of 1 to the text and an attention weight of 2 to the lowest level title. The attention weight increases by 1 for each higher title level, and the attention weight of the highest level title is set to L. This stratification method can better capture the structure and hierarchical information of the text.

[0122] Text decomposition and sentence segmentation: The content of each document is segmented into sentences to obtain a sentence set. Then, the Word2Vec algorithm is used to convert each sentence into a vector representation, and the cosine similarity between each pair of sentences is calculated to construct a sentence similarity matrix. According to the preset merge threshold, sentences with similarity higher than the threshold are merged to form a sentence set.

[0123] Content analysis and label matching: Preprocess each sentence set, including removing stop words and punctuation marks. Use the Word2Vec algorithm to convert each sentence in the sentence set into a vector representation, average the sentence vectors in each sentence set, and obtain the vector representation of the sentence set. Vectorize the preset labels, calculate the cosine similarity between each sentence set and the classification label, and generate the classification label of the sentence set.

[0124] Macro statistics and classification determination: Count the number of characters in the text under each title, count the number of characters in each sentence set, and calculate the proportion of each sentence set under each title and in the full text. Insert these proportions into the classification label, and calculate the importance coefficient of each label in the full text through macro statistics of the full text. Cluster all labels, calculate the comprehensive importance coefficient of all labels in each clustering result, set the threshold δ to judge the clustering effect, and ensure the accuracy and reliability of the classification results. If the central label of the clustering result meets the threshold condition, it will be used as the final classification result; otherwise, reclassify it according to the importance coefficient of each label in the full text to ensure the objectivity of the result. The final classification result not only reflects the main content of the text, but also improves the efficiency of subsequent information retrieval through the generated retrieval labels. The recorded data is shown in Table 1.

[0125] Table 1 Data record table

[0126]

[0127]

[0128] Through manual verification, the output results of the classification labels of the present invention are completely consistent with the manual judgment.

[0129] It can be seen from the experimental data that the present invention has shown significant advantages in the classification of different types of documents. First, the number of sentence sets after merging is significantly reduced. For example, technical documents A and B merge 200 and 180 sentences into 50 and 45 sentence sets respectively, which greatly simplifies the content analysis process and improves processing efficiency. Secondly, the number of classification tags and the importance coefficient show the superiority of this method in processing complex texts. For example, the number of classification tags of scientific paper D is 7, and the importance coefficient is as high as 0.92, indicating that this method has significant advantages in texts with strong academic nature and complex information. In contrast, the number of classification tags and importance coefficients of news reports E and F are relatively low, indicating that the structure of news texts is relatively simple. When processing professional documents and texts with complex structures, the present invention can more accurately identify and distinguish key information in the text, and the output results of the classification tags are completely consistent with manual judgment, which verifies the high accuracy and reliability of this method and significantly improves the accuracy and efficiency of text classification.

[0130] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present invention, which should all be included in the scope of the claims of the present invention.

Claims

1. A method for classifying job labels based on text content analysis, characterized in that: include: Collect text content information; Disassemble and analyze the text content, and insert labels into each disassembled part according to the analysis results; The tags are counted and the statistical results are analyzed to obtain the classification results of the text content.

2. The method for classifying job labels based on text content analysis according to claim 1, characterized in that: The text content includes importing the text into the system, the system identifying the text information, and preprocessing the text; The preprocessing includes using a pre-trained BERT model to identify each level of title and text in the text, and stratify according to the title level; During the stratification process, the titles are arranged from high-level titles to low-level titles according to their levels; the main text is given an attention weight of 1, and the lowest-level title is given an attention weight of 2. According to the order of arrangement, the attention weight increases by 1 for each higher title level. The attention weight of the highest-level title is set to L. After the stratification is completed, the text content of the title and body content are disassembled separately.

3. The method for classifying job labels based on text content analysis according to claim 2, characterized in that: The text content decomposition includes, for the content of the same paragraph, dividing the text content into sentences to obtain a sentence set {S1, S2, ..., S n }; Using the Word2Vec algorithm, each sentence S i Convert to vector representation V(S i ); Among them, W(S i ) represents sentence S i The word set in, v(w) represents the word vector of word w; Select sentence j in the text that is adjacent to sentence i and calculate the cosine similarity θ(S i , S j ): Construct sentence similarity matrix M, M[i][j] = θ(S i , S j ); Where M[i][j] represents the element in the i-th row and j-th column of the matrix M; Assume the merge threshold θ, if θ(S i , S j )≤θ, then sentence i and sentence j are merged into the same sentence set; if θ(S i , S j )>θ, then sentence i and sentence j are split so that sentence i and sentence j are in two different sentence sets respectively; After the decomposition is completed, content analysis is performed on each set of decomposition sentences.

4. The method for classifying job labels based on text content analysis according to claim 3 is characterized in that: The content analysis includes preprocessing the sentences in each sentence set, including removing stop words and punctuation marks; Use the Word2Vec algorithm to convert each sentence in the sentence set into a vector representation, and take the average of the sentences in each sentence set to obtain the vector representation of the sentence set; Among them, C k represents the k-th sentence set, V(S i ) represents sentence S i A vector of Vectorize the preset labels and calculate the cosine similarity between each sentence set and the classification label; Among them, L m Represents the vector representation of the mth preset label, similarity(C k , L m ) represents the k-th sentence set and label L m similarity; take similarity(C k , L m ) is used as the label selection basis to generate the classification label of the sentence set.

5. The method for classifying job labels based on text content analysis according to claim 4, characterized in that: The content analysis also includes counting the number of characters in the text under each title, recorded as D; counting the number of characters in each sentence set, recorded as J; Calculate the proportion of each sentence set under its respective title: Calculate the proportion of the title of each sentence set in the full text: Where T represents the number of characters in the full text; Insert the proportion P of the sentence set under the title into the classification label corresponding to each sentence set, and complete the final classification through macro statistics of the full text.

6. The method for classifying job labels based on text content analysis according to claim 5, characterized in that: The macro statistics of the whole text includes counting all tags in the whole text to obtain N tag types; Calculate the importance coefficient of each tag in the full text: Among them, P q Indicates the proportion of the sentence set containing the qth tag in the current tag category under the corresponding title; F q Indicates the attention weight of the title of the qth tag in the current tag category; Z q Indicates the proportion of the content matched by the qth tag in the current tag type in the full text; Q indicates the number of tags in the current tag type; Cluster all the tags in the text to obtain M clustering results, and calculate the importance coefficients of all tags in each clustering result in the full text: Among them, P u Indicates the proportion of the sentence set containing the u-th label in the corresponding title in the current clustering result; F u Indicates the attention weight of the title of the u-th tag in the current tag category; Z u It indicates the proportion of the content matched by the u-th tag in the current tag type in the whole text; U indicates the number of tags in the current clustering result.

7. The method for classifying job labels based on text content analysis according to claim 6, characterized in that: The final classification includes a preset threshold value δ as a standard for judging whether the clustering effect is available; if Y>δ, the classification label of the cluster center of the clustering result is judged as the final classification result; If Y≤δ, the clustering effect is judged to be unavailable. The text is classified according to the importance coefficient ZY of each label in the full text, and the classification label corresponding to the maximum value of ZY is selected as the final classification result; The final classification result is output as the total classification result of the text content; at the same time, All tags of are used as searchable tags for text content; When searching, you can locate the current text by entering the tag content.

8. A job label classification system based on text content analysis using the method according to any one of claims 1 to 7, characterized in that: A collection unit, collecting text content information; The label unit disassembles and analyzes the text content and inserts labels into each disassembled part according to the analysis results; The classification unit counts the tags and analyzes the statistical results to obtain the classification results of the text content.

9. A computer device comprising: Memory and processor; The memory stores a computer program, which is characterized in that when the processor executes the computer program, the steps of the function label classification method based on text content analysis are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of a method for classifying job functions labels based on text content analysis are implemented.

Images