A large model assisted text automatic labeling method based on reverse thinking reasoning
By employing a large-scale model-assisted text automatic labeling method based on reverse reasoning, the difficulties in labeling text in standardized documents and complex industries have been resolved. This method achieves efficient association between text blocks and atomic problems, improves the accuracy and recall of labels, and meets the actual needs of the knowledge base.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- POWERCHINA HUADONG ENG CORP LTD
- Filing Date
- 2026-01-28
- Publication Date
- 2026-06-05
AI Technical Summary
Existing technologies cannot effectively perform automatic text tagging in industries with numerous standard documents and difficulties in extracting industry information triples, leading to difficulties in building knowledge bases.
We employ a large model-assisted automatic text labeling method based on reverse reasoning. This method generates text blocks through semantic segmentation, extracts keywords and constructs atomic questions, uses a large model to generate hypothetical questions and stores them in vector form, and finally constructs an association system of text blocks, atomic questions and tags.
It improves the accuracy and recall of text tags, meets the actual needs of the knowledge base in standardized documents and complex industries, and enhances the standardization and accuracy of tags.
Smart Images
Figure CN122153066A_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of artificial intelligence, specifically relating to a large-scale model-assisted automatic text labeling method based on reverse reasoning. Background Technology
[0002] Large models primarily refer to Large Language Models (LLMs). Large models typically refer to a class of artificial intelligence models with a large number of parameters, built from artificial neural networks. Large models are usually trained on massive amounts of data through self-supervised or semi-supervised learning. With technological advancements, the concepts of large visual models, multimodal large models, and large models for basic sciences have gradually expanded. Currently, large models generally refer to models with tens of billions, hundreds of billions, or even trillions of parameters. Furthermore, it is also common to refer to models with billions of parameters that have undergone large-scale pre-training (significantly more training data than traditionally pre-trained models) as large models.
[0003] Tags refer to the labels that are added to each piece of content after the knowledge base has acquired it. These labels are used to accurately locate the corresponding content when the knowledge base conducts knowledge retrieval.
[0004] Currently, various knowledge base frameworks on the market have their own tagging methods. However, for industries with many specification documents and difficulties in extracting industry information triples, there is often a problem that general tagging methods cannot fully cover the relevant content and cannot find the correct tags. A large model-assisted tagging method based on reverse reasoning can solve this pain point. Therefore, a large model-assisted automatic text tagging method based on reverse reasoning is necessary for knowledge base construction projects in industries with many specification documents and difficulties in extracting triples. Summary of the Invention
[0005] The main objective of this invention is to provide a method for automatic text labeling based on a large model assisted by reverse reasoning, addressing the problems mentioned above.
[0006] Therefore, the above-mentioned objective of the present invention is achieved through the following technical solution:
[0007] A large-scale model-assisted automatic text labeling method based on reverse reasoning includes the following steps:
[0008] S1. Receive the raw text to be processed, perform semantic block processing on the raw text, and obtain multiple independent text blocks;
[0009] S2. Based on reverse thinking logic, generate corresponding hypothetical questions for each text block. The unique answer to the hypothetical question is its corresponding text block. All possible hypothetical questions are stored as atomic questions of the text block.
[0010] S3. Construct a tagging system based on the core information features of text blocks; match the atomic questions corresponding to each text block generated in S2 to the target tags in the tagging system to complete the one-to-one correspondence and labeling of text blocks, atomic questions, and tags; verify the completed association and labeling results to check the uniqueness of the matching between atomic questions and text blocks and the rationality of the association between tags and questions; correct the labeling results that do not meet the verification conditions to obtain the final set of text block atomic question tags.
[0011] While adopting the above technical solutions, the present invention may also adopt or combine the following technical solutions:
[0012] As a preferred technical solution of the present invention: In step S1, during the process of dividing text blocks, an upper limit for the size of the text block is set. First, the text is divided into the smallest units according to the specified splitting symbols. Then, the smallest units, except for text statements whose length is greater than the upper limit of the text block size, are combined in sequence to generate text blocks. If the length of the current text block plus the statement text of the next smallest unit exceeds the upper limit of the text block size, then the statement text is used as the starting point to continue adding statement text as a new text block until all statement texts are divided.
[0013] As a preferred technical solution of the present invention: after the sentences are concatenated and divided according to the upper limit length, the large model is first used to organize the summary of each text block. Then, starting from the first text block, it is concatenated with the second text block to form a whole text block. The summary of the first text block is provided to the large model, which extracts the semantically relevant content from the concatenated text block based on the summary. The first text block of the final result is used to extract the semantically irrelevant parts and merge them with the remaining text into a whole. The text is then divided into the smallest units, the sentences are concatenated into text blocks, and a summary is generated. The first two text blocks are selected to extract a final text block according to the semantics of the summary. This process is repeated until all texts are divided.
[0014] As a preferred technical solution of the present invention, step S2 specifically includes the following steps:
[0015] S21. Extract keywords from the text block, including entities, relationships, attribute names, attribute values, etc.
[0016] S22. Determine whether there are multiple keywords among the keywords extracted in step S21 that can form a complete semantic statement, and store the complete semantic statement in the system.
[0017] S23. For the complete semantic statement extracted in step S22, without changing the keywords contained therein, convert the affirmative statement into a judgment question, and ensure that the correct judgment can be made through the text block content. Perform the judgment question extraction operation on each text block one by one.
[0018] S24. Obtain the file ID and user ID, use the MD5 encryption algorithm to generate a unique ID for the text block, and store all relevant information of the text block in the text block table;
[0019] S25. Use the MD5 encryption algorithm to generate a unique ID for the obtained hypothetical question, and use the embedding model to generate a field vector. Treat the hypothetical question as an atomic question of the text block and store it in the atomic question table along with relevant information.
[0020] S26. Associate the text block table with the atomic problem table, and add a chunkid field to the atomic problem table to store the ID of the corresponding text block.
[0021] As a preferred technical solution of the present invention: the text block table stores the following content: 1). _id, 2). _index, 3). _score, 4). doc_id, 5). kb_id, 6). docnm_kwd, 7). title_tks, 8). title_sm_tks, 9). content_with_weight, 10). content_ltks, 11). content_sm_ltks, 12). page_num_int, 13). position_int, 14). top_int, 15). create_time, 16). create_timestamp_flt, 17). img_id, 18). q_1024_vec.
[0022] As a preferred technical solution of the present invention, the atomic question table stores the following contents: 1). _id, 2). _index, 3). _score, 4). chunkid, 5). content, 6). doc_id, 7). atomquestion, 8). q_1024_vec.
[0023] As a preferred technical solution of the present invention: Step S3 specifically involves, after obtaining the atomic problem results, addressing issues such as missing keywords and unclear judgments that may occur during the extraction of atomic problems from some text blocks, by specifically changing the calling prompt words of the large model, regenerating the atomic problems for the corresponding text, and automatically updating them to the atomic problem table.
[0024] Compared with the prior art, the present invention has the following beneficial effects:
[0025] This invention leverages the language understanding capabilities of a large model. Unlike the commonly used text tagging method that directly divides text into blocks of size and then vectorizes them for recall, this invention uses reverse thinking, transforming answers into questions, starting from small to large, and constructing from keywords to structure the text content into atomic questions. Finally, vectorization is used as the tag, resulting in a significant improvement in performance compared to the commonly used method, thus meeting practical needs. Attached Figure Description
[0026] Figure 1 The flowchart shows the method for auxiliary labeling of large models based on reverse reasoning provided by this invention.
[0027] Figure 2 A flowchart for segmenting text blocks.
[0028] Figure 3 This is a JSON format storage of a text block in the example.
[0029] Figure 4 This is the table of contents for the first 20 text blocks in the example.
[0030] Figure 5 This is a text block information table in the embodiment.
[0031] Figure 6 This is an atomic problem information table in the embodiment.
[0032] Figure 7 This is a JSON format storage of a certain atom in the example. Detailed Implementation
[0033] The present invention will now be described in further detail with reference to the accompanying drawings and specific embodiments.
[0034] like Figure 1 As shown, a large-scale model-assisted automatic text labeling method based on reverse reasoning includes the following steps:
[0035] S1. Receive the raw text to be processed, perform semantic block processing on the raw text, and obtain multiple independent text blocks;
[0036] During the process of dividing text blocks, an upper limit for the size of the text block is set. First, the text is divided into the smallest units according to the specified splitting symbols. Then, the smallest units, except for text statements whose length is greater than the upper limit of the text block size, are combined in turn to generate text blocks. If the length of the current text block plus the text of the next smallest unit exceeds the upper limit of the text block size, then the text of this text block is used as the starting point to continue adding text statements until all text statements are divided.
[0037] Text is divided into its smallest units based on line breaks and punctuation marks. Punctuation marks used for division include periods, question marks, exclamation marks, semicolons, and ellipses, but not commas. Tables of all sizes are not divided. Units containing no content are deleted.
[0038] The maximum text block size is set to 512, with no minimum size. When concatenating the smallest units into a text block, if the first smallest unit exceeds the maximum text block size, it will not be split or further concatenated; it will be output as a single text block. For the smallest units generated from separate tables, they will not be concatenated and will be output as a single text block.
[0039] After segmenting the sentences according to the upper limit length, the large model first organizes the summaries of each text block. Then, starting from the first text block, it is concatenated with the second text block to form a whole text block. The summary of the first text block is provided to the large model, which extracts the semantically relevant content from the concatenated text block based on the summary, and uses it as the first text block of the final result. The semantically irrelevant parts are extracted and merged back into a whole with the remaining text. The text is then further segmented into the smallest units, and the sentences are concatenated into text blocks to generate summaries. The first two text blocks are selected and a final text block is extracted according to the semantics of the summary. This process is repeated until all text is segmented.
[0040] S2. Based on reverse thinking logic, generate corresponding hypothetical questions for each text block. The unique answer to the hypothetical question is its corresponding text block. All possible hypothetical questions are stored as atomic questions of the text block.
[0041] The process of generating atomic questions that can be answered by text blocks based on keywords is as follows: adjacent semantically continuous keywords cannot be split into multiple keywords, but are treated as a single long keyword. There is no limit to the number of keywords in a text block; all concepts, formulas, and classifications are expressed in the keywords, ensuring accuracy. Pronouns such as "he," "she," "they," "company," and "person" will not appear in the keywords; instead, referential terms will be used.
[0042] In the generated atomic questions, the subject, predicate, and object are all extracted keywords. All atomic questions are interrogative sentences. The atomic questions are completely related to the knowledge of the relevant field. For a text block, the extracted atomic question group contains all the keywords extracted from the text block. When a keyword not included is detected, a new atomic question is automatically extracted based on that keyword and added to the atomic question group.
[0043] Both text blocks and atomic problems are encrypted and vectorized before storage. The MD5 encryption algorithm is selected to generate unique IDs. During the vectorization process, the same rerank model is selected to ensure that similar content has a consistent trend in the vector.
[0044] S21. Extract keywords from the text block, including entities, relationships, attribute names, attribute values, etc.
[0045] S22. Determine whether there are multiple keywords among the keywords extracted in step S21 that can form a complete semantic statement, and store the complete semantic statement in the system.
[0046] S23. For the complete semantic statement extracted in step S22, without changing the keywords contained therein, convert the affirmative statement into a judgment question, and ensure that the correct judgment can be made through the text block content. Perform the judgment question extraction operation on each text block one by one.
[0047] S24. Obtain the file ID and user ID, use the MD5 encryption algorithm to generate a unique ID for the text block, and store all relevant information of the text block in the text block table;
[0048] S25. Use the MD5 encryption algorithm to generate a unique ID for the obtained hypothetical question, and use the embedding model to generate a field vector. Treat the hypothetical question as an atomic question of the text block and store it in the atomic question table along with relevant information.
[0049] S26. Associate the text block table with the atomic problem table, and add a chunkid field to the atomic problem table to store the ID of the corresponding text block.
[0050] The text block table stores the following content: 1) _id: Unique ID corresponding to the text block; 2) _index: Text block table name; 3) _score: Rating; 4) doc_id: Article ID; 5) kb_id: Knowledge base ID; 6) docnm_kwd: Article name; 7) title_tks: Filename segmentation result; 8) title_sm_tks: Further segmentation based on filename segmentation result; 9) content_with_weight: Main text; 10) content_ltks: Main text segmentation result; 11) content_sm_ltks: Further segmentation based on main text segmentation result; 12) page_num_int: Page number; 13) position_int: Position (page number and horizontal and vertical start and end coordinates); 14) top_int: Vertical start value; 15) create_time: Creation time; 16) create_timestamp_flt: Creation timestamp ID; 17) img_id: Image corresponding to OCR; 18) q_1024_vec chunk vector.
[0051] The atomic question table stores the following information: 1) _id: the unique ID of the atomic question; 2) _index: the name of the atomic question table; 3) _score: the score; 4) chunkid: the ID of the corresponding text block; 5) content: the content of the corresponding text block; 6) doc_id: the article ID; 7) atomquestion: the content of the atomic question; 8) q_1024_vec: the vector of atomic questions.
[0052] S3. Construct a tagging system based on the core information features of text blocks; match the atomic questions corresponding to each text block generated in S2 to the target tags in the tagging system to complete the one-to-one correspondence and labeling of text blocks, atomic questions, and tags; verify the completed association and labeling results to check the uniqueness of the matching between atomic questions and text blocks and the rationality of the association between tags and questions; correct the labeling results that do not meet the verification conditions to obtain the final set of text block atomic question tags.
[0053] The specification document contains a large amount of professional information, among which the tables of regional type classifications and corresponding labor unit prices are quite complex, such as... Figure 3 As shown, this embodiment transforms standard text into atomic problem information usable by the knowledge base. It selects the processing of table content in classic standards as an example of auxiliary labels for large models, and provides a detailed description of the technical solution of this invention in conjunction with the accompanying drawings.
[0054] The technical solution of this invention includes three main processes: receiving the original text, dividing the text into blocks according to semantics, generating and storing atomic questions based on keywords, and fine-tuning the prompt words based on user feedback.
[0055] S1. Receive the raw text and divide it into multiple text blocks according to semantics. The specific process is as follows: Figure 2 As shown. In this embodiment, the original specification text is in PDF format. After being parsed using OCR, the text is divided into the smallest units according to line breaks and punctuation marks, totaling 659 items, including 82 table items. The maximum text block size is set to 512. In the first round, 120 text blocks are generated, including 38 plain text items and 82 table items. After 119 rounds of division according to the abstract content, 116 text blocks are finally obtained, including 34 plain text items and 82 table items. The content of the first 20 text blocks is as follows: Figure 4 show.
[0056] S2. Keyword extraction is performed on the 116 text blocks divided into standard text instances. Atomic questions are generated based on the keywords. Section 5.2.1 in the text is processed as a text block instance, resulting in 18 atomic questions. Among them, the keywords "procurement fee" and "storage fee" are two independent words, but they appear in pairs in the context and can be combined into a complete semantic. Therefore, they are used as the merged keywords according to semantics and are recorded as "procurement and storage fee".
[0057] When storing text blocks and atomic problems, extract and store them into the tables according to the attribute content required by the two tables. The content parameters of the text block table are as follows: Figure 5 As shown, there are 18 items: 1) _id: Unique ID corresponding to the text block; 2) _index: Table name of the text block; 3) _score: Score; 4) doc_id: Article ID; 5) kb_id: Knowledge base ID; 6) docnm_kwd: Article name; 7) title_tks: Filename segmentation result; 8) title_sm_tks: Further segmentation based on filename segmentation result; 9) content_with_weight: Main text; 10) content_ltks: Main text segmentation result; 11) content_sm_ltks: Further segmentation based on main text segmentation result; 12) page_num_int: Page number; 13) position_int: Position (page number and horizontal and vertical start and end coordinates); 14) top_int: Vertical starting value; 15) create_time: Creation time; 16) create_timestamp_flt: Creation timestamp ID; 17) img_id: Image corresponding to OCR; 18) q_1024_vec Chunk vector. Atomic problem attributes such as... Figure 6 As shown, there are 8 items: 1) _id: the unique ID of the atomic problem; 2) _index: the name of the atomic problem table; 3) _score: the score; 4) chunkid: the ID of the corresponding text chunk; 5) content: the content of the corresponding text chunk; 6) doc_id: the article ID; 7) atomquestion: the content of the atomic problem; 8) q_1024_vec: the atomic problem vector.
[0058] In this embodiment, the text block table contains 116 entries, and the atomic problem table contains 3528 entries. The number of atomic problems corresponding to each text block ranges from 8 to 72, with the average number of atomic problems generated by the table being 35.
[0059] In this embodiment, the bge-large-zh-1.5 model was used for both text blocks and atomic questions during vectorization. The similarity calculation results for similar sentences were good, improving the text recall accuracy in subsequent question answers by 50%. Both text blocks and atomic questions were stored in a table in JSON format. Figure 3 This is the JSON storage content of a certain text block in the embodiment. Figure 7 This is the JSON storage content of a certain atomic problem in the embodiment.
[0060] S3. In this embodiment, the prompt words when calling the large model are fine-tuned based on feedback from professional users. For the corresponding professional categories, system prompt words are added to the prompt words, allowing the large model to perform text summarization and keyword extraction as a new energy expert. The segmentation results are more professional than when the identity is not clearly defined, the keyword understanding is more accurate, and the generation of atomic problems is more professional.
[0061] The technical solution of the present invention has been described in conjunction with the specific experimental procedures shown in the accompanying drawings. However, the scope of protection of the present invention is not limited to these specific embodiments. Without departing from the principles of the present invention, those skilled in the art can make equivalent changes or substitutions to the relevant technical features, and the technical solutions resulting from such changes or substitutions will all fall within the scope of protection of the present invention.
Claims
1. A large-scale model-assisted automatic text labeling method based on reverse reasoning, characterized in that, Includes the following steps: S1. Receive the raw text to be processed, perform semantic block processing on the raw text, and obtain multiple independent text blocks; S2. Based on reverse thinking logic, generate corresponding hypothetical questions for each text block. The unique answer to the hypothetical question is its corresponding text block. All possible hypothetical questions are stored as atomic questions of the text block. S3. Construct a tagging system based on the core information features of text blocks; match the atomic questions corresponding to each text block generated in S2 to the target tags in the tagging system to complete the one-to-one correspondence and labeling of text blocks, atomic questions, and tags; verify the completed association and labeling results to check the uniqueness of the matching between atomic questions and text blocks and the rationality of the association between tags and questions; correct the labeling results that do not meet the verification conditions to obtain the final set of text block atomic question tags.
2. The method according to claim 1, characterized in that: In step S1, during the process of dividing text blocks, an upper limit for the size of the text block is set. First, the text is divided into the smallest units according to the specified splitting symbols. Then, the smallest units, except for text statements whose length is greater than the upper limit of the text block size, are combined in sequence to generate text blocks. If the length of the current text block plus the statement text of the next smallest unit exceeds the upper limit of the text block size, then the statement text is used as the starting point to continue adding statement text as a new text block until all statement texts are divided.
3. The method according to claim 2, characterized in that: After segmenting the sentences according to the upper limit length, the large model first organizes the summaries of each text block. Then, starting from the first text block, it is concatenated with the second text block to form a whole text block. The summary of the first text block is provided to the large model, which extracts the semantically relevant content from the concatenated text block based on the summary, and uses it as the first text block of the final result. The semantically irrelevant parts are extracted and merged back into a whole with the remaining text. The text is then further segmented into the smallest units, and the sentences are concatenated into text blocks to generate summaries. The first two text blocks are selected and a final text block is extracted according to the semantics of the summary. This process is repeated until all text is segmented.
4. The method according to claim 1, characterized in that: Step S2 specifically includes the following steps: S21. Extract keywords from the text block, including entities, relationships, attribute names, and attribute values; S22. Determine whether there are multiple keywords among the keywords extracted in step S21 that can form a complete semantic statement, and store the complete semantic statement in the system. S23. For the complete semantic statement extracted in step S22, without changing the keywords contained therein, convert the affirmative statement into a judgment question, and ensure that the correct judgment can be made through the text block content. Perform the judgment question extraction operation on each text block one by one. S24. Obtain the file ID and user ID, use the MD5 encryption algorithm to generate a unique ID for the text block, and store all relevant information of the text block in the text block table; S25. Use the MD5 encryption algorithm to generate a unique ID for the obtained hypothetical question, and use the embedding model to generate a field vector. Treat the hypothetical question as an atomic question of the text block and store it in the atomic question table along with relevant information. S26. Associate the text block table with the atomic problem table, and add a chunkid field to the atomic problem table to store the ID of the corresponding text block.
5. The method according to claim 4, characterized in that: The text block table stores the following content: 1) _id, 2) _index, 3) _score, 4) doc_id, 5) kb_id, 6) docnm_kwd, 7) title_tks, 8) title_sm_tks, 9) content_with_weight, 10) content_ltks, 11) content_sm_ltks, 12) page_num_int, 13) position_int, 14) top_int, 15) create_time, 16) create_timestamp_flt, 17) img_id, 18) q_1024_vec.
6. The method according to claim 4, characterized in that: The atomic question table stores the following contents: 1). _id, 2). _index, 3). _score, 4). chunkid, 5). content, 6). doc_id, 7). atomquestion, 8). q_1024_vec.
7. The method according to claim 1, characterized in that: Step S3 specifically involves, after obtaining the atomic problem results, addressing potential issues such as missing keywords and unclear judgments during the extraction of atomic problems from certain text blocks. This is achieved by specifically modifying the calling prompts of the large model, regenerating the atomic problems for the corresponding text, and automatically updating them in the atomic problem table.