Document title tree construction method and device, terminal equipment and storage medium
By combining the title, target title and text content of the subtitle text tree path in the document title tree construction model, semantic recognition is carried out to determine the hierarchical relationship, and the problems of low efficiency in the title tree construction and inaccurate hierarchical structure in the prior art are solved. Achieve more accurate and efficient title tree construction.
Patent Information
- Application Number
- CN202411442347.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-10-16
- Publication Date
- 2025-02-18
- Estimated Expiration
- 2044-10-16
AI Technical Summary
The existing document title tree construction technology is inefficient and the hierarchical structure is not accurate enough, which can easily lead to bias.
During non-first construction, the title on the path from the root node to the rightmost leaf node in the subtitle text tree is input together with the target title and text content into the title tree construction model, and the hierarchical relationship between the target titles is determined through semantic recognition. .
Improves the accuracy and efficiency of title tree construction, ensures coherence at the title level and reduces bias.
Smart Images

Figure CN119476269A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of computer technology, and in particular to a method, device, terminal equipment and storage medium for constructing a document title tree. Background Art
[0002] Document title recognition is a crucial technology in document processing, which mainly relies on machine learning and natural language processing (NLP) technology in the field of artificial intelligence (AI). The principle of document title recognition is mainly based on machine learning and natural language processing technology. Through a large amount of text data training, machine learning models can learn how to extract key information from documents, such as topics, keywords, etc., and generate or identify appropriate titles based on this information. Natural language processing technology is responsible for processing and understanding the language structure, semantic relationships, etc. in the document, so as to extract key information more accurately.
[0003] However, the current document title tree construction technology is to gradually build the title tree by comparing the titles at the node level, that is, by judging the parent-child relationship between the current title and each node in the title tree. This requires a large number of preset rules, resulting in a very large amount of calculation and low efficiency in building the document title tree. Moreover, the current document title tree construction method relies on the results of pairwise comparison to build a hierarchical structure, which is limited to pairwise comparison of two titles, lacks a global perspective, and is prone to incorrect associations. Therefore, a biased hierarchical structure may be generated. This may affect users' understanding and use of information. Summary of the invention
[0004] The embodiments of the present invention provide a method, apparatus, terminal device and storage medium for constructing a document title tree. When the title tree construction operation is not performed for the first time, several first titles on the path from the root node to the rightmost leaf node in the existing subtitle text tree are input into a title tree construction model together with a target title to be identified and a target text content, so that the title tree construction model determines the hierarchical relationship between the target titles according to the hierarchical relationship of the first titles, so as to ensure the hierarchical coherence between titles of different groups, thereby overcoming the defects of the current document title tree construction technology that the title hierarchy structure is not accurate or has deviations.
[0005] An embodiment of the present invention provides a method for constructing a document title tree, comprising:
[0006] Get the original file of the title tree to be built;
[0007] Convert the original file into a preset plain text format to generate a file to be identified;
[0008] According to the file to be identified, identifying a number of titles in the file to be identified, and text content corresponding to each title;
[0009] According to the plurality of titles and a preset title tree construction model, the title tree construction operation is repeatedly performed until a document title tree of the original file is generated;
[0010] The title tree construction operation includes:
[0011] According to the order of the titles in the to-be-recognized file, a plurality of target titles for which the title tree construction operation has not been performed are obtained, and the target text content corresponding to each target title is obtained;
[0012] Determine whether the title tree building operation is currently being performed for the first time;
[0013] If yes, the target title and the target text content are input into the title tree construction model, so that the title tree construction model can perform semantic recognition on the target title and the target text content, determine the hierarchical relationship between the target titles, and generate a subtitle text tree;
[0014] If not, then obtain several first titles on the path from the root node to the rightmost leaf node of the current subtitle text tree, and input the target title, the target text content, and the first title into the title tree construction model, so that the title tree construction model performs semantic recognition on the target title and the target text content, and determines the hierarchical relationship between the target titles according to the hierarchical relationship of the first titles, and then inserts the target title into the subtitle text tree according to the hierarchical relationship;
[0015] When it is confirmed that the title tree construction operation has been performed on all the titles, the latest subtitle text tree is used as the document title tree of the original file;
[0016] When it is confirmed that there are titles for which the title tree construction operation has not been performed, the title tree construction operation is repeatedly performed.
[0017] Furthermore, the step of identifying, based on the file to be identified, a number of titles in the file to be identified, and text content corresponding to each title, includes:
[0018] Get the preset prefix character matching list;
[0019] Identify each line of text in the to-be-identified file, and determine a number of prefix characters in each line of text;
[0020] Matching a number of the prefix characters with the prefix character matching list;
[0021] When it is determined that a prefix character matches successfully, the target line corresponding to the prefix character is used as the title, and the text between the target line and the next adjacent target line is used as the text content corresponding to the title.
[0022] Furthermore, the title tree construction model performs semantic recognition on the target title and the target text content, determines the hierarchical relationship between the target titles, and generates a subtitle text tree, including:
[0023] Performing semantic recognition on each of the target titles and the corresponding target text content to determine the text meaning of each target title, and determining irrelevant text in each of the target titles and the corresponding target text content that is inconsistent with the text meaning based on the text meaning;
[0024] Deleting the irrelevant text from the target title and the corresponding target text content, and generating a second title and second text content that match the meaning of the text;
[0025] According to the second titles and the second text content, the hierarchical relationship between the second titles is determined, and according to the hierarchical relationship, a subtitle text tree is generated.
[0026] Furthermore, the step of inputting the target title, the target text content, and the first title into a title tree construction model includes:
[0027] Generate a large model prompt word according to the hierarchical relationship between the first titles;
[0028] The first title and the large model prompt word are first input into the title tree construction model, and then the target title and the target text content are input into the title tree construction model.
[0029] Another embodiment of the present invention provides a device for constructing a document title tree, comprising:
[0030] A file acquisition module, used to obtain the original file of the title tree to be constructed;
[0031] A format conversion module, used to convert the original file into a preset plain text format to generate a file to be identified;
[0032] A title recognition module, used to recognize a plurality of titles in the file to be recognized and text content corresponding to each title according to the file to be recognized;
[0033] A title tree construction module, used to repeatedly perform the title tree construction operation according to a plurality of titles and a preset title tree construction model, until a document title tree of the original file is generated;
[0034] The title tree construction operation includes:
[0035] According to the order of the titles in the to-be-recognized file, a plurality of target titles for which the title tree construction operation has not been performed are obtained, and the target text content corresponding to each target title is obtained;
[0036] Determine whether the title tree building operation is currently being performed for the first time;
[0037] If yes, the target title and the target text content are input into the title tree construction model, so that the title tree construction model can perform semantic recognition on the target title and the target text content, determine the hierarchical relationship between the target titles, and generate a subtitle text tree;
[0038] If not, then obtain several first titles on the path from the root node to the rightmost leaf node of the current subtitle text tree, and input the target title, the target text content, and the first title into the title tree construction model, so that the title tree construction model performs semantic recognition on the target title and the target text content, and determines the hierarchical relationship between the target titles according to the hierarchical relationship of the first titles, and then inserts the target title into the subtitle text tree according to the hierarchical relationship;
[0039] When it is confirmed that the title tree construction operation has been performed on all the titles, the latest subtitle text tree is used as the document title tree of the original file;
[0040] When it is confirmed that there are titles for which the title tree construction operation has not been performed, the title tree construction operation is repeatedly performed.
[0041] Furthermore, the step of identifying, based on the file to be identified, a number of titles in the file to be identified, and text content corresponding to each title, includes:
[0042] Get the preset prefix character matching list;
[0043] Identify each line of text in the to-be-identified file, and determine a number of prefix characters in each line of text;
[0044] Matching a number of the prefix characters with the prefix character matching list;
[0045] When it is determined that a prefix character matches successfully, the target line corresponding to the prefix character is used as the title, and the text between the target line and the next adjacent target line is used as the text content corresponding to the title.
[0046] Furthermore, the title tree construction model performs semantic recognition on the target title and the target text content, determines the hierarchical relationship between the target titles, and generates a subtitle text tree, including:
[0047] Performing semantic recognition on each of the target titles and the corresponding target text content to determine the text meaning of each target title, and determining irrelevant text in each of the target titles and the corresponding target text content that is inconsistent with the text meaning based on the text meaning;
[0048] Deleting the irrelevant text from the target title and the corresponding target text content, and generating a second title and second text content that match the meaning of the text;
[0049] According to the second titles and the second text content, the hierarchical relationship between the second titles is determined, and according to the hierarchical relationship, a subtitle text tree is generated.
[0050] Furthermore, the step of inputting the target title, the target text content, and the first title into a title tree construction model includes:
[0051] Generate a large model prompt word according to the hierarchical relationship between the first titles;
[0052] The first title and the large model prompt word are first input into the title tree construction model, and then the target title and the target text content are input into the title tree construction model.
[0053] Another embodiment of the present invention provides a terminal device, including a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, and when the processor executes the computer program, a method for constructing a document title tree as described in any one of the embodiments is implemented.
[0054] Another embodiment of the present invention provides a storage medium, which includes a stored computer program, wherein when the computer program is running, the device where the storage medium is located is controlled to execute a method for constructing a document title tree as described in any one of the above embodiments.
[0055] The following beneficial effects are achieved by implementing the present invention:
[0056] The present invention discloses a method, device, terminal device and storage medium for constructing a document title tree. The method converts the original file into a preset plain text format to improve the subsequent title recognition efficiency and text segmentation efficiency in a unified format. Further, according to the order of the titles in the file to be recognized, the title tree construction operation is performed in batches to prevent the title tree construction model from being unable to capture and memorize key information in the text, thereby improving the accuracy of semantic recognition of the title tree construction model. And when the title tree construction operation is not performed for the first time, a number of first titles on the path from the root node to the rightmost leaf node in the existing subtitle text tree are input into the title tree construction model together with the target title to be recognized and the target text content, so that the title tree construction model determines the hierarchical relationship between the target titles according to the hierarchical relationship of the first titles, so as to ensure the hierarchical coherence between the titles of different groups, and overcome the defects of the title hierarchical structure of the current document title tree construction technology that is not accurate or has deviations. BRIEF DESCRIPTION OF THE DRAWINGS
[0057] Figure 1 The figure is a flowchart of a method for constructing a document title tree provided by an embodiment of the present invention.
[0058] Figure 2 It is a structural schematic diagram of a device for constructing a document title tree provided by an embodiment of the present invention.
[0059] Figure 3 Another flowchart of a method for constructing a document title tree provided by an embodiment of the present invention is as follows DETAILED DESCRIPTION
[0060] In order to make the purpose, technical solutions and advantages of this application clearer, the technical solutions in this application will be clearly and completely described below in conjunction with the drawings in the embodiments of this application. Obviously, the described embodiments are part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.
[0061] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by technicians in the technical field to which this application belongs; the terms used herein are only for the purpose of describing specific embodiments and are not intended to limit this application; the terms "including" and "having" in the specification and claims of this application and the above-mentioned figure descriptions and any variations thereof are intended to cover non-exclusive inclusions.
[0062] In the description of the embodiments of the present application, the technical terms "first", "second", etc. are only used to distinguish different objects, and cannot be understood as indicating or implying relative importance or implicitly indicating the number, specific order or primary and secondary relationship of the indicated technical features. In the description of the embodiments of the present application, the meaning of "multiple" is more than two, unless otherwise clearly and specifically defined.
[0063] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.
[0064] In the description of the embodiments of the present application, the term "and / or" is only a description of the association relationship of the associated objects, indicating that there may be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article generally indicates that the associated objects before and after are in an "or" relationship.
[0065] In the description of the embodiments of the present application, the term "multiple" refers to more than two (including two). Similarly, "multiple groups" refers to more than two groups (including two groups), and "multiple pieces" refers to more than two pieces (including two pieces).
[0066] In the description of the embodiments of the present application, unless otherwise clearly specified and limited, technical terms such as "installed", "connected", "connected", "fixed" and the like should be understood in a broad sense. For example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection or an indirect connection through an intermediate medium, and it can be the internal connection of two elements or the interaction relationship between two elements. For those of ordinary skill in the art, the specific meanings of the above terms in the embodiments of the present application can be understood according to the specific circumstances.
[0067] See also Figure 1 , is a flow chart of a method for constructing a document title tree provided by an embodiment of the present invention, comprising:
[0068] S1. Obtain the original file of the title tree to be constructed;
[0069] In a preferred embodiment of the present invention, the original file is a file in any format with text content, such as: .txt, .pdf, .docx, .html, etc.
[0070] S2, converting the original file into a preset plain text format to generate a file to be identified;
[0071] In a preferred embodiment of the present invention, the preset plain text format is Markdown format. It should be noted that the core of format conversion lies in how to support more document types and ensure that the extracted results are still understandable to humans. This part of the work mainly relies on traditional document parsing capabilities such as OCR. However, due to the wide variety of file formats in real scenarios, this part of the challenge is very large and is a process that needs to be carried out step by step in combination with requirements and technical realities. In this embodiment, the mainstream RAG development frameworks Langchain[4] and LlamaIndex[5] are adopted, both of which have different components to meet the basic functions.
[0072] S3. Identify several titles in the to-be-identified file and the text content corresponding to each title according to the to-be-identified file;
[0073] Preferably, the step of identifying several titles in the to-be-identified file and the text content corresponding to each title according to the to-be-identified file includes:
[0074] S31. Obtain a preset prefix character matching list;
[0075] S32. Identify each line of text in the to-be-identified file and determine several prefix characters in each line of text;
[0076] S33. Match the several prefix characters with the prefix character matching list;
[0077] S34. When it is determined that a prefix character matches successfully, use the target line corresponding to the prefix character as the title, and use the text between the target line and the next adjacent target line as the text content corresponding to the title.
[0078] In a preferred embodiment of the present invention, based on a preset prefix character matching rule, it is determined whether each independent line is a title. An example prefix character matching list is as follows: ["§", "第", "一", "二", "三", "四", "五", "六", "七", "八", "九", "十", "零", "〇", "1", "2", "3", "4", "5", "6", "7", "8", "9", "0", "(一", "(二", "(三", "(四", "(五", "(六", "(七", "(八", "(九", "(十"]. Here, the prefix character matching rule can be customized and modified to regular matching or a more complex method such as LLM Chain according to the content of the original file.
[0079] It should be noted that in the current document title tree technology, the "title format restrictions" are set relatively strictly, such as punctuation restrictions, text length restrictions, and special symbol restrictions, which makes it easy to miss many correct titles; therefore, this embodiment adopts a more relaxed title recognition rule through the title recognition stage to make the recall rate of document titles higher and will not miss important potential titles. In addition, based on the powerful text semantic understanding ability of the subsequent title tree construction model, it can remove irrelevant erroneous titles in the title tree construction stage, while ensuring the accuracy of the title.
[0080] S4, repeatedly executing the title tree construction operation according to the plurality of titles and a preset title tree construction model, until a document title tree of the original file is generated;
[0081] The title tree construction operation includes:
[0082] S41, according to the order of the titles in the to-be-recognized file, obtaining a number of target titles for which the title tree construction operation has not been performed, and obtaining the target text content corresponding to each target title;
[0083] S42, determining whether the title tree construction operation is currently being performed for the first time;
[0084] S43, if yes, input the target title and the target text content into the title tree construction model, so that the title tree construction model performs semantic recognition on the target title and the target text content, determines the hierarchical relationship between the target titles, and generates a subtitle text tree;
[0085] Preferably, the title tree construction model performs semantic recognition on the target title and the target text content, determines the hierarchical relationship between the target titles, and generates a subtitle text tree, including:
[0086] S431, performing semantic recognition on each of the target titles and the corresponding target text content, determining the text meaning of each target title, and determining irrelevant text that does not match the text meaning in each of the target titles and the corresponding target text content according to the text meaning;
[0087] S432, deleting the irrelevant text from the target title and the corresponding target text content, and generating a second title and second text content that are consistent with the meaning of the text;
[0088] S433: Determine the hierarchical relationship between the second titles according to the second titles and the second text content, and generate a subtitle text tree according to the hierarchical relationship.
[0089] S44, if not, then obtain several first titles on the path from the root node to the rightmost leaf node of the current subtitle text tree, and input the target title, the target text content, and the first title into the title tree construction model, so that the title tree construction model performs semantic recognition on the target title and the target text content, and determines the hierarchical relationship between the target titles according to the hierarchical relationship of the first titles, and then inserts the target title into the subtitle text tree according to the hierarchical relationship;
[0090] Preferably, the step of inputting the target title, the target text content, and the first title into a title tree construction model comprises:
[0091] S441, generating a large model prompt word according to the hierarchical relationship between the first titles;
[0092] S442. The first title and the large model prompt word are preferentially input into the title tree construction model, and then the target title and the target text content are input into the title tree construction model.
[0093] S45, when it is confirmed that the title tree construction operation has been performed on all the titles, the latest subtitle text tree is used as the document title tree of the original file;
[0094] S46. When it is confirmed that there are titles for which the title tree construction operation has not been performed, repeat the title tree construction operation.
[0095] In a preferred embodiment of the present invention, the title tree construction model is a pre-trained large language model (Large Language Model 1, LLM for short). For the potential title list obtained by title recognition, such as Figure 3 As shown, this embodiment designs a fusion task based on a large-scale pre-trained generation model.
[0096] a) Group the title list in order (the default size of each group can be set to 20) and input it into the large-scale pre-trained generative model, requiring the model to identify the hierarchy between titles according to the title content and return it in the form of markdown, and directly discard the content that is not a title in the result. Then parse the result returned by the model and integrate it into the current title tree.
[0097] b) In order to ensure the consistency between the levels of different groups, we designed a "throwing a brick to attract jade" strategy. Before each new title group is input into the model, the title contained in the rightmost subpath of the current title is placed in front of the current group, and its corresponding hierarchical result is included in the large model prompt word.
[0098] (4) Document tree construction: Based on the title tree and Markdown format files, the corresponding document tree is constructed.
[0099] It is understandable that in actual scenarios, LLM structural analysis (LLM Based Method) may extract a large number of titles. Feeding all of them to LLM at once will cause two risks: (1) If the input length is too long and exceeds LLM's input length, the model will fail to respond; (2) When the input is long, how can LLM ensure the consistency of the previous and next levels in the result? For this reason, we have involved corresponding strategies in the strategy:
[0100] (1) Long input leads to model failure: limit the number of titles that can be input at one time, which is currently set to 20 in the actual solution.
[0101] (2) Consistency of the result levels: In each prompt result input, the rightmost path of the constructed Title Tree is added as a "primer" to stimulate further discussion, thereby ensuring consistency of the result levels.
[0102] The final results show that even when using a local quantization model (qwen-72b-int8) with poor model performance, the final effect of the above two-stage extraction strategy is still very impressive.
[0103] The present embodiment provides a method for constructing a document title tree, by converting the original file into a preset plain text format, improving the subsequent title recognition efficiency and text segmentation efficiency with a unified format, and further performing the title tree construction operation in batches according to the order of the titles in the file to be recognized, so as to prevent the title tree construction model from being unable to capture and memorize key information in the text, and improve the accuracy of semantic recognition of the title tree construction model. And when the title tree construction operation is not performed for the first time, several first titles on the path from the root node to the rightmost leaf node in the existing subtitle text tree are input into the title tree construction model together with the target title to be recognized and the target text content, so that the title tree construction model determines the hierarchical relationship between the target titles according to the hierarchical relationship of the first titles, so as to ensure the hierarchical coherence between titles of different groups, and overcome the defects of the title hierarchy structure of the current document title tree construction technology that is not accurate or has deviations.
[0104] See also Figure 2 , is a schematic diagram of a structure of a device for constructing a document title tree provided by an embodiment of the present invention, comprising:
[0105] A file acquisition module, used to obtain the original file of the title tree to be constructed;
[0106] A format conversion module, used to convert the original file into a preset plain text format to generate a file to be identified;
[0107] A title recognition module, used to recognize a plurality of titles in the file to be recognized and text content corresponding to each title according to the file to be recognized;
[0108] A title tree construction module, used to repeatedly perform the title tree construction operation according to a plurality of titles and a preset title tree construction model, until a document title tree of the original file is generated;
[0109] The title tree construction operation includes:
[0110] According to the order of the titles in the to-be-recognized file, a plurality of target titles for which the title tree construction operation has not been performed are obtained, and the target text content corresponding to each target title is obtained;
[0111] Determine whether the title tree building operation is currently being performed for the first time;
[0112] If yes, the target title and the target text content are input into the title tree construction model, so that the title tree construction model can perform semantic recognition on the target title and the target text content, determine the hierarchical relationship between the target titles, and generate a subtitle text tree;
[0113] If not, then obtain several first titles on the path from the root node to the rightmost leaf node of the current subtitle text tree, and input the target title, the target text content, and the first title into the title tree construction model, so that the title tree construction model performs semantic recognition on the target title and the target text content, and determines the hierarchical relationship between the target titles according to the hierarchical relationship of the first titles, and then inserts the target title into the subtitle text tree according to the hierarchical relationship;
[0114] When it is confirmed that the title tree construction operation has been performed on all the titles, the latest subtitle text tree is used as the document title tree of the original file;
[0115] When it is confirmed that there are titles for which the title tree construction operation has not been performed, the title tree construction operation is repeatedly performed.
[0116] Preferably, the step of identifying, based on the file to be identified, a number of titles in the file to be identified, and text content corresponding to each title, includes:
[0117] Get the preset prefix character matching list;
[0118] Identify each line of text in the to-be-identified file, and determine a number of prefix characters in each line of text;
[0119] Matching a number of the prefix characters with the prefix character matching list;
[0120] When it is determined that a prefix character matches successfully, the target line corresponding to the prefix character is used as the title, and the text between the target line and the next adjacent target line is used as the text content corresponding to the title.
[0121] Preferably, the title tree construction model performs semantic recognition on the target title and the target text content, determines the hierarchical relationship between the target titles, and generates a subtitle text tree, including:
[0122] Performing semantic recognition on each of the target titles and the corresponding target text content to determine the text meaning of each target title, and determining irrelevant text in each of the target titles and the corresponding target text content that is inconsistent with the text meaning based on the text meaning;
[0123] Deleting the irrelevant text from the target title and the corresponding target text content, and generating a second title and second text content that match the meaning of the text;
[0124] According to the second titles and the second text content, the hierarchical relationship between the second titles is determined, and according to the hierarchical relationship, a subtitle text tree is generated.
[0125] Preferably, the step of inputting the target title, the target text content, and the first title into a title tree construction model comprises:
[0126] Generate a large model prompt word according to the hierarchical relationship between the first titles;
[0127] The first title and the large model prompt word are first input into the title tree construction model, and then the target title and the target text content are input into the title tree construction model.
[0128] It should be noted that the device embodiments described above are merely schematic, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. In addition, in the accompanying drawings of the device embodiments provided by the present invention, the connection relationship between the modules indicates that there is a communication connection between them, which may be specifically implemented as one or more communication buses or signal lines. A person of ordinary skill in the art may understand and implement it without paying any creative effort.
[0129] Those skilled in the art can clearly understand that for the sake of convenience and brevity, the specific working process of the device described above can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here.
[0130] Another preferred embodiment of the present invention provides a terminal device, including a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, wherein when the processor executes the computer program, a method for constructing a document title tree as described in any one of the above embodiments is implemented.
[0131] The terminal device may be a computing device such as a desktop computer, a notebook, a PDA, a cloud server, etc. The terminal device may include, but is not limited to, a processor and a memory.
[0132] The processor may be a central processing unit (CPU), other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor, etc. The processor is the control center of the terminal device, and uses various interfaces and lines to connect various parts of the entire terminal device.
[0133] The memory can be used to store the computer program, and the processor realizes various functions of the terminal device by running or executing the computer program stored in the memory and calling the data stored in the memory. The memory can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, at least one application required for a function, etc.; the data storage area can store data created according to the use of the mobile phone, etc. In addition, the memory can include a high-speed random access memory, and can also include a non-volatile memory, such as a hard disk, a memory, a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), at least one disk storage device, a flash memory device, or other volatile solid-state storage devices.
[0134] Another preferred embodiment of the present invention provides a storage medium, the storage medium is a computer-readable storage medium, the computer program is stored in the computer-readable storage medium, and the computer program, when executed by the processor, can implement the steps of the above-mentioned various method embodiments. Wherein, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form, etc. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, USB flash drive, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), electrical carrier signal, telecommunication signal and software distribution medium, etc.
[0135] The above is a preferred embodiment of the present invention. It should be pointed out that a person skilled in the art can make several improvements and modifications without departing from the principle of the present invention. These improvements and modifications are also considered to be within the scope of protection of the present invention.
Claims
1. A method for constructing a document title tree, characterized in that: include: Get the original file of the title tree to be built; Convert the original file into a preset plain text format to generate a file to be identified; According to the file to be identified, identifying a number of titles in the file to be identified, and text content corresponding to each title; According to the plurality of titles and a preset title tree construction model, the title tree construction operation is repeatedly performed until a document title tree of the original file is generated; The title tree construction operation includes: According to the order of the titles in the to-be-recognized file, a plurality of target titles for which the title tree construction operation has not been performed are obtained, and the target text content corresponding to each target title is obtained; Determine whether the title tree building operation is currently being performed for the first time; If yes, the target title and the target text content are input into the title tree construction model, so that the title tree construction model can perform semantic recognition on the target title and the target text content, determine the hierarchical relationship between the target titles, and generate a subtitle text tree; If not, then obtain several first titles on the path from the root node to the rightmost leaf node of the current subtitle text tree, and input the target title, the target text content, and the first title into the title tree construction model, so that the title tree construction model performs semantic recognition on the target title and the target text content, and determines the hierarchical relationship between the target titles according to the hierarchical relationship of the first titles, and then inserts the target title into the subtitle text tree according to the hierarchical relationship; When it is confirmed that the title tree construction operation has been performed on all the titles, the latest subtitle text tree is used as the document title tree of the original file; When it is confirmed that there are titles for which the title tree construction operation has not been performed, the title tree construction operation is repeatedly performed.
2. A method for constructing a document title tree as claimed in claim 1, characterized in that: The step of identifying, according to the file to be identified, a plurality of titles in the file to be identified, and text content corresponding to each title, includes: Get the preset prefix character matching list; Identify each line of text in the to-be-identified file, and determine a number of prefix characters in each line of text; Matching a number of the prefix characters with the prefix character matching list; When it is determined that a prefix character matches successfully, the target line corresponding to the prefix character is used as the title, and the text between the target line and the next adjacent target line is used as the text content corresponding to the title.
3. A method for constructing a document title tree as claimed in claim 2, characterized in that: The title tree construction model performs semantic recognition on the target title and the target text content, determines the hierarchical relationship between the target titles, and generates a subtitle text tree, including: Performing semantic recognition on each of the target titles and the corresponding target text content to determine the text meaning of each target title, and determining irrelevant text in each of the target titles and the corresponding target text content that is inconsistent with the text meaning based on the text meaning; Deleting the irrelevant text from the target title and the corresponding target text content, and generating a second title and second text content that match the meaning of the text; According to the second titles and the second text content, the hierarchical relationship between the second titles is determined, and according to the hierarchical relationship, a subtitle text tree is generated.
4. A method for constructing a document title tree as claimed in claim 3, characterized in that: The step of inputting the target title, the target text content, and the first title into a title tree construction model comprises: Generate a large model prompt word according to the hierarchical relationship between the first titles; The first title and the large model prompt word are first input into the title tree construction model, and then the target title and the target text content are input into the title tree construction model.
5. A device for constructing a document title tree, characterized in that: include: A file acquisition module, used to obtain the original file of the title tree to be constructed; A format conversion module, used to convert the original file into a preset plain text format to generate a file to be identified; A title recognition module, used to recognize a plurality of titles in the file to be recognized and text content corresponding to each title according to the file to be recognized; A title tree construction module, used to repeatedly perform the title tree construction operation according to a plurality of titles and a preset title tree construction model, until a document title tree of the original file is generated; The title tree construction operation includes: According to the order of the titles in the to-be-recognized file, a plurality of target titles for which the title tree construction operation has not been performed are obtained, and a target text content corresponding to each target title is obtained; Determine whether the title tree building operation is currently being performed for the first time; If yes, the target title and the target text content are input into the title tree construction model, so that the title tree construction model can perform semantic recognition on the target title and the target text content, determine the hierarchical relationship between the target titles, and generate a subtitle text tree; If not, then obtain several first titles on the path from the root node to the rightmost leaf node of the current subtitle text tree, and input the target title, the target text content, and the first title into the title tree construction model, so that the title tree construction model performs semantic recognition on the target title and the target text content, and determines the hierarchical relationship between the target titles according to the hierarchical relationship of the first titles, and then inserts the target title into the subtitle text tree according to the hierarchical relationship; When it is confirmed that the title tree construction operation has been performed on all the titles, the latest subtitle text tree is used as the document title tree of the original file; When it is confirmed that there are titles for which the title tree construction operation has not been performed, the title tree construction operation is repeatedly performed.
6. A document title tree construction device as claimed in claim 5, characterized in that: The step of identifying, according to the file to be identified, a plurality of titles in the file to be identified, and text content corresponding to each title, includes: Get the preset prefix character matching list; Identify each line of text in the to-be-identified file, and determine a number of prefix characters in each line of text; Matching a number of the prefix characters with the prefix character matching list; When it is determined that a prefix character matches successfully, the target line corresponding to the prefix character is used as the title, and the text between the target line and the next adjacent target line is used as the text content corresponding to the title.
7. A document title tree construction device as claimed in claim 6, characterized in that: The title tree construction model performs semantic recognition on the target title and the target text content, determines the hierarchical relationship between the target titles, and generates a subtitle text tree, including: Performing semantic recognition on each of the target titles and the corresponding target text content to determine the text meaning of each target title, and determining irrelevant text in each of the target titles and the corresponding target text content that is inconsistent with the text meaning based on the text meaning; Deleting the irrelevant text from the target title and the corresponding target text content, and generating a second title and second text content that match the meaning of the text; According to the second titles and the second text content, the hierarchical relationship between the second titles is determined, and according to the hierarchical relationship, a subtitle text tree is generated.
8. A document title tree construction device as claimed in claim 7, characterized in that: The step of inputting the target title, the target text content, and the first title into a title tree construction model comprises: Generate a large model prompt word according to the hierarchical relationship between the first titles; The first title and the large model prompt word are first input into the title tree construction model, and then the target title and the target text content are input into the title tree construction model.
9. A terminal device, characterized in that: The method comprises a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, wherein when the processor executes the computer program, a method for constructing a document title tree as claimed in any one of claims 1 to 4 is implemented.
10. A storage medium, characterized in that: The storage medium includes a stored computer program, wherein when the computer program is executed, the device where the storage medium is located is controlled to execute a method for constructing a document title tree as claimed in any one of claims 1 to 4.
Citation Information
Patent Citations
Document title tree construction method and device, electronic equipment and storage medium
CN111460083A
Knowledge base automatic rapid construction method, system and device based on large language model technology
CN118152520A
Document directory extraction method and system based on generative language model
CN118170863A
Document data extraction method and system based on large model and extraction target
CN118468814A
Natural language processing applications using large language models
US20240095463A1
Cited By
Document chapter jumping method and device, equipment and storage medium
CN120821523A