A text chunking method and device based on HTML node path resolution

By using a text segmentation method based on HTML node path parsing, combined with tag information and length thresholds to process HTML documents, the problem of the correspondence between structure and content in document segmentation is solved, achieving efficient and accurate document segmentation, and applicable to segmentation processing of various document types.

CN122153120APending Publication Date: 2026-06-05CHINA EVERBRIGHT BANK

Patent Information

Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
CHINA EVERBRIGHT BANK
Filing Date
2026-01-29
Publication Date
2026-06-05

Smart Images

  • Figure CN122153120A_ABST
    Figure CN122153120A_ABST
Patent Text Reader

Abstract

Embodiments of the present specification relate to the technical field of text processing, and provide a text blocking method and device based on HTML node path analysis, comprising: performing initial blocking on the HTML document to obtain an initial HTML document blocking result, and recording an XPath path expression corresponding to each piece of text in the target HTML document and an initial HTML document block to which the text belongs; performing preprocessing on each initial HTML document block to obtain a plurality of initial pure text document blocks; performing merging or segmentation operations on the initial pure text document blocks to obtain a plurality of final pure text document blocks; and blocking the target HTML document according to the XPath path expression corresponding to each piece of text in the final pure text document blocks and the initial HTML document block to which the text belongs, to obtain a final HTML document blocking result. Through the embodiments of the present specification, the accuracy of HTML text blocking can be improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This specification relates to the field of text processing technology, and in particular to a text segmentation method and apparatus based on HTML node path parsing. Background Technology

[0002] In the field of information technology, especially in applications such as natural language processing, information retrieval, and knowledge management, efficiently and accurately segmenting structured documents into content blocks is a fundamental and crucial prerequisite task. Effective document segmentation aims to break down lengthy and complex documents into semantically coherent, structurally complete, and appropriately sized independent units, facilitating subsequent vectorization processing, semantic understanding, precise retrieval, or efficient ingestion by large language models.

[0003] Existing document segmentation methods can be mainly categorized as follows: First, mechanical segmentation based on fixed lengths or sliding windows. This method is simple to implement but easily fragments complete sentences, paragraphs, or tables, disrupting inherent semantic units and logical structures, leading to information fragmentation and context loss. Second, segmentation based on simple rules (such as line breaks and punctuation). While it considers natural paragraphs to some extent, it struggles to adapt to diverse document layouts and complex nested structures. Third, with the development of pre-trained models, dynamic segmentation techniques based on semantic similarity have emerged. These methods better ensure semantic consistency within blocks, but when processing documents with explicit hierarchical structures and rich formatting information, such as source code, rich text, or HTML, they discard the original visual cues, hierarchical relationships, and structured data (such as tables and lists), resulting in distorted segmentation results that cannot correspond to the logical organization of the original document. Summary of the Invention

[0004] To address the aforementioned problems in the prior art, the purpose of the embodiments in this specification is to provide a text segmentation method and apparatus based on HTML node path parsing, so as to solve the problem of poor text segmentation accuracy in the prior art.

[0005] To solve the above-mentioned technical problems, the specific technical solutions of the embodiments in this specification are as follows: On the one hand, embodiments of this specification provide a text segmentation method based on HTML node path parsing, the method comprising: The HTML document is divided into blocks based on the tag information in the target HTML document to obtain the initial HTML document block result; Record the XPath path expression corresponding to each piece of text in the target HTML document and the initial HTML document block to which it belongs; Each initial HTML document block in the initial HTML document segmentation result is preprocessed to obtain multiple initial plain text document blocks. Based on the length of each initial plain text document block and a first preset threshold, the initial plain text document blocks are merged or split to obtain multiple final plain text document blocks. The target HTML document is divided into blocks based on the XPath path expression corresponding to each text segment in the final plain text document block and the initial HTML document block to which it belongs, to obtain the final HTML document block result.

[0006] Further, the tag information includes first tag information and second tag information; dividing the HTML document into blocks based on the tag information in the target HTML document includes: Based on the first tag information, determine whether the target HTML document contains multi-level tag information; If it contains multi-level tag information, then the tag information in the target HTML document is traversed; Determine the highest-level tag in the multi-level tag information; Starting from the initial position of the target HTML document, traverse downwards. When the first top-level tag is reached, the document content before the top-level tag is divided into the first initial HTML document block. Starting from the highest-level tag, continue traversing downwards, dividing the highest-level tag and its subsequent document content into the second initial HTML document block. Stop when the second highest-level tag is reached, and repeat the above block division process with the second highest-level tag as the new starting point until the target HTML document is completely traversed, resulting in multiple initial HTML document blocks.

[0007] Furthermore, dividing the HTML document into blocks based on tag information in the target HTML document also includes: If the target HTML document does not contain multi-level tag information, then the target HTML document is treated as an initial HTML document block, and a preprocessing step is performed on the initial HTML document block.

[0008] Further, each initial HTML document block in the initial HTML document segmentation result is preprocessed, including: Convert the tables in each initial HTML document block to Markdown format; and Remove the second tag information, spaces, and line breaks from each initial HTML document block.

[0009] Further, a merging operation is performed on the initial plain text document blocks based on the length of each initial plain text document block and a first preset threshold, including: When the length of the initial plain text document block does not exceed the first preset threshold, compare its last tag level with the first tag level in the next initial plain text document block; If the former is greater than or equal to the latter, then the two initial plain text document blocks will be merged. Determine whether the length of the merged plain text document block exceeds the first preset threshold; If so, then perform a splitting operation on the merged plain text document block; If not, repeat the steps above to compare label levels.

[0010] Further, the initial plain text document block is segmented based on its length and a first preset threshold, including: When the length of the initial plain text document block exceeds a first preset threshold, the initial plain text document block is divided into multiple sentences; Each sentence is converted into a vector using a vectorization model, resulting in a vector sequence. Calculate the semantic similarity between adjacent sentences in the vector sequence to obtain a similarity sequence; The segmentation point is determined based on the similarity sequence and the second preset threshold; The initial plain text document block is split into multiple sub-blocks based on the segmentation points, wherein the semantic similarity between sentences in each sub-block is higher than the second preset threshold.

[0011] Further, the target HTML document is divided into blocks based on the XPath path expression corresponding to each text segment in the final plain text document block and the initial HTML document block to which it belongs, including: For each of the final plain text document blocks, perform the following operations: Based on the initial HTML document block to which each text segment in the final plain text document block belongs, determine the set of nodes corresponding to it in the target HTML document; Based on the XPath path expressions corresponding to the first and last text segments in the final plain text document block, determine the node range in the node set; Based on the HTML document content included in the node interval, the final HTML document segmentation result corresponding to the final plain text document block is obtained; The final HTML document segmentation result is obtained based on each final HTML document segment.

[0012] On the other hand, embodiments of this specification provide a text segmentation device based on HTML node path parsing, the device comprising: The first segmentation module is used to segment the HTML document into segments based on the tag information in the target HTML document to obtain an initial HTML document segmentation result; The recording module is used to record the XPath path expression corresponding to each piece of text in the target HTML document and the initial HTML document block to which it belongs; The preprocessing module is used to preprocess each initial HTML document block in the initial HTML document segmentation result to obtain multiple first plain text document blocks. The merging or splitting module is used to perform merging or splitting operations on the first plain text document block according to the length of each initial plain text document block and a first preset threshold, to obtain multiple second plain text document blocks. The second segmentation module is used to segment the target HTML document into blocks based on the XPath path expression corresponding to each text segment in the second plain text document block and the initial HTML document block to which it belongs, so as to obtain the final HTML document segmentation result.

[0013] In another aspect, embodiments of this specification also provide a computer device, including a memory, a processor, and a computer program stored in the memory, wherein the computer program, when executed by the processor, performs instructions of any of the methods described above.

[0014] In another aspect, embodiments of this specification also provide a computer-readable storage medium having a computer program stored thereon, the computer program being executed by a processor of a computer device to perform instructions for any of the methods described above.

[0015] By adopting the above technical solution, the text segmentation method based on HTML node path parsing provided in this specification first completes the initial segmentation based on HTML tag information and associates and records the XPath path expression of each text segment with its corresponding initial block. Then, after preprocessing, the initial plain text document blocks are obtained, and merging or splitting operations are performed based on the length and a preset threshold. Finally, the final HTML document segmentation result is obtained through reverse mapping. On the one hand, it can preserve the correspondence between text content and HTML document structure during the segmentation process, avoiding the problem of plain text and structural information being disconnected in traditional segmentation methods, and ensuring the structural integrity of the segmentation result. On the other hand, the granularity of the plain text document blocks can be flexibly adjusted by controlling the length threshold, effectively solving the problems of content fragmentation caused by overly fine segmentation or information redundancy caused by overly coarse segmentation, improving the adaptability of the segmentation result to subsequent text retrieval, semantic analysis and other application scenarios. At the same time, relying on the mapping of XPath paths, it ensures that the final HTML segmentation result has both the coherence of plain text content and retains the original structural features of the HTML document, significantly improving the accuracy and practicality of HTML text segmentation.

[0016] The above description is merely an overview of some embodiments of the technical solutions in this specification. In order to better understand the technical means of some embodiments of this specification and to implement them in accordance with the content of the specification, and to make the above and other objects, features and advantages of the embodiments of this specification more apparent and understandable, preferred embodiments are described below in detail with reference to the accompanying drawings. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in the embodiments or prior art of this specification, the drawings used in the description of the embodiments or prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this specification. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 The diagram illustrates a system implementation of a text segmentation method based on HTML node path parsing, as described in some embodiments of this specification. Figure 2 The diagram illustrates the steps of a text segmentation method based on HTML node path parsing in some embodiments of this specification. Figure 3 This specification illustrates the steps of performing a chunking process on an HTML document in some embodiments. Figure 4 This specification illustrates the steps of preprocessing an initial HTML document block in some embodiments. Figure 5 This specification illustrates the steps of merging initial plain text document blocks in some embodiments; Figure 6 This specification illustrates the steps for segmenting an initial plain text document block in some embodiments; Figure 7 This specification illustrates the steps of performing secondary chunking on an HTML document in some embodiments. Figure 8 This specification shows a schematic diagram of the structure of a text segmentation device based on HTML node path parsing in some embodiments; Figure 9 A schematic diagram of the structure of a computer device is shown in this specification.

[0019] Explanation of symbols in the attached drawings: 801, First Block Module; 802. Recording module; 803. Preprocessing module; 804. Merge or split modules; 805, Second Sub-module; 902. Computer equipment; 904, Processor; 906. Memory; 908. Drive mechanism; 910. Input / Output Module; 912. Input devices; 914. Output devices; 916. Presentation equipment; 918. Graphical User Interface; 920. Network interface; 922. Communication link; 924. Communication bus. Detailed Implementation

[0020] The technical solutions in the embodiments of this specification will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this specification, and not all embodiments. Based on the embodiments in this specification, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this specification.

[0021] It should be noted that the terms "first," "second," etc., used in this specification, claims, and the foregoing drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, apparatus, product, or device that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or devices.

[0022] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties. Furthermore, the acquisition, storage, use, and processing of data in the technical solutions described in the embodiments of this application all comply with relevant regulations.

[0023] It should be noted that in the embodiments of this specification, certain software, components, models and other existing solutions in the industry may be mentioned. These should be regarded as exemplary and are only intended to illustrate the feasibility of implementing the technical solutions of the embodiments of this specification. However, it does not mean that the applicant has used or necessarily used such solutions.

[0024] like Figure 1 The diagram illustrates an implementation system for a text segmentation method based on HTML node path parsing, as described in this specification. The system includes a terminal 101 and a server 102, with a communication connection established between them to enable data interaction. The terminal 101 collects and sends a target HTML document to the server 102. The server 102 receives the target HTML document and processes it according to the method described in this specification. Specifically, the system includes: segmenting the HTML document into blocks based on tag information to obtain initial HTML document segmentation results; recording the XPath path expression corresponding to each text segment in the target HTML document and its associated initial HTML document block; preprocessing each initial HTML document block in the initial HTML document segmentation results to obtain multiple initial plain text document blocks; performing merging or splitting operations on the initial plain text document blocks based on their length and a first preset threshold to obtain multiple final plain text document blocks; and segmenting the target HTML document into blocks based on the XPath path expression corresponding to each text segment in the final plain text document blocks and its associated initial HTML document block to obtain final HTML document segmentation results. Server 102 returns the final HTML document chunked result to terminal 101.

[0025] In the embodiments of this specification, the server 102 may be an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDN), and big data and artificial intelligence platforms.

[0026] In an optional embodiment, terminal 101 may be an electronic device, including but not limited to self-service terminal equipment, desktop computers, tablet computers, laptops, smart wearable devices, etc. Optionally, the operating system running on the electronic device may include, but is not limited to, Android, iOS, Linux, Windows, etc. Of course, terminal 101 is not limited to the aforementioned physical electronic devices; it may also be software running on the aforementioned electronic devices.

[0027] In addition, it should be noted that, Figure 1 The example shown is merely one application environment provided by this disclosure. In practical applications, it may include multiple terminals 101, and this specification does not impose any restrictions.

[0028] Figure 2 This diagram illustrates the steps of a text segmentation method based on HTML node path parsing, as provided in the embodiments of this specification. While this specification provides the operational steps of the method described in the embodiments or flowcharts, more or fewer steps may be included based on conventional or non-inventive methods. The order of steps listed in the embodiments is merely one possible execution order among many and does not represent the only possible order. In actual system or device products, the methods shown in the embodiments or accompanying drawings can be executed sequentially or in parallel. Specifically, as shown... Figure 2 As shown, when applied to the server side described above, the method may include: S201: Divide the HTML document into blocks according to the tag information in the target HTML document to obtain the initial HTML document block result; S202: Record the XPath path expression corresponding to each piece of text in the target HTML document and the initial HTML document block to which it belongs; S203: Preprocess each initial HTML document block in the initial HTML document segmentation result to obtain multiple initial plain text document blocks; S204: Perform a merging or splitting operation on the initial plain text document block according to the length of each initial plain text document block and a first preset threshold to obtain multiple final plain text document blocks; S205: Divide the target HTML document into blocks according to the XPath path expression corresponding to each text segment in the final plain text document block and the initial HTML document block to which it belongs, and obtain the final HTML document block result.

[0029] By adopting the above technical solution, the text segmentation method based on HTML node path parsing provided in this specification first completes the initial segmentation based on HTML tag information and associates and records the XPath path expression of each text segment with its corresponding initial block. Then, after preprocessing, the initial plain text document blocks are obtained, and merging or splitting operations are performed based on the length and a preset threshold. Finally, the final HTML document segmentation result is obtained through reverse mapping. On the one hand, it can preserve the correspondence between text content and HTML document structure during the segmentation process, avoiding the problem of plain text and structural information being disconnected in traditional segmentation methods, and ensuring the structural integrity of the segmentation result. On the other hand, the granularity of the plain text document blocks can be flexibly adjusted by controlling the length threshold, effectively solving the problems of content fragmentation caused by overly fine segmentation or information redundancy caused by overly coarse segmentation, improving the adaptability of the segmentation result to subsequent text retrieval, semantic analysis and other application scenarios. At the same time, relying on the mapping of XPath paths, it ensures that the final HTML segmentation result has both the coherence of plain text content and retains the original structural features of the HTML document, significantly improving the accuracy and practicality of HTML text segmentation.

[0030] The text segmentation method based on node path parsing provided in this specification is not limited to HTML documents. For other document formats such as Word, PDF, and XML, they can first be converted to a standard HTML document structure using a format conversion tool, and then this method can be applied. Specifically, initial segmentation is performed based on the tag information of the converted HTML document. The XPath path expression corresponding to each text segment and its associated initial block are recorded. The initial blocks are preprocessed to obtain plain text document blocks. Merging or splitting operations are then performed based on the length and a preset threshold. Finally, the final segmentation result of the target document is obtained through reverse mapping. This extended application method does not require significant adjustments to the core logic of this method, achieving efficient segmentation for multiple document types and effectively improving the method's universality and practical application value.

[0031] The text segmentation method based on HTML node path parsing provided in the embodiments of this specification can be applied to the field of natural language processing, such as analyzing documents using large language models. Since large language models have a limited input length, they struggle to generate any effective information when the input length exceeds this limit. Therefore, by reasonably dividing the document into multiple parts, retaining the necessary contextual content within a defined text length range, and ensuring that the length of the text input to the model is within the maximum acceptable text length range, the corresponding block in the HTML document is found based on the position of each plain text block, resulting in segmented HTML document blocks. By segmenting the HTML document, each plain document block with a limited text length is fed into the model, avoiding the impact of the model's input length limit, expanding the types of documents that can be used, ensuring the orderly structure of each text block fed into the model, and guaranteeing that each document block contains sufficient information, the returned HTML format document blocks can be displayed on the webpage front-end and correspond to the generated content.

[0032] In other embodiments, besides its application in the field of natural language processing, the HTML text segmentation method based on node path parsing described above can also provide well-structured text units for text retrieval, semantic analysis, summary generation, and intelligent question answering, significantly improving the efficiency and accuracy of natural language processing tasks. Furthermore, it can be extended to the field of digital content management, performing standardized segmentation processing on documents converted from various formats to achieve automated classification, archiving, and rapid retrieval of massive amounts of documents. It can also play a role in information extraction and knowledge base construction, significantly broadening the application scenarios of the method.

[0033] HTML, or HyperText Markup Language, is a markup language used to build the structure of web pages. It doesn't directly define the visual style of a page; instead, it uses a series of predefined tags to structurally mark up document content. Each tag corresponds to a specific content module, such as headings, paragraphs, lists, and images. Tags can carry attributes to supplement the module's characteristic information. The entire HTML document presents a tree structure formed by nested tags. Based on this structure, XPath path expressions can be used to accurately locate the position of any piece of text content and its corresponding structural module. Browsers can then interpret the semantics of the tags and render the corresponding web page content.

[0034] In the embodiments of this specification, the tag information in the HTML document is specifically divided into first tag information and second tag information, wherein the first tag information is the heading level tag, for example... <h1>、< / h1> <h2>、< / h2> <h3> to< / h3> <h6>Tags. The second tag information includes all tags in the HTML document except for heading tags, such as image tags. Paragraph tags Container label List tags 、 In-line tags Conventional functional labels, etc. In the embodiments of this specification, refer to... Figure 3 Based on the above tag information, the HTML document is segmented into blocks, including the following steps: S301: Determine whether the target HTML document contains multi-level tag information based on the first tag information; S302: If multi-level tag information is included, the tag information in the target HTML document is traversed. S303: Determine the highest-level tag in the multi-level tag information; S304: Starting from the initial position of the target HTML document, traverse downwards. When the first top-level tag is reached, divide the document content before the top-level tag into the first initial HTML document block. S305: Starting from the highest-level tag, continue traversing downwards, dividing the highest-level tag and its subsequent document content into the second initial HTML document block. Stop when the second highest-level tag is reached, and repeat the above block division process with the second highest-level tag as the new starting point until the target HTML document is traversed, resulting in multiple initial HTML document blocks.

[0035] Understandably, the first step is to determine whether two or more different levels of heading tags exist simultaneously in the target HTML document based on the first tag information. If the document contains both, then... <h1> and< / h1> <h2>、< / h2> <h2> and< / h2> <h3> If there are different levels of heading tags, the document is considered to contain multi-level tag information; if the document only contains a single level of heading tags, such as only having...< / h3> <h2> Labels, or none< / h2> <h1> to< / h1> <h6> If the heading level tag is not found, then the document is determined not to contain multi-level tag information. For example, if the target HTML document contains both heading level tags and multi-level tags, then the document is determined not to contain multi-level tag information.< / h6> <h1> Technical solution< / h1> and <h2> Blocking steps< / h2> If the document contains multiple levels of tags, then it is determined that the document contains multi-level tag information; if the document only contains multiple levels of tags, then it is determined that the document contains multiple levels of tags. Paragraph tags and A container tag without any heading level tags is considered not to contain multi-level tag information. After determining that the document contains multi-level tag information, a complete and ordered traversal of the entire content of the target HTML document is performed. During the traversal, all tag information within the document is scanned, including all tags. <h1> to< / h1> <h6>The first tag information also includes 、 The purpose of this second tag information traversal is to completely extract the specific location, tag level, and corresponding text content of all heading level tags in the document, ensuring that no heading tags are missed during subsequent segmentation and guaranteeing the integrity of the segmentation. For example, during the traversal, it will sequentially identify and record the heading tags appearing in the document. <h1> 1 Introduction< / h1> , <h2> 1.1 Purpose< / h2> , <h2> 1.2 Scope< / h2> , <h1> 2 System Requirements< / h1> , <h2> 2.1 System Overview< / h2> , <h2> 2.2 System Function List< / h2> The position and hierarchy information of all heading level tags. HTML documents have clear rules for classifying heading level tags by level. In the embodiments of this specification, <h1> This is the highest level heading tag.< / h1> <h2> Secondly, and so on.< / h2> <h6> The lowest level heading tag. Based on the traversal results from the previous step, the highest level heading tag is selected from all extracted heading tags of different levels. After determining the highest level tag, the traversal begins from the very beginning of the target HTML document, i.e., the beginning of the HTML document root node, and proceeds from top to bottom according to the inherent order of the document content. Upon locating the first occurrence of the highest level tag during the traversal, the traversal stops immediately, and all HTML document content before the location of this highest level tag is divided into an independent first initial HTML document block. This part of the content is generally the document's preface, description, table of contents, or basic content without hierarchical classification, and contains no highest level heading tags. Then, using the first highest level tag located in the previous step as the new traversal starting point, the highest level tag itself, along with all document content after that tag, is added sequentially to the second initial HTML document block. The traversal continues downwards along the document order. When the second highest level tag of the same level is reached, the current block division is terminated immediately. At this point, all content between the two highest level tags of the same level constitutes a complete initial HTML document block. Next, using this second highest-level tag as a new starting point for traversal, the above chunking process is repeated, and so on in a loop. This continues until the target HTML document has been completely traversed, at which point the entire document can be divided into multiple initial HTML document chunks matching the highest-level tags. For example, if the highest-level tag is...< / h6> <h1> The document contains< / h1> <h1> 1 Introduction< / h1> , <h2> 1.1 Purpose< / h2> , <h2> 1.2 Scope< / h2> , <h1> 2 System Requirements< / h1> , <h2> 2.1 System Overview< / h2> , <h2> 2.2 System Function List< / h2> Tags such as "etc." will be assigned by the first one. <h1> 1 Introduction< / h1> Starting from the second one, traverse to the next one. <h1> 2 System Requirements< / h1> Stop at a certain time; all content within this interval forms the first initial HTML document block, and then... <h1> 2 Implementation Steps< / h1> Starting from the beginning, traverse to the end of the document. This part is the second initial HTML document block, completing the initial block division of the entire document. That is, the content from 1. Introduction to 1.2. Scope is one block, and the content from 2. System Requirements to 2.2 System Function List is another block.

[0036] In the embodiments of this specification, if the target HTML document does not contain multi-level tag information, the target HTML document is treated as an initial HTML document block, and subsequent preprocessing steps are performed on the initial HTML document block.

[0037] In some embodiments of this specification, the step S202, which records the XPath path expression of each text segment in the HTML structure and the document block to which it belongs, specifically refers to the fact that an HTML document is essentially a tree structure starting from the root node, with the root node being the tag, and each text segment belonging to the tag being... Label, Tags, etc., the tags corresponding to the images are Tags, and all tags and text, have a unique position within this tree structure. XPath path expressions are standardized path identifiers that can precisely locate any node in the HTML tree structure. Starting with the root tag of the HTML document, they sequentially construct the tag paths from parent to child nodes according to the hierarchical nesting relationship, ultimately pointing to the specific tag node carrying the text. Through this XPath path expression, the physical location and structural hierarchy of the text within the target HTML document can be accurately located, and each text segment's XPath path expression is unique; XPath path expressions for different text segments are not repeated. After recording the XPath path expression for each text segment, it is necessary to simultaneously label the initial HTML document block to which the text belongs. Specifically, based on the division results of the aforementioned initial block segmentation steps, it is determined which initial HTML document block the current text belongs to, and the identification information of that initial HTML document block, such as block number and block code, is associated and recorded with the text and XPath path expression. If the target HTML document is divided into a single initial HTML document block due to the lack of multi-level tag information, then all text segments within the document are marked as belonging to that unique initial HTML document block.

[0038] In other embodiments of this specification, when the target HTML document contains image elements, it is also necessary to perform the same recording operation as the text content on each image in the document, that is, to record the XPath path expression corresponding to each image in the target HTML document and the initial HTML document block to which the image belongs. The specific recording method is consistent with the recording method of the text content.

[0039] After completing the initial segmentation of the target HTML document through the aforementioned steps, each resulting initial HTML document block retains the original HTML tag structure, i.e., it contains... 、 If second-level tag information, table tags, and image tags are directly retained, it will lead to a mixture of text and formatting information. On the other hand, redundant non-text content such as extra spaces and invalid line breaks generated by tag nesting and formatting will not only interfere with the accuracy of subsequent text length statistics but also disrupt the coherence of the plain text content, making merging and splitting operations inaccurate and thus affecting the accuracy of the final block division. Therefore, to retain the core valid content, the formatted HTML document blocks are converted into plain text document blocks that can be directly used for subsequent merging and splitting, referring to... Figure 4 The specific preprocessing steps are as follows: S401: Convert the tables in each initial HTML document block to markdown format; S402: Remove the second tag information, spaces, and line breaks from each initial HTML document block.

[0040] Understandably, the table content in an HTML document is based on... 、 、 、 Dedicated table tags enable structured row and column display. Directly deleting these tags would result in disorganized cell content, loss of row and column relationships, and failure to fully preserve the core information of the table. This specification's embodiments convert HTML tables to Markdown format. This removes the original HTML tags while preserving the table's row and column hierarchy and cell content relationships through Markdown's standardized text format, ensuring the integrity of the table information. Furthermore, Markdown is a plain text format that will not interfere with subsequent text length calculations, merging, or splitting operations. The specific operation method is: recognizing the tags within the initial HTML document block... 、 、 The table tags are used to separate the content of each row of cells with vertical lines, following the original row and column order. Cells in different rows are separated by newline characters. For HTML tables with headers, horizontal lines are added between the header and the content rows. This completes the conversion from HTML table to Markdown table. In this embodiment, step S402 removes the second tag information, spaces, and newline characters from each initial HTML document block. Specifically, it means first removing the second tag information from each initial HTML document block, since the tags are... Similar pairs exist. After removing the relevant tags, there are still extra spaces and line breaks between the text. For example, there was originally only one blank line between paragraph 1 and paragraph 2. After deleting the tags, the paragraph occupied by the tags becomes a blank line. Therefore, it is necessary to remove these extra symbols and blank lines to obtain the same content structure as the original document text content.

[0041] After the above preprocessing operations, each initial HTML document block has been converted into an initial plain text document block containing only text content and no redundant formatting. Since the text lengths of each initial plain text document block vary, there are problems such as fragmented content due to excessively short text segments and difficulty in information integration due to excessively long text blocks. To solve this problem, this embodiment of the specification merges or splits each initial plain text document block according to its actual text length and a preset first threshold, to obtain a final plain text document block with appropriate text length and coherent, complete content. In this embodiment, reference is made to... Figure 5 The specific merging steps are as follows: S501: When the length of the initial plain text document block does not exceed the first preset threshold, compare its last tag level with the first tag level in the next initial plain text document block; S502: If the former is greater than or equal to the latter, then merge the two initial plain text document blocks. S503: Determine whether the length of the merged plain text document block exceeds the first preset threshold; S504: If so, perform a splitting operation on the merged plain text document block; S505: If not, repeat the steps above to compare label levels.

[0042] Understandably, the first preset threshold is the pre-defined maximum adaptable length of the text block, which can be adjusted according to the actual application scenario. Tag level refers to the first tag information retained in the initial plain text document block, i.e., the level of the heading level tag. Heading levels generally range from level one to level six, with lower numbers indicating higher levels. The last tag level of the initial plain text document block refers to the level of the last valid heading tag at the end of the current initial plain text document block. The first tag level in the next initial plain text document block refers to the level of the first valid heading tag at the beginning of the next initial plain text document block adjacent to the current block. If the last tag level of the current initial plain text document block is greater than or equal to the first tag level of the next initial plain text document block, it indicates that the content of the two document blocks has a subordinate relationship and strong content coherence. The next document block is considered a submodule of the current document block. Merging these two document blocks will not destroy semantic integrity, therefore, a merge operation is performed. For example, there are currently three document blocks. The first document block contains 1. Introduction and 1.1. Purpose; the second document block contains 1.2. Scope; and the third document block contains 2. System Requirements, 2.1 System Overview, and 2.2 System Function List. In the first document block, the heading level of "Purpose" is the same as that of "Scope". Therefore, the first and second blocks are merged. After merging, there are currently two text blocks. We continue to judge these two text blocks. If the heading level of "Scope" is lower than that of "System Requirements", then these two blocks are not merged.

[0043] In the embodiments of this specification, in order to avoid the merged document block being too long and affecting subsequent information integration, after the merge operation is performed, the length of the merged plain text document block needs to be compared with a first preset threshold. If the former is longer than the latter, a splitting operation is performed to split the excessively long merged block into sub-blocks of suitable length, thereby avoiding information redundancy.

[0044] In the embodiments described herein, reference is made to... Figure 6 The specific segmentation steps are as follows: S601: When the length of the initial plain text document block exceeds a first preset threshold, the initial plain text document block is divided into multiple sentences; S602: Use a vectorization model to convert each sentence into a vector, resulting in a vector sequence; S603: Calculate the semantic similarity between adjacent sentences in the vector sequence to obtain a similarity sequence; S604: Determine the segmentation point based on the similarity sequence and the second preset threshold; S605: The initial plain text document block is split into multiple sub-blocks according to the segmentation point, wherein the semantic similarity between sentences in each sub-block is higher than the second preset threshold.

[0045] Understandably, the segmentation step is performed on initial plain text document blocks whose length exceeds a first preset threshold. It primarily splits the document based on a vectorization model and sentence semantic similarity, avoiding semantic breaks caused by mechanically segmenting by character count and ensuring semantic coherence within each sub-block. First, Chinese sentence-ending punctuation, such as periods, exclamation marks, and question marks, is used as the dividing criterion to split the entire initial plain text document block into independent sentences that retain complete semantics. It should be noted that the content within a Markdown format table should be considered a complete semantic unit and not segmented into sentences to avoid disrupting the table structure. The vectorization model converts natural language sentences into high-dimensional numerical vectors. The distance between vectors in space accurately represents the semantic similarity between sentences; the closer the distance, the more similar the semantics. Some embodiments in this specification use the bge-m3 vectorization model to ensure that the vectors completely preserve the semantic features of the sentences. Other embodiments may also select vectorization models such as Sentence-BERT and Word2Vec as needed. Step S602 inputs each independent sentence obtained from the aforementioned split into a vectorization model. The model outputs a fixed-dimensional vector corresponding to each sentence, and arranges all vectors according to the order of the sentences in the original document block to form an ordered vector sequence. This embodiment uses a cosine similarity algorithm to calculate the semantic similarity between adjacent sentence vectors. The cosine similarity value ranges from -1 to 1; the closer the value is to 1, the stronger the semantic association between the two sentences; the closer the value is to 0, the weaker the semantic association. Step S603 calculates the cosine similarity between adjacent sentences sequentially according to the vector sequence, and arranges the calculation results in order to form a similarity sequence. In step S604, the second preset threshold is a pre-set semantic similarity threshold used to determine whether adjacent sentences belong to the same semantic unit. If the similarity between adjacent sentences is less than the second preset threshold, it indicates a weak semantic association, and this is a split point. If the similarity is greater than the second preset threshold, it indicates semantic coherence, and no split is performed. Step S605 divides the original plain text document block into multiple sub-blocks, using the split point as the boundary. Within each sub-block, the semantic similarity of all adjacent sentences must be greater than a second preset threshold to ensure semantic coherence. The text length of each sub-block must be less than or equal to a first preset threshold to ensure length suitability. If a sub-block still exceeds the first preset threshold after splitting, the process S601-S605 is repeated until all sub-blocks meet the requirements.

[0046] After the above merging and segmentation operations, all initial plain text document blocks have been optimized into final plain text document blocks with a length that meets the first preset threshold and whose internal sentences are semantically coherent. Since these final plain text document blocks are in unformatted plain text form, they cannot be directly restored to HTML document blocks with the original structure. Therefore, relying on the XPath paths of the text and images retained throughout the process and the initial block association information, each final plain text document block is reverse-mapped to the corresponding position in the target HTML document to restore the HTML tag structure, image elements, markdown format tables, etc., thus obtaining the final HTML document block corresponding to the plain text document block. In the embodiments of this specification, reference is made to... Figure 7 The target HTML document is divided into blocks based on the XPath path expression corresponding to each text segment in the final plain text document block and the initial HTML document block to which it belongs. The specific steps include: For each of the final plain text document blocks, perform the following operations: S701: Determine the set of nodes corresponding to each text segment in the target HTML document based on the initial HTML document block to which each text segment belongs in the final plain text document block; S702: Determine the node range in the node set based on the XPath path expressions corresponding to the first and last text segments in the final plain text document block; S703: Based on the HTML document content included in the node interval, obtain the final HTML document block corresponding to the final plain text document block; The final HTML document segmentation result is obtained based on each final HTML document segment.

[0047] It is understood that all tags, text, and tables in the target HTML document correspond to nodes in a tree structure. Step S701, node set, refers to the ordered set of all nodes corresponding to a certain initial HTML document block selected from the full set of nodes in the target HTML document. In the embodiments of this specification, determining the node set corresponding to each text segment in the final plain text document block in the target HTML document specifically includes: extracting the identifier of the initial HTML document block to which all text segments in the current final plain text document block belong; based on the identifier, selecting all HTML nodes belonging to these initial blocks from the full node library of the target HTML document; and integrating the selected nodes according to their order in the original document to form the node set corresponding to the final plain text document block. Step S702, node interval, refers to the range of all continuous nodes in the node set, starting from the node corresponding to the XPath of the first text segment and ending at the node corresponding to the XPath of the last text segment. This interval is the structural range of the final HTML document block. The specific steps are as follows: Extract the XPath path expression of the first segment of text in the current final plain text document block, denoted as XPath1, and the XPath path expression of the last segment of text, denoted as XPath2. In the node set, locate the first node corresponding to XPath1 and the last node corresponding to XPath2 through the XPath path. Starting from the first node and ending at the last node, determine the continuous node interval in the node set, including the first node, the last node, and all nodes between them. If the first and last nodes belong to node sets of different initial blocks, they are still connected into continuous intervals according to the original document order. Step S703 obtains the final HTML document block corresponding to the final plain text document block based on the HTML document content included in the node interval. The specific steps are as follows: First, obtain all the original HTML content within the node interval, including tag structure, text content, image tags, markdown format tables, attribute information, etc., without modifying the original format and order of the content. Then, integrate the extracted content into a complete HTML document block, which is the final HTML document block corresponding to the current final plain text document block. Traverse all final plain text document blocks, repeating steps S701-S703. All integrated HTML document blocks together constitute the final segmentation result of the target HTML document.

[0048] For example, if a final plain text document block includes text segments 1 and 2 of the initial HTML document block A, and text segment 3 of the initial HTML document block B, first determine the set of nodes corresponding to it in the target HTML document. Filter out all nodes corresponding to the initial HTML document block A from the target HTML document, such as all nodes under / body / div[1], and all nodes corresponding to the initial HTML document block B, such as all nodes under / body / div[2]. Concatenate the above two types of nodes according to the original document order to form the node set of the final plain text document block. Next, determine the node range. If the XPath path expression of the first segment of the final plain text document block is: / html / body / div[1] / h2, corresponding to the initial HTML document block A... <h2>The XPath path expression for the tag node and the end text is: / html / body / div[2] / p[3], corresponding to the initial HTML document block B. Tag nodes. First, locate the first and last nodes in the node set. Then, take all consecutive nodes from / html / body / div[1] / h2 to / html / body / div[2] / p[3] as the node range. Finally, extract all HTML content within the node range from / html / body / div[1] / h2 to / html / body / div[2] / p[3], including tag structure, text content, image tags, markdown format tables, attribute information, etc. This content is the final HTML document block corresponding to the final plain text document block.

[0049] Thus, the HTML text segmentation method in this embodiment parses the HTML document structure sequentially, initially segmenting the document into blocks based on heading levels. During the merging of document blocks, the heading levels of consecutive blocks are compared, ensuring the accurate order of the segmented document content and the arrangement of heading levels from highest to lowest within the same block. The segmentation process considers semantic similarity within the text, merging semantically similar paragraphs together as much as possible. Setting length limits for text blocks also improves the speed and quality of model output when applied to large oracle model scenarios. Furthermore, by sending the segmented HTML document back to the webpage frontend, users can intuitively see the document segmentation results and edit the document blocks that need modification online in real time.

[0050] Based on the above-described text segmentation method based on HTML node path parsing, this specification also provides a corresponding text segmentation device based on HTML node path parsing. The device may include a system (including a distributed system), software (application), module, component, server, client, etc., using the method described in this specification, combined with necessary hardware implementation. Based on the same innovative concept, the devices in one or more embodiments provided in this specification are as described in the following embodiments. Since the implementation schemes and methods for solving the problem are similar, the implementation of specific devices in this specification can refer to the implementation of the aforementioned method, and repeated details will not be repeated. As used below, the terms "unit" or "module" can refer to a combination of software and / or hardware that implements a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, hardware implementation, or a combination of software and hardware, is also possible and contemplated.

[0051] Specifically, Figure 8 This is a schematic diagram of the module structure of one embodiment of a text segmentation device based on HTML node path parsing provided in this specification. (Refer to...) Figure 8 As shown in the embodiments of this specification, a text segmentation device based on HTML node path parsing includes: The first segmentation module 801 is used to segment the HTML document into segments based on the tag information in the target HTML document to obtain an initial HTML document segmentation result; The recording module 802 is used to record the XPath path expression corresponding to each piece of text in the target HTML document and the initial HTML document block to which it belongs; Preprocessing module 803 is used to preprocess each initial HTML document block in the initial HTML document segmentation result to obtain multiple first plain text document blocks. The merging or splitting module 804 is used to perform merging or splitting operations on the first plain text document block according to the length of each initial plain text document block and a first preset threshold to obtain multiple second plain text document blocks. The second segmentation module 805 is used to segment the target HTML document into blocks based on the XPath path expression corresponding to each text segment in the second plain text document block and the initial HTML document block to which it belongs, so as to obtain the final HTML document segmentation result.

[0052] The beneficial effects obtained by the apparatus provided in the embodiments of this specification are consistent with the beneficial effects obtained by the methods described above, and will not be repeated here.

[0053] Reference Figure 9 As shown, based on the above-described text segmentation method based on HTML node path parsing, an embodiment of this specification also provides a computer device 902, wherein the above method operates on the computer device 902. The computer device 902 may include one or more processors 904, such as one or more central processing units (CPUs), each processing unit implementing one or more hardware threads. The computer device 902 may also include any memory 906 for storing any kind of information such as code, settings, data, etc. Non-limitingly, for example, the memory 906 may include any type of RAM, any type of ROM, flash memory, hard disk, optical disk, etc. More generally, any memory can use any technology to store information. Further, any memory can provide volatile or non-volatile retention of information. Further, any memory can represent a fixed or removable component of the computer device 902. In one case, when the processor 904 executes associated instructions stored in any memory or combination of memories, the computer device 902 can perform any operation of the associated instructions. The computer device 902 also includes one or more drive mechanisms 908 for interacting with any memory, such as a hard disk drive mechanism, an optical disk drive mechanism, etc.

[0054] Computer device 902 may also include an input / output module 910 (I / O) for receiving various inputs (via input device 912) and providing various outputs (via output device 914). A specific output mechanism may include a presentation device 916 and an associated graphical user interface (GUI) 918. In other embodiments, the input / output module 910 (I / O), input device 912, and output device 914 may be omitted, and the device may function solely as a computer device within a network. Computer device 902 may also include one or more network interfaces 920 for exchanging data with other devices via one or more communication links 922. One or more communication buses 924 couple the components described above together.

[0055] Communication link 922 can be implemented in any way, such as via a local area network, a wide area network (e.g., the Internet), a point-to-point connection, or any combination thereof. Communication link 922 may include any combination of hardwired links, wireless links, routers, gateway functions, name servers, etc., governed by any protocol or combination of protocols.

[0056] Corresponding to, for example Figures 2 to 7 In addition to the method shown, embodiments of this specification also provide a computer-readable storage medium storing a computer program that, when executed by a processor, performs the steps of the above-described method.

[0057] This specification also provides computer-readable instructions, wherein when a processor executes the instructions, the program therein causes the processor to perform the following... Figures 2 to 7 The method shown.

[0058] This specification also provides a computer program product, including at least one instruction or at least one program segment, wherein the at least one instruction or the at least one program segment is loaded and executed by a processor to achieve the following: Figures 2 to 7 The method shown.

[0059] It should be understood that in the various embodiments of this specification, the sequence number of each process does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this specification.

[0060] It should also be understood that, in the embodiments of this specification, the term "and / or" is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. Additionally, the character " / " in this specification generally indicates that the preceding and following related objects have an "or" relationship.

[0061] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed in this specification can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of each example have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this specification.

[0062] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0063] In the several embodiments provided in this specification, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the couplings or direct couplings or communication connections shown or discussed may be indirect couplings or communication connections through some interfaces, devices, or units, or they may be electrical, mechanical, or other forms of connection.

[0064] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of the embodiments described in this specification, depending on actual needs.

[0065] Furthermore, the functional units in the various embodiments of this specification can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0066] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this specification, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this specification. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0067] This specification uses specific embodiments to illustrate the principles and implementation methods of this specification. The descriptions of the above embodiments are only for the purpose of helping to understand the methods and core ideas of this specification. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this specification. Therefore, the content of this specification should not be construed as a limitation of this specification. < / h2> < / h6> < / h6>

Claims

1. A text segmentation method based on HTML node path parsing, characterized in that, The method includes: The HTML document is divided into blocks based on the tag information in the target HTML document to obtain the initial HTML document block result; Record the XPath path expression corresponding to each piece of text in the target HTML document and the initial HTML document block to which it belongs; Each initial HTML document block in the initial HTML document segmentation result is preprocessed to obtain multiple initial plain text document blocks. Based on the length of each initial plain text document block and a first preset threshold, the initial plain text document blocks are merged or split to obtain multiple final plain text document blocks. The target HTML document is divided into blocks based on the XPath path expression corresponding to each text segment in the final plain text document block and the initial HTML document block to which it belongs, to obtain the final HTML document block result.

2. The method according to claim 1, characterized in that, The tag information includes first tag information and second tag information; the HTML document is divided into blocks according to the tag information in the target HTML document, including: Based on the first tag information, determine whether the target HTML document contains multi-level tag information; If it contains multi-level tag information, then the tag information in the target HTML document is traversed; Determine the highest-level tag in the multi-level tag information; Starting from the initial position of the target HTML document, traverse downwards. When the first top-level tag is reached, the document content before the top-level tag is divided into the first initial HTML document block. Starting from the highest-level tag, continue traversing downwards, dividing the highest-level tag and its subsequent document content into the second initial HTML document block. Stop when the second highest-level tag is reached, and repeat the above block division process with the second highest-level tag as the new starting point until the target HTML document is completely traversed, resulting in multiple initial HTML document blocks.

3. The method according to claim 2, characterized in that, The HTML document is divided into blocks based on tag information in the target HTML document, and the process also includes: If the target HTML document does not contain multi-level tag information, then the target HTML document is treated as an initial HTML document block, and a preprocessing step is performed on the initial HTML document block.

4. The method according to claim 2, characterized in that, Preprocessing is performed on each initial HTML document block in the initial HTML document segmentation result, including: Convert the tables in each initial HTML document block to Markdown format; and Remove the second tag information, spaces, and line breaks from each initial HTML document block.

5. The method according to claim 1, characterized in that, The initial plain text document blocks are merged based on their length and a first preset threshold, including: When the length of the initial plain text document block does not exceed the first preset threshold, compare its last tag level with the first tag level in the next initial plain text document block; If the former is greater than or equal to the latter, then the two initial plain text document blocks will be merged. Determine whether the length of the merged plain text document block exceeds the first preset threshold; If so, then perform a splitting operation on the merged plain text document block; If not, repeat the steps above to compare label levels.

6. The method according to claim 5, characterized in that, The initial plain text document block is segmented based on its length and a first preset threshold, including: When the length of the initial plain text document block exceeds a first preset threshold, the initial plain text document block is divided into multiple sentences; Each sentence is converted into a vector using a vectorization model, resulting in a vector sequence. Calculate the semantic similarity between adjacent sentences in the vector sequence to obtain a similarity sequence; The segmentation point is determined based on the similarity sequence and the second preset threshold; The initial plain text document block is split into multiple sub-blocks based on the segmentation points, wherein the semantic similarity between sentences in each sub-block is higher than the second preset threshold.

7. The method according to claim 1, characterized in that, The target HTML document is divided into blocks based on the XPath path expression corresponding to each text segment in the final plain text document block and the initial HTML document block to which it belongs, including: For each of the final plain text document blocks, perform the following operations: Based on the initial HTML document block to which each text segment in the final plain text document block belongs, determine the set of nodes corresponding to it in the target HTML document; Based on the XPath path expressions corresponding to the first and last text segments in the final plain text document block, determine the node range in the node set; Based on the HTML document content included in the node interval, the final HTML document segmentation result corresponding to the final plain text document block is obtained; The final HTML document segmentation result is obtained based on each final HTML document segment.

8. A text segmentation device based on HTML node path parsing, characterized in that, The device includes: The first segmentation module is used to segment the HTML document into segments based on the tag information in the target HTML document to obtain an initial HTML document segmentation result; The recording module is used to record the XPath path expression corresponding to each piece of text in the target HTML document and the initial HTML document block to which it belongs; The preprocessing module is used to preprocess each initial HTML document block in the initial HTML document segmentation result to obtain multiple first plain text document blocks. The merging or splitting module is used to perform merging or splitting operations on the first plain text document block according to the length of each initial plain text document block and a first preset threshold, to obtain multiple second plain text document blocks. The second segmentation module is used to segment the target HTML document into blocks based on the XPath path expression corresponding to each text segment in the second plain text document block and the initial HTML document block to which it belongs, so as to obtain the final HTML document segmentation result.

9. A computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the method as described in any one of claims 1 to 7.