A hierarchical structure perception-based specification file semantic slicing method, system, device and medium

Abstract

The application discloses a specification file semantic slicing method, system, device and medium based on hierarchical structure perception, the method comprises the following steps: identifying the association relationship and logical relationship between each term entity, and constructing a semantic association graph according to the identification result; generating a slicing rule according to the hierarchical structure tree and the semantic association graph, and determining the slicing boundary according to the slicing rule to obtain an initial slice; verifying the initial slice according to the semantic association graph, and adjusting the slicing boundary according to the verification result; collecting feedback data of users when using the final slice for business application and performance data of a large model when processing the final slice, and optimizing the slicing rule according to the feedback data and the performance data. Through analysis of the specification file structure and the term semantic relationship, the application guarantees semantic integrity, can also select an adaptive mechanism according to a scene to perform slicing processing, and can continuously optimize the slicing rule through user feedback and large model performance data.

Description

Technical Field

[0001] This invention relates to the field of natural language processing technology, and in particular to a method, system, device and medium for semantic slicing of normative documents based on hierarchical structure awareness. Background Technology

[0002] As enterprises advance digitalization, normative documents, as important carriers of enterprise management systems, technical standards, and business processes, are increasingly numerous and complex. In recent years, large language models have demonstrated strong application potential in scenarios such as intelligent question answering, knowledge retrieval, and compliance review. However, the token processing capabilities of large models have limitations. Therefore, how to reasonably divide lengthy and complex normative documents into semantic units suitable for large model processing has become a problem that needs to be considered.

[0003] Existing document slicing technologies primarily employ simple segmentation methods with fixed lengths or character counts. This mechanical approach ignores the chapter and clause structure characteristics of normative documents. In practical applications, it has been found that such slicing methods easily fragment complete clause content at slice boundaries, resulting in the same clause content being scattered across different slices. Furthermore, the relationships and logical connections between terminology entities are broken due to the mechanical division of slice boundaries. More importantly, different business scenarios have varying requirements for slicing granularity, and existing technologies cannot select an appropriate slicing mechanism for each business scenario. They also lack a mechanism for continuous optimization of slicing strategies based on feedback from actual applications. This makes it difficult for the slicing quality to meet actual business needs, impacting the effectiveness of large models in normative document processing. Summary of the Invention

[0004] In view of the aforementioned existing problems, the present invention is proposed.

[0005] Therefore, this invention provides a hierarchical structure-aware method, system, device, and medium for semantic slicing of normative documents to address the problems of existing document slicing methods lacking hierarchical structure awareness, failing to guarantee semantic integrity, and lacking a selection and adaptation mechanism based on the scenario.

[0006] To solve the above-mentioned technical problems, the present invention provides the following technical solution: In a first aspect, the present invention provides a method for semantic slicing of normative documents based on hierarchical structure awareness, comprising: parsing the format content of the normative document to obtain several structural elements, and constructing a hierarchical structure tree based on the format features and subordinate relationships of the structural elements; extracting terminology entities from the normative document, and identifying the association and logical relationships between the terminology entities, and constructing a semantic association graph based on the identification results; generating slicing rules based on the hierarchical structure tree and the semantic association graph, and determining slicing boundaries according to the slicing rules, performing slicing processing on the normative document based on the slicing boundaries to obtain initial slices; verifying the initial slices according to the semantic association graph, adjusting the slicing boundaries according to the verification results, and generating final slices based on the adjusted slicing boundaries.

[0007] As a preferred embodiment of the hierarchical structure-aware semantic slicing method for normative documents described in this invention, the step of constructing a hierarchical structure tree based on the format features and subordinate relationships of structural elements includes: identifying the document format of the normative document; selecting an appropriate parsing method to extract the format features of the structural elements based on the document format; classifying the structural elements hierarchically based on the format features; determining the subordinate relationships between structural elements at each level; and constructing the hierarchical structure tree.

[0008] The beneficial effects of this preferred technical solution are as follows: By selecting the appropriate parsing method based on the document format, it can handle normative documents in various formats such as Word, PDF, and Excel, avoiding the limitations of a single parsing method. The extraction of format features covers information such as heading style, font size, and numbering format, enabling hierarchical classification to distinguish structural elements at different levels, such as chapter, clause, subheading, and paragraph. The determination of subordinate relationships is achieved through a comprehensive judgment of numbering information and format features. The constructed hierarchical structure tree fully reflects the chapter and clause organization logic of the normative document. The document's hierarchical structure is expressed in the form of a tree data structure, facilitating the determination of slice boundaries according to the actual structure of the document.

[0009] As a preferred embodiment of the hierarchical structure-aware semantic slicing method for normative documents described in this invention, the step of constructing a semantic association graph based on the recognition results includes: extracting terminology entities from the normative document using named entity recognition technology, and identifying the association and logical relationships between the terminology entities using dependency parsing technology; identifying key information based on the marker symbols in the normative document, and constructing a semantic association graph according to the terminology entities, association relationships, logical relationships, and key information.

[0010] The beneficial effects of this preferred technical solution are as follows: Named entity recognition technology can extract different types of terminology entities from normative documents, such as industry terms, standard numbers, and core responsibilities. Dependency parsing technology, based on this, identifies the relationships between terminology entities and the causal, parallel, conditional, and adversative relationships between sentences, enabling a structured expression of semantic connections within the document. The identification of marker symbols incorporates key information marked with asterisks or underscores into the semantic association graph, ensuring that important content is identified within the graph. The constructed semantic association graph adopts a graph data structure, using terminology entities and key information as nodes, and relationships and logical connections as edges, fully presenting the semantic network of the normative document. This allows the determination of slice boundaries to be based on semantic associations, avoiding the fragmentation of related terminology entities and logical relationships.

[0011] As a preferred embodiment of the hierarchical structure-aware semantic slicing method for normative documents described in this invention, the step of generating slicing rules based on the hierarchical structure tree and semantic association graph includes: determining the hierarchical priority rules and semantic integrity rules of the slice based on the hierarchical structure tree and semantic association graph respectively, setting a slice size threshold based on the token processing capability of the large model, and generating scenario-based rules according to business scenario requirements; and generating the slicing rules by combining the hierarchical priority rules, semantic integrity rules, slice size threshold, and scenario-based rules.

[0012] The beneficial effects of this preferred technical solution are as follows: The hierarchical priority rules and semantic integrity rules are obtained from the hierarchical structure tree and semantic association graph, respectively, ensuring that the slicing rules simultaneously consider both the structural and semantic features of the document, avoiding slicing quality issues caused by relying on only a single dimension; the slice size threshold is set based on the token processing capability of the large model, ensuring that the generated slices can be processed by the large model without exceeding its processing limit; the introduction of scenario-based rules allows the slicing rules to adapt to the needs of different business scenarios, enabling differentiated processing of slice granularity and slice boundary settings in scenarios such as intelligent question answering, normative document checking, and large model fine-tuning. The combination of these four rules allows slicing processing to meet the requirements of different application scenarios while ensuring semantic integrity.

[0013] As a preferred embodiment of the hierarchical structure-aware semantic slicing method for normative documents described in this invention, the step of obtaining the initial slice includes: determining the initial boundary from the hierarchical structure tree according to the hierarchical priority rule, and slicing the normative document according to the initial boundary to obtain several candidate slices; determining whether the number of tokens in each candidate slice exceeds the slice size threshold, and if it does not exceed the threshold, retaining it; if it exceeds the threshold, re-slicing the candidate slices according to the semantic integrity rule to finally obtain the initial slice.

[0014] The beneficial effects of this preferred technical solution are as follows: Determining the initial boundary based on hierarchical priority rules ensures that slice processing is based on the actual document structure, avoiding structural damage caused by arbitrarily setting slice boundaries; the generation of candidate slices follows the document's chapter and clause organization logic, ensuring that most candidate slices maintain the integrity of their structural units. The token count judgment mechanism ensures that candidate slices can adapt to the processing capabilities of large models; candidate slices with a token count below the threshold are directly retained, reducing unnecessary secondary processing. For candidate slices with a token count exceeding the threshold, re-slicing is performed using semantic integrity rules. This hierarchical processing strategy controls slice size while ensuring that semantic connections are not severed. The generation of initial slices satisfies the token processing capability requirements of large models while maintaining the quality of slices based on document structure and semantic features.

[0015] As a preferred embodiment of the hierarchical structure-aware semantic slicing method for normative documents described in this invention, the step of adjusting the slice boundaries according to the verification results includes: performing semantic coherence verification on each initial slice based on the semantic association graph, detecting whether there are any fragmented associations or logical relationships between term entities at the boundaries of the initial slices; performing integrity verification on key information on each initial slice, detecting whether the key information is completely contained in a single initial slice; for initial slices that fail the semantic coherence verification or integrity verification, adjusting the slice boundaries until related term entities are merged into the same slice and the key information is completely preserved in a single slice, and generating the final slice based on the adjusted slice boundaries.

[0016] The beneficial effects of this preferred technical solution are as follows: Semantic coherence verification detects whether the semantic association graph breaks the association or logical relationship between term entities at the slice boundary, so that semantic breakage problems in the initial slice can be identified; Integrity verification detects key information to ensure that important content such as technical indicators and compliance standards are not split into different initial slices due to the setting of slice boundaries. This dual verification mechanism controls the quality of the initial slice from two dimensions: semantic coherence and information integrity, making up for the shortcomings that may exist in generating initial slices by relying solely on slice rules; For initial slices that fail verification, related term entities are merged into the same slice by adjusting the slice boundary and ensuring the integrity of key information, so that the final slice maintains semantic coherence and the integrity of key information while meeting the large model token processing capacity.

[0017] As a preferred embodiment of the hierarchical structure-aware semantic slicing method for normative documents described in this invention, the method further includes: iterating the slicing rules through an optimization mechanism; wherein the optimization mechanism includes the following steps: collecting feedback data and performance data when the large model processes the final slices; analyzing the feedback data and performance data to determine the problem type corresponding to each set of feedback data and performance data; determining the degree of correlation between the problem type and hierarchical priority rules, semantic integrity rules, slice size thresholds, and contextual rules; and adjusting the correlation rules to optimize the slicing rules.

[0018] The beneficial effects of this preferred technical solution are as follows: Analysis of feedback and performance data can identify problems in slices during actual business applications. Determining the problem type transforms user feedback and model performance into locatable slice quality issues. By judging the correlation between problem type and hierarchical priority rules, semantic integrity rules, slice size thresholds, and scenario-based rules, the specific rules causing the problem can be accurately located, avoiding the waste of resources caused by blindly adjusting all rules. Adjusting the correlation rules specifically addresses the slice quality issues exposed in actual applications, enabling slice rules to be improved based on real application effects. This optimization mechanism based on actual application feedback allows slice quality to continuously improve as application scenarios deepen, and slice rules gradually adapt to the actual needs of different business scenarios.

[0019] Secondly, the present invention provides a hierarchical structure-aware semantic slicing system for normative documents, comprising: Hierarchical structure parsing module: used to parse the format content of normative documents, extract structural elements, and construct a hierarchical structure tree based on the format characteristics and subordinate relationships of the structural elements; Semantic association analysis module: used to extract term entities from the normative documents, identify the association and logical relationships between term entities, and construct a semantic association graph; Slicing rule generation module: used to generate slicing rules based on the hierarchical structure tree and semantic association graph; The intelligent slicing execution module is used to slice the normative document according to the slicing rules to obtain the initial slice, and to verify and adjust the initial slice according to the semantic association graph to generate the final slice. Slicing optimization module: Used to collect feedback data and performance data of the final slice, and optimize the slicing rules based on the feedback data and performance data.

[0020] Thirdly, the present invention provides an electronic device, comprising: Memory and processor; The memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions, which, when executed by the processor, implement the steps of a hierarchical structure-aware normative document semantic slicing method.

[0021] Fourthly, the present invention provides a computer-readable storage medium storing computer-executable instructions that, when executed by a processor, implement the steps of the hierarchical structure-aware normative document semantic slicing method.

[0022] Compared with existing technologies, the advantages of this invention are as follows: By constructing a hierarchical structure tree and a semantic association graph, a deep analysis of the chapter and clause structure and semantic relationships of terms in normative documents is achieved. During the generation of slicing rules, hierarchical priority rules ensure that slicing is based on meaningful structural units such as clause-level or subheading-level units, avoiding the mechanical fragmentation of document structure caused by fixed-length slicing methods. Semantic integrity rules constrain slice boundaries through the associations and logical relationships in the semantic association graph, preventing the semantic connections between terminology entities from being interrupted by slice boundaries. More importantly, the setting of slice size thresholds fully considers the token processing capabilities of large models, achieving adaptation to the processing capabilities of large models while ensuring semantic integrity. The introduction of scenario-based rules further addresses the specific needs of different business scenarios such as intelligent question answering, compliance checks, and large model fine-tuning, generating differentiated slicing strategies so that the same normative document can achieve optimal slicing results in different application scenarios.

[0023] During the slicing execution phase, a dual verification mechanism—semantic coherence verification and key information integrity verification—is used to finely adjust the initial slices. This ensures that each slice neither severs related terminology entities due to improper boundary settings nor leads to the splitting or omission of important information such as key technical indicators and compliance standards due to slice division. More importantly, this invention also establishes a continuous optimization mechanism based on user feedback data and large model performance data. By analyzing the types of problems in practical applications and locating the corresponding slicing rules, dynamic adjustment and iterative optimization of the slicing strategy are achieved. This closed-loop optimization mechanism allows the slice quality to continuously improve as the application scenario deepens, thereby comprehensively improving the application effect and user experience of the large model in normative document processing. Attached Figure Description

[0024] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the following description of the embodiments will be briefly introduced. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0025] Figure 1 This is a schematic diagram of the overall process of a hierarchical structure-aware semantic slicing method for normative documents according to an embodiment of the present invention. Detailed Implementation

[0026] To make the above-mentioned objects, features, and advantages of the present invention more apparent and understandable, specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of them. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the protection scope of the present invention.

[0027] Example 1, referring to Figure 1 As an embodiment of the present invention, a method for semantic slicing of normative documents based on hierarchical structure awareness is provided, including steps S100 to S500: S100. Parse the format content of the normative document to obtain several structural elements, and construct a hierarchical structure tree based on the format characteristics and subordinate relationships of the structural elements.

[0028] S200. Extract terminology entities from normative documents, identify the relationships and logical connections between terminology entities, and construct a semantic association graph based on the identification results.

[0029] S300. Generate slicing rules based on the hierarchical structure tree and semantic association graph, determine the slicing boundaries according to the slicing rules, and perform slicing processing on the normative document based on the slicing boundaries to obtain the initial slices.

[0030] S400. Verify the initial slices based on the semantic association graph, adjust the slice boundaries according to the verification results, and generate the final slices based on the adjusted slice boundaries.

[0031] S500 iterates the slicing rules through an optimization mechanism.

[0032] It should be noted that in practical applications, normative documents are often lengthy and complex in structure, containing multiple levels of chapters and clauses, making semantic understanding and information retrieval difficult. Furthermore, due to the limited token processing capabilities of large models, directly inputting the complete normative document will exceed its processing capacity, leading to processing failures or poor results. In addition, simple fixed-length slicing will disrupt the semantic integrity of normative documents, destroy the logical relationships between chapters and clauses, and affect the accuracy of business applications such as intelligent question answering and compliance checks. Therefore, semantic slicing processing of normative documents is crucial.

[0033] Therefore, to address the aforementioned issues of semantic fragmentation and poor business application effectiveness, steps S100-S500 are employed to construct a hierarchical structure tree reflecting the organizational relationships of chapters and clauses in the normative document, and a semantic association graph reflecting the relationships and logical connections between terminology entities. This generates slicing rules adapted to the large model's token processing capabilities and business scenario requirements. This achieves semantically complete slicing of the normative document, ensuring that slice boundaries do not sever the relationships and logical connections between terminology entities, and guaranteeing that key information is fully contained within a single slice. Simultaneously, slicing rules are continuously optimized based on user feedback data and large model performance data to improve slice quality and business application effectiveness, thereby enhancing the accuracy of intelligent question answering and the precision of normative document checks.

[0034] Example 2, refer to Figure 1 As an embodiment of the present invention, based on the above embodiment, a method for semantic slicing of normative documents based on hierarchical structure awareness is provided.

[0035] In the embodiments of this application, step S100 involves parsing the format content of the normative document to obtain several structural elements, and constructing a hierarchical structure tree based on the format characteristics and subordinate relationships of the structural elements. In this embodiment, taking the "Technical Specification for Key Technologies of Intelligent Design and Application Optimization of Power Enterprise Architecture" of Guangxi Power Grid Co., Ltd. as an example, this normative document is in Word format and includes chapters such as general principles, technical requirements, research and development deliverables, and project schedule. By parsing the document format content, extracting structural elements such as titles, numbers, and paragraphs, identifying the format characteristics and subordinate relationships of each structural element, a complete hierarchical structure tree is constructed. This hierarchical structure tree contains 6 chapter-level nodes, 18 clause-level nodes, 45 sub-item-level nodes, and 156 paragraph-level nodes, fully reflecting the hierarchical structure and content organization relationship of the normative document.

[0036] The steps for constructing a hierarchical structure tree based on the format characteristics and subordinate relationships of structural elements include A1~A2: A1. Identify the document format of normative documents, and select the appropriate parsing method to extract the format features of structural elements based on the document format; First, the document format is identified as Word by its file extension. The Apache POI library is used to parse the document structure and extract paragraph style information, including heading styles, font size, bold attributes, and numbering formats. In this embodiment, the heading "Chapter 2 Technical Requirements" is a first-level heading with a 16-point font size, bold, and "Chapter X" numbering format, identified as a chapter-level structural element; the heading "2.3 Technical Specifications Requirements" is a second-level heading with a 14-point font size, bold, and "XX" numbering format, identified as a clause-level structural element; and the heading "2.3.1 Supported Number of Multi-Turn Dialogue Rounds" is a third-level heading with a 12-point font size, bold, and "XXX" numbering format, identified as a subheading-level structural element. Through this parsing method, structural elements in the normative document are extracted, including heading text, numbering information, paragraph content, and formatting features, resulting in several structural elements.

[0037] A2. Classify structural elements hierarchically according to format characteristics, determine the subordinate relationships between structural elements at each level, and construct a hierarchical structure tree; First, each structural element is categorized into chapter-level, clause-level, subheading-level, and paragraph-level. In this embodiment, the title "Chapter 2 Technical Requirements" is categorized as chapter-level, the title "2.3 Technical Specifications Requirements" as clause-level, the title "2.3.1 Supported Number of Multi-Turn Dialogue Rounds" as subheading-level, and the content paragraph "Supported Number of Multi-Turn Dialogue Rounds ≥ 10" as paragraph-level. Then, the hierarchical relationship between structural elements at each level is determined based on the numbering information. The clause-level structural element "2.3 Technical Specifications Requirements" is subordinate to the chapter-level structural element "Chapter 2 Technical Requirements," the subheading-level structural element "2.3.1 Supported Number of Multi-Turn Dialogue Rounds" is subordinate to the clause-level structural element "2.3 Technical Specifications Requirements," and the paragraph-level structural element "Supported Number of Multi-Turn Dialogue Rounds ≥ 10" is subordinate to the subheading-level structural element "2.3.1 Supported Number of Multi-Turn Dialogue Rounds." Finally, a hierarchical structure tree is constructed according to the hierarchical relationship. The hierarchical structure tree is represented by a tree data structure, with the root node being the document title of the normative document, and the structural elements at each level are organized into a tree structure according to the hierarchical relationship.

[0038] In this embodiment of the application, step S200 involves extracting terminology entities from normative documents, identifying the association and logical relationships between terminology entities, and constructing a semantic association graph based on the identification results. The steps for constructing a semantic association graph based on the recognition results include B1~B2: B1. Use named entity recognition technology to extract terminology entities from normative documents, and use dependency parsing technology to identify the relationships and logical connections between terminology entities; First, terminology entities are extracted from the paragraph-level structural elements in the hierarchical structure tree constructed in step A2. The extracted terminology entities include three categories: industry terms, standard numbers, and core responsibilities. In this embodiment, the industry terms extracted from the clause-level structural element "2.3 Technical Requirements" include terminology entities such as "number of rounds in multi-turn dialogue," "question-answer accuracy," "LoRA fine-tuning," and "token quantity." The standard numbers extracted from the clause-level structural element "1.2 Basis for Compilation" include terminology entities such as "GB / T22239-2019" and "Q / CSG2041002-2023." The core responsibilities extracted from the clause-level structural element "4.1 Project Organization" include terminology entities such as "project leader" and "technical leader." Then, dependency parsing techniques are used to identify the relationships between terminology entities and the logical relationships between sentences. The logical relationships include causal relationships, parallel relationships, conditional relationships, and adversative relationships. For example, it was found that there is a definitional relationship between "LoRA fine-tuning" and "efficient fine-tuning technology for large model parameters", a causal relationship exists in the sentence "reducing model training resource consumption through LoRA fine-tuning technology", and a parallel relationship exists in the sentence "question answering accuracy ≥90% and user concurrency ≥200".

[0039] B2. Identify key information based on the marking symbols in the normative documents, and construct a semantic association graph according to term entities, relationships, logical relationships, and key information; First, the paragraph-level structural elements in the hierarchical structure tree constructed in step A2 are scanned level by level to identify content marked with an asterisk (★) or an underline, and these are marked as key information. In this embodiment, key technical indicators marked with an asterisk are identified from the clause-level structural element "2.3 Technical Indicator Requirements," including "★ Question-answering accuracy ≥ 90%", "★ Number of concurrent users ≥ 200", and "★ Answer traceability," etc. Key compliance requirements marked with an underline are identified, including "The system should comply with the requirements of GB / T22239-2019 standard," etc. Then, according to the terminology entities extracted in step B1, the identified associations and logical relationships, and the identified key information, a semantic association graph is constructed. The semantic association graph is represented using a graph data structure, with nodes including terminology entity nodes and key information nodes, and edges including association edges and logical relationship edges. For example, there is a definitional relationship edge between the term entity node "question answer accuracy" and the key information node "★question answer accuracy ≥ 90%", and there is a causal relationship edge between the term entity node "LoRA fine-tuning" and the term entity node "model training resource consumption".

[0040] In an optional implementation, the construction of the semantic association graph based on the recognition results in step S200 can also be performed through statistical analysis of term entity co-occurrence relationships. Specifically, after extracting term entities, the co-occurrence of each term entity in the normative document is statistically analyzed, and the co-occurrence frequency and co-occurrence distance between term entities are calculated: when two term entities appear in the same paragraph-level structural element, it is recorded as one co-occurrence, and the word distance between the two term entities is calculated as the co-occurrence distance. The association strength between term entities is determined based on the co-occurrence frequency and co-occurrence distance; the higher the co-occurrence frequency and the smaller the co-occurrence distance, the stronger the association strength. Then, an association strength threshold is set, and association edges are established for term entity pairs whose association strength exceeds the threshold. At the same time, logical relationships are identified by recognizing logical connectors and punctuation marks in sentences. When a sentence contains words such as "through," "therefore," and "because," it is identified as a causal relationship; when it contains words such as "and," "and," and "as well as," it is identified as a parallel relationship; when it contains words such as "when," "if," and "if," it is identified as a conditional relationship; and when it contains words such as "but" and "however," it is identified as a contrastive relationship. For example, the term entities "safety production responsibility system" and "safety production investment" have a high co-occurrence frequency, thus establishing an association edge; the sentence "establish and improve the safety production responsibility system and ensure the effective implementation of safety production investment" contains "and", which is identified as a parallel relationship. Through co-occurrence statistics and logical connector identification, a semantic association graph containing both association and logical relationships is finally constructed.

[0041] In another optional implementation, the semantic association graph constructed in step S200 based on the identification results can also be performed through hierarchical positional relationship and citation relationship analysis. Specifically, the association relationship is determined based on the positional relationship of the term entities in the hierarchical structure tree. When two term entities are located in the same clause-level structural element or adjacent sub-item-level structural elements, an association relationship edge is established. At the same time, citation marks in the normative document are identified, including citation expressions such as "see Chapter X", "refer to Clause XX", "as described above", and "as shown below". When the paragraph containing term entity A cites the paragraph containing term entity B, an association relationship edge is established between the two term entities. For the identification of logical relationships, the order and numbering relationships between paragraph-level structural elements are used for judgment. Adjacent paragraphs with consecutive numbers are identified as parallel relationships, and paragraphs that are consecutive and contain progressive numbering are identified as causal or conditional relationships. For example, the term entity "principal responsible person" is located in the clause-level structural element "1.1", and the term entity "safety production management personnel" is located in the clause-level structural element "2.1". The two establish an association relationship edge through the citation mark "see Clause 1.1". By analyzing hierarchical positions and reference relationships, a semantic association graph reflecting the structural characteristics of normative documents is constructed.

[0042] In this embodiment of the application, step S300 involves generating slicing rules based on the hierarchical structure tree and semantic association graph, determining slicing boundaries according to the slicing rules, and performing slicing processing on the normative document based on the slicing boundaries to obtain initial slices. The steps for generating slicing rules based on the hierarchical structure tree and semantic association graph include C1~C2: C1. Determine the hierarchical priority rules and semantic integrity rules of the slices based on the hierarchical structure tree and semantic association graph, set the slice size threshold according to the token processing capacity of the large model, and generate scenario-based rules according to business scenario requirements. First, based on the hierarchical structure tree constructed in step A2, the hierarchical priority rules for slices are determined. The hierarchical priority rules stipulate that slices should preferentially use clause-level structural elements as basic units. When the number of tokens of clause-level structural elements exceeds a threshold, sub-item-level structural elements are used as basic units. When the number of tokens of sub-item-level structural elements still exceeds a threshold, paragraph-level structural elements are used as basic units. Then, based on the semantic association graph constructed in step B2, the semantic integrity rules for slices are determined. The semantic integrity rules stipulate that slice boundaries should not sever the association and logical relationships between term entities, ensuring that each slice contains complete semantic units. In this embodiment, for the clause-level structural element "2.3 Technical Indicator Requirements", the semantic integrity rule requires that the name, required value, and evaluation method of each technical indicator must be included in the same slice and cannot be split. Next, a slice size threshold is set based on the token processing capacity of the large model. For example, if the large model's token processing capacity is 32K, a single slice token number threshold of 20K~25K is set to reserve redundant space. Finally, scenario-based rules are generated according to business scenario requirements. For example, for intelligent question-answering scenarios, scenario-based rules require slices to focus on question-answer related semantic units, merging technical indicator requirements and evaluation methods into one slice to improve question-answer matching efficiency. For normative document inspection scenarios, scenario-based rules require slices to be split according to inspection dimensions, with document coverage checks, document conflict checks, and document timeliness checks corresponding to independent slices to adapt to the positioning requirements of the inspection algorithm. For large model fine-tuning scenarios, scenario-based rules require slices to retain complete domain knowledge units, including the complete process of automatic corpus annotation technology in the same slice.

[0043] C2. Combine hierarchical priority rules, semantic integrity rules, slice size thresholds, and contextual rules to generate slice rules; Based on the hierarchical priority rules, semantic integrity rules, slice size thresholds, and scenario-based rules determined in step C1, slice rules are integrated and generated. In this embodiment, the generated slice rules include: prioritizing slices based on clause-level structural elements as the basic unit; controlling the number of tokens in a single slice within the range of 20K to 25K; ensuring that slice boundaries do not sever the association and logical relationships between term entities; and generating corresponding scenario-based rules according to the needs of different business scenarios.

[0044] In an optional implementation, the generation of slicing rules based on the hierarchical structure tree and semantic association graph in step S300 can also be performed by analyzing historical slicing data. First, slicing data from previously processed normative documents is read, and the average number of tokens for different hierarchical structural elements, the optimal granularity of slicing under different business scenarios, and the boundary features of slicing with good user feedback are statistically analyzed. Based on the statistical results, recommended values ​​for hierarchical priority rules and slicing size thresholds are determined. For example, if the statistics show that the average number of tokens for clause-level structural elements is 18K and user feedback is high, the slicing size threshold is set to 15K~20K. If the statistics show that slicing with sub-item-level structural elements as the basic unit has higher accuracy in intelligent question-answering scenarios, the hierarchical priority rules are adjusted to prioritize sub-item-level structural elements. Through historical data analysis, slicing rules that better meet actual application needs are generated, improving slicing quality and user satisfaction.

[0045] In another optional implementation, the generation of slicing rules based on the hierarchical structure tree and semantic association graph in step S300 can also be performed through a rule template library matching method. Specifically, a slicing rule template library is pre-established, which contains slicing rule templates for different types of normative documents. Each slicing rule template contains preset parameters for hierarchical priority rules, semantic integrity rules, slice size thresholds, and scenario-based rules. Based on the number of levels in the hierarchical structure tree, the distribution of the number of structural elements at each level, and the number of term entities and the density of association relationships in the semantic association graph, the matching degree between the current normative document and each rule template is calculated, and the rule template with the highest matching degree is selected as the base template. Then, the base template is fine-tuned according to the specific characteristics of the current normative document. For example, the slice size threshold is adjusted according to the token processing capability of the large model, and the scenario-based rules are adjusted according to the business scenario requirements. Finally, slicing rules adapted to the current normative document are generated, improving the efficiency and accuracy of slicing rule generation.

[0046] The steps for obtaining the initial slice include D1~D2: D1. Determine the initial boundary from the hierarchical structure tree according to the hierarchical priority rules, and slice the normative document according to the initial boundary to obtain several candidate slices; First, based on the hierarchical priority rules in the slicing rules generated in step C2, clause-level structural elements are selected as basic units from the hierarchical structure tree constructed in step A2 to determine the initial boundaries. In this embodiment, for the chapter-level structural element "Chapter 2 Technical Requirements," this chapter includes clause-level structural elements such as "2.1 General Requirements," "2.2 Functional Requirements," "2.3 Technical Indicator Requirements," and "2.4 Performance Requirements." The start and end positions of each clause-level structural element are used as the initial boundaries. Then, the normative document is sliced ​​according to the initial boundaries. The content of the clause-level structural element "2.1 General Requirements" is selected as candidate slice 1, the content of the clause-level structural element "2.2 Functional Requirements" is selected as candidate slice 2, the content of the clause-level structural element "2.3 Technical Indicator Requirements" is selected as candidate slice 3, and the content of the clause-level structural element "2.4 Performance Requirements" is selected as candidate slice 4, resulting in several candidate slices.

[0047] D2. Determine whether the number of tokens in each candidate slice exceeds the slice size threshold. If it does not exceed the threshold, retain it. If it does exceed the threshold, re-slice the candidate slice according to the semantic integrity rule to finally obtain the initial slice. First, the token count of each candidate slice obtained in step D1 is counted. In this embodiment, candidate slice 1 has 18K tokens, candidate slice 2 has 22K tokens, candidate slice 3 has 35K tokens, and candidate slice 4 has 16K tokens. Then, the token count of each candidate slice is compared with the slice size threshold (20K~25K) in the slice rules generated in step C2. The token counts of candidate slice 1, candidate slice 2, and candidate slice 4 are all within the threshold range or do not exceed the upper limit, and are therefore retained. The token count of candidate slice 3 is... The token count is 35K, exceeding the slice size threshold, requiring further slicing. At this point, candidate slice 3 is re-sliced ​​according to the semantic integrity rule. Candidate slice 3 contains 15 technical indicators. Based on the relationships in the semantic association graph, the first 10 technical indicators (including "number of multi-turn dialogue rounds" to "ROUGE score") are merged into one slice with 22K tokens. The last 5 technical indicators (including "number of high-quality sample data" to "answer traceability") are merged into another slice with 13K tokens. Both slices meet the slice size threshold requirement and maintain semantic integrity. Finally, five initial slices are obtained: candidate slice 1, candidate slice 2, the two slices resulting from splitting candidate slice 3, and candidate slice 4.

[0048] In this embodiment of the application, step S400 involves verifying the initial slice based on the semantic association graph, adjusting the slice boundary according to the verification result, and generating the final slice based on the adjusted slice boundary. The steps for adjusting the slice boundaries according to the verification results include E1 to E3: E1. Perform semantic coherence verification on each initial slice based on the semantic association graph, and detect whether there are any disconnected association or logical relationships between term entities at the boundaries of the initial slice. First, the initial slices obtained in step D2 are read, and semantic coherence is checked at the boundaries of each initial slice. In this embodiment, initial slice 1 (corresponding to the clause-level structural element "2.1 General Requirements") is checked. Based on the semantic association graph constructed in step B2, it is detected whether there is an association relationship between the end boundary of the initial slice and the content of the next slice. Then, it is checked whether the boundary of the initial slice severs the association or logical relationship between term entities. For example, it is detected that the term entity "LoRA fine-tuning" exists at the end boundary of initial slice 2 (corresponding to the clause-level structural element "2.2 Functional Requirements"). This term entity has a causal relationship with the term entity "model training resource consumption" in initial slice 3, but the two term entities are separated by the slice boundary, and the semantic coherence check fails. At the same time, it is detected that at the end boundary of the first slice after the initial slice 3 is split, the term entity "question-answer accuracy" has a parallel relationship with the term entity "answer traceability" in the next slice, but it is separated by the slice boundary, and the semantic coherence check fails.

[0049] E2. Perform a completeness check on the key information of each initial slice to detect whether the key information is completely contained in a single initial slice. First, the key information identified in step B2 is read, including content marked with an asterisk (★) or underlined. Key information includes key clauses, technical indicators, and compliance standards. In this embodiment, key information related to technical indicators includes "★ Question-answering accuracy ≥ 90%", "★ User concurrency ≥ 200", and "★ Answer traceability," while key information related to compliance standards includes "The system should comply with the requirements of GB / T22239-2019 standard." Then, it is checked whether each initial slice completely contains the key information. For example, the first slice after splitting initial slice 3 is detected to contain the indicator name and required value of the key information "★ Question-answering accuracy ≥ 90%", but the evaluation method for this key information, "Evaluated using ROUGE scoring," is located in the next paragraph outside the slice boundary. The key information is split, and the integrity check fails. Simultaneously, it is detected that initial slice 4 (corresponding to the clause-level structural element "2.4 Performance Requirements") completely contains the indicator name, required value, and evaluation method of the key information "★ User concurrency ≥ 200", and the integrity check passes.

[0050] E3. For initial slices that fail semantic coherence or integrity checks, adjust the slice boundaries until related term entities are merged into the same slice and key information is fully preserved in a single slice. Generate the final slice based on the adjusted slice boundaries. First, for the initial slice 2 that failed the semantic coherence check in step E1, the end boundary was adjusted by merging adjacent paragraphs. The paragraph containing the term entity "model training resource consumption," which has a causal relationship with the term entity "LoRA fine-tuning," was merged into the initial slice 2. The adjusted slice 2 has 24K tokens, meeting the slice size threshold requirement. Then, for the first slice after splitting the initial slice 3 that failed the integrity check in step E2, the end boundary was adjusted accordingly. The evaluation method paragraph containing the key information "★Question-answering accuracy ≥ 90%" was merged into this slice. The adjusted slice has 23K tokens, meeting the slice size threshold requirement and ensuring the key information is complete. For individual slices with a token count exceeding the threshold, adjustments were made by splitting the non-core semantic parts of excessively long clauses. In this embodiment, if the token count of a single slice reaches 28K, the example description part in that single slice is split into an independent slice, while retaining the core technical requirements in the original slice. After adjusting all the initial slices, a total of 5 final slices were generated, and each final slice passed both semantic coherence and integrity checks. Finally, metadata tags are added to each final slice, including slice ID, hierarchy, corresponding document page number, core terms, and business scenario tags. For example, metadata tags are added to final slice 1: slice ID is "Chunk_2.1", hierarchy is "clause level", corresponding document page number is "pages 5-6", core terms are "general requirements, system architecture", and business scenario tags are "intelligent Q&A, compliance inspection". At the same time, a slice index library is established to support quick retrieval by hierarchy, terminology, and scenario. Users can quickly retrieve all slices containing the term by entering "Q&A accuracy".

[0051] In this embodiment of the application, step S500 involves iterating the slicing rules through an optimization mechanism; The optimization mechanism includes steps F1 to F2: F1. Collect feedback data and performance data when the large model processes the final slice. Analyze the feedback data and performance data to determine the problem type corresponding to each feedback data and performance data. Determine the degree of correlation between the problem type and the hierarchical priority rule, semantic integrity rule, slice size threshold and scenario rule, and determine the correlation rule. First, feedback data from users during business applications and performance data from the large model processing the final slices are collected. In this embodiment, the collected feedback data includes user feedback in the intelligent question-answering scenario such as "the response is very slow when querying technical indicator requirements," and user feedback in the normative document checking scenario such as "the compliance check results are not accurate enough." The collected performance data includes data such as the question-answering accuracy and processing speed of the large model when processing relevant slices. Then, based on the feedback content, the corresponding final slice is located, and the problem type corresponding to each feedback data and performance data is analyzed and determined. For example, "slow response when querying technical indicator requirements" is located in the final slice 1.1, and the problem type is determined to be "slice granularity is too large". "Inaccurate compliance check results" is located in the final slice 2.3, and the problem type is determined to be "slice semantic incompleteness". Next, the degree of correlation between the problem type and the hierarchical priority rule, semantic integrity rule, slice size threshold and scenario rule is judged. It is found that "slice granularity is too large" is highly correlated with slice size threshold and hierarchical priority rule, and "slice semantic incompleteness" is highly correlated with semantic integrity rule. Finally, the correlation rules are determined according to the degree of correlation. For example, slice size threshold, hierarchical priority rule and semantic integrity rule are determined as correlation rules.

[0052] F2. Adjust the association rules to optimize the slicing rules; In this embodiment, to address the issue of "excessively large slice granularity," the slice size threshold is reduced from 20K~25K to 15K~20K. Simultaneously, the hierarchical priority rule is adjusted, prioritizing slicing based on sub-item-level structural elements as the basic unit. To address the issue of "incomplete slice semantics," the judgment criteria for semantic integrity rules are adjusted, increasing the identification of indirect relationships between terminology entities to ensure that slice boundaries do not sever indirectly related terminology entities. To address the issue of "insufficient slice semantic association," the scenario-based rules are adjusted. For intelligent question-answering scenarios, question-answering related technical indicators are separated into a single slice, reducing slice granularity and focusing on question-answer semantic units. After these adjustments, steps S100~S400 are re-executed to generate the optimized final slices. New feedback data and performance data are collected for verification, thus optimizing the slicing rules and improving slice quality and business application effectiveness.

[0053] In summary, by constructing a hierarchical structure tree and a semantic association graph, a deep analysis of the chapter and clause structure and semantic relationships of terms in normative documents was achieved. During the generation of slicing rules, hierarchical priority rules ensure that slices are based on meaningful structural units such as clause or subheading levels, avoiding the mechanical fragmentation of document structure caused by fixed-length slicing. Semantic integrity rules, on the other hand, constrain slice boundaries through the relationships and logical connections in the semantic association graph, preventing the semantic connections between terminology entities from being broken by slice boundaries. More importantly, the setting of slice size thresholds fully considers the token processing capabilities of large models, achieving adaptation to the processing capabilities of large models while ensuring semantic integrity. The introduction of scenario-based rules further addresses the specific needs of different business scenarios such as intelligent question answering, compliance checks, and large model fine-tuning, generating differentiated slicing strategies so that the same normative document can achieve optimal slicing results in different application scenarios.

[0054] During the slicing execution phase, a dual verification mechanism—semantic coherence verification and key information integrity verification—is used to finely adjust the initial slices. This ensures that each slice neither severs related terminology entities due to improper boundary settings nor leads to the splitting or omission of important information such as key technical indicators and compliance standards due to slice division. More importantly, this invention also establishes a continuous optimization mechanism based on user feedback data and large model performance data. By analyzing the types of problems in practical applications and locating the corresponding slicing rules, dynamic adjustment and iterative optimization of the slicing strategy are achieved. This closed-loop optimization mechanism allows the slice quality to continuously improve as the application scenario deepens, thereby comprehensively improving the application effect and user experience of the large model in normative document processing.

[0055] Example 3 illustrates a schematic scheme for a hierarchical structure-aware semantic slicing method for normative documents. It should be noted that the technical solution of this hierarchical structure-aware semantic slicing system for normative documents belongs to the same concept as the technical solution of the hierarchical structure-aware semantic slicing method for normative documents described above. Details not described in detail in the technical solution of the hierarchical structure-aware semantic slicing system for normative documents in this embodiment can be found in the description of the technical solution of the hierarchical structure-aware semantic slicing method for normative documents described above.

[0056] This embodiment also provides a hierarchical structure-aware semantic slicing system for normative documents, including: Hierarchical structure parsing module: used to parse the format content of normative documents, extract structural elements, and construct a hierarchical structure tree based on the format characteristics and subordinate relationships of the structural elements; Semantic association analysis module: used to extract term entities from normative documents, identify the association and logical relationships between term entities, and construct a semantic association graph; Slicing rule generation module: used to generate slicing rules based on the hierarchical structure tree and semantic association graph; The intelligent slicing execution module is used to slice the normative document according to the slicing rules to obtain the initial slice, and to verify and adjust the initial slice according to the semantic association graph to generate the final slice. Slicing optimization module: Used to collect feedback data and performance data of the final slices, and optimize slicing rules based on the feedback data and performance data.

[0057] This embodiment also provides an electronic device applicable to the case of hierarchical structure-aware semantic slicing of normative documents, including: a memory and a processor; the memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions to implement the hierarchical structure-aware semantic slicing method for normative documents as proposed in the above embodiment.

[0058] This embodiment also provides a storage medium on which a computer program is stored. When the program is executed by a processor, it implements the hierarchical structure-aware semantic slicing method for normative documents as proposed in the above embodiments.

[0059] The storage medium proposed in this embodiment and the hierarchical structure-aware semantic slicing method for normative documents proposed in the above embodiments belong to the same inventive concept. Technical details not described in detail in this embodiment can be found in the above embodiments, and this embodiment has the same beneficial effects as the above embodiments.

[0060] Based on the above description of the implementation methods, those skilled in the art can clearly understand that the present invention can be implemented using software and necessary general-purpose hardware, and of course, it can also be implemented using hardware. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as a computer floppy disk, read-only memory (ROM), random access memory (RAM), flash memory, hard disk, or optical disk, etc., including several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods of the various embodiments of the present invention.

[0061] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A method for semantic slicing of normative documents based on hierarchical structure awareness, characterized in that, include: Parse the format content of the normative document to obtain several structural elements, and construct a hierarchical structure tree based on the format characteristics and subordinate relationships of the structural elements; The terminology entities are extracted from the normative documents, and the relationships and logical connections between the terminology entities are identified. A semantic association graph is constructed based on the identification results. Slicing rules are generated based on the hierarchical structure tree and semantic association graph, and slice boundaries are determined according to the slicing rules. The normative document is then sliced ​​based on the slice boundaries to obtain the initial slices. The initial slice is validated based on the semantic association graph, the slice boundaries are adjusted according to the validation results, and the final slice is generated based on the adjusted slice boundaries.

2. The hierarchical structure-aware semantic slicing method for normative documents as described in claim 1, characterized in that, The steps for constructing a hierarchical structure tree based on the format characteristics and hierarchical relationships of structural elements include: Identify the document format of the normative document, and select the appropriate parsing method to extract the format features of the structural elements based on the document format; The structural elements are classified hierarchically according to the format features, and the subordinate relationships between structural elements at each level are determined to construct the hierarchical structure tree.

3. The hierarchical structure-aware semantic slicing method for normative documents as described in claim 2, characterized in that, The steps for constructing a semantic association graph based on the recognition results include: Named entity recognition technology is used to extract terminology entities from the normative documents, and dependency parsing technology is used to identify the association and logical relationships between the terminology entities. Identify key information based on the marking symbols in the normative documents, and construct a semantic association graph according to the term entities, relationships, logical relationships, and key information.

4. The hierarchical structure-aware semantic slicing method for normative documents as described in claim 3, characterized in that, The steps for generating slicing rules based on the hierarchical structure tree and semantic association graph include: The hierarchical priority rules and semantic integrity rules of the slices are determined based on the hierarchical structure tree and semantic association graph, and the slice size threshold is set according to the token processing capability of the large model. At the same time, scenario-based rules are generated according to business scenario requirements. The slicing rules are generated by combining the hierarchical priority rules, semantic integrity rules, slice size thresholds, and contextual rules.

5. The hierarchical structure-aware semantic slicing method for normative documents as described in claim 4, characterized in that, The steps to obtain the initial slice include: The initial boundary is determined from the hierarchical structure tree according to the hierarchical priority rule, and the normative document is sliced ​​according to the initial boundary to obtain several candidate slices; Determine whether the number of tokens in each candidate slice exceeds the slice size threshold. If it does not exceed the threshold, retain it. If it does exceed the threshold, re-slice the candidate slice according to the semantic integrity rule to finally obtain the initial slice.

6. The hierarchical structure-aware semantic slicing method for normative documents as described in claim 5, characterized in that, The steps for adjusting the slice boundaries according to the verification results include: Based on the semantic association graph, the semantic coherence of each initial slice is checked to detect whether there are any association or logical relationships between the term entities at the boundary of the initial slice. Perform a completeness check on the key information of each initial slice to detect whether the key information is completely contained in a single initial slice; For initial slices that fail semantic coherence or integrity checks, adjust the slice boundaries until related term entities are merged into the same slice and key information is fully preserved in a single slice. The final slice is then generated based on the adjusted slice boundaries.

7. The hierarchical structure-aware semantic slicing method for normative documents as described in claim 1 or 6, characterized in that, Also includes: The slicing rules are iterated through an optimization mechanism; The optimization mechanism includes the following steps: Collect feedback data and performance data when the large model processes the final slice, analyze the feedback data and performance data, determine the problem type corresponding to each feedback data and performance data, and determine the degree of correlation between the problem type and the hierarchical priority rule, semantic integrity rule, slice size threshold and scenario rule, and determine the correlation rule; The association rules are adjusted to optimize the slicing rules.

8. A hierarchical structure-aware semantic slicing system for normative documents, employing the method described in any one of claims 1-7, characterized in that, include: Hierarchical structure parsing module: used to parse the format content of normative documents, extract structural elements, and construct a hierarchical structure tree based on the format characteristics and subordinate relationships of the structural elements; Semantic association analysis module: used to extract term entities from the normative documents, identify the association and logical relationships between term entities, and construct a semantic association graph; Slicing rule generation module: used to generate slicing rules based on the hierarchical structure tree and semantic association graph; The intelligent slicing execution module is used to slice the normative document according to the slicing rules to obtain the initial slice, and to verify and adjust the initial slice according to the semantic association graph to generate the final slice. Slicing optimization module: Used to collect feedback data and performance data of the final slice, and optimize the slicing rules based on the feedback data and performance data.

9. An electronic device, comprising: Memory and processor; The memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions, which, when executed by the processor, implement the steps of the hierarchical structure-aware normative document semantic slicing method according to any one of claims 1 to 7.

10. A computer-readable storage medium storing computer-executable instructions that, when executed by a processor, implement the steps of the hierarchical structure-aware normative document semantic slicing method according to any one of claims 1 to 7.

Images