Intelligent question-answering system for normative documents based on mixed vectors

Abstract

The application belongs to the technical field of data processing, and discloses a normative file intelligent question and answer system based on a mixed vector, which comprises a structure analysis module, extracts natural hierarchical information, and splits a university normative file into a plurality of clause atomic segments; for each clause atomic segment, a corresponding hierarchical path identifier and an original position index are generated, and the clause atomic segments are packaged and constructed into a clause atomic segment set through association; a knowledge base construction module performs multi-dimensional semantic modeling on the clause atomic segment set, and generates dense vector representation and sparse vector representation respectively; a mixed vector index is constructed based on the dense vector representation and the sparse vector representation; and fine structure modeling of the university normative file is realized, and clause retrieval accuracy is improved.

Description

Technical Field

[0001] This invention relates to the field of data processing technology, and more specifically, to a standardized document intelligent question-answering system based on hybrid vectors. Background Technology

[0002] Existing intelligent question-and-answer systems for university normative documents mainly suffer from the following problems:

[0003] The search primarily employs full-text or large-paragraph search modes, failing to analyze the natural hierarchical structure of university normative documents and thus unable to extract the smallest semantic unit—the atomic fragment of a clause. This mode, limited by the overall nature of the text, easily incorporates a large amount of irrelevant and redundant information during the search process, leading to search bias and making it difficult to accurately match the core needs of users.

[0004] Without configuring hierarchical path identifiers and original location indexes for the search units, it is impossible to represent the logical hierarchical relationship of clauses in the normative system, making it difficult to support the generation of structured answers; it is also impossible to achieve accurate tracing of search results to original documents, failing to meet the core requirements of universities for authoritative sources and location certainty in normative document searches, thus limiting the credibility and practicality of search results.

[0005] Many retrieval methods employ a single vector representation model: dense vectors struggle to accurately match key terms and literal features, leading to missed term detections; sparse vectors fail to capture deep semantics and contextual relationships, resulting in semantic bias. Furthermore, retrieval methods based on single vector indexes cannot simultaneously guarantee high recall while improving retrieval precision, making it difficult to accurately identify user query intent and match the most relevant terms.

[0006] In view of this, the present invention proposes a hybrid vector-based intelligent question-answering system for normative documents to solve the above problems. Summary of the Invention

[0007] To overcome the aforementioned shortcomings of the prior art and to achieve the above objectives, the present invention provides the following technical solution: a prescriptive document intelligent question-answering system based on hybrid vectors, comprising:

[0008] The structure parsing module extracts natural hierarchical information to break down university normative documents into several clause atomic fragments; for each clause atomic fragment, it generates a corresponding hierarchical path identifier and original position index, and constructs a set of clause atomic fragments through association and encapsulation.

[0009] The knowledge base construction module performs multi-dimensional semantic modeling on the set of atomic fragments of terms, generating dense vector representations and sparse vector representations respectively; based on the dense vector representations and sparse vector representations, a hybrid vector index is constructed.

[0010] The dual-stream hybrid retrieval module receives and parses user query commands, extracts query intent and key terms to generate query feature vectors, and performs parallel retrieval on the hybrid vector index based on the query feature vectors to obtain an initial set of candidate terms.

[0011] The conflict resolution module detects clauses in the initial candidate clause set that have conflicts or mutual exclusions between old and new clauses, automatically removes expired or replaced clauses, and moves the current clauses to the highest position to form a clause sequence after conflict resolution.

[0012] The structured generation module is used to select target clauses from the clause sequence after conflict resolution and generate structured intelligent question-and-answer results by combining the natural hierarchical information of the target clauses.

[0013] Preferably, the method for extracting natural hierarchy information includes:

[0014] The system acquires the text of normative documents from universities and colleges to be processed, performs standardization preprocessing on the content of the text, removes non-text content, standardizes the numbering format, and preserves the original text order, thereby forming parsable normative text data; based on preset normative document structure rules, it identifies the natural hierarchical structure markers in the normative text data; natural hierarchical structure markers include chapters, sections, articles, and structural identifiers below the article level;

[0015] By traversing the identified natural hierarchical structure markers, and combining the nesting relationships and order of appearance between the hierarchical structure markers, the starting and ending positions of different hierarchical structures are determined, and a tree-like hierarchical path representing the membership relationship between each hierarchical structure is established, thus completing the acquisition of natural hierarchical information.

[0016] Preferably, the method for obtaining the set of clause atomic fragments includes:

[0017] The normative text data is traversed based on a tree-like hierarchical path. The clauses of the normative documents of universities are divided into several clause atomic fragments according to the natural hierarchical structure markers as the segmentation units. Each clause atomic fragment contains the corresponding text content of the normative documents of universities.

[0018] For each clause atomic fragment, the corresponding complete hierarchical path is extracted from the tree-like hierarchical path, and the hierarchical path is encoded according to a preset format to generate the corresponding hierarchical path identifier; at the same time, the position information of each clause atomic fragment in the university normative documents is recorded to form the original position index; the generated clause atomic fragments are associated one by one with the corresponding hierarchical path identifiers and original position indexes, and encapsulated into a clause atomic fragment set.

[0019] Preferably, the method for generating dense vector representations and sparse vector representations includes:

[0020] For any clause atomic fragment in the set of clause atomic fragments, the corresponding text of the university normative document is text-encoded to generate a text semantic vector. At the same time, a hierarchical path embedding vector is constructed based on the hierarchical path identifier to which the clause atomic fragment belongs. The obtained text semantic vector and the hierarchical path embedding vector are jointly modeled to generate a dense vector representation that reflects the semantics of the clause content and the semantics of the structure.

[0021] Based on a pre-defined set of normative terms, terminology recognition and weighting are performed on the text of the normative documents of universities corresponding to the atomic fragments of clauses. By detecting the normative terms appearing in the atomic fragments of clauses and their corresponding frequencies, the normative terms are assigned corresponding weight values, thereby constructing a sparse vector representation for keyword retrieval.

[0022] Preferably, the method for constructing the hybrid vector index includes:

[0023] For each clause atomic fragment, the corresponding dense vector representation and sparse vector representation are stored together. At the same time, corresponding vector index structures are constructed for the dense vector representation and sparse vector representation respectively. The two vector index structures are uniformly associated with the same clause atomic fragment through index mapping relationship, thereby forming a hybrid vector index that supports the parallel execution of semantic search and keyword search.

[0024] Preferably, the method for obtaining the query feature vector includes:

[0025] The system receives user query commands and parses them, including word segmentation, stop word removal, part-of-speech tagging, and word form restoration. It uses natural language processing technology to identify the core elements in the query, including the target terms, legal concepts and requirements, thereby extracting the query intent and key terms.

[0026] The TF-IDF algorithm maps each key term in the query to a corresponding word vector. By combining the word vectors in the query and using weighted averaging or vector concatenation, a query feature vector for retrieval is generated.

[0027] Preferably, the method for obtaining the initial candidate clause set includes:

[0028] Based on the query feature vector, parallel retrieval is performed on the hybrid vector index, simultaneously performing semantic retrieval and keyword retrieval. During each query, a set of supporting terms is generated, and the stable support of the terms is calculated based on the set of supporting terms generated for each query.

[0029] Stable support is based on the frequency of a clause's appearance in the query. A preset stable support threshold is used. If a clause's stable support is greater than or equal to the preset stable support threshold, the clause is considered a stable clause and included in the final search results, thus obtaining an initial set of candidate clauses after stable support filtering.

[0030] Preferably, the method for detecting clauses in the initial candidate clause set that have conflicts or mutual exclusions includes:

[0031] By analyzing clause version information, clause content similarity, and hierarchical relationships between clauses, the validity and evolution information of each candidate clause in the candidate clause set are extracted. Based on the validity and evolution information of each candidate clause, conflict detection is performed on all candidate clauses in the candidate clause set. By judging whether there are expired, replaced, or mutually exclusive situations among the candidate clauses, the new and old conflict relationship, substitution relationship, or mutual exclusion relationship between the candidate clauses is identified.

[0032] Preferably, the method for forming the conflict-resolved clause sequence includes:

[0033] Candidate clauses that have conflicting relationships and have expired, as well as candidate clauses that have been substituted and have been replaced, are removed from the candidate clause set. For candidate clauses that are mutually exclusive, based on the validity information of the candidate clauses, the candidate clauses that have taken effect later and are still valid are moved to the highest position and given higher ranking priority. The remaining candidate clauses in the mutually exclusive relationships are removed, thus forming a clause sequence after the conflict is resolved.

[0034] Preferably, the method for generating structured intelligent question-answering results includes:

[0035] Based on the matching degree between the user's query intent and the terms content, the terms most relevant to the query are selected from the conflict-resolved terms sequence as the target terms. The matching degree is calculated based on the semantic similarity between the query feature vector and the terms content.

[0036] By utilizing the natural hierarchical information of the clauses, the hierarchical structure of the target clause in the document is parsed to obtain the hierarchical path and corresponding contextual information of the target clause, so as to clarify the position and function of the clause in the document; combined with the hierarchical path, number and content of the target clause, a structured intelligent question and answer result is generated.

[0037] Compared with the prior art, the present invention has the following beneficial effects:

[0038] This invention extracts information from natural hierarchical levels and breaks down university normative documents into atomic fragments of clauses, breaking the text integrity limitation of traditional full-text retrieval. It achieves retrieval and matching at the smallest semantic unit granularity, avoiding retrieval bias caused by redundant information in the full text.

[0039] By generating hierarchical path identifiers and original location indexes for each clause's atomic fragment, not only is the logical position and subordinate relationship of the clause in the normative system accurately represented, but a structural basis is also provided for subsequent structured answers; at the same time, it realizes accurate tracing of clause atomic fragments and original documents, meeting the core needs of universities for source authority and location certainty in normative document queries;

[0040] The generated dense vector representation can effectively capture the deep semantics and contextual relationships of the atomic fragments of the clauses, while the sparse vector representation can accurately match key terms and literal features. The two representation forms are complementary and solve the limitations of single vector representation in semantic understanding or term matching. At the same time, parallel retrieval based on hybrid vector index can simultaneously use semantic similarity and term matching degree to filter the initial candidate clause set. Compared with the traditional single retrieval method, it significantly improves the retrieval accuracy while ensuring high recall, ensuring that the user's query intent can be accurately identified and matched with the most relevant clauses.

[0041] By calculating the stability support of clauses and filtering stable clauses based on their performance in multiple queries, the system effectively avoids random errors from single queries, improving the stability and consistency of search results. Combining semantic and keyword parallel retrieval allows for the rapid identification of stable clauses highly relevant to the query intent, improving retrieval efficiency while ensuring the accuracy of search results. Attached Figure Description

[0042] Figure 1 This is a schematic diagram of the structure of the intelligent question-answering system for normative documents based on hybrid vectors according to the present invention;

[0043] Figure 2 This is a schematic diagram of the intelligent question-answering method for normative documents based on hybrid vectors according to the present invention. Detailed Implementation

[0044] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0045] Example 1

[0046] Please see Figure 1As shown, this embodiment provides an intelligent question-answering system for normative documents based on hybrid vectors, specifically including the following steps:

[0047] The structure parsing module extracts natural hierarchical information to break down university normative documents into several clause atomic fragments; for each clause atomic fragment, it generates a corresponding hierarchical path identifier and original position index, and constructs a set of clause atomic fragments through association and encapsulation.

[0048] The knowledge base construction module performs multi-dimensional semantic modeling on the set of atomic fragments of terms, generating dense vector representations and sparse vector representations respectively; based on the dense vector representations and sparse vector representations, a hybrid vector index is constructed.

[0049] The dual-stream hybrid retrieval module receives and parses user query commands, extracts query intent and key terms to generate query feature vectors, and performs parallel retrieval on the hybrid vector index based on the query feature vectors to obtain an initial set of candidate terms.

[0050] The conflict resolution module detects clauses in the initial candidate clause set that have conflicts or mutual exclusions between old and new clauses, automatically removes expired or replaced clauses, and moves the current clauses to the highest position to form a clause sequence after conflict resolution.

[0051] The structured generation module is used to select target clauses from the conflict-resolved clause sequence and generate structured intelligent question-and-answer results by combining the natural hierarchical information of the target clauses.

[0052] Methods for extracting natural hierarchical information include:

[0053] The system acquires the text of normative documents from higher education institutions, performs standardization preprocessing on the content of the text to remove non-text content, standardize the numbering format, and retain the original text order, thereby forming parsable normative text data; based on preset normative document structure rules, it identifies the natural hierarchical structure markers in the normative text data; natural hierarchical structure markers include chapters, sections, articles, and structural identifiers below the article level (sections, items, or headings).

[0054] By traversing the identified natural hierarchical structure markers, and combining the nesting relationships and order of appearance between the hierarchical structure markers, the starting and ending positions of different hierarchical structures are determined, and a tree-like hierarchical path representing the membership relationship between each hierarchical structure is established, thus completing the acquisition of natural hierarchical information.

[0055] Methods for obtaining the set of atomic fragments of a clause include:

[0056] The normative text data is traversed based on a tree-like hierarchical path. The clauses of the normative documents of universities are divided into several clause atomic fragments according to the natural hierarchical structure markers as the segmentation units. Each clause atomic fragment contains the corresponding text content of the normative documents of universities.

[0057] For each clause atomic fragment, the corresponding complete hierarchical path is extracted from the tree-like hierarchical path, and the hierarchical path is encoded according to a preset format to generate the corresponding hierarchical path identifier; at the same time, the position information of each clause atomic fragment in the university normative documents is recorded to form the original position index; the generated clause atomic fragments are associated one by one with the corresponding hierarchical path identifiers and original position indexes, and encapsulated into a clause atomic fragment set, which serves as the basic input data for subsequent knowledge base construction modules to perform multi-dimensional semantic modeling and hybrid vector index construction;

[0058] The hierarchical path identifier is composed of the chapter, section, and article, as well as the structural identifiers below the article level, where the corresponding clause atomic fragment is located, in the order of their hierarchical relationship. It is used to uniquely represent the position of the clause atomic fragment in the structure system of normative documents of higher education institutions.

[0059] The original location index contains the start and end positions of clause atomic fragments in the university's normative documents, which are determined by character sequence numbers, line numbers, or paragraph numbers. It is used to construct a precise mapping relationship between clause atomic fragments and the original normative document text.

[0060] Methods for generating dense and sparse vector representations include:

[0061] For any clause atomic fragment in the set of clause atomic fragments, the corresponding text of the university normative document is text-encoded to generate a text semantic vector. At the same time, a hierarchical path embedding vector is constructed based on the hierarchical path identifier to which the clause atomic fragment belongs. The obtained text semantic vector and the hierarchical path embedding vector are jointly modeled to generate a dense vector representation that reflects the semantics of the clause content and the semantics of the structure.

[0062] Specifically: First, the main text of the clause atomic fragments is text-encoded to generate a text semantic vector representation. ;in, This represents the encoding function used to extract semantic features from the clause text; Indicates the first The content of the normative document of the university corresponding to each clause's atomic fragment; Indicates the first The text semantic vector corresponding to each clause atomic fragment; For the index of the atomic fragment of the clause;

[0063] Simultaneously, based on the hierarchical path identifier of the clause atomic fragment, the corresponding complete hierarchical path is identified. Construct hierarchical path embedding vectors ;in, Indicates the first Hierarchical path identifiers; Indicates the first Embedded representation of hierarchical path identifiers; This is the hierarchical weighting coefficient, used to reflect the degree of influence of different hierarchical structures on the semantics of clauses; Indicates the first Hierarchical path embedding vectors corresponding to each clause atomic fragment; Indicates the first Hierarchical path identifiers For hierarchical indexes, ;

[0064] Based on this, the text semantic vector and the hierarchical path embedding vector are concatenated or weighted and fused to generate a dense vector representation. Among them, symbols This indicates a vector concatenation or weighted fusion operation; Indicates the first Dense vector representation of each clause's atomic fragment;

[0065] Based on a pre-defined set of normative terms, terminology recognition and weighting are performed on the text of the normative documents of universities corresponding to the atomic fragments of clauses. By detecting the normative terms appearing in the atomic fragments of clauses and their corresponding frequencies, corresponding weight values ​​are assigned to the normative terms, thereby constructing a sparse vector representation for keyword retrieval. The sparse vector representation is used to characterize the matching features of the atomic fragments of clauses in the terminology dimension, so as to support the judgment of keyword matching relationships in the subsequent retrieval process.

[0066] It should be noted that the preset set of normative terms is used to characterize terms with clear institutional meaning or binding force in university normative documents. It can be obtained based on historical university normative documents, management system texts or manual rules, and will be continuously updated during system operation.

[0067] The text of the relevant university normative documents corresponding to the atomic fragments of the clauses is compared item by item according to the normative terminology set to detect whether there are any terms that match the terms in the normative terminology set; when a normative term is detected, the occurrence of the corresponding normative term in the current atomic fragment of the clause is recorded;

[0068] The detected normative terms are statistically analyzed for frequency of occurrence. The frequency of occurrence is used to reflect the importance of the corresponding normative term in the clause atomic fragment. The frequency of occurrence can be determined by counting the number of times or the density of occurrence of the normative term in the current clause atomic fragment.

[0069] Based on the frequency of occurrence of normative terms, each normative term is assigned a corresponding weight value. The weight value is used to reflect the semantic contribution of the normative term in the current clause atomic fragment. Normative terms that appear more frequently in the clause atomic fragment can be assigned a relatively higher weight value, while normative terms that appear less frequently can be assigned a relatively lower weight value. This allows the constructed sparse vector representation to highlight the features of normative terms that contribute significantly to the meaning of the system in the clause atomic fragment, thus supporting subsequent retrieval and judgment based on keyword matching relationships.

[0070] For example, for a certain clause, the corresponding text of the university's normative document is: "Students who violate examination discipline will be given a warning, a serious warning, or a demerit, depending on the severity of the circumstances; in serious cases, they will be placed on probation or expelled from the university." The pre-set set of normative terms includes normative terms such as "punishment," "warning," "demerit," "probation," and "expulsion."

[0071] When performing term recognition and weighting on the atomic fragments of the clauses, the system detected that the normative terms "disciplinary action" appeared twice; "warning" appeared twice; "demerit" appeared once; "probation" appeared once; and "expulsion" appeared once.

[0072] Based on the above frequency statistics, different weight values ​​are assigned to each normative term. For example, the normative terms "punishment" and "warning" that appear more frequently are assigned higher weight values ​​(e.g., 29%).

[0073] The less frequently used normative terms "demerit", "probation", and "expulsion" are assigned a lower weight value (e.g., 14%).

[0074] Methods for constructing hybrid vector indices include:

[0075] For each clause atomic fragment, the corresponding dense vector representation and sparse vector representation are stored together. At the same time, corresponding vector index structures are constructed for the dense vector representation and sparse vector representation respectively. The two vector index structures are uniformly associated with the same clause atomic fragment through index mapping relationship, thereby forming a hybrid vector index that supports the parallel execution of semantic search and keyword search.

[0076] Methods for obtaining query feature vectors include:

[0077] The system receives user query commands and parses them, including word segmentation, stop word removal, part-of-speech tagging, and lemmatization. It uses natural language processing (NLP) technology to identify the core elements in the query, including the target terms, legal concepts and requirements, thereby extracting the query intent and key terms.

[0078] The TF-IDF algorithm maps each key term in the query to a corresponding word vector. By combining the word vectors in the query and using weighted averaging or vector concatenation, a query feature vector for retrieval is generated.

[0079] Methods for obtaining the initial candidate clause set include:

[0080] Based on the query feature vector, parallel retrieval is performed on the hybrid vector index, simultaneously performing semantic retrieval and keyword retrieval. During each query, a set of supporting terms is generated, and the stable support of the terms is calculated based on the set of supporting terms generated for each query.

[0081] Stable support level: ;in, Terms and Conditions Stable support; Indicates the first Support set for this query; This indicates an indicator function used to determine the terms. Does it belong to the first The support set for this query, when the terms Appeared in the When querying the support set. Otherwise, it is 0; An index variable representing the number of queries; Indicates the total number of queries; Indicates the first Support set for this query;

[0082] Stable support is based on the frequency of a clause's appearance in the query. A preset stable support threshold is used. If a clause's stable support is greater than or equal to the preset stable support threshold, the clause is considered a stable clause and included in the final search results, thus obtaining an initial set of candidate clauses after stable support filtering.

[0083] This paper addresses the following technical problems existing in current technologies: Traditional retrieval systems largely rely on the matching degree of a single query to determine the relevance of results. However, this method is easily affected by the query itself or retrieval conditions, leading to unstable or inconsistent retrieval results. Traditional retrieval systems only rely on the matching of query feature vectors with the index, without considering the performance and stability of terms across multiple queries. This may result in the return of terms that are rarely queried or perform poorly in certain queries, affecting the reliability of the final results. In multiple queries, some terms may appear briefly in some queries but lack sustained relevance. Traditional methods fail to effectively distinguish these terms, resulting in unstable terms being mixed into the retrieval results, thus affecting the overall system performance.

[0084] The advantages over existing technologies are as follows: By calculating the stability support of clauses and filtering stable clauses based on their performance across multiple queries, the system effectively avoids random errors in a single query, ensuring that clauses in the search results maintain stable performance across multiple queries, thereby improving the stability and consistency of the search results. Introducing a stability support threshold allows for more precise clause filtering. Clauses with high matching rates in most queries are considered stable clauses and included in the final search results. This enables the system to more accurately identify truly relevant and stable clauses, rather than relying solely on the matching rate of a single query to determine the validity of a clause. Since stability support is calculated based on the frequency of a clause's occurrence across multiple queries, the system can effectively exclude low-quality clauses that only appear in a few queries, preventing low-frequency, high-noise clauses from affecting the final results. By executing semantic and keyword searches in parallel, combined with the stability support filtering mechanism, the system can quickly identify clauses that are highly relevant and stable to the query intent, thereby effectively improving search efficiency. Simultaneously, the stability support-based filtering mechanism improves the accuracy of search results, ensuring that the clauses presented to the user are the most suitable for their query needs.

[0085] Methods for detecting conflicts or mutual exclusions between new and old clauses in the initial candidate clause set include:

[0086] By analyzing clause version information, clause content similarity, and hierarchical relationships between clauses, the validity and evolution information of each candidate clause in the candidate clause set are extracted. Based on the validity and evolution information of each candidate clause, conflict detection is performed on all candidate clauses in the candidate clause set. By judging whether there are expired, replaced, or mutually exclusive situations among the candidate clauses, the new and old conflict relationship, substitution relationship, or mutual exclusion relationship between the candidate clauses is identified.

[0087] Determining the conflict between old and new: For candidate clauses in the same university normative document or different university normative documents that are related to revision, if the candidate clauses have the same or highly similar themes, and the corresponding candidate clause version information indicates that the effective date of one candidate clause is later than the expiration or revision date of another candidate clause, then it can be determined that there is a conflict between old and new.

[0088] In the above circumstances, candidate clauses that have an earlier effective date and have expired (been amended or repealed) can be judged as old clauses, while candidate clauses that have a later effective date and are currently in effect can be judged as new clauses.

[0089] Determination of substitution relationship: When candidate clauses meet at least one of the following conditions, a substitution relationship can be determined: First, the evolution information of the candidate clauses clearly identifies one candidate clause as a revised or substituted clause of another candidate clause; Second, the content similarity between the candidate clauses is higher than the preset similarity threshold, and their candidate clause hierarchical paths have an inheritance or correspondence relationship, while the candidate clause version information indicates that one of the candidate clauses is a subsequently released version.

[0090] If a substitution relationship is determined, the candidate clause that is substituted can be marked as a non-current clause, and the substituted clause can be given priority in participating in subsequent sorting and generation.

[0091] Determining mutual exclusion: When there are obvious contradictions or exclusive descriptions between candidate clauses in terms of applicable conditions, binding rules or processing results, and they cannot be applied simultaneously through candidate clause content similarity analysis and keyword comparison, it can be determined that there is a mutual exclusion relationship between the candidate clauses.

[0092] Furthermore, when candidate clauses originate from different normative documents, but their hierarchical relationship indicates that they cannot be applied concurrently within the same scope, a mutually exclusive relationship can also be determined. In cases of mutual exclusivity, candidate clauses can be screened or reordered based on their expiration date, hierarchical priority, or scope of application.

[0093] Methods for forming a sequence of clauses after conflict resolution include:

[0094] Candidate clauses that have conflicting relationships and have expired, as well as candidate clauses that have been substituted and have been replaced, are removed from the candidate clause set. For candidate clauses that are mutually exclusive, based on the validity information of the candidate clauses, the candidate clauses that have taken effect later and are still valid are moved to the highest position and given higher ranking priority. The remaining candidate clauses in the mutually exclusive relationships are removed, thus forming a clause sequence after the conflict is resolved.

[0095] Methods for generating structured intelligent question-answering results include:

[0096] Based on the matching degree between the user's query intent and the terms content, the terms most relevant to the query are selected from the conflict-resolved terms sequence as the target terms. The matching degree is calculated based on the semantic similarity between the query feature vector and the terms content.

[0097] By utilizing the natural hierarchical information of the clauses, the hierarchical structure of the target clause in the document is parsed to obtain the hierarchical path of the target clause and the corresponding contextual information, so as to clarify the position and function of the clause in the document; combined with the hierarchical path, number and content of the target clause, a structured intelligent question and answer result is generated.

[0098] The preset stable support threshold is set by staff. By collecting different stable support values, the average of multiple stable support values ​​is taken as the preset stable support threshold. Similarly, the preset similarity threshold is set.

[0099] In this embodiment, by extracting information at the natural level, the normative documents of universities are broken down into atomic fragments of clauses, breaking the text integrity limitation of traditional full-text retrieval, realizing retrieval and matching at the smallest semantic unit granularity, and avoiding retrieval deviations caused by redundant information in the full text;

[0100] By generating hierarchical path identifiers and original location indexes for each clause's atomic fragment, not only is the logical position and subordinate relationship of the clause in the normative system accurately represented, but a structural basis is also provided for subsequent structured answers; at the same time, it realizes accurate tracing of clause atomic fragments and original documents, meeting the core needs of universities for source authority and location certainty in normative document queries;

[0101] The generated dense vector representation can effectively capture the deep semantics and contextual relationships of the atomic fragments of the clauses, while the sparse vector representation can accurately match key terms and literal features. The two representation forms are complementary and solve the limitations of single vector representation in semantic understanding or term matching. At the same time, parallel retrieval based on hybrid vector index can simultaneously use semantic similarity and term matching degree to filter the initial candidate clause set. Compared with the traditional single retrieval method, it significantly improves the retrieval accuracy while ensuring high recall, ensuring that the user's query intent can be accurately identified and matched with the most relevant clauses.

[0102] By calculating the stability support of clauses and filtering stable clauses based on their performance in multiple queries, the system effectively avoids random errors from single queries, improving the stability and consistency of search results. Combining semantic and keyword parallel retrieval allows for the rapid identification of stable clauses highly relevant to the query intent, improving retrieval efficiency while ensuring the accuracy of search results.

[0103] Example 2

[0104] Please see Figure 2 As shown, parts not described in detail in this embodiment are described in Embodiment 1. A method for intelligent question answering of normative documents based on hybrid vectors is provided, including:

[0105] S1. By extracting natural hierarchical information, the normative documents of universities are divided into several clause atomic fragments; for each clause atomic fragment, a corresponding hierarchical path identifier and original location index are generated, and a set of clause atomic fragments is constructed through association and encapsulation.

[0106] S2. Perform multidimensional semantic modeling on the set of atomic fragments of the clauses, and generate dense vector representation and sparse vector representation respectively; construct a hybrid vector index based on the dense vector representation and sparse vector representation;

[0107] S3. Receive and parse the user's query command, extract the query intent and key terms to generate a query feature vector; based on the query feature vector, perform parallel retrieval on the hybrid vector index to obtain an initial candidate clause set;

[0108] S4. Detect clauses in the initial candidate clause set that have conflicts or mutual exclusions between old and new clauses, automatically remove expired or replaced clauses, and at the same time, move the current clauses to the highest position to form a clause sequence after conflict resolution.

[0109] S5 is used to select target clauses from the clause sequence after conflict resolution and generate structured intelligent question-and-answer results by combining the natural hierarchical information of the target clauses.

[0110] Since the electronic device described in this embodiment is the electronic device used to implement the intelligent question-and-answer system for normative documents based on hybrid vectors in this application embodiment, those skilled in the art can understand the specific implementation methods and various variations of the electronic device in this embodiment based on the intelligent question-and-answer system for normative documents based on hybrid vectors described in this application embodiment. Therefore, how the electronic device implements the method in this application embodiment will not be described in detail here. Any electronic device used by those skilled in the art to implement the intelligent question-and-answer system for normative documents based on hybrid vectors in this application embodiment falls within the scope of protection of this application.

[0111] The above formulas are all dimensionless calculations. The formulas are derived from software simulations based on a large amount of collected data to obtain the most recent real-world results. The preset parameters and thresholds in the formulas are set by those skilled in the art according to the actual situation.

[0112] The above description is merely a preferred embodiment of the present invention, and the scope of protection of the present invention is not limited to the above embodiments. All technical solutions falling within the scope of the present invention's concept are within the scope of protection of the present invention. It should be noted that for users of ordinary technical skills, any improvements and modifications made without departing from the principles of the present invention should also be considered within the scope of protection of the present invention.

Claims

1. A prescriptive document intelligent question-answering system based on hybrid vectors, characterized in that, include: The structure parsing module extracts natural hierarchical information to break down university normative documents into several clause atomic fragments; for each clause atomic fragment, it generates a corresponding hierarchical path identifier and original position index, and constructs a set of clause atomic fragments through association and encapsulation. The method for extracting natural hierarchy information includes: The system acquires the text of normative documents from universities and colleges to be processed, performs standardization preprocessing on the content of the text, removes non-text content, standardizes the numbering format, and preserves the original text order, thereby forming parsable normative text data; based on preset normative document structure rules, it identifies the natural hierarchical structure markers in the normative text data; natural hierarchical structure markers include chapters, sections, articles, and structural identifiers below the article level; By traversing the identified natural hierarchical structure markers, and combining the nesting relationship and occurrence order between the hierarchical structure markers, the starting and ending positions of different hierarchical structures are determined, and a tree-like hierarchical path representing the membership relationship between each hierarchical structure is established, thus completing the acquisition of natural hierarchical information. The method for obtaining the set of atomic fragments of the terms includes: For each clause atomic fragment, the corresponding complete hierarchical path is extracted from the tree-like hierarchical path, and the hierarchical path is encoded according to a preset format to generate the corresponding hierarchical path identifier; at the same time, the position information of each clause atomic fragment in the university normative documents is recorded to form the original position index; the generated clause atomic fragments are associated one by one with the corresponding hierarchical path identifiers and original position indexes, and encapsulated into a clause atomic fragment set. The knowledge base construction module performs multi-dimensional semantic modeling on the set of atomic fragments of terms, generating dense vector representations and sparse vector representations respectively; based on the dense vector representations and sparse vector representations, a hybrid vector index is constructed. The methods for generating dense and sparse vector representations include: For any clause atomic fragment in the set of clause atomic fragments, the corresponding text of the university normative document is text-encoded to generate a text semantic vector. At the same time, a hierarchical path embedding vector is constructed based on the hierarchical path identifier to which the clause atomic fragment belongs. The obtained text semantic vector and the hierarchical path embedding vector are jointly modeled to generate a dense vector representation that reflects the semantics of the clause content and the semantics of the structure. Based on a pre-defined set of normative terms, term recognition and weighting are performed on the main text of the normative documents of universities corresponding to the atomic fragments of clauses. By detecting the normative terms that appear in the atomic fragments of clauses and their corresponding frequencies, the normative terms are assigned corresponding weight values, thereby constructing a sparse vector representation for keyword retrieval. The method for constructing the hybrid vector index includes: For each clause atomic fragment, the corresponding dense vector representation and sparse vector representation are stored together. At the same time, corresponding vector index structures are constructed for the dense vector representation and sparse vector representation respectively. The two vector index structures are uniformly associated with the same clause atomic fragment through the index mapping relationship, thereby forming a hybrid vector index that supports the parallel execution of semantic search and keyword search. The dual-stream hybrid retrieval module receives and parses user query commands, extracts query intent and key terms to generate query feature vectors, and performs parallel retrieval on the hybrid vector index based on the query feature vectors to obtain an initial set of candidate terms. The conflict resolution module detects clauses in the initial candidate clause set that have conflicts or mutual exclusions between old and new clauses, automatically removes expired or replaced clauses, and moves the current clauses to the highest position to form a clause sequence after conflict resolution. The structured generation module is used to select target clauses from the conflict-resolved clause sequence and generate structured intelligent question-and-answer results by combining the natural hierarchical information of the target clauses.

2. The intelligent question-answering system for normative documents based on hybrid vectors according to claim 1, characterized in that, The method for obtaining the set of atomic fragments of the clauses also includes: The normative text data is traversed based on a tree-like hierarchical path. The clauses of the normative documents of universities are divided into several clause atomic fragments according to the natural hierarchical structure markers. Each clause atomic fragment contains the corresponding text of the university normative document.

3. The intelligent question-answering system for normative documents based on hybrid vectors according to claim 2, characterized in that, The method for obtaining the query feature vector includes: The system receives user query commands and parses them, including word segmentation, stop word removal, part-of-speech tagging, and word form restoration. It uses natural language processing technology to identify the core elements in the query, including the target terms, legal concepts and requirements, thereby extracting the query intent and key terms. The TF-IDF algorithm maps each key term in the query to a corresponding word vector. By combining the word vectors in the query and using weighted averaging or vector concatenation, a query feature vector for retrieval is generated.

4. The intelligent question-answering system for normative documents based on hybrid vectors according to claim 3, characterized in that, The method for obtaining the initial candidate clause set includes: Based on the query feature vector, parallel retrieval is performed on the hybrid vector index, simultaneously performing semantic retrieval and keyword retrieval. During each query, a set of supporting terms is generated, and the stable support of the terms is calculated based on the set of supporting terms generated for each query. Stable support is based on the frequency of a clause's appearance in the query. A preset stable support threshold is used. If a clause's stable support is greater than or equal to the preset stable support threshold, the clause is considered a stable clause and included in the final search results, thus obtaining an initial set of candidate clauses after stable support filtering.

5. The intelligent question-answering system for normative documents based on hybrid vectors according to claim 4, characterized in that, The method for detecting clauses in the initial candidate clause set that have conflicts or mutual exclusions includes: By analyzing clause version information, clause content similarity, and hierarchical relationships between clauses, the validity and evolution information of each candidate clause in the candidate clause set are extracted. Based on the validity and evolution information of each candidate clause, conflict detection is performed on all candidate clauses in the candidate clause set. By judging whether there are expired, replaced, or mutually exclusive situations among the candidate clauses, the new and old conflict relationship, substitution relationship, or mutual exclusion relationship between the candidate clauses is identified.

6. The intelligent question-answering system for normative documents based on hybrid vectors according to claim 5, characterized in that, The method for forming the conflict-resolved clause sequence includes: Candidate clauses that have conflicting relationships and have expired, as well as candidate clauses that have been substituted and have been replaced, are removed from the candidate clause set. For candidate clauses that are mutually exclusive, based on the validity information of the candidate clauses, the candidate clauses that have taken effect later and are still valid are moved to the highest position and given higher ranking priority. The remaining candidate clauses in the mutually exclusive relationships are removed, thus forming a clause sequence after the conflict is resolved.

7. The intelligent question-answering system for normative documents based on hybrid vectors according to claim 6, characterized in that, The method for generating structured intelligent question-answering results includes: Based on the matching degree between the user's query intent and the terms content, the terms most relevant to the query are selected from the conflict-resolved terms sequence as the target terms. The matching degree is calculated based on the semantic similarity between the query feature vector and the terms content. By utilizing the natural hierarchical information of the clauses, the hierarchical structure of the target clause in the document is parsed to obtain the hierarchical path of the target clause and the corresponding contextual information, so as to clarify the position and function of the clause in the document; combined with the hierarchical path, number and content of the target clause, a structured intelligent question and answer result is generated.

Images