Data processing method and system

Abstract

The embodiments of the present specification provide a data processing method and system, wherein the data processing method comprises: determining problem data of a target project, calling a question and answer model, and generating at least one query task corresponding to the problem data based on document hierarchical data of the target project, wherein the document hierarchical data is generated based on document coding data and project data of the target project, and the document coding data is generated based on project documents of the target project; performing the at least one query task based on the document coding data, obtaining query data corresponding to the at least one query task respectively; reconstructing the query data corresponding to the at least one query task respectively to obtain target query data, and inputting the document hierarchical data and the target query data into the question and answer model to obtain answer data corresponding to the problem data.

Description

Technical Field

[0001] The embodiments in this specification relate to the field of big data technology, and in particular to data processing methods and systems. Background Technology

[0002] With the deepening of enterprise digital transformation, the scale of business documents and data is growing exponentially. When using question-answering models for knowledge retrieval and question answering, efficiently processing massive amounts of unstructured data becomes a key challenge. Existing technologies typically perform full-text searches or simple vector matching directly based on the original documents. This approach often ignores the inherent structured hierarchical features of project documents, resulting in search results containing a large amount of redundant information or noisy data. Furthermore, when faced with complex business problems, single-dimensional queries struggle to accurately locate key information, and interference from similar data can easily lead to model illusions or the output of low-quality answers. This not only increases the consumption of computing resources but also severely impacts the accuracy of answers and response efficiency. Therefore, a more effective data processing method is urgently needed to solve these problems. Summary of the Invention

[0003] In view of the above, embodiments of this specification provide a data processing method. One or more embodiments of this specification also relate to a data processing apparatus, a computing device, a computer-readable storage medium, and a computer program product, to address the technical deficiencies existing in the prior art.

[0004] According to a first aspect of the embodiments of this specification, a data processing method is provided, comprising: Identify the question data associated with the target project, invoke the question-answering model, and generate at least one query task corresponding to the question data based on the document hierarchy data of the target project, wherein the document hierarchy data is generated based on the document encoding data and project data of the target project, and the document encoding data is generated based on the project documents of the target project; Execute the at least one query task based on the document encoding data to obtain query data corresponding to the at least one query task respectively; The query data corresponding to each of the at least one query task is reconstructed to obtain target query data. The document-level data and the target query data are then input into the question-answering model to obtain the answer data corresponding to the question data.

[0005] According to a second aspect of the embodiments of this specification, a data processing system is provided, including a client and a server, comprising: The client is used to submit problem data related to the target project to the server; The server is configured to invoke a question-answering model to generate at least one query task corresponding to the question data based on the document-level data of the target project. The document-level data is generated based on the document-encoded data and project data of the target project, and the document-encoded data is generated based on the project documents of the target project. The server executes the at least one query task based on the document-encoded data to obtain query data corresponding to each of the at least one query task. The server reconstructs the query data corresponding to each of the at least one query task to obtain target query data. The server then inputs the document-level data and the target query data into the question-answering model to obtain answer data corresponding to the question data, and sends the answer data to the client.

[0006] According to a third aspect of the embodiments of this specification, a data processing apparatus is provided, comprising: The determination module is configured to determine the question data associated with the target project, call the question-answering model, and generate at least one query task corresponding to the question data based on the document hierarchy data of the target project, wherein the document hierarchy data is generated based on the document encoding data and project data of the target project, and the document encoding data is generated based on the project document of the target project; The execution module is configured to execute at least one query task based on the document encoding data, and obtain query data corresponding to each of the at least one query task; The input module is configured to reconstruct the query data corresponding to the at least one query task to obtain target query data, and input the document-level data and the target query data into the question-answering model to obtain the answer data corresponding to the question data.

[0007] According to a fourth aspect of the embodiments of this specification, a computing device is provided, comprising: Memory and processor; The memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions, which, when executed by the processor, implement the steps of the above-described data processing method.

[0008] According to a fifth aspect of the embodiments of this specification, a computer-readable storage medium is provided that stores computer-executable instructions, which, when executed by a processor, implement the steps of the data processing method described above.

[0009] According to a sixth aspect of the embodiments of this specification, a computer program product is provided, including a computer program or instructions that, when executed by a processor, implement the steps of the data processing method described above.

[0010] One embodiment of this specification describes a data processing method that identifies question data associated with a target project, invokes a question-and-answer model, and generates at least one query task corresponding to the question data based on the target project's document-level data. The document-level data is generated based on the target project's document-encoded data and project data, and the document-encoded data is generated based on the target project's project documents. The method involves hierarchically modeling the target project's project documents and project data to obtain document-level data. Query tasks are constructed based on the target project's document-level data, decomposing complex queries into at least one query task. At least one query task is executed based on the document-encoded data to obtain query data corresponding to each of the at least one query task. Executing at least one query task based on the document-encoded data reduces the impact of similar data on the query results. The query data corresponding to each of the at least one query task is reconstructed to obtain target query data. The document-level data and target query data are then input into the question-and-answer model to obtain answer data corresponding to the question data. The reconstructed target query data ensures that the data input into the question-and-answer model does not contain redundant data, reducing the model's resource consumption and improving the accuracy of the answer data. Attached Figure Description

[0011] Figure 1 This is a schematic diagram of a data processing method provided in one embodiment of this specification; Figure 2 This is a flowchart illustrating a data processing method provided in one embodiment of this specification; Figure 3 This is a topology tree diagram illustrating a data processing method provided in one embodiment of this specification; Figure 4 This is a schematic diagram illustrating the execution of a query task in a data processing method provided in one embodiment of this specification; Figure 5 This is a schematic diagram illustrating the generation of answer data for a data processing method provided in one embodiment of this specification; Figure 6 This is a flowchart illustrating the processing procedure of a data processing method provided in one embodiment of this specification. Figure 7 This is a schematic diagram of the structure of a data processing system provided in one embodiment of this specification; Figure 8 This is a question-and-answer flowchart of a data processing method provided in one embodiment of this specification; Figure 9 This is a schematic diagram of the structure of a data processing apparatus provided in one embodiment of this specification; Figure 10 This is a structural block diagram of a computing device provided in one embodiment of this specification. Detailed Implementation

[0012] Many specific details are set forth in the following description to provide a full understanding of this specification. However, this specification can be implemented in many other ways than those described herein, and those skilled in the art can make similar extensions without departing from the spirit of this specification. Therefore, this specification is not limited to the specific implementations disclosed below.

[0013] The terminology used in one or more embodiments of this specification is for the purpose of describing particular embodiments only and is not intended to be limiting of the one or more embodiments of this specification. The singular forms “a,” “described,” and “the” as used in one or more embodiments of this specification and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used in one or more embodiments of this specification refers to and includes any or all possible combinations of one or more associated listed items.

[0014] It should be understood that although the terms first, second, etc., may be used to describe various information in one or more embodiments of this specification, such information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another. For example, first may also be referred to as second without departing from the scope of one or more embodiments of this specification, and similarly, second may also be referred to as first. Depending on the context, the word "if" as used herein may be interpreted as "when," "when," or "in response to a determination."

[0015] Furthermore, it should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in one or more embodiments of this specification are all information and data authorized by the user or fully authorized by all parties. Moreover, the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and corresponding operation entry points are provided for users to choose to authorize or refuse.

[0016] First, the terms and concepts used in one or more embodiments of this specification will be explained.

[0017] Aspose is a well-known enterprise-level document and image processing SDK. It allows developers to directly create, edit, convert, render, and extract metadata from various file formats such as Word, Excel, PowerPoint, PDF, images, and CAD files within their own applications using code.

[0018] Figure 1This diagram illustrates a data processing method according to an embodiment of this specification. The method involves identifying question data associated with a target project, invoking a question-and-answer model, and generating at least one query task corresponding to the question data based on the target project's document-level data. The document-level data is generated based on the target project's document-encoded data and project data, and the document-encoded data is generated based on the target project's project documents. The method involves hierarchically modeling the target project's project documents and project data to obtain document-level data. Query tasks are constructed based on the target project's document-level data, decomposing complex queries into at least one query task. At least one query task is executed based on the document-encoded data to obtain query data corresponding to each of the at least one query task. Executing at least one query task based on the document-encoded data reduces the impact of similar data on the query data acquisition results. The query data corresponding to each of the at least one query task is reconstructed to obtain target query data. The document-level data and target query data are then input into the question-and-answer model to obtain answer data corresponding to the question data. The reconstructed target query data ensures that the data input into the question-and-answer model does not contain redundant data, reducing the resource consumption of the question-and-answer model and improving the accuracy of the answer data.

[0019] This specification provides a data processing method, and also relates to a data processing apparatus, a computing device, a computer-readable storage medium, and a computer program product, which will be described in detail in the following embodiments.

[0020] See Figure 2 , Figure 2 A flowchart of a data processing method according to an embodiment of this specification is shown, which specifically includes the following steps.

[0021] Step 202: Determine the question data associated with the target project, invoke the question-answering model, and generate at least one query task corresponding to the question data based on the document-level data of the target project. The document-level data is generated based on the document-encoded data and project data of the target project, and the document-encoded data is generated based on the project documents of the target project.

[0022] Specifically, the target project can be a financial business project in the financial field. Project documents are unstructured documents in the financial field. Project data includes, but is not limited to, financial terminology, historical defect cases, and system operating environment status. Question data refers to user query text; the question-answering model can be a large language model. Document-encoded data refers to multi-dimensional heterogeneous feature encoding built based on the project documents. It is data obtained by linearly fusing and encoding the multi-dimensional heterogeneous features of the project documents, reflecting the hierarchical structure of the content in the project documents, as well as the characteristics of objects such as tables and illustrations. Document-level data is a five-layer memory organization structure built on the basis of document-encoded data and combined with the project data of the target project. At least one query task corresponding to the question data can be represented in the form of an ordered task plan table, which records at least one query task.

[0023] Based on this, when a user has a question-and-answer requirement for a target project, they can submit question data for that project. After determining the question data associated with the target project, the question-and-answer model is invoked to generate at least one query task corresponding to the question data based on the document hierarchy data of the target project. The question data is input into the question-and-answer model, which reads the document hierarchy data and provides an answer. The document hierarchy data is generated based on the document encoding data and project data of the target project. The document encoding data is generated based on the project documents of the target project. The document encoding data reflects the hierarchical structure of the content in the project documents, as well as the characteristics of objects such as tables and illustrations in the project documents. It is a multi-dimensional heterogeneous feature encoding corresponding to the project documents.

[0024] Furthermore, considering that project documents typically have a hierarchical structure, the data within them reflects high-dimensional information such as chapter logic and business attributes. To avoid ignoring this high-dimensional information during subsequent queries, a multi-dimensional heterogeneous feature encoding can be constructed for the project documents. This encoding serves as the document encoding data for the project documents, reflecting their high-dimensional hierarchical information and the object characteristics of objects within them. The specific implementation is as follows: The project document is hierarchically divided to obtain the hierarchical structure data of the project document, and the document pages of the project document are parsed to obtain the object metadata of the project document; the hierarchical structure data and the object metadata are encoded to obtain the semantic vector of the project document; the document encoded data is generated based on the hierarchical structure data, the object metadata and the semantic vector.

[0025] Specifically, hierarchical partitioning of project documents refers to dividing the content of the project document into hierarchical levels according to the hierarchical logical structure of the data within the project document. A document page refers to a content page within the project document, used to identify objects contained within the document. Object metadata refers to the descriptive data of objects within the project document. Objects in the project document can be tables, illustrations, formulas, etc., and correspondingly, object description data can include table names, illustration titles, and formula numbers. Encoding hierarchical structure data and object metadata can be achieved through a pre-trained encoder. The pre-trained encoder independently represents the hierarchical structure data and object metadata, encoding them into heterogeneous semantic vectors.

[0026] Based on this, the project document is divided into chapters to obtain its hierarchical structure data. The document pages are parsed to identify objects such as tables, illustrations, and formulas, and the object metadata is determined based on these objects and their descriptions within the document pages. The hierarchical structure data and object metadata are encoded separately to obtain the semantic vector of the project document. Document encoding data is then generated based on the hierarchical structure data, object metadata, and semantic vector; these elements can be used as the document encoding data.

[0027] For example, when the target project is a company's business project, the project documents can be project drafts, including two documents: "Prospectus of Company A" and "Prospectus of Company B". The project documents are then hierarchically divided according to chapter levels to obtain hierarchical structure data representing the chapter levels, thus parsing the unstructured project document set into a four-level logical topology. The document pages of each project document are parsed to extract object description data for objects such as tables, illustrations, and formulas, i.e., table names, illustration titles, formula numbers, and key index terms. The hierarchical structure data and object metadata are encoded separately to obtain a multidimensional heterogeneous semantic vector for the project documents.

[0028] The extraction of object metadata can be achieved using a three-tiered progressive approach: First, prioritize calling the Aspose interface to extract metadata such as the object's position coordinates and style tags; Second, for complex tables containing merged cells across rows and columns, irregular borders, or nested sub-tables, introduce a complex table recognition model to re-extract structured metadata based on table line detection and cell segmentation algorithms; Third, when the table complexity exceeds the model's processing capacity, call a multimodal large language model to perform semantic-level parsing of the table image to ensure comprehensive metadata coverage.

[0029] In summary, document encoding data is generated based on hierarchical structure data, object metadata, and semantic vectors. This constructs a multidimensional heterogeneous feature encoding for project documents, representing their high-dimensional hierarchical information and object features. This avoids the loss of high-dimensional hierarchical information such as the project document's context, as well as the semantics of specific structural objects within the project document, during subsequent queries.

[0030] Furthermore, considering that the hierarchical structure data originates from project documents, to improve generalization ability, document content data, document metadata, and document chapter data can be determined for the hierarchical structure data, and page vectors, metadata vectors, and chapter vectors can be generated respectively. The specific implementation is as follows: Document content data, document metadata, and document chapter data are determined from the hierarchical structure data. The document content data, document metadata, and document chapter data are encoded respectively to obtain page vectors, metadata vectors, and chapter vectors. The object metadata is encoded to obtain object vectors. The content vector, metadata vector, chapter vector, and object vector are used as the semantic vectors of the project document.

[0031] Specifically, the hierarchical structure data contains four levels of indexes, restoring the knowledge in the project document into a logically deep topological network. These four levels form a four-level topological structure: Level 1: T0 global root node; Level 2: T1 business domain node; Level 3: T2 chapter directory node; and Level 4: T3 atomic page node. The four levels of the hierarchical structure data are determined based on the project document's document directory. Based on these four levels of indexes, document content data, document metadata, and document chapter data can be constructed. Document content data is generated by encoding the project document pages corresponding to the fourth-level (T3) atomic page nodes, creating content vectors that represent the full text information of the project document pages. Document metadata is obtained by extracting business domain information and global paths from the four levels of the hierarchical structure data, then encoding these metadata vectors to represent the project document's business domain and global path attributes. Document chapter data is generated by encoding the chapter information of the project document corresponding to the third-level (T2) chapter directory node. Object metadata refers to data such as table names, illustration titles, formula numbers, and key index terms in project documents. Metadata vectors are object vectors obtained by encoding object metadata.

[0032] Based on this, the hierarchical structure data contains four hierarchical indices. These indices determine the document content data, document metadata, and document chapter data of the project document. A pre-trained encoder is then used to encode the document content data, document metadata, and document chapter data respectively, obtaining page vectors, metadata vectors, and chapter vectors. The pre-trained encoder can be a general vector model. The object metadata is then encoded using the pre-trained encoder to obtain object vectors. These content vectors, metadata vectors, chapter vectors, and object vectors are used as the semantic vectors of the project document. These semantic vectors are heterogeneous semantic vectors corresponding to the four dimensions of the project document.

[0033] Continuing with the previous example, after traditional flattening and slicing, the vectors of the relevant paragraphs on "subsidiary profit situation" in two documents are highly similar. Searching for "subsidiary profit of Company A" might incorrectly recall content from Company B. After introducing the T0-T3 topology tree, each T3 page node carries complete hierarchical coordinates (investment banking documents / Company A's prospectus / Chapter 4 Subsidiary Information / Page 38). During retrieval, each segment is concatenated with global information from the chapter / table of contents, allowing for precise differentiation based on business domain and document affiliation, eliminating cross-document confusion. For example... Figure 3 As shown, an unstructured project document set can be parsed into a four-level logical topology: T0 global root node, T1 business domain node, T2 chapter directory node, and T3 atomic page node. Each T3 atomic page node naturally carries the complete hierarchical coordinates (business domain / document name / chapter / page number) of its ancestor node. During the page parsing stage, the document parsing operator is used to automatically identify and extract object-level metadata within the page. This metadata includes object metadata such as table names, illustration titles, formula numbers, and key index terms. Based on the T0-T3 topology tree, the document content data, document metadata, and document chapter data of the project document are determined. A pre-trained encoder is used to encode the document content data, document metadata, and document chapter data respectively to obtain page vectors (content vector Ep represents the full text information of the page), metadata vectors (Emeta represents the business domain and global path attributes), and chapter vectors (Etoc represents the chapter title logic). The content vectors, metadata vectors, chapter vectors, and object vectors (Eobj represents the table name and figure name features within the page) are used as the semantic vectors of the project document.

[0034] The chapter directory information corresponding to the chapter directory node and the page layout information corresponding to the atomic page node can be extracted using a three-layer progressive approach: First layer: Utilize Aspose to parse the chapter directory structure of Word / PDF, extract heading levels and paragraph hierarchical relationships, and construct an initial document tree. Second layer: When the document has non-standard layouts such as column layouts or PPT-converted cover pages, resulting in missing directory levels, a self-developed page layout analysis model is introduced to reconstruct the hierarchical topology through visual features and spatial relationships. Third layer: For extreme cases that still cannot be effectively handled, a multimodal large language model is invoked to complete the final repair and completion of the hierarchical structure using combined text and image understanding capabilities.

[0035] The content vector Ep, object vector Eobj, chapter vector Etoc, and meta-vector Emeta can each correspond to the first encoder. The inputs for each encoder are as follows: Ep's encoder input: the complete text of the T3 atomic page node. That is, the plain text content of the page after document parsing, excluding structured annotations such as chart titles and formula numbers. Eobj's encoder input: object metadata, including table names, illustration titles, formula numbers, and key index terms. These fields are fed into the encoder as concatenated strings, encoded independently from the text, to avoid object name features being overwhelmed by the semantics of the text. Etoc's encoder input: the title path string of the T2 chapter directory node (e.g., "Chapter 4 Subsidiary Situation > 4.2 Major Subsidiary Financial Data"), expressing the semantic position of the page in the document's chapter structure. Emeta's encoder input: the T1 business domain node label and the global hierarchical path string of the page (e.g., "Investment Banking Documents / Company A Prospectus / Page 38"), expressing the business affiliation and document coordinates of the page.

[0036] In summary, using content vectors, metadata vectors, chapter vectors, and object vectors as semantic vectors for project documents—heterogeneous semantic vectors used to represent the four dimensions of information in project documents—can improve generalization ability.

[0037] Furthermore, considering that question-answering models typically require answering questions based on project documents, and that answering questions usually necessitates deep retrieval of project documents, and that answers cannot be obtained directly from the document data, constructing document-level data of project documents can enable the self-evolution of the question-answering strategy. The specific implementation is as follows: The project terminology, historical project data, project operation data, and session data associated with the target project are determined, and the project terminology, historical project data, project operation data, and session data are used as the project data; the document encoding data and the project data are input into the data processing model to obtain the document hierarchy data.

[0038] Specifically, the project terminology associated with the target project refers to the professional terminology of the target project's field, which can be determined based on a relevant professional terminology dictionary, corresponding to the fact layer data. The historical project data of the target project can be operational layer (SOP) data such as historical badcase records and repair strategies, recall failure modes, and cross-page table processing rules. Project runtime data can be runtime layer data such as the current document set status, vector library index status, system time, and runtime environment resources of the target project. Session data can be session layer data such as historical execution session records, current session context, and current task progress of the target project. Document encoding data is used to construct the index layer data. The data processing model can be a large language model used to construct document-level data. Document-level data includes the document index layer, fact layer, SOP layer, session layer, and runtime layer.

[0039] Based on this, the project terminology, historical project data, project operation data, and session data associated with the target project are identified, and these are used as the project data. Document-encoded data and project data are input into the data processing model to obtain document-level data. Project terminology corresponds to the L2 fact layer, historical project data corresponds to the L3 SOP layer, document-encoded data corresponds to the L1 document index layer, session data corresponds to the L4 session layer, and project operation data corresponds to the L5 operation layer. Layers L1-L5 constitute a five-layer memory organization structure.

[0040] Following the previous example, the document-level data includes a document index layer, a fact layer, a SOP layer, a conversation layer, and an operational layer. The L1 document index layer stores a summary of the document directory structure, high-frequency query entries, and key business terminology mappings. It serves as the starting point for each decision by the question-answering model, providing a global navigation map of the document, allowing users to locate target chapters and key objects without traversing the entire text. The data source is cold-start loading, automatically built and written by the document parsing process during system initialization, remaining static within a single runtime cycle. The update mechanism is manual reconstruction triggered by the administrator with each document library version iteration, and does not dynamically change due to user interaction. The L2 fact layer stores user role and position information (analyst / investment banker / compliance, etc.), historical query preferences, and a financial terminology dictionary. It provides verified and stable facts to help the model understand user intent and professional background. The data source is hybrid—the terminology dictionary is pre-built for cold starts; user roles and preferences are personal dialogue data, written in real-time with user interaction; and human-confirmed terminology correction records are organizational memory. The update mechanism is automatic updating of personal preferences with each conversation. The L3 SOP layer stores historical badcase records and repair strategies, recall failure modes, and cross-page table processing rules. It stores reusable procedural knowledge, transforming historical failure experiences into future decision-making basis, supporting the ability to learn from mistakes and apply them to other situations. Data source: Organizational memory accumulation type—the large model continuously monitors interaction quality during operation. When it detects substantial user corrections, repeated errors of similar issues, or users marking high-value answers, it automatically generates candidate entries and pushes them to the human review queue. The update mechanism is: candidate entries are formally written after confirmation by authorized personnel; the large model performs timeliness assessments at preset intervals (e.g., quarterly), marking expired or conflicting entries, which are then archived or refreshed after human confirmation. The L4 session layer stores historical session records, current session context, and current task progress, providing session-level persistence and traceability, supporting contextual coherence in multi-turn dialogues. Data source: Personal dialogue data type, automatically written in real-time with user interaction, its lifecycle bound to the user session. The update mechanism is: automatically archived to history after the session ends; high-value content across sessions can be identified by the large model and submitted for human confirmation, accumulating in L2 or L3. The L5 runtime layer stores the current document set state, vector library index state, system time, and runtime environment resources, providing real-time environment awareness to ensure that model decisions are based on the current actual system state. Data is collected in real-time by the system and automatically injected by the runtime environment, independent of specific users. The update mechanism is real-time refresh as the system state changes, requiring no manual intervention.

[0041] In summary, by inputting document-encoded data and project data into the data processing model, document-level data is obtained, and a multi-layered memory organization structure corresponding to the target project is constructed. This provides a high signal-to-noise ratio decision context for data retrieval in the subsequent question-answering model. The subsequent question-answering model can read the multi-layered memory organization structure, which includes document directory structure, user role information, a terminology dictionary, historical badcases, and session records. It can autonomously decide the retrieval path and ranking strategy for at least one query task in the question-answering data, and achieve self-evolution of the query strategy through continuous feedback.

[0042] Furthermore, considering that different problem data correspond to different problem types, and when the problem type information of the problem data corresponds to a document query type, it is necessary to obtain the answer data of the problem data by retrieving multiple project documents of the target project. Given the complexity of the problem data, at least one query task can be constructed for the problem data to achieve comprehensive retrieval of query data related to the problem data. The specific implementation is as follows: The question-answering model is invoked to determine the question type information corresponding to the question data based on the document hierarchy data of the target project; if the question type information corresponds to a document query type, the question semantic information of the question data is determined, and the at least one query task is constructed according to the document hierarchy structure contained in the document hierarchy data.

[0043] Specifically, the question type information indicates the query type of the question data. The question type information corresponds to the document query type, indicating that the answer to the question data needs to be determined by querying the project document. The question semantic information indicates the query semantics of the question data, such as: profit filtering.

[0044] Based on this, a question-answering model is invoked to determine the question type information corresponding to the question data based on the document hierarchy data of the target project. Given that the question type information corresponds to a document query type, the question semantic information of the question data is determined, clarifying the query semantics of the question data. The question-answering model obtains the document hierarchy structure contained in the document hierarchy data and constructs at least one query task corresponding to the question data based on the document hierarchy structure and question semantic information. This separates the complex question data queries into at least one independently executable retrieval task. The constructed at least one query task can correspond to an ordered task plan list, and the task arrangement order corresponding to the at least one query task can be determined based on the ordered task plan list. The task arrangement order represents the data query order of the question data. The at least one query task includes, but is not limited to, chapter location in the project document, exact keyword matching, and document page integrity verification.

[0045] Continuing with the previous example, when a user queries, "Which subsidiaries had profits exceeding 1 billion in 2024?", the question type information of the question data corresponds to the document query type. The question-answering model reads hierarchical memory (document-level data): L1 layer learns that the document contains a table of contents entry "Chapter 4 Subsidiary Information", corresponding to one query task; L2 layer identifies the user as an investment banking analyst who prefers financial data, corresponding to another query task; L3 layer records the historical bad case "Profit statement spanning multiple pages caused incomplete recall, requiring window expansion", corresponding to a third query task.

[0046] In summary, given that the question type information corresponds to the document query type, the semantic information of the question data is determined, and at least one query task is constructed based on the document hierarchy structure contained in the document hierarchy data. This breaks down the complex question data answering task into at least one query task, which can then be executed concurrently to improve data retrieval efficiency.

[0047] Step 204: Execute the at least one query task based on the document encoding data to obtain the query data corresponding to the at least one query task respectively.

[0048] Specifically, after identifying the question data related to the target project, the question-answering model is invoked. At least one query task corresponding to the question data is generated based on the document-level data of the target project. The document-level data is generated from the document-encoded data and project data of the target project. After the document-encoded data is generated from the project documents of the target project, at least one query task can be executed based on the document-encoded data to obtain the query data corresponding to each query task. The at least one query task can be executed in parallel to retrieve query data. The at least one query task forms an ordered task sequence for answering the question data, and the at least one query task is stored in an ordered task plan table corresponding to the question data. Query data refers to at least one data fragment retrieved from the project documents by executing the query task.

[0049] Based on this, after determining the question data related to the target project, the question-answering model is invoked. At least one query task corresponding to the question data is generated based on the document-level data of the target project. The document-level data is generated from the target project's document encoding data and project data. After the document encoding data is generated from the target project's project documents, at least one query task is executed based on the document encoding data to obtain the query data corresponding to each of the at least one query task. The document encoding data contains multiple types of data, including encoding vectors, the directory tree of the project documents, and object metadata text corresponding to the project documents. When at least one query task is executed, each query task can correspond to at least one type of data in the document encoding data for data retrieval.

[0050] Furthermore, considering the diversity of problem type information in the problem data, which can correspond to interface query types or cross-domain query types, different query tasks can be constructed using large models or intelligent agents to achieve query intent routing for different query types. The specific implementation is as follows: When the problem type information corresponds to an interface query type, the interface query task corresponding to the problem data is executed based on the interface database; when the problem type information corresponds to a cross-domain query type, the query task corresponding to the problem data is split into at least one query subtask, and the at least one query subtask is executed.

[0051] Specifically, the question type information corresponds to the interface query type, indicating that the question data corresponds to the interface query question. The question type information corresponds to the cross-domain query type, indicating that the question data corresponds to a comprehensive query task, which requires retrieval of knowledge across multiple domains to obtain the answer.

[0052] Based on this, when the question type information corresponds to an interface query type, the question-answering model directly guides the question data to a dedicated interface database and executes the corresponding interface query task based on the interface database. When the question type information corresponds to a cross-domain query type, it indicates that the question data corresponds to a comprehensive query task. The query task corresponding to the question data is split into at least one query subtask, and after executing at least one query subtask, the execution results of each query subtask are integrated to obtain the answer data for the question data.

[0053] Continuing with the previous example, the question-answering model receives user queries, reads the hierarchical memory modules (L1~L5), and determines the query type. If it's an API query, it routes to the API documentation library. When the question-answering model identifies that the question is asking how to use a specific "function button" or "data interface" (e.g., "What are the parameters of the user login interface?" or "What is the URL for order query?"), it directly guides the user to the dedicated API documentation library. API interfaces typically have standard documentation. These documents precisely record the interface address, request method, required parameters, and return examples. Precisely routing such requests to the API documentation library allows for the retrieval of the most accurate and structured technical details within seconds, avoiding interference from irrelevant information. If it's a cross-domain comprehensive query, it's broken down into sub-tasks for separate processing. When the question data corresponds to a large, complex question that requires knowledge across multiple domains to answer, a single knowledge base cannot provide a direct answer. In this case, the question data is broken down into multiple query sub-tasks, which are then executed separately. Finally, the query results from each sub-task are summarized, deduplicated, and logically organized to synthesize a comprehensive and complete answer.

[0054] In summary, question type information can correspond to interface query types or cross-domain query types. For different query types, different query tasks can be constructed using large models or intelligent agents to achieve query intent routing. This ensures that different query types correspond to effective query processing methods.

[0055] Furthermore, considering the relationships between at least one query task, and to ensure high data recall coverage for subsequent tasks, a query task list can be constructed. Then, based on this list and the task sequence, the query path corresponding to the problematic data can be determined. By executing the path node tasks included in the query path, precise data recall can be achieved. The specific implementation is as follows: A query task list is constructed based on the at least one query task, and a query path matching the task sequence corresponding to the query task list is determined; the path node tasks included in the query path are determined, and the path node tasks are executed based on the document encoding data to obtain the query data corresponding to the at least one query task respectively.

[0056] Specifically, the query task list can be generated by arranging at least one query task in the order of task execution, such as arranging chapter location query tasks before exact match query tasks. The task sequence corresponding to the query task list consists of at least one query task arranged in the order of task execution. There can be at least one query path matching the task sequence, that is, each query task in the task sequence can correspond to one query path. The query tasks in the query task list need to be executed sequentially according to the order of the task sequence. The query path contains at least one path node task, and the at least one path node task contained in the query path can be executed in parallel for parallel data retrieval. The task types of path node tasks include, but are not limited to, semantic retrieval, logical navigation retrieval, and exact match retrieval. Semantic retrieval corresponds to path A, logical navigation retrieval corresponds to path B, and exact match retrieval corresponds to path C. The four-level topology structure (topology tree of T0-T3) in the document encoding data is used for logical navigation retrieval query tasks, the semantic vectors in the document encoding data are used for semantic retrieval query tasks, and the object metadata in the document encoding data is used for exact match retrieval query tasks.

[0057] Based on this, a query task list is constructed according to the task execution order corresponding to at least one query task, and query paths matching the task sequence corresponding to the query task list are determined. The query tasks in the query task list are executed sequentially according to the task execution order corresponding to at least one query task. Each query task can correspond to at least one query path, and each query path contains at least one path node task. For any given query task, the path node tasks contained in the query path corresponding to the query task are determined, and the path node tasks are executed based on the topology tree, semantic vector, and / or object metadata contained in the document encoding data to obtain the query data corresponding to at least one query task.

[0058] Continuing with the previous example, such as Figure 4 As shown, after determining the user query text "Which subsidiaries had profits exceeding 1 billion in 2024?", the query text is input into the question-answering model. The model reads document-level data and, if the query type is determined to be a document or table query, generates a query task list containing at least one query task. This query task list is the query plan table, which includes tasks to be executed sequentially, such as locating the subsidiary section in Chapter 4, precisely matching "Table 4-1 Subsidiary Profits", and verifying the integrity of cross-page table continuations. Each query task corresponds to at least one query path, and each query path contains at least one path node task. For the query task "Precisely matching 'Table 4-1 Subsidiary Profits'", the corresponding query path includes path A (semantic recall), path B (logical navigation recall), and path C (precise matching recall). The path node tasks for each path can be executed in parallel, achieving three-way parallel recall.

[0059] Path A (Semantic Recall): High-precision semantic matching is performed based on BM25 / Vhp vectors (semantic vectors), prioritizing the recall of fragments aligned with the business domain, and adept at handling queries with ambiguous intents and synonymous expressions. The agent feeds the user query text into the same pre-trained encoder (e.g., BGE-M3) used during database construction to generate a query vector Vq. Cosine similarity is calculated between Vq and the fused vectors Vhp of each L3 page in the vector library, and hybrid search is performed in conjunction with BM25 sparse retrieval results, prioritizing the recall of fragments whose business domain labels are aligned with the query intent. Vhp (semantic vectors) are linearly fused from four-dimensional heterogeneous vectors (Ep / Eobj / Etoc / Emeta), so semantic recall naturally perceives the business domain and chapter coordinates of the page, rather than relying solely on the similarity of the main text. This allows "Profit of Subsidiary of Company A" and "Profit of Subsidiary of Company B" to have a distinguishable distance in the vector space due to their different Emeta (business domain affiliation), even if the main text semantics are similar. Path A outputs: Top-K candidate L3 page fragments, along with similarity scores and hierarchical coordinates. For the query "Which subsidiaries had profits exceeding 1 billion in 2024?", Path A uses Vhp vector matching to semantically match "subsidiary profits," recalling page 38 (the main text of the profit summary table) and page 39 (continued). However, since Chapter 2 (Overview) and Chapter 6 (Risk Factors) in the same document also contain paragraphs related to "subsidiaries," Path A may include them as candidates as well.

[0060] Path B (Logical Navigation Recall): The agent performs deterministic pathfinding based on the T0-T2 directory tree, ensuring that the recall results conform to the document organization logic. It excels at chapter location and structure-aware queries. The agent does not rely on vector similarity but semantically aligns the user query with the document directory structure in the L1 index layer, locates the T2 node of the target chapter, and then performs deterministic top-down pathfinding along the T0→T1→T2→T3 topology tree to assemble a complete directory. After pathfinding, the directory titles of each level along the way are assembled into a complete hierarchical path string, which is input into the large model along with the T3 page text content. This ensures that the large model is always aware of the document coordinates and chapter affiliation of the current content when generating responses. For cross-page content (such as continued tables), Path B automatically expands the recall window through the sequential relationship between T3 atomic page nodes, ensuring the semantic integrity of cross-page tables or consecutive chapters. The pathfinding process for path B is as follows: T0 (Global Root Node): XX Investment Bank Documents; T1 (Business Domain Node): Company A Prospectus; T2 (Chapter Directory Node): Chapter 4 Subsidiary Information > Section 4.2 Major Subsidiary Financial Data; T3 (Atomic Page Node): Page 38 (Profit Summary Table) Page 39 (Continued, automatically expands window). After assembly, the hierarchical path string for the large model is: XX Investment Bank Documents / Company A Prospectus / Chapter 4 Subsidiary Information / Section 4.2 Major Subsidiary Financial Data / Pages 38-39. The large model generates answers at this coordinate system, with source footnotes precisely pointing to the original document location, eliminating the need for secondary positioning.

[0061] Path C (Precise Keyword Recall): The agent performs precise string matching on proper nouns, table names, figure names, formula numbers, etc. in the query, and directly hits object-level metadata (such as "Table 1-1 Main Financial Indicators" and "Equation (3-1) Valuation Model") based on the inverted index, bypassing vector approximation errors and achieving zero-biased precise positioning, complementing the semantic recall of Path A. During the document parsing phase of the project document, the object-level metadata (table names, figure titles, formula numbers, and key index terms) within each T3 atomic page node has been independently extracted and written into the inverted index, stored separately from the main text vector library. Path C involves the agent performing named entity recognition on the user query, extracting proper nouns and object identifiers, and then performing a retrieval on the inverted index using precise string matching or regular expressions, directly hitting the corresponding T3 atomic page node, completely bypassing the errors caused by vector approximation calculations. The hit result of path C is cross-validated with the recall results of paths A and B: when all three paths hit the same T3 atomic page node; when only path A hits but path C does not, the segment is judged to be semantically similar but not an exact target, and its ranking weight is reduced. Path C output: the exact hit T3 atomic page node, with attached object metadata fields. Querying "Which subsidiaries had profits exceeding 1 billion in 2024?", the agent identifies the implicit object intent "Subsidiary Profit Summary Table" from the query, retrieves the inverted index using the regular expression table \d+-\d+.profit, and hits the page 38 containing "Table 4-1 Subsidiary Profit Summary" with zero error. After obtaining the query data corresponding to each query task, the results can be fused and re-ranked to obtain the answer data for the question data.

[0062] In summary, by executing path node tasks based on document-encoded data, query data corresponding to at least one query task can be obtained. The path node tasks, including semantic vectors, logical navigation, and precise keywords, complement and validate each other. Semantic paths cover ambiguous intents, logical paths ensure structural coherence, and precise paths pinpoint proper nouns and object numbers. The synergy of these three elements significantly improves recall coverage and precision in complex query scenarios.

[0063] Furthermore, considering that tables in the project document may span multiple pages, the data augmentation task in the task sequence may hit only part of the table, leading to incomplete table retrieval. To address this, a semantic supplementation path needs to be constructed to achieve expanded document page retrieval, thus enabling data augmentation. The specific implementation is as follows: If the task sequence includes a data augmentation task, determine the semantic supplementation path corresponding to the data augmentation task; use the semantic supplementation path as the query path.

[0064] Specifically, the data augmentation task refers to the task of recalling and expanding document pages. When the data augmentation task was executed, page 38 of the project document was recalled. If it was determined that there was a table spanning multiple pages on page 38, a semantic supplementary path needed to be generated to recall the relevant document pages on page 38.

[0065] Based on this, when a data augmentation task is included in the task sequence, it indicates that the recalled document pages contain issues with tables or structured data spanning multiple pages during the execution of the data augmentation task. A semantic supplementation path corresponding to the data augmentation task is then determined. This semantic supplementation path is used to remedy the issue of tables or structured data spanning multiple pages in the document pages. The semantic supplementation path is then used as the query path.

[0066] Using the previous example, when a user queries: "Which subsidiaries had profits exceeding 1 billion in 2024?", the question-answering model reads the layered memory: Layer L1 recognizes that the document contains the "Chapter 4 Subsidiary Information" table of contents entry; Layer L2 identifies the user as an investment banking analyst who prefers financial data; Layer L3 records the historical bad case "Profit table spanning multiple pages caused incomplete recall, requiring window expansion." Based on this, the model makes the following autonomous decision: Activate path B (logical navigation to Chapter 4) + path C (exact match "Table 4-1 Subsidiary Profit Summary") + path A (semantic supplementation); if the recall results are insufficient, the window is automatically expanded, ultimately tracing back to Chapter 4 / Section 4.2 / Page 38 / Table 4-1. After the data augmentation task hits page 38 of the project document, the window is automatically expanded along the order of the T3 atomic page nodes to: page 37 (preceding context, optional); page 38 ← exact hit page; page 39 ← expanded window inclusion (continued); page 40 (following context, optional). The window size (how many pages to expand forward / backward) is dynamically determined by the agent based on the historical badcase records of the L3 SOP layer. When the SOP layer records "this document contains a table spanning multiple pages," the window expansion strategy is automatically triggered; otherwise, it is not expanded by default to avoid introducing irrelevant content and increasing token consumption.

[0067] In summary, when the task sequence includes data augmentation tasks, semantic supplementation paths are used as query paths to ensure automatic window expansion and broaden the page recall scope when recall results are insufficient. Document-level data drives the question-answering model to make autonomous retrieval decisions, and the model autonomously determines the query path, improving the accuracy of subsequently generated answer data.

[0068] Furthermore, considering the possibility of query task failure during the execution of at least one query task, to ensure the successful execution of each query task, the task status of each query task can be checked after at least one query task has been completed. Based on the task status, it can be determined whether to generate a supplementary task. The specific implementation is as follows: Based on the task status corresponding to the at least one query task, determine the task to be executed in the at least one query task; generate supplementary tasks for the task to be executed, and execute the task to be executed and the supplementary tasks.

[0069] Specifically, the task status indicates the execution status of the query task, including but not limited to executing, completed, failed, and not executed. A task to be executed can be at least one query task that has not yet been completed. A supplementary task refers to a task added to a task to be executed; that is, a supplementary task is created based on the results of a task to be executed, and is used to complete the execution result by executing the supplementary task.

[0070] Based on this, according to the task status corresponding to at least one query task, tasks with an incomplete execution status are identified as pending tasks within at least one query task. Supplementary tasks are generated for these pending tasks and appended to improve the execution results of the pending tasks. The pending tasks and supplementary tasks are executed until each query task is completed.

[0071] Continuing with the previous example, such as Figure 4 As shown, at least one query task includes locating the subsidiary section in Chapter 4, exactly matching "Table 4-1 Subsidiary Profits", and verifying the integrity of cross-page continuation tables. The symbols in parentheses before each task indicate the task status. [√] indicates the task has been completed. Tasks without a checkmark in parentheses are pending tasks. For the pending task "Verify the integrity of cross-page continuation tables", a supplementary task "Recall the continuation table on page 30" can be created. Continue executing pending tasks and supplementary tasks until at least one query task has been completed.

[0072] In summary, supplementary tasks are generated for tasks to be executed, and both tasks and supplementary tasks are executed. This ensures that if the results obtained from at least one query task are insufficient, supplementary tasks are added to ensure the comprehensiveness and completeness of the retrieved query data.

[0073] Step 206: Reconstruct the query data corresponding to the at least one query task to obtain target query data, and input the document level data and the target query data into the question-answering model to obtain the answer data corresponding to the question data.

[0074] Specifically, after executing at least one query task based on the document-encoded data and obtaining the query data corresponding to each of the at least one query task, the query data corresponding to each of the at least one query task can be reconstructed to obtain the target query data. The document-level data and the target query data are then input into the question-answering model to obtain the answer data corresponding to the question data. The reconstruction of the query data corresponding to each of the at least one query task includes, but is not limited to, deduplicating the query data corresponding to each of the at least one query task, dynamically determining the physical page retrieval window of the large model, removing low-relevance redundant segments, and processing the cross-page tables contained in the query data corresponding to each of the at least one query task, merging the cross-page tables into complete tables. This enables adaptive context reconstruction of the query data corresponding to each of the at least one query task using the large model, ensuring the semantic integrity of cross-page tables or continuous chapters, while effectively reducing the token consumption fed into the large model.

[0075] Based on this, after executing at least one query task based on the document-encoded data and obtaining the query data corresponding to each of the at least one query task, the query data corresponding to each of the at least one query task is reconstructed to obtain the target query data. It is ensured that the target query data does not contain redundant fragments and that cross-page tables in the target query data are merged into complete tables. The document-level data and the target query data are then input into the question-answering model to obtain the answer data corresponding to the question data; the answer data is the final answer to the question data.

[0076] In practical applications, such as Figure 5 As shown, after determining the candidate segments recalled by paths A, B, and C respectively, adaptive context reconstruction can be performed, namely, deduplication of candidate segments (redundancy removal, low-relevance segment filtering and noise reduction) and merging of cross-page tables (pages 38-39 are merged into a complete table). The large model can dynamically determine the physical page retrieval window based on the chapter semantic density distribution, remove low-relevance redundant segments, ensure the semantic integrity of cross-page tables or consecutive chapters, and effectively reduce the token consumption fed into the large model.

[0077] Furthermore, considering that the answer data corresponding to the question data is determined by integrating and analyzing target query data containing multiple data fragments, in order to ensure the traceability of the answer data, answer hint information can be generated at the same time as generating the answer information. The answer hint information is used to represent the complete hierarchical tracing path, and the specific implementation is as follows: The document-level data and the target query data are input into the question-answering model to obtain at least one answer and answer hints; the at least one answer and the answer hints are used as the answer data corresponding to the question data.

[0078] Specifically, answer hints can contain at least one sub-answer hint, with one answer hint corresponding to one sub-answer. The answer information represents the specific answer to the question, while the answer hints indicate the hierarchical position of the specific answer within the project document, such as: L2 1.1 Company Overview. Answer hints can also represent the complete source path of at least one answer, such as: Investment Banking Documents / Company A's Prospectus / Chapter 4, Section 4.2 / Pages 38-39 / Table 4-1 Subsidiary Profits.

[0079] Based on this, document-level data and target query data are input into the question-answering model to obtain at least one answer message representing the answer to the question data, as well as answer hints for at least one answer message. The answer hints indicate the specific data source location of the answer message within the project document. The at least one answer message and the answer hints are used as the answer data corresponding to the question data. Along with the answer, source information is provided, containing a complete hierarchical source path to ensure that each answer message can be traced back to its specific data source location within the project document.

[0080] Following the previous example, the document-level data and target query data are input into the question-answering model to obtain at least one answer and answer hints, constituting the answer data. The answer data includes the final answer and a source footnote. The final answer (at least one answer): "Among Company A's subsidiaries, three had profits exceeding 1 billion in 2024: Subsidiary XX (profit XX billion)..."; the source footnote (answer hint): Investment bank working papers / Company A's prospectus / Chapter 4.2 / Pages 38-39 / Table 4-1 Subsidiary Profits. Detection precision is improved by 35%, recall by 40%, and token efficiency by 50%.

[0081] In summary, using at least one answer and one answer hint as the answer data corresponding to the question data improves the traceability of the answer data and ensures that each answer can be traced back to the specific data source location in the project document.

[0082] One embodiment of this specification describes a data processing method that identifies question data associated with a target project, invokes a question-and-answer model, and generates at least one query task corresponding to the question data based on the target project's document-level data. The document-level data is generated based on the target project's document-encoded data and project data, and the document-encoded data is generated based on the target project's project documents. The method involves hierarchically modeling the target project's project documents and project data to obtain document-level data. Query tasks are constructed based on the target project's document-level data, decomposing complex queries into at least one query task. At least one query task is executed based on the document-encoded data to obtain query data corresponding to each of the at least one query task. Executing at least one query task based on the document-encoded data reduces the impact of similar data on the query results. The query data corresponding to each of the at least one query task is reconstructed to obtain target query data. The document-level data and target query data are then input into the question-and-answer model to obtain answer data corresponding to the question data. The reconstructed target query data ensures that the data input into the question-and-answer model does not contain redundant data, reducing the model's resource consumption and improving the accuracy of the answer data.

[0083] The following is in conjunction with the appendix Figure 6 Taking the application of the data processing method provided in this specification in domain knowledge question answering as an example, the data processing method will be further explained. Figure 6 A flowchart illustrating the processing procedure of a data processing method according to an embodiment of this specification is shown, specifically including the following steps.

[0084] Step 602: Determine the question data associated with the target project, call the question-answering model, and determine the question type information corresponding to the question data based on the document hierarchy data of the target project.

[0085] Step 604: Given that the question type information corresponds to the document query type, determine the question semantic information of the question data, and construct a query task list containing at least one query task based on the document hierarchy structure contained in the document hierarchy data. The document hierarchy data is generated based on the target project's document encoding data and project data, and the document encoding data is generated based on the target project's project documents.

[0086] In practical applications, in this embodiment, the target project can be a financial business project in the financial field. Project documents are unstructured documents in the financial field. Project data includes, but is not limited to, financial terminology, historical defect cases, system operating environment status, etc. Question data refers to user query text, such as: "Which subsidiaries had profits exceeding 1 billion in 2024?" The question-answering model can be a large language model. Question type information refers to the query type of the question data, which includes, but is not limited to, document or table queries, API interface queries, and cross-domain comprehensive queries. Question semantic information refers to the query semantics of the question data.

[0087] Document-encoded data refers to multi-dimensional heterogeneous feature encoding constructed based on project documents. For example... Figure 7 As shown, unstructured project documents include, but are not limited to, PDF, Word, and mixed text and image documents. Hierarchical topology parsing is performed on unstructured project documents, resulting in a four-level logical topology structure: T0 global root node, T1 business domain node, T2 chapter directory node, and T3 atomic page node. Atomic object metadata is extracted from the project documents. During the page parsing phase, document parsing operators are used to automatically identify and extract object-level metadata within the page. Object-level metadata includes, but is not limited to, table names, illustration titles, formula numbers, and key index terms. A pre-trained encoder is used to independently represent information in different dimensions (object-level metadata and the four-level logical topology structure), generating four-dimensional heterogeneous semantic vectors: the content vector Ep represents the full text information of the page; the object vector Eobj represents the table and figure name features within the page; the chapter vector Etoc represents the logical structure of the chapter title; and the meta-vector Emeta represents the business domain and global path attributes.

[0088] Document-level data is a five-layer memory organization structure built upon document-encoded data and combined with project data from the target project. Specifically, it includes: L1 Index Layer (resident): Document directory structure summary, high-frequency query entry points, and key business terminology mapping. The large model is always planned at the top level in a high signal-to-noise ratio environment, only diving deeper when necessary. L2 Fact Layer: User roles and positions (analysts / investment bankers / compliance, etc.), historical query preferences, and a dictionary of financial terminology. Validated and stable facts, valid for a long time. L3 SOP Layer: Historical badcase records and repair strategies, failure recall modes, cross-page table handling rules, common failure cases and recovery strategies. Reusable procedural knowledge, supporting generalization. L4 Session Layer: Historical execution session records and current session context, used for persistence and traceability. L5 Runtime Layer: Current document set state, vector library index state, and runtime environment resources, providing real-time environment awareness.

[0089] Furthermore, when the issue type information corresponds to an interface query type, the interface query task corresponding to the issue data is executed based on the interface database. When the issue type information corresponds to a cross-domain query type, the query task corresponding to the issue data is split into at least one query subtask, and at least one query subtask is executed.

[0090] Step 606: Determine the query path that matches the task sequence corresponding to the query task list.

[0091] Step 608: Determine the path node tasks included in the query path, and execute the path node tasks based on the document encoding data to obtain query data corresponding to at least one query task.

[0092] In practice, when the question type information corresponds to a document query type or a table query type, the semantic information of the question data is determined. The question data and semantic information are then input into the question-answering model. Based on the query semantics and the document structure information in hierarchical memory, the question-answering model autonomously generates an ordered task plan list, decomposing complex queries into several independently executable retrieval subtasks. The ordered task plan list contains at least one query task. For example, for the query "Which subsidiaries had profits exceeding 1 billion in 2024?", the question-answering model generates an ordered task plan list, locating the subsidiary section in Chapter 4; precisely matching "Table 4-1 Summary of Subsidiary Profits"; and verifying the completeness of cross-page continuation. The ordered task plan list is dynamically refreshed by the large model after each round of recall: fully completed tasks are marked as […]. If the result is insufficient, additional subtasks are added, and the process continues until all tasks are completed.

[0093] In practical applications, after reading the hierarchical memory, the question-answering model sequentially makes the following decisions, driving the corresponding subsequent steps: The question-answering model reads the L1 index layer (document directory structure) and the L2 fact layer (user role preferences), decomposing complex queries into an ordered task plan list, clarifying the retrieval target and priority of each task. Based on preset analysis logic, the question-answering model autonomously determines which paths to activate. For complex queries, the question-answering model can activate multiple paths simultaneously and execute them in parallel, without the need for fixed weight rules. For example, when a user queries "Which subsidiaries had profits exceeding 1 billion in 2024?", the L1-L5 layers of memory are written into the question-answering model context all at once: [L1 Index Layer] The document contains the table of contents entry "Chapter 4 Subsidiary Information"; Key term mapping: "Table 4-1 Summary of Subsidiary Profits"; [L2 Fact Layer] Current user: Investment banking analyst; Preference: Precise financial data results; Terminology: Underwriting / Quotation / Lock-up period; [L3 SOP Layer] Bad case record: "Profit statement page crossover caused incomplete recall, handling strategy: expand window"; [L4 Session Layer] This session has queried: "Company A's overall revenue" (Company A's prospectus has been recalled); [L5 Runtime Layer] Vector library status: Normal; Company A's prospectus: Full indexing completed. The question-answering model integrates the above information within the same context and outputs a complete decision at once: generating a query task list: ① logical navigation to Chapter 4 ② exact match "Table 4-1 Subsidiary Profit Summary" ③ verify the completeness of cross-page table continuation; activating paths: path B (chapter navigation) + path C (exact match Table 4-1) + path A (semantic supplement fallback); due to the L3 record cross-page bad case, the window expansion strategy is automatically set.

[0094] Step 610: Reconstruct the query data corresponding to at least one query task to obtain the target query data.

[0095] For each query task, the retrieved query data includes: ① Merging the deduplicated candidate fragment set (page 38 + page 39 continued); ② Path B and path C are cross-validated, both hitting page 38; ③ The PlanList is dynamically refreshed: [ ① Navigate logically to "Chapter Four: Subsidiary Information," locating Section 4.2; ②Exactly match "Table 4-1 Summary of Subsidiary Profits"; ③ Verify the integrity of cross-page continuation tables (pages 38-39 have been recalled and are complete). Within the scope of Company A's documents, perform context trimming and reconstruction. Based on the semantic density distribution of chapters, the large model dynamically determines the retrieval window of the physical page, eliminates low-relevance redundant fragments, ensures the semantic integrity of cross-page tables or continuous chapters, and effectively reduces the token consumption sent to the large model.

[0096] Step 612: Input the document-level data and target query data into the question-answering model to obtain at least one answer and answer hint information, and generate the answer data corresponding to the question data based on the at least one answer and answer hint information.

[0097] The large model generates the final answer data based on the trimmed context, and includes a complete hierarchical tracing path (folder / document name / page number / object name) in the footnotes of the answer data. The tracing path is the answer hint information, used to ensure that each conclusion can be traced back to a specific location in the original document.

[0098] In summary, the data processing method provided in this embodiment targets "object vectors" of specific objects such as figures and tables within a document. By injecting table names and figure names as strong semantic features into the encoding, it achieves a leap from "approximately in this section" to "precisely locating this table," greatly improving the retrieval experience in scenarios such as financial terms and technical specifications. Through T0-T3 hierarchical indexing, knowledge is restored into a topological network with logical depth. Each micro-level data slice carries the macro-level business background and chapter coordinates, fundamentally solving the retrieval offset problem caused by highly similar semantics within professional fields. The three-way recall (semantic vector, logical navigation, and precise keywords) complement and verify each other. The semantic path covers ambiguous intents, the logical path ensures structural coherence, and the precise path locks in proper nouns and object numbers. The synergy of these three significantly improves the recall coverage and precision in complex query scenarios.

[0099] One embodiment of this specification describes a data processing method that identifies question data associated with a target project, invokes a question-and-answer model, and generates at least one query task corresponding to the question data based on the target project's document-level data. The document-level data is generated based on the target project's document-encoded data and project data, and the document-encoded data is generated based on the target project's project documents. The method involves hierarchically modeling the target project's project documents and project data to obtain document-level data. Query tasks are constructed based on the target project's document-level data, decomposing complex queries into at least one query task. At least one query task is executed based on the document-encoded data to obtain query data corresponding to each of the at least one query task. Executing at least one query task based on the document-encoded data reduces the impact of similar data on the query results. The query data corresponding to each of the at least one query task is reconstructed to obtain target query data. The document-level data and target query data are then input into the question-and-answer model to obtain answer data corresponding to the question data. The reconstructed target query data ensures that the data input into the question-and-answer model does not contain redundant data, reducing the model's resource consumption and improving the accuracy of the answer data.

[0100] Corresponding to the above method embodiments, this specification also provides data processing system embodiments. Figure 8 A schematic diagram of the structure of a data processing system provided in one embodiment of this specification is shown. Figure 8 As shown, the data processing system 800 includes a client 810 and a server 820. The client 810 is used to submit question data related to a target project to the server 820. The server 820 is used to invoke a question-answering model to generate at least one query task corresponding to the question data based on the document-level data of the target project, wherein the document-level data is generated based on the document-encoded data and project data of the target project, and the document-encoded data is generated based on the project document of the target project. The server executes the at least one query task based on the document-encoded data to obtain query data corresponding to each of the at least one query task. The server reconstructs the query data corresponding to each of the at least one query task to obtain target query data, and inputs the document-level data and the target query data into the question-answering model to obtain answer data corresponding to the question data, and sends the answer data to the client 810.

[0101] This specification describes a data processing system comprising a client and a server, according to one embodiment. The client submits question data related to a target project to the server. The server determines the question data related to the target project, invokes a question-answering model, and generates at least one query task corresponding to the question data based on the target project's document-level data. The document-level data is generated based on the target project's document-encoded data and project data, and the document-encoded data is generated based on the target project's project documents. Hierarchical modeling is performed on the target project's project documents and project data to obtain document-level data. Query tasks are constructed based on the target project's document-level data, decomposing complex queries into at least one query task. At least one query task is executed based on the document-encoded data to obtain query data corresponding to each of the at least one query task. Executing at least one query task based on the document-encoded data reduces the impact of similar data on the query results. The query data corresponding to each of the at least one query task is reconstructed to obtain target query data. The document-level data and target query data are then input into the question-answering model to obtain answer data corresponding to the question data, which is then sent to the client. The reconstructed target query data ensures that the data input into the question-answering model does not contain redundant data, reducing the resource consumption of the question-answering model and improving the accuracy of the answer data.

[0102] Corresponding to the above method embodiments, this specification also provides data processing apparatus embodiments. Figure 9 A schematic diagram of the structure of a data processing apparatus according to one embodiment of this specification is shown. Figure 9 As shown, the device includes: The determination module 902 is configured to determine the question data associated with the target project, call the question-answering model, and generate at least one query task corresponding to the question data based on the document hierarchy data of the target project, wherein the document hierarchy data is generated based on the document encoding data and project data of the target project, and the document encoding data is generated based on the project document of the target project; Execution module 904 is configured to execute at least one query task based on the document encoding data, and obtain query data corresponding to each of the at least one query task; The input module 906 is configured to reconstruct the query data corresponding to the at least one query task to obtain target query data, and input the document level data and the target query data into the question-answering model to obtain the answer data corresponding to the question data.

[0103] In one optional embodiment, the generation of the document encoded data includes: The project document is hierarchically divided to obtain the hierarchical structure data of the project document, and the document pages of the project document are parsed to obtain the object metadata of the project document; The hierarchical structure data and the object metadata are encoded to obtain the semantic vector of the project document; The document encoding data is generated based on the hierarchical structure data, the object metadata, and the semantic vector.

[0104] In an optional embodiment, encoding the hierarchical structure data and the object metadata to obtain the semantic vector of the project document includes: Document content data, document metadata, and document chapter data are determined from the hierarchical structure data, and the document content data, document metadata, and document chapter data are encoded respectively to obtain page vector, metadata vector, and chapter vector; The object metadata is encoded to obtain an object vector, and the content vector, the metadata vector, the chapter vector, and the object vector are used as the semantic vector of the project document.

[0105] In one optional embodiment, the generation of the document hierarchy data includes: Determine the project terms, historical project data, project operation data, and session data associated with the target project, and use the project terms, historical project data, project operation data, and session data as the project data; The document encoding data and the project data are input into the data processing model to obtain the document hierarchical data.

[0106] In one optional embodiment, the invocation of the question-answering model, based on the document-level data of the target project, generates at least one query task corresponding to the question data, including: The question-answering model is invoked to determine the question type information corresponding to the question data based on the document hierarchy data of the target project. When the question type information corresponds to a document query type, the question semantic information of the question data is determined, and the at least one query task is constructed based on the document hierarchy structure contained in the document hierarchy data.

[0107] In an optional embodiment, the method further includes: When the problem type information corresponds to the interface query type, the interface query task corresponding to the problem data is executed based on the interface database; In the case where the question type information corresponds to a cross-domain query type, the query task corresponding to the question data is split into at least one query subtask, and the at least one query subtask is executed.

[0108] In an optional embodiment, the step of performing the at least one query task based on the document encoding data to obtain query data corresponding to each of the at least one query task includes: A query task list is constructed based on the at least one query task, and a query path matching the task sequence corresponding to the query task list is determined. The path node tasks included in the query path are determined, and the path node tasks are executed based on the document encoding data to obtain the query data corresponding to the at least one query task.

[0109] In an optional embodiment, determining the query path that matches the task sequence corresponding to the query task list includes: If the task sequence includes a data augmentation task, determine the semantic supplementation path corresponding to the data augmentation task; The semantic supplement path is used as the query path.

[0110] In an optional embodiment, after executing the at least one query task based on the document encoding data to obtain query data corresponding to the at least one query task, the method further includes: Based on the task status corresponding to the at least one query task, determine the task to be executed in the at least one query task; A supplementary task is generated for the task to be executed, and the task to be executed and the supplementary task are executed.

[0111] In an optional embodiment, the step of inputting the document-level data and the target query data into the question-answering model to obtain the answer data corresponding to the question data includes: The document hierarchy data and the target query data are input into the question-answering model to obtain at least one answer and answer hint information. The at least one answer and the answer hint are used as the answer data corresponding to the question data.

[0112] An embodiment of this specification describes a data processing apparatus that determines question data associated with a target project, invokes a question-and-answer model, and generates at least one query task corresponding to the question data based on the target project's document-level data. The document-level data is generated based on the target project's document-encoded data and project data, and the document-encoded data is generated based on the target project's project documents. The apparatus performs hierarchical modeling on the target project's project documents and project data to obtain document-level data. Query tasks are constructed based on the target project's document-level data, decomposing complex queries into at least one query task. At least one query task is executed based on the document-encoded data to obtain query data corresponding to each of the at least one query task. Executing at least one query task based on the document-encoded data reduces the impact of similar data on the query data acquisition results. The query data corresponding to each of the at least one query task is reconstructed to obtain target query data. The document-level data and target query data are then input into the question-and-answer model to obtain answer data corresponding to the question data. The reconstructed target query data ensures that the data input into the question-and-answer model does not contain redundant data, reducing the resource consumption of the question-and-answer model and improving the accuracy of the answer data.

[0113] The above is an illustrative scheme of a data processing apparatus according to this embodiment. It should be noted that the technical solution of this data processing apparatus and the technical solution of the data processing method described above belong to the same concept. For details not described in detail in the technical solution of the data processing apparatus, please refer to the description of the technical solution of the data processing method described above.

[0114] Figure 10 A structural block diagram of a computing device 1000 according to one embodiment of this specification is shown. The components of the computing device 1000 include, but are not limited to, a memory 1010 and a processor 1020. The processor 1020 is connected to the memory 1010 via a bus 1030, and a database 1050 is used to store data.

[0115] The computing device 1000 also includes an access device 1040, which enables the computing device 1000 to communicate via one or more networks 1060. Examples of these networks include Public Switched Telephone Network (PSTN), Local Area Network (LAN), Wide Area Network (WAN), Personal Area Network (PAN), or combinations of communication networks such as the Internet. The access device 1040 may include one or more of any type of wired or wireless network interface (e.g., a network interface card (NIC)), such as an IEEE 802.11 Wireless Local Area Network (WLAN) wireless interface, a Wi-MAX (Worldwide Interoperability for Microwave Access) interface, an Ethernet interface, a Universal Serial Bus (USB) interface, a cellular network interface, a Bluetooth interface, or a Near Field Communication (NFC) interface.

[0116] In one embodiment of this specification, the above-described components of the computing device 1000 and Figure 10 Other components, not shown, can also be connected to each other, for example, via a bus. It should be understood that... Figure 10 The block diagram of the computing device shown is for illustrative purposes only and is not intended to limit the scope of this specification. Those skilled in the art can add or replace other components as needed.

[0117] The computing device 1000 can be any type of stationary or mobile computing device, including mobile computers or mobile computing devices (e.g., tablet computers, personal digital assistants, laptop computers, notebook computers, netbooks, etc.), mobile phones (e.g., smartphones), wearable computing devices (e.g., smartwatches, smart glasses, etc.) or other types of mobile devices, or stationary computing devices such as desktop computers or personal computers (PCs). The computing device 1000 can also be a mobile or stationary server.

[0118] The processor 1020 is configured to execute the following computer-executable instructions, which, when executed by the processor, implement the steps of the above-described data processing method.

[0119] The above is an illustrative scheme of a computing device according to this embodiment. It should be noted that the technical solution of this computing device and the technical solution of the data processing method described above belong to the same concept. For details not described in detail in the technical solution of the computing device, please refer to the description of the technical solution of the data processing method described above.

[0120] An embodiment of this specification also provides a computer-readable storage medium storing computer-executable instructions that, when executed by a processor, implement the steps of the above-described data processing method.

[0121] The above is an illustrative scheme of a computer-readable storage medium according to this embodiment. It should be noted that the technical solution of this storage medium and the technical solution of the data processing method described above belong to the same concept. For details not described in detail in the technical solution of the storage medium, please refer to the description of the technical solution of the data processing method described above.

[0122] An embodiment of this specification also provides a computer program product, including a computer program or instructions that, when executed by a processor, implement the steps of the above-described data processing method.

[0123] The above is an illustrative scheme of a computer program product according to this embodiment. It should be noted that the technical solution of this computer program product and the technical solution of the data processing method described above belong to the same concept. For details not described in detail in the technical solution of the computer program product, please refer to the description of the technical solution of the data processing method described above.

[0124] The foregoing has described specific embodiments of this specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in a different order than that shown in the embodiments and may still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily require the specific or sequential order shown to achieve the desired result. In some embodiments, multitasking and parallel processing are possible or may be advantageous.

[0125] The computer instructions include computer program code, which may be in the form of source code, object code, executable file, or certain intermediate forms. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording media, USB flash drive, portable hard drive, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc. It should be noted that the content included in the computer-readable medium may be appropriately added or removed according to the requirements of patent practice. For example, in some regions, according to patent practice, computer-readable media may not include electrical carrier signals and telecommunication signals.

[0126] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that the embodiments in this specification are not limited to the described order of actions, because according to the embodiments in this specification, some steps can be performed in other orders or simultaneously. Furthermore, those skilled in the art should also understand that the embodiments described in this specification are all preferred embodiments, and the actions and modules involved are not necessarily essential to the embodiments in this specification.

[0127] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0128] The preferred embodiments disclosed above are merely illustrative of this specification. The optional embodiments do not exhaustively describe all details, nor do they limit the invention to the specific implementations described. Clearly, many modifications and variations can be made based on the embodiments described in this specification. These embodiments are selected and specifically described in this specification to better explain the principles and practical applications of the embodiments, thereby enabling those skilled in the art to better understand and utilize this specification.

Claims

1. A data processing method, characterized in that, include: Identify the question data associated with the target project, invoke the question-answering model, and generate at least one query task corresponding to the question data based on the document hierarchy data of the target project, wherein the document hierarchy data is generated based on the document encoding data and project data of the target project, and the document encoding data is generated based on the project documents of the target project; A query task list is constructed based on the at least one query task. If the task sequence corresponding to the query task list includes a data augmentation task, a semantic supplementation path corresponding to the data augmentation task is determined. The path node tasks included in the semantic supplementation path are executed based on the document encoded data to obtain the query data corresponding to the at least one query task. The query data corresponding to each of the at least one query task is reconstructed to obtain target query data. The document-level data and the target query data are then input into the question-answering model to obtain the answer data corresponding to the question data.

2. The data processing method according to claim 1, characterized in that, The generation of the document encoded data includes: The project document is hierarchically divided to obtain the hierarchical structure data of the project document, and the document pages of the project document are parsed to obtain the object metadata of the project document; The hierarchical structure data and the object metadata are encoded to obtain the semantic vector of the project document; The document encoding data is generated based on the hierarchical structure data, the object metadata, and the semantic vector.

3. The data processing method according to claim 2, characterized in that, The process of encoding the hierarchical structure data and the object metadata to obtain the semantic vector of the project document includes: Document content data, document metadata, and document chapter data are determined from the hierarchical structure data, and the document content data, document metadata, and document chapter data are encoded respectively to obtain page vector, metadata vector, and chapter vector; The object metadata is encoded to obtain an object vector, and the page vector, the metadata vector, the chapter vector, and the object vector are used as the semantic vector of the project document.

4. The data processing method according to claim 1, characterized in that, The generation of the document hierarchy data includes: Determine the project terms, historical project data, project operation data, and session data associated with the target project, and use the project terms, historical project data, project operation data, and session data as the project data; The document encoding data and the project data are input into the data processing model to obtain the document hierarchical data.

5. The data processing method according to claim 1, characterized in that, The invoked question-answering model generates at least one query task corresponding to the question data based on the document-level data of the target project, including: The question-answering model is invoked to determine the question type information corresponding to the question data based on the document hierarchy data of the target project. When the question type information corresponds to a document query type, the question semantic information of the question data is determined, and the at least one query task is constructed based on the document hierarchy structure contained in the document hierarchy data.

6. The data processing method according to claim 5, characterized in that, The method further includes: When the problem type information corresponds to the interface query type, the interface query task corresponding to the problem data is executed based on the interface database; When the question type information corresponds to a cross-domain query type, the query task corresponding to the question data is split into at least one query subtask, and the at least one query subtask is executed.

7. The data processing method according to claim 1, characterized in that, After executing the path node tasks contained in the semantic supplementary path based on the document encoding data to obtain the query data corresponding to each of the at least one query task, the method further includes: Based on the task status corresponding to the at least one query task, determine the task to be executed in the at least one query task; A supplementary task is generated for the task to be executed, and the task to be executed and the supplementary task are executed.

8. The data processing method according to claim 1, characterized in that, The step of inputting the document-level data and the target query data into the question-answering model to obtain the answer data corresponding to the question data includes: The document hierarchy data and the target query data are input into the question-answering model to obtain at least one answer and answer hint information. The at least one answer and the answer hint are used as the answer data corresponding to the question data.

9. A data processing system, characterized in that, Including both client and server sides, including: The client is used to submit problem data related to the target project to the server; The server is configured to invoke a question-answering model to generate at least one query task corresponding to the question data based on the document-level data of the target project. The document-level data is generated based on the document-encoded data and project data of the target project, and the document-encoded data is generated based on the project documents of the target project. A query task list is constructed based on the at least one query task. If the task sequence corresponding to the query task list includes a data augmentation task, a semantic supplementation path corresponding to the data augmentation task is determined. The path node tasks included in the semantic supplementation path are executed based on the document-encoded data to obtain query data corresponding to each of the at least one query task. The query data corresponding to each of the at least one query task is reconstructed to obtain target query data. The document-level data and the target query data are then input into the question-answering model to obtain the answer data corresponding to the question data.

10. A computing device, characterized in that, include: Memory and processor; The memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions, which, when executed by the processor, implement the steps of the data processing method according to any one of claims 1 to 8.

11. A computer-readable storage medium, characterized in that, It stores computer-executable instructions that, when executed by a processor, implement the steps of the data processing method according to any one of claims 1 to 8.

12. A computer program product, characterized in that, It includes a computer program or instructions that, when executed by a processor, implement the steps of the data processing method according to any one of claims 1 to 8.

Images