A method for establishing a dataset for multi-modal data fusion extraction in the field of solid waste disposal

Abstract

This invention discloses a method for constructing a dataset through multimodal data fusion and extraction in the field of solid waste disposal. The method includes: first, converting the original document into a sequence of multimodal objects; second, constructing a document-level context index and entity knowledge base, extracting entity relationships, and constructing a global knowledge base and index; next, using the global knowledge base and index, performing semantic classification and dynamic routing on the multimodal object sequence, performing multimodal divide-and-conquer extraction by assembling contextual information packets, and outputting discrete record fragments; finally, generating feature identifiers based on experimental conditions, reconstructing cross-modal data through a confidence priority mechanism, and generating a standard dataset. This invention effectively solves the attention drift and task confusion problems when large language models process complex long documents, overcomes the challenge of entity referencing resolution in cross-paragraph contexts, and overcomes the difficulties of information incompleteness and alignment in multimodal table and image data, significantly improving the quality of dataset construction.

Description

Technical Field

[0001] This invention relates to the field of solid waste disposal technology, specifically a method for establishing a dataset through multimodal data fusion and extraction in the field of solid waste disposal. Background Technology

[0002] With the advancement of the "dual carbon" goals, the resource utilization of solid waste (such as sludge pyrolysis and biomass gasification) has become a research hotspot. A vast amount of experimental data has been published in various scientific journals, often in unstructured formats, including complex text descriptions, diverse table formats, and graphs containing key trend information. Constructing a high-quality, structured dataset for solid waste disposal is of great significance for machine learning-assisted material design, process optimization, and data modeling.

[0003] Currently, the construction of datasets in the field of solid waste disposal mainly relies on manual extraction by researchers. This involves reading research literature one by one, manually identifying key parameters (such as raw material types and reaction temperatures) in the experimental materials and methods sections, as well as various attribute values ​​of raw materials and products, and manually entering them into tables or databases. This method is extremely inefficient and prone to errors.

[0004] With the rise of large language models, some fields, such as medicine and materials science, have begun to use them for batch data extraction from literature. However, there is currently less relevant content in the solid waste field. Documents in the solid waste field are typically lengthy, involve inconsistent naming of raw material entities, and have data mainly concentrated in figures and tables. Directly applying large language models to process long scientific documents can easily lead to attention drift and task confusion, making it difficult to maintain consistency across paragraph contexts. Furthermore, it cannot effectively solve the problems of automatic alignment and deep fusion of multimodal data (figures, tables, and text), resulting in insufficient completeness and standardization of the output data.

[0005] However, practice has shown that existing technologies have the following problems:

[0006] Firstly, conventional methods are extremely inefficient, and the standards of different personnel are subjective and inconsistent, resulting in a chaotic format of the aggregated dataset, making it difficult to use directly for machine learning and data modeling.

[0007] Secondly, when directly applying large language models to process lengthy scientific literature on solid waste, the structure of scientific literature is loose. The complete definitions, abbreviations, and detailed descriptions of key entities (such as raw materials) are usually located at the beginning of the article, while specific experimental data are scattered in the figures and tables in the following text. It is difficult to maintain the consistency of cross-paragraph context and cannot effectively solve the problem of automatic alignment and deep fusion of multimodal data of figures, tables, and text. This results in insufficient completeness and standardization of output data, making it difficult to maintain the uniqueness and continuity of entity identity in long-distance contexts. Consequently, it is impossible to correctly attribute the data extracted from the following text to specific entities in the preceding text.

[0008] Third, image data is often semantically incomplete or slightly biased, showing only some variables (such as trend curves), while key experimental conditions (such as heating rate and atmosphere) are only mentioned in the text or figure captions. Current technologies lack effective cross-modal indexing and fusion mechanisms, failing to automatically and accurately align these incomplete image data with experimental conditions in text or tables. This results in a large amount of high-value chart data remaining unused, and the data's completeness and standardization are severely inadequate. Summary of the Invention

[0009] The purpose of this section is to outline some aspects of the embodiments of the present invention and to briefly describe some preferred embodiments. Simplifications or omissions may be made in this section, as well as in the abstract and title of this application, to avoid obscuring the purpose of these documents; however, such simplifications or omissions should not be construed as limiting the scope of the invention.

[0010] 1. Technical problems to be solved:

[0011] To address the aforementioned issues such as attention drift and task confusion when large language models process complex long documents, entity referencing resolution and identity tracking in cross-paragraph contexts, and information incompleteness and alignment difficulties in multimodal data (especially charts), this invention is proposed.

[0012] Therefore, the purpose of this invention is to provide a method for establishing a dataset by fusing and extracting multimodal data in the field of solid waste disposal. This invention can effectively solve the problems of attention drift and task confusion when large language models process complex long documents; secondly, it solves the problems of entity alignment and referential resolution in cross-paragraph contexts, as well as the problems of information incompleteness and alignment difficulties in multimodal data (especially charts).

[0013] 2. Technical Solution:

[0014] To address the aforementioned technical problems, according to one aspect of the present invention, the present invention provides the following technical solution:

[0015] Includes the following steps:

[0016] S1: Document structuring and serialization preprocessing transforms multi-source raw documents in the field of solid waste disposal into a multimodal object sequence containing text paragraphs, tables, and image objects that can be processed by computers;

[0017] S2: Document-level context indexing and entity knowledge base construction: Scan the multimodal object sequence, filter entity association corpus and extract entity relationships and entity naming rules, construct a global knowledge base for subsequent multimodal extraction, and identify and extract the reference association features of text paragraphs to tables and images, and establish a cross-modal bidirectional index between text paragraph objects and table objects and image objects.

[0018] S3: Multimodal divide-and-conquer extraction based on semantic classification and dynamic routing. The multimodal objects are classified according to their semantic attributes. The global knowledge base and index built in step S2 are used to dynamically assemble context information packages for each type of multimodal object. The multimodal objects and their corresponding context information packages are routed to the corresponding classification data parsing model for deep parsing. The output is a discrete record fragment containing specific dimension data fields. The multidimensional data fields include experimental condition parameters, raw material attribute data, or product attribute data.

[0019] S4: Cross-modal data reconstruction based on feature identifiers: extract experimental condition parameters from the discrete record segments and generate feature identifiers; instantiate the experimental run master record object with the feature identifiers as the primary key; and attach the discrete record segments with the same feature identifiers to the corresponding experimental run master record object to aggregate them into a standard dataset for the solid waste disposal field.

[0020] As a preferred embodiment of the method for multimodal data fusion and extraction to establish a dataset in the field of solid waste disposal according to the present invention, step S1 specifically includes:

[0021] S1.1: Document data acquisition: Through a preset database interface and local storage path, scientific research documents in the field of solid waste disposal are acquired as input data sources.

[0022] S1.2: Structured parsing and serialization cleaning: The input data source is read using a document parsing tool. Based on the document structure parsing and layout analysis algorithm, non-content noise information such as headers, footers, page numbers, and formatting is identified and removed. The cleaned document content is then serialized into the multimodal object sequence.

[0023] As a preferred embodiment of the method for establishing a dataset through multimodal data fusion and extraction in the field of solid waste disposal according to the present invention, step S2 specifically includes:

[0024] S2.1: Noise removal and entity association corpus screening: Based on the preset solid waste domain document structure features, background noise paragraphs are filtered out from the multimodal object sequence, and the first semantic reasoning model is called to screen the remaining text paragraph objects, and objects containing specific entity features are marked as target entity paragraph objects.

[0025] S2.2: Extraction and standardization of cross-modal reference relationships. When the first semantic reasoning model is called in S2.1 to filter text paragraph objects, the natural language references to tables and pictures are identified simultaneously. The extracted table and picture identifiers are standardized and stored in the metadata field of the corresponding text paragraph objects, thereby establishing a bidirectional index association between text paragraph objects and table and picture objects.

[0026] S2.3: Entity association corpus aggregation, which concatenates the marked target entity paragraph objects according to their position order in the original document to construct an aggregated text block;

[0027] S2.4: Global entity recognition and naming rule extraction. The aggregated text block constructed in S2.3 is input into the second semantic reasoning model to perform entity alignment and entity naming rule extraction tasks, generating a document-level global raw material entity table and naming rule set as the global knowledge base.

[0028] As a preferred embodiment of the method for multimodal data fusion and extraction to establish a dataset in the field of solid waste disposal according to the present invention, step S3 specifically includes:

[0029] S3.1: Semantic classification and dynamic routing: Traverse all multimodal objects to be processed, use the third semantic reasoning model to determine the content attributes, mark them as semantic classification labels of "experimental condition source", "raw material attribute source", "product attribute source" and "other", and route them to the classification data parsing model that matches the semantic classification label.

[0030] S3.2: Dynamic assembly of context information package. Before extraction, a structured context information package is dynamically constructed for the multimodal object to be processed. The global knowledge base constructed in S2 is used as the global standard entity dictionary for entity recognition and the logical basis for parsing non-standard reference symbols.

[0031] For table and image objects in multimodal objects, the index built by S2 is also used to automatically retrieve and inject related text paragraph objects that refer to the table or image in the text, as cross-modal supplementary context data to supplement experimental conditions;

[0032] S3.3: Each classification data parsing model, based on the context information package assembled in S3.2, identifies and extracts specific dimension data fields corresponding to the semantic classification labels, and generates the discrete record fragments containing experimental condition parameters, raw material attribute data, or product attribute data.

[0033] As a preferred embodiment of the method for establishing a dataset through multimodal data fusion and extraction in the field of solid waste disposal according to the present invention, step S4 specifically includes:

[0034] S4.1: Data cleaning and dimensional normalization: Using a mapping algorithm based on keyword priority, non-standardized product and raw material terms in the literature are mapped to standard physical phases or standard raw material types; regular expressions are used to identify and remove non-standard symbols in numerical data and convert them into standard floating-point number format.

[0035] S4.2: Feature identifier generation and error correction, extracting experimental condition parameters from the discrete recording segments, and establishing experimental condition feature identifiers for cross-modal numerical values ​​based on the confidence priority mechanism;

[0036] S4.3: Master record instantiation and data fusion: Using the experimental condition feature identifier as an index key, match the existing experimental run master record object. If there is no corresponding experimental run master record object, instantiate a new experimental run master record object based on the experimental condition feature identifier; if it already exists, insert the data fields in the discrete record fragment into the experimental run master record object to complete cross-modal aggregation.

[0037] As a preferred embodiment of the method for multimodal data fusion and extraction to establish a dataset in the field of solid waste disposal according to the present invention, in step S4.2, the reliability priority mechanism is to establish the condition parameters from text or table sources as anchor points, align and correct the condition parameters from image sources within a preset tolerance range, and generate a unique experimental condition feature identifier based on the finally determined parameters; the preset reliability priorities are as follows: the parameters from the main text source have the highest priority, the parameters from the table source have the second highest priority, and the parameters from the image source have the lowest priority.

[0038] As a preferred embodiment of the method for multimodal data fusion and extraction to establish a dataset in the field of solid waste disposal according to the present invention, the first semantic reasoning model, the second semantic reasoning model, the third semantic reasoning model and the classification data parsing model are all constructed based on the large language model interface integrated in the processor.

[0039] The processor is pre-configured with a set of task constraint instructions for different modal extraction tasks in the solid waste field. When executing each model task, the processor extracts a few sample examples and constraint instructions for the corresponding task, and performs structured concatenation with the input data to generate a sequence of prompt words, thereby dynamically instantiating the large language model interface into a functional unit that performs the corresponding single computation task.

[0040] 3. Beneficial effects:

[0041] Compared with the prior art, the beneficial effects of the present invention are:

[0042] This method for establishing a dataset through multimodal data fusion and extraction in the field of solid waste disposal:

[0043] 1. This invention addresses the problems of time-consuming, subjective, and inconsistent manual processing by establishing a method for fusing and extracting multimodal data in the field of solid waste disposal to create a dataset;

[0044] 2. This invention categorizes multimodal objects in long documents based on semantic attributes, establishes an index, document-level entity tables, and naming rule tables, and distributes them to a dedicated classification data parsing model through dynamic routing. This significantly reduces the cognitive load on the model. Combined with dynamically assembled contextual information packets, it avoids logical gaps and solves the attention drift and task confusion problems that arise when large language models process long documents.

[0045] 3. This invention constructs a document-level global knowledge base (including a raw material entity table and a naming rule set) and a cross-modal bidirectional index. This enables the model to call global information for entity alignment at any time when processing local paragraphs or charts, ensuring that entity identities can be correctly parsed at any position in a long document and eliminating information gaps in cross-paragraph contexts;

[0046] 4. This invention utilizes experimental conditions to generate unique feature identifiers as anchor points and introduces a confidence priority mechanism to solve the problem of accuracy differences, thereby achieving automatic alignment and aggregation of graph, table, and text data, constructing a logically complete and high-quality standardized dataset, and realizing accurate alignment of cross-modal heterogeneous data. Attached Figure Description

[0047] To more clearly illustrate the technical solutions of the embodiments of the present invention, the present invention will be described in detail below with reference to the accompanying drawings and detailed embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort. Wherein:

[0048] Figure 1 This is an overall flowchart of a method for establishing a dataset through multimodal data fusion and extraction in the field of solid waste disposal, according to the present invention.

[0049] Figure 2 This is a schematic diagram of the multimodal divide-and-conquer extraction process for semantic classification and dynamic routing in this invention;

[0050] Figure 3 This is a schematic diagram of cross-modal data alignment based on feature identifiers and confidence priority in this invention. Detailed Implementation

[0051] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0052] This invention is described in detail with reference to the schematic diagrams. When describing the embodiments of this invention, for ease of explanation, the cross-sectional views illustrating the device structure may be partially enlarged, not according to the usual scale. Furthermore, the schematic diagrams are merely examples and should not be construed as limiting the scope of protection of this invention. In actual fabrication, the three-dimensional spatial dimensions of length, width, and depth should be included.

[0053] The orientation or positional relationship indicated in the terminology is based on the orientation or positional relationship shown in the accompanying drawings and is only for the convenience of describing the invention and simplifying the description, and is not intended to indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of the invention.

[0054] The term "connection method" should be interpreted broadly. For example, "connection" can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection or an indirect connection through an intermediate medium; it can be a connection within two components. Those skilled in the art can understand the specific meaning of the above terms in this invention according to the specific circumstances.

[0055] The embodiments of the present invention will now be described in further detail with reference to the accompanying drawings.

[0056] This invention provides a schematic diagram of the overall structure of an embodiment of a method for multimodal data fusion and extraction to establish a dataset in the field of solid waste disposal, comprising:

[0057] Please see Figure 1-3 This embodiment of a method for establishing a dataset through multimodal data fusion and extraction in the field of solid waste disposal includes the following steps:

[0058] S1: Document structuring and serialization preprocessing transforms multi-source raw documents in the field of solid waste disposal into a multimodal object sequence containing text paragraphs, tables, and image objects that can be processed by computers;

[0059] S2: Document-level context indexing and entity knowledge base construction: Scan the multimodal object sequence, filter entity association corpus and extract entity relationships and entity naming rules, construct a global knowledge base for subsequent multimodal extraction, and identify and extract the reference association features of text paragraphs to tables and images, and establish a cross-modal bidirectional index between text paragraph objects and table objects and image objects.

[0060] S3: Multimodal divide-and-conquer extraction based on semantic classification and dynamic routing. The multimodal objects are classified according to their semantic attributes. The global knowledge base and index built in step S2 are used to dynamically assemble context information packages for each type of multimodal object. The multimodal objects and their corresponding context information packages are routed to the corresponding classification data parsing model for deep parsing. The output is a discrete record fragment containing specific dimension data fields. The multidimensional data fields include experimental condition parameters, raw material attribute data, or product attribute data.

[0061] S4: Cross-modal data reconstruction based on feature identifiers: extract experimental condition parameters from the discrete record segments and generate feature identifiers; instantiate the experimental run master record object with the feature identifiers as the primary key; and attach the discrete record segments with the same feature identifiers to the corresponding experimental run master record object to aggregate them into a standard dataset for the solid waste disposal field.

[0062] It is worth noting that, specifically, step S1 includes:

[0063] S1.1: Document data acquisition: Through a preset database interface and local storage path, scientific research documents in the field of solid waste disposal are acquired as input data sources.

[0064] S1.2: Structured parsing and serialization cleaning: The input data source is read using a document parsing tool. Based on the document structure parsing and layout analysis algorithm, non-content noise information such as headers, footers, page numbers, and formatting is identified and removed. The cleaned document content is then serialized into the multimodal object sequence.

[0065] Next, specifically, step S2 includes:

[0066] S2.1: Noise removal and entity association corpus screening: Based on the preset solid waste domain document structure features, background noise paragraphs are filtered out from the multimodal object sequence, and the first semantic reasoning model is called to screen the remaining text paragraph objects, and objects containing specific entity features are marked as target entity paragraph objects.

[0067] S2.2: Extraction and standardization of cross-modal reference relationships. When the first semantic reasoning model is called in S2.1 to filter text paragraph objects, the natural language references to tables and pictures are identified simultaneously. The extracted table and picture identifiers are standardized and stored in the metadata field of the corresponding text paragraph objects, thereby establishing a bidirectional index association between text paragraph objects and table and picture objects.

[0068] S2.3: Entity association corpus aggregation, which concatenates the marked target entity paragraph objects according to their position order in the original document to construct an aggregated text block;

[0069] S2.4: Global entity recognition and naming rule extraction. The aggregated text block constructed in S2.3 is input into the second semantic reasoning model to perform entity alignment and entity naming rule extraction tasks, generating a document-level global raw material entity table and naming rule set as the global knowledge base.

[0070] Furthermore, specifically, step S3 includes:

[0071] S3.1: Semantic classification and dynamic routing: Traverse all multimodal objects to be processed, use the third semantic reasoning model to determine the content attributes, mark them as semantic classification labels of "experimental condition source", "raw material attribute source", "product attribute source" and "other", and route them to the classification data parsing model that matches the semantic classification label.

[0072] S3.2: Dynamic assembly of context information package. Before extraction, a structured context information package is dynamically constructed for the multimodal object to be processed. The global knowledge base constructed in S2 is used as the global standard entity dictionary for entity recognition and the logical basis for parsing non-standard reference symbols.

[0073] For table and image objects in multimodal objects, the index built by S2 is also used to automatically retrieve and inject related text paragraph objects that refer to the table or image in the text, as cross-modal supplementary context data to supplement experimental conditions;

[0074] S3.3: Each classification data parsing model, based on the context information package assembled in S3.2, identifies and extracts specific dimension data fields corresponding to the semantic classification labels, and generates the discrete record fragments containing experimental condition parameters, raw material attribute data, or product attribute data.

[0075] Next, specifically, step S4 includes:

[0076] S4.1: Data cleaning and dimensional normalization: Using a mapping algorithm based on keyword priority, non-standardized product and raw material terms in the literature are mapped to standard physical phases or standard raw material types; regular expressions are used to identify and remove non-standard symbols in numerical data and convert them into standard floating-point number format.

[0077] S4.2: Feature identifier generation and error correction, extracting experimental condition parameters from the discrete recording segments, and establishing experimental condition feature identifiers for cross-modal numerical values ​​based on the confidence priority mechanism;

[0078] S4.3: Master record instantiation and data fusion: Using the experimental condition feature identifier as an index key, match the existing experimental run master record object. If there is no corresponding experimental run master record object, instantiate a new experimental run master record object based on the experimental condition feature identifier; if it already exists, insert the data fields in the discrete record fragment into the experimental run master record object to complete cross-modal aggregation.

[0079] Meanwhile, specifically in step S4.2, the reliability priority mechanism establishes the condition parameters from the text or table source as anchor points, aligns and corrects the condition parameters from the image source within a preset tolerance range, and generates a unique experimental condition feature identifier based on the finally determined parameters; the preset reliability priorities are as follows: the parameters from the main text source have the highest priority, the parameters from the table source have the second highest priority, and the parameters from the image source have the lowest priority.

[0080] Finally, specifically, the first semantic reasoning model, the second semantic reasoning model, the third semantic reasoning model, and the classification data parsing model are all built on the large language model interface integrated in the processor;

[0081] The processor is pre-configured with a set of task constraint instructions for different modal extraction tasks in the solid waste field. When executing each model task, the processor extracts a few sample examples and constraint instructions for the corresponding task, and performs structured concatenation with the input data to generate a sequence of prompt words, thereby dynamically instantiating the large language model interface into a functional unit that performs the corresponding single computation task.

[0082] Example 1:

[0083] S1: Document structuring and serialization preprocessing.

[0084] S1.1: Document Data Acquisition. Using the API provided by the pre-defined Elsevier academic database, obtain research documents in HTML or XML format related to sludge pyrolysis as input data sources.

[0085] S1.2: Structured Parsing and Serialization Cleaning. Using parsing tools such as a DOM tree parser, the input data source is read. By traversing the DOM tree nodes, non-content noise information such as headers, footers, page numbers, and formatting is identified and removed. Subsequently, based on the document's logical structure, the document content is serialized into an ordered sequence of multimodal objects. Each paragraph object contains at least the text content and its position index information in the original document.

[0086] S2: Document-level context indexing and entity knowledge base construction.

[0087] S2.1: Noise Removal and Entity Association Corpus Filtering. The text paragraph objects in the multimodal object sequence generated in S1 are traversed. Based on preset solid waste domain document structure features (such as title features like "Abstract" and "References"), background noise paragraphs such as the preface, acknowledgments, and references are filtered out from the multimodal object sequence. Subsequently, the first semantic reasoning model is invoked to filter the remaining text paragraph objects.

[0088] Preferably, the first semantic reasoning model is built based on a large language model interface (such as Tongyi Qianwen, GPT4, etc. API) integrated in the processor. The processor is pre-configured with task constraint instructions for entity screening tasks. During the screening process, the processor extracts the task constraint instructions, which include definitions of specific entity features in the solid waste field (i.e., specific sludge names, abbreviation definitions, or reaction process descriptions), and structurally concatenates them with the text paragraphs to be processed to generate a prompt word sequence, which is then input into the large language model interface. The model logically judges the text content based on the prompt words. If the text contains the aforementioned features, it marks the text paragraph object as a "target entity paragraph object," thereby constructing a candidate corpus sequence that has eliminated redundant information.

[0089] S2.2: Extraction and Standardization of Cross-Modal Reference Relationships. In S2.1, when the first semantic reasoning model is called to filter text paragraph objects, natural language references to tables and images (such as "Fig." or "Table" and their numbers) are identified simultaneously. The extracted table and image identifiers are standardized and uniformly mapped to formats such as "FIG_1" and "TAB_2", and stored as key-value pairs in the metadata fields of the corresponding text paragraph objects, thereby establishing a bidirectional index relationship between text paragraph objects and table and image objects.

[0090] S2.3: Entity-related corpus aggregation. The marked target entity paragraph objects are concatenated according to their position order in the original document to construct aggregated text blocks.

[0091] S2.4: Global entity recognition and naming rule extraction. The aggregated text block constructed in S2.3 is input into the second semantic reasoning model to perform entity alignment and entity naming rule extraction tasks, generating a document-level global raw material entity table and naming rule set, which serves as the global knowledge base.

[0092] Preferably, the second semantic reasoning model is built based on a large language model interface integrated in the processor. The processor configuration includes role setting, entity extraction and rule parsing tasks, constraints, and structured prompts for few-shot examples, which are concatenated with the aggregated text block and input into the model. The model performs two tasks: first, it identifies and merges synonymous entities (such as full names and abbreviations); second, it parses the naming rules of experimental objects, and finally outputs a standardized global knowledge base.

[0093] S3: Multimodal divide-and-conquer extraction based on semantic classification and dynamic routing.

[0094] S3.1: Semantic Classification and Dynamic Routing. Traverse all multimodal objects to be processed, use a third semantic reasoning model to determine content attributes, and label them as semantic classification tags of "experimental condition source", "raw material attribute source", "product attribute source" and "other".

[0095] Based on the semantic classification labels and modal types (text / table / image) of objects, a dynamic routing mechanism is established to distribute objects to the corresponding classification data parsing models (for example, distributing text containing experimental parameters to the conditional extraction parsing model, and distributing images containing product data to the product attribute-image extraction parsing model).

[0096] Preferably, the third semantic reasoning model is built based on a large language model interface integrated in the processor. The processor extracts the content features of the object to be processed, calls the model to determine its semantic attributes, and sets classification labels. Based on the output labels, the processor automatically routes the object to the corresponding classification data parsing model.

[0097] Preferably, the classification data parsing model is also built based on the large language model interface, and its essence is a functional unit dynamically instantiated by the processor according to different task requirements. For each semantic tag, the processor pre-configures differentiated task constraint instructions and few-sample examples. For example, when routed to the "experimental condition extraction model", the processor loads the prompt word template for extracting experimental conditions; when routed to the "product attribute extraction model", it loads the prompt word template for extracting product attributes.

[0098] S3.2: Dynamic assembly of the context information package. Before extraction, a structured context information package is dynamically constructed for the current multimodal object to be processed. This package uses the global knowledge base constructed in S2 as the global standard entity dictionary for entity recognition and the logical basis for parsing non-standard referential symbols.

[0099] For table and image objects in multimodal objects, the index built by S2 is also used to automatically retrieve and inject related text paragraph objects that reference the table or image in the main text, as cross-modal supplementary context data to supplement experimental conditions.

[0100] S3.3: Each classification data parsing model, based on the context information package assembled in S3.2, identifies and extracts specific dimension data fields corresponding to the semantic classification labels, and generates the discrete record fragments containing experimental condition parameters, raw material attribute data, or product attribute data.

[0101] For objects labeled as "experimental condition sources", the model is driven by semantic matching instructions to identify control parameters and their units (control parameters include, but are not limited to, temperature, pressure, residence time, and heating rate).

[0102] For objects labeled as "raw material attribute source" or "product attribute source," the classification data parsing model is first instructed to determine the specific data type (e.g., elemental analysis, industrial analysis, calorific value, or yield) contained in the target object, thereby activating the corresponding extraction mode. Then, the classification data parsing model executes differentiated extraction logic based on modal features and comprehensively calls upon the prompts in the context information package: In the text modality, entity association extraction based on referential resolution is performed. The large language model is guided to identify "attribute-value" key-value pairs in sentences and forces the model to associate the value with a specific entity (raw material or product), thereby extracting scattered attribute data; in the table modality, semantic mapping extraction based on row and column structure is performed. Combining the injected global raw material entity table, product naming rule library, and related paragraph text, the model is instructed to perform structured cognition of the table, mapping the sample codes in the table header or row index back to globally unique raw material entity names, and using row and column cross-location to enable the model to accurately capture structured values ​​in cells; in the image modality, cross-modal parsing combining visual and textual methods is performed. By inputting the injected global raw material entity table, product naming rule library, and related paragraph text auxiliary information into the multimodal large model, the model is instructed to logically complete and align the visually recognized coordinate points or curve trends with the fixed experimental conditions (such as raw material type and reaction temperature) in the text prompts, and to convert the visually recognized coordinate points into data with clear physical meaning.

[0103] Ultimately, a series of semantically clear and structured discrete record fragments are produced, providing standardized input for subsequent fusion steps.

[0104] S4: Cross-modal data reconstruction based on feature identifiers.

[0105] S4.1: Data Cleaning and Dimensional Normalization. A mapping algorithm based on keyword priority is used to uniformly map non-standardized, multi-source descriptions of product and raw material terms in the literature to standard physical phases (solid, liquid, or gaseous) or standard raw material types. Simultaneously, regular expressions are used to automatically clean non-standard symbols in the numerical data, converting them to standard floating-point format to ensure data standardization.

[0106] S4.2: Feature identifier generation and error correction.

[0107] Each cleaned discrete record segment is analyzed to extract the core experimental condition set (including pyrolysis final temperature, heating rate, residence time, and initial moisture content of raw materials). All experimental condition parameters in the same record are constructed into a multi-dimensional experimental condition vector. To address the alignment failure problem caused by small precision differences between multimodal data sources (such as experimental condition values ​​extracted from images and experimental condition values ​​extracted from text) (e.g., a text description of "500℃" while an image extraction value of "499℃"), a confidence-priority vector tolerance normalization mechanism is introduced. Since data from text and tables typically have high confidence, while image extraction data is limited by visual recognition accuracy and thus has biases, parameters from the main text source are given the highest priority, followed by those from tables, and then those from images. Simultaneously, the condition vectors extracted from text and tables are established as a set of high-confidence anchor vectors. For record vectors from image sources, they are iterated and compared with the anchor vector set. The difference between the image record vector and the corresponding component of the anchor vector is calculated in each dimension (e.g., temperature dimension, time dimension). A match is considered successful only when all dimensional components of the image record vector simultaneously fall within the preset tolerance range of the corresponding dimension of the anchor vector (e.g., temperature error < ±5℃ and time error < ±1 minute). If the match is successful, all values ​​in the image record vector are forcibly corrected to the corresponding anchor vector values ​​to eliminate visual errors. If the match fails, the image record is determined to represent an independent experimental run or special experimental conditions, and all its original values ​​are retained without correction. After the alignment process is completed, the final determined condition vector parameters are concatenated with the raw material ID in a preset order to generate a unique feature identifier.

[0108] S4.3: Data Fusion.

[0109] Using the experimental condition feature identifier as an index key, an existing experimental run master record object is matched. If no corresponding experimental run master record object exists, a new experimental run master record object is instantiated based on the experimental condition feature identifier. If it already exists, the data fields in the discrete record fragment are inserted into the experimental run master record object to complete cross-modal aggregation.

[0110] This step achieves automatic alignment of cross-modal data through feature identifier matching, ensuring that data extracted from different sources (text, images, tables) can be automatically aggregated into the same experimental record as long as they have the same experimental condition feature identifiers, thereby completing the reconstruction from fragmented information to a structured dataset.

[0111] It is worth noting that, specifically, in this embodiment, the first semantic reasoning model, the second semantic reasoning model, the third semantic reasoning model, and the classification data parsing model are all built based on the large language model interface integrated in the processor. However, in practical applications, the model architecture is not limited to a single model and can be layered according to the difficulty of the task. For example, simple entity recognition can call a lightweight model, while complex semantic dependency analysis or image understanding can call a large-scale generative model.

[0112] Next, the processor is pre-configured with a set of task constraint instructions for different modal extraction tasks in the solid waste field. When executing each model task, the processor extracts the few sample examples and constraint instructions of the corresponding task, and performs structured concatenation with the input data to generate a sequence of prompt words, thereby dynamically instantiating the large language model interface into a functional unit that performs the corresponding single computation task.

[0113] Although the present invention has been described above with reference to embodiments, various modifications can be made and components can be replaced with equivalents without departing from the scope of the invention. In particular, as long as there is no structural conflict, the features in the disclosed embodiments can be combined with each other in any manner. The lack of an exhaustive description of these combinations in this specification is merely for the sake of brevity and resource conservation. Therefore, the present invention is not limited to the specific embodiments disclosed herein, but includes all technical solutions falling within the scope of the claims.

Claims

1. A method for establishing a dataset through multimodal data fusion and extraction in the field of solid waste disposal, characterized in that, Includes the following steps: S1: Document structuring and serialization preprocessing transforms multi-source raw documents in the field of solid waste disposal into a multimodal object sequence containing text paragraphs, tables, and image objects that can be processed by computers; S2: Document-level context indexing and entity knowledge base construction: Scan the multimodal object sequence, filter entity association corpus and extract entity relationships and entity naming rules, construct a global knowledge base for subsequent multimodal extraction, and identify and extract the reference association features of text paragraphs to tables and images, and establish a cross-modal bidirectional index between text paragraph objects and table objects and image objects. S3: Multimodal divide-and-conquer extraction based on semantic classification and dynamic routing. The multimodal objects are classified according to their semantic attributes. The global knowledge base and index built in step S2 are used to dynamically assemble context information packages for each type of multimodal object. The multimodal objects and their corresponding context information packages are routed to the corresponding classification data parsing model for deep parsing. The output is a discrete record fragment containing specific dimension data fields. The multidimensional data fields include experimental condition parameters, raw material attribute data, or product attribute data. S4: Cross-modal data reconstruction based on feature identifiers: extract experimental condition parameters from the discrete record segments and generate feature identifiers; instantiate the experimental run master record object with the feature identifiers as the primary key; and attach the discrete record segments with the same feature identifiers to the corresponding experimental run master record object to aggregate them into a standard dataset for the solid waste disposal field.

2. The method according to claim 1, characterized in that, Step S1 specifically includes: S1.1: Document data acquisition: Through a preset database interface and local storage path, scientific research documents in the field of solid waste disposal are acquired as input data sources. S1.2: Structured parsing and serialization cleaning: The input data source is read using a document parsing tool. Based on the document structure parsing and layout analysis algorithm, non-content noise information such as headers, footers, page numbers, and formatting is identified and removed. The cleaned document content is then serialized into the multimodal object sequence.

3. The method according to claim 1, characterized in that, Step S2 specifically includes: S2.1: Noise removal and entity association corpus screening: Based on the preset solid waste domain document structure features, background noise paragraphs are filtered out from the multimodal object sequence, and the first semantic reasoning model is called to screen the remaining text paragraph objects, and objects containing specific entity features are marked as target entity paragraph objects. S2.2: Extraction and standardization of cross-modal reference relationships. When the first semantic reasoning model is called in S2.1 to filter text paragraph objects, the natural language references to tables and pictures are identified simultaneously. The extracted table and picture identifiers are standardized and stored in the metadata field of the corresponding text paragraph objects, thereby establishing a bidirectional index association between text paragraph objects and table and picture objects. S2.3: Entity association corpus aggregation, which concatenates the marked target entity paragraph objects according to their position order in the original document to construct an aggregated text block; S2.4: Global entity recognition and naming rule extraction. The aggregated text block constructed in S2.3 is input into the second semantic reasoning model to perform entity alignment and entity naming rule extraction tasks, generating a document-level global raw material entity table and naming rule set as the global knowledge base.

4. The method according to claim 1, characterized in that, Step S3 specifically includes: S3.1: Semantic classification and dynamic routing: Traverse all multimodal objects to be processed, use the third semantic reasoning model to determine the content attributes, mark them as semantic classification labels of "experimental condition source", "raw material attribute source", "product attribute source" and "other", and route them to the classification data parsing model that matches the semantic classification label. S3.2: Dynamic assembly of context information package. Before extraction, a structured context information package is dynamically constructed for the multimodal object to be processed. The global knowledge base constructed in S2 is used as the global standard entity dictionary for entity recognition and the logical basis for parsing non-standard reference symbols. For table and image objects in multimodal objects, the index built by S2 is also used to automatically retrieve and inject related text paragraph objects that refer to the table or image in the text, as cross-modal supplementary context data to supplement experimental conditions; S3.3: Each classification data parsing model, based on the context information package assembled in S3.2, identifies and extracts specific dimension data fields corresponding to the semantic classification labels, and generates the discrete record fragments containing experimental condition parameters, raw material attribute data, or product attribute data.

5. The method according to claim 1, characterized in that, Step S4 specifically includes: S4.1: Data cleaning and dimensional normalization: Using a mapping algorithm based on keyword priority, non-standardized product and raw material terms in the literature are mapped to standard physical phases or standard raw material types; regular expressions are used to identify and remove non-standard symbols in numerical data and convert them into standard floating-point number format. S4.2: Feature identifier generation and error correction, extracting experimental condition parameters from the discrete recording segments, and establishing experimental condition feature identifiers for cross-modal numerical values ​​based on the confidence priority mechanism; S4.3: Master record instantiation and data fusion: Using the experimental condition feature identifier as an index key, match the existing experimental run master record object. If there is no corresponding experimental run master record object, instantiate a new experimental run master record object based on the experimental condition feature identifier; if it already exists, insert the data fields in the discrete record fragment into the experimental run master record object to complete cross-modal aggregation.

6. The method according to claim 5, characterized in that, In step S4.2, the reliability priority mechanism establishes the condition parameters from the text or table source as anchor points, aligns and corrects the condition parameters from the image source within a preset tolerance range, and generates a unique experimental condition feature identifier based on the finally determined parameters. The preset reliability priorities are as follows: parameters from the main text source have the highest priority, parameters from the table source have the second highest priority, and parameters from the image source have the lowest priority.

7. The method according to claim 3 or 4, characterized in that, The first semantic reasoning model, the second semantic reasoning model, the third semantic reasoning model, and the classification data parsing model are all built based on the large language model interface integrated in the processor; The processor is pre-configured with a set of task constraint instructions for different modal extraction tasks in the solid waste field. When executing each model task, the processor extracts a few sample examples and constraint instructions for the corresponding task, and performs structured concatenation with the input data to generate a sequence of prompt words, thereby dynamically instantiating the large language model interface into a functional unit that performs the corresponding single computation task.

Images