Ship project data automatic extraction method based on document type and attribute mapping

Abstract

The present application relates to the technical field of ship project data processing, and particularly relates to a ship project data automatic extraction method based on document type and attribute mapping, which extracts features to obtain a document type through hierarchical classification by constructing a mapping rule library, extracts attribute values after searching in three domains of a text, a table and a chapter in parallel and cross-domain mutual verification according to the document type, fills the attribute values into a structured data table and records traceability information, realizes automatic and efficient extraction of ship project data, and improves accuracy and traceability.

Description

Technical Field

[0001] This invention relates to the field of ship project data processing technology, and in particular to an automated method for extracting ship project data based on document type and attribute mapping. Background Technology

[0002] In the process of ship design, construction, and operation management, project documents (such as design drawings, bills of materials, and technical specifications) typically contain a large amount of structured or semi-structured key data. Current technologies generally rely on manual review and data entry of each document item, or on the use of template-based parsing tools to automatically extract data based on preset field positions and keyword rules. While these approaches are feasible in scenarios with uniform document formats and stable content layouts, their limitations become increasingly apparent when dealing with the massive, heterogeneous, and frequently changing document systems in actual shipbuilding projects.

[0003] The main drawback of conventional methods lies in their lack of adaptive document type recognition capabilities. Ship documents from different sources exhibit significant differences in chapter divisions, table structures, and terminology. Traditional methods are only effective for one or a few fixed templates; once the document format changes or unexpected types are mixed in, the parsing process is prone to failure, leading to significant data omissions or misalignments. Furthermore, existing methods typically perform searches only within a single information area (such as the main text or tables), failing to consider cross-domain information verification and error correction. For example, when the same attribute value contradicts itself in different chapters or tables, the system lacks an effective conflict detection and confidence assessment mechanism, blindly adopting the initial matching result, making it difficult to meet the stringent reliability requirements of shipbuilding engineering.

[0004] The aforementioned issues have led to a heavy reliance on manual verification and secondary correction in current shipbuilding project data extraction processes. This is not only inefficient but also makes it difficult to trace the root causes of errors, creating potential problems for subsequent design changes, supply chain management, and compliance reviews. The industry urgently needs a data extraction solution that can automatically adapt to document types and enable collaborative verification across multiple data sources. Summary of the Invention

[0005] This invention provides a method and system for automated extraction of ship project data based on document type and attribute mapping, which can solve the problems in the prior art.

[0006] A first aspect of this invention provides a method for automated extraction of ship project data based on document type and attribute mapping, comprising: Construct a mapping rule base that stores document type definitions and data attribute definitions, as well as mapping records that associate document type identifiers with target attribute identifiers and configure search strategies and data transformation rules; Receive ship project documents, extract the document's metadata and text features, perform hierarchical classification processing based on the document's metadata and text features to assign document type identifiers, and generate standardized document objects; Based on the document type identifier, the associated mapping records are retrieved from the mapping rule base to generate an extraction task list. For standardized document objects, parallel searches are performed in three domains according to the search strategy: in the text area, regular expression matching is performed using keywords and variant keywords; in the table area, cell value extraction is performed using semantic similarity of table headers and row and column cross-location; and in the chapter area, keyword location search is performed after constraining the search scope by matching chapter title levels. Cross-domain mutual verification is performed on the search results of the three domains. Based on the semantic consistency between the extraction results of each search domain, mutual confirmation or conflict status is determined, and the confidence is adjusted accordingly before selecting the optimal match. Each standardized attribute value is populated into a structured data table according to the attribute definition. For each attribute value, the source document identifier, original text location information, matching search domain identifier, and extraction time are recorded to form a data traceability chain.

[0007] Building a mapping rule base includes: A priority score is calculated for each mapping record. The priority score is obtained by weighting and summing the document source authority score, historical extraction success rate score, and rule timeliness score according to preset weight coefficients. The source authority dimension score is determined according to the sorting position of the document type associated with the mapping record in the preset authority level sequence and the segmented mapping rules. The historical extraction success rate dimension score is the ratio of the cumulative number of successful extractions to the total number of extraction attempts for the mapping record. The timeliness dimension score is assigned according to the time interval between the most recent update time of the mapping record and the current time, based on a preset decay coefficient. Based on the attribute dependencies parsed between extraction tasks, a directed acyclic dependency graph is constructed. The extraction tasks are divided into multiple execution batches by topological sorting. Independent extraction tasks without attribute dependencies are assigned to the first batch for parallel execution, while extraction tasks that depend on previous results are assigned to subsequent batches for sequential execution after the dependency attribute values ​​are obtained. Extraction tasks within the same batch are arranged in descending order of priority score and a preemptive termination strategy is adopted: when the extraction task with the highest priority in the same batch is successfully executed and the confidence score reaches the preset main threshold, the remaining extraction tasks in the batch are skipped.

[0008] Based on the document's metadata and text features, a hierarchical classification process is performed to assign document type identifiers and generate standardized document objects, including: Perform the first layer of rule-based deterministic keyword matching: extract the file name text and perform edit distance matching with the document type alias library, scan the front area of ​​the document text to detect deterministic feature words, and if at least a preset threshold of deterministic feature words are hit, directly output the document type identifier and set the classification confidence to a deterministic value; If the first-level matching fails, the second-level classification based on text vectorization and a linear classifier is executed: after preprocessing the document text by word segmentation and stop word removal, the inverse document frequency vector features of the word frequency are extracted according to multi-granularity features at the character level and word level. The vector features are input into a linear classifier, which outputs probability estimates for each candidate class using a one-to-many strategy. If the highest probability value exceeds the first probability threshold and the difference between the highest probability and the second highest probability is not lower than the preset difference threshold, then the candidate category with the highest probability will be used as the document type identifier. If the second layer does not meet the output conditions, the third layer is executed for classification based on a pre-trained semantic model: the filename and the truncated part of the text are concatenated into the input sequence, and the semantic vector is extracted by the pre-trained semantic model. The semantic vector is input into the classification output layer to calculate the probability distribution of each candidate category. The category with the highest probability is taken as the document type identifier. If the highest probability is lower than the second probability threshold, the document is marked as awaiting manual judgment.

[0009] Perform parallel searches across three domains on standardized document objects according to the configured search strategy, including: Perform regular expression matching searches based on keywords and their variants in the text area, including: Retrieve the main keyword specified by the search strategy and the set of variant keywords derived from the thesaurus. Iterates through the document's main text blocks, performing exact matches against the primary keyword. When a match is found, the context window is captured, and regular expressions are applied sequentially for pattern capture. If a regular expression fails to capture the pattern, the window is expanded and captured again. If none of the main keywords are matched, initiate variant keyword matching and execute regular expression capture. Perform cell value extraction search within the table area based on semantic similarity of the table header and row / column intersection location, including: Traverse the document tables to extract candidate header areas, and perform semantic similarity calculations between the header text and the header keyword set specified by the search strategy to select the target table. After constructing the column name to column index mapping relationship of the target table, locate the target row and column according to the row matching pattern and column matching pattern specified by the search strategy, and extract the text value of the cross cell according to the extraction position strategy; Perform keyword-based location searches within the chapter area, constraining the search scope after matching chapter title levels, including: The document chapter structure tree is parsed to form a chapter-level index. The target chapter node is matched in the chapter tree according to the chapter matching pattern specified by the search strategy. Keyword matching and regular expression capture extraction are performed within the text range corresponding to the target chapter.

[0010] Perform cross-domain mutual verification on the search results from the three domains and adjust the confidence level accordingly to select the best match, including: Obtain the candidate extraction results and their initial confidence scores returned by the main text area, table area and chapter area respectively. Select two different search domains from the three search domains to form a domain pair combination. Extract standardized attribute values ​​from the candidate extraction results of each domain pair combination. Calculate the difference measure between the two standardized attribute values ​​as the inter-domain consistency measure. If the inter-domain consistency metric of the domain pair combination meets the preset consistency judgment condition, then the two search domains of the domain pair are determined to be in a mutually confirmed state, and a preset positive gain value is applied to the initial confidence score of the candidate extraction results in the mutually confirmed state. If the inter-domain consistency metric of the domain pair combination does not meet the preset consistency judgment condition, the two search domains of the domain pair are determined to be in conflict. After reducing the confidence score of the candidate results involved in the conflict state, the deep semantic arbitration process is triggered. Deep semantic arbitration backtracks the original context in the source document and extracts the extended context semantic vector for each candidate extraction result of the conflict state. The extended context semantic vector of each candidate extraction result is independently semantically compared with the semantic vector of the target data attribute description, and the semantic similarity is calculated. The candidate extraction result with the highest semantic similarity is selected as the final output.

[0011] The search strategy for mapping records in the mapping rule base is dynamically and adaptively updated according to the following mechanism: Monitor the extraction failure rate and average confidence score of each mapping record within the preset evaluation period. When the extraction failure rate of a mapping record exceeds the preset warning threshold or the average confidence score is lower than the preset lower limit threshold, trigger the adaptive adjustment process of the search strategy for that mapping record. The adaptive adjustment process attempts the following steps: after expanding the keyword variant derivation range of the mapping record search strategy to the preset expanded range, the indicator recovery status is re-evaluated. If the recovery is not achieved, the regular expression matching mode of the search strategy is downgraded from the exact mode to the loose mode and then re-evaluated. If the recovery is still not achieved, the context window width of the search strategy is expanded to the preset expanded width and then re-evaluated. After each adjustment, the adjusted search strategy is repeatedly verified with a preset number of verifications. If the evaluation metric returns to the normal range, the adjustment process is terminated and the adjusted parameters are persistently updated to the search strategy configuration of the mapping record. If the evaluation metric still does not return after all adjustment operations are performed in sequence, the mapping record is marked as pending manual revision and the administrator is notified to intervene in the revision.

[0012] A second aspect of this invention provides an automated data extraction system for ship projects based on document type and attribute mapping, comprising: The rule base unit is used to build a mapping rule base, which stores document type definitions and data attribute definitions, as well as mapping records that associate document type identifiers with target attribute identifiers and configure search strategies and data transformation rules; The document identification unit is used to receive ship project documents, extract the metadata and text features of the documents, perform hierarchical classification processing based on the metadata and text features of the documents to assign document type identification, and generate standardized document objects. The three-domain verification unit is used to retrieve associated mapping records from the mapping rule base based on document type identifiers to generate an extraction task list. It performs parallel searches in the three domains according to the search strategy for standardized document objects: in the text area, it performs regular expression matching search using keywords and variant keywords; in the table area, it performs cell value extraction search using semantic similarity of table headers and row and column cross-location; in the chapter area, it performs keyword location search after constraining the search range by matching chapter title levels; it performs cross-domain mutual verification on the search results of the three domains; it determines the mutual confirmation or conflict status based on the semantic consistency between the extraction results of each search domain, adjusts the confidence level accordingly, and selects the optimal match. The data traceability unit is used to populate the standardized attribute values ​​into the structured data table according to the attribute definition. For each attribute value, the source document identifier, original text location information, matching search domain identifier and extraction time are recorded to form a data traceability chain.

[0013] A third aspect of the present invention provides an electronic device, comprising: processor; Memory used to store processor-executable instructions; The processor is configured to invoke instructions stored in the memory to execute the aforementioned method.

[0014] A fourth aspect of the present invention provides a computer-readable storage medium having stored thereon computer program instructions that, when executed by a processor, implement the aforementioned method.

[0015] This method achieves flexible association between document type definitions and attribute definitions by constructing a mapping rule base, significantly improving its adaptability to processing various types of ship project documents. Hierarchical progressive classification processing comprehensively utilizes metadata and text features, greatly improving the accuracy of document type identification and laying a solid foundation for subsequent accurate extraction. The generation of standardized document objects unifies the data source format, effectively reducing the processing complexity and error probability caused by heterogeneous documents.

[0016] The three-domain parallel search strategy employs differentiated search techniques tailored to the characteristics of the main text, tables, and chapters, significantly improving the speed and accuracy of attribute value localization. The main text region uses regular expression matching to support keywords and variations, flexibly capturing various expressions; the table region accurately extracts structured data units based on semantic similarity of table headers and row / column cross-location; and the chapter region narrows the search scope through hierarchical matching, effectively avoiding interference from irrelevant content. A cross-domain mutual verification mechanism integrates the semantic consistency of the three-domain extraction results for mutual confirmation or conflict determination, dynamically adjusting the confidence level to select the optimal match. This fundamentally avoids omissions or misjudgments that might occur with a single search domain, making the final extraction results more reliable and complete. Attached Figure Description

[0017] Figure 1 This is a flowchart illustrating an automated data extraction method for ship projects based on document type and attribute mapping. Figure 2 This is the processing interface for three-domain parallel extraction and mutual verification in this embodiment of the invention; Figure 3 This is the processing interface of the document recognition center in this embodiment of the invention. Detailed Implementation

[0018] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0019] The technical solution of the present invention will be described in detail below with reference to specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments.

[0020] refer to Figures 1 to 3 , Figure 1 This is a flowchart illustrating the automated data extraction method for ship projects based on document type and attribute mapping according to an embodiment of the present invention, including: Construct a mapping rule base that stores document type definitions and data attribute definitions, as well as mapping records that associate document type identifiers with target attribute identifiers and configure search strategies and data transformation rules; Receive ship project documents, extract the document's metadata and text features, perform hierarchical classification processing based on the document's metadata and text features to assign document type identifiers, and generate standardized document objects; Based on the document type identifier, retrieve associated mapping records from the mapping rule base to generate an extraction task list, and perform parallel searches on standardized document objects in three domains according to the configured search strategy: In the text area, regular expression matching is performed using keywords and variant keywords. In the table area, cell value extraction is performed using semantic similarity of table headers and row and column cross-location. In the chapter area, keyword location search is performed after constraining the search range by matching chapter title levels. Cross-domain mutual verification is performed on the search results of the three domains. Based on the semantic consistency between the extraction results of each search domain, mutual confirmation or conflict status is determined and the confidence is adjusted accordingly before selecting the optimal match. Each standardized attribute value is populated into a structured data table according to the attribute definition. For each attribute value, the source document identifier, original text location information, matching search domain identifier, and extraction time are recorded to form a data traceability chain.

[0021] Figure 2 This is the processing interface for parallel extraction and mutual verification of three domains in this embodiment of the invention. The system performs searches in the main text, table and chapter areas for the host continuous power attribute, displays the candidate values ​​of each domain, the initial confidence level and the pairwise consistency judgment results, and uses 18,600 kW after multi-domain confirmation as the final structured output; the conflict arbitration and dependent task batch execution status are also shown below.

[0022] Figure 3 In this embodiment of the invention, the document recognition center's processing interface receives a ship project document, extracts the document's metadata and text features, determines the document type through a three-level progressive classification mechanism consisting of deterministic keyword matching, text classification, and semantic models, generates a standardized document object, and creates a corresponding attribute extraction task accordingly.

[0023] In one alternative implementation, building the mapping rule base includes: A priority score is calculated for each mapping record. The priority score is obtained by weighting and summing the document source authority score, historical extraction success rate score, and rule timeliness score according to preset weight coefficients. The source authority dimension score is determined according to the sorting position of the document type associated with the mapping record in the preset authority level sequence and the segmented mapping rules. The historical extraction success rate dimension score is the ratio of the cumulative number of successful extractions to the total number of extraction attempts for the mapping record. The timeliness dimension score is assigned according to the time interval between the most recent update time of the mapping record and the current time, based on a preset decay coefficient. Based on the attribute dependencies parsed between extraction tasks, a directed acyclic dependency graph is constructed. The extraction tasks are divided into multiple execution batches by topological sorting. Independent extraction tasks without attribute dependencies are assigned to the first batch for parallel execution, while extraction tasks that depend on previous results are assigned to subsequent batches for sequential execution after the dependency attribute values ​​are obtained. Within the same batch, extraction tasks are sorted in descending order of priority score and a preemptive termination strategy is adopted: If the highest priority extraction task in the same batch is successfully executed and the confidence score reaches the preset main threshold, the remaining extraction tasks in that batch will be skipped.

[0024] The construction of the mapping rule base includes not only document type definition, data attribute definition, and static storage of mapping records, but also assigning a dynamic priority score to each mapping record to ensure orderly scheduling based on rule quality differences during actual extraction. The priority score is derived by weighted merging of sub-scores from three dimensions: document source authority, historical extraction success rate, and rule timeliness. Let the preset weight coefficients for the three dimensions be... , , And satisfy The sub-scores for the three dimensions are as follows: , , Then the priority score of a certain mapping record It is calculated in the following way: The weighting coefficients can be adjusted at the configuration layer according to the project stage and data quality requirements. For example, they can be appropriately reduced when historical data accumulation is insufficient in the early stages of a project. The proportion of data sources can be appropriately increased in scenarios with diverse data sources. The proportion of.

[0025] Source authority rating The determination of authority levels relies on a pre-defined hierarchy. This hierarchy arranges common document types in shipbuilding projects according to their information authority, such as classification society approvals, shipowner technical specifications, design institute drawings, and supplier technical specifications, each corresponding to a different authority level. Under the segmented mapping rule, the hierarchy is divided into several intervals, each corresponding to a fixed score. Document types falling into higher authority intervals will receive higher scores. The score is assigned to document types with lower authority, and lower scores are assigned to document types with lower authority. This segmented mapping method avoids the problem of amplifying small differences caused by linear scoring, and ensures that document types within the same authority level have the same starting point for competition in this dimension.

[0026] Historical extraction success rate dimension score Directly retrieve the cumulative number of successful extractions for this mapping record. Total number of extraction attempts The ratio, i.e. When a mapping record has not yet accumulated any historical execution records, It is zero at this time. Use a preset default value (usually an intermediate value, such as 0.5) to avoid new rules being overly suppressed or over-prioritized due to a lack of historical data. After each extraction task is completed, regardless of success or failure, the result must be synchronously written back to the mapping rule base to update the corresponding mapping record. and Thus It evolves dynamically over time, gradually reflecting the rule's actual applicability in real documents.

[0027] Timeliness Dimension Score The calculation is based on the time interval between the most recent update time of the mapped record and the current time. Let the time interval be... (Unit: days), preset attenuation coefficient is ,but Assigning values ​​according to exponential decay: When the mapping record has just been updated Approaching zero A value approaching 1 indicates that the rule has high timeliness; as time goes on, The decay coefficient gradually decreases, indicating that the rule may no longer accurately reflect changes in the current document format or data organization, and requires timely review and updating. Different configurations can be set for different document types. For example, a larger value can be set for supplier quotation documents that have frequent format changes. The value can be set to a smaller value for classification society certificates, which have relatively stable formats. value.

[0028] Beyond the priority scoring system, the construction of the mapping rule base also involves parsing the attribute dependencies between extraction tasks and establishing a scheduling structure. In the scenario of ship project data extraction, the extraction of some attributes depends on the prior acquisition of other attribute values. For example, the unit conversion of equipment power depends on the identification result of the system category, and the calculation of cabin volume depends on the extraction result of ship type parameters. For these dependencies, it is necessary to parse the dependencies between tasks after generating the extraction task list and construct a directed acyclic dependency graph based on this. Each node in the graph represents an extraction task. The execution requires a task If the output is used as input, then in the graph from Towards Add a directed edge to represent yes The precursor mission.

[0029] After constructing the directed acyclic dependency graph, a topological sorting algorithm is used to traverse the nodes in the graph. All nodes with an in-degree of zero (i.e., independent extraction tasks that do not depend on the output of any other task) are assigned to the first execution batch. These tasks can be executed in full parallel, provided resources allow, fully utilizing multi-threaded or distributed computing resources to shorten the overall extraction time. Once the tasks in the first batch are completed and their results are written to the intermediate attribute cache, the in-degree of subsequent tasks that depend on the output of these tasks decreases accordingly. When all predecessor tasks of a task are completed and its in-degree drops to zero, that task can be added to the next execution batch. This process iterates until all extraction tasks are assigned to their corresponding batches, forming a multi-layered serial-parallel hybrid execution structure. This topological sorting-based batch partitioning method ensures strict adherence to attribute dependencies at the execution level while maximizing the parallelism of tasks within the same batch.

[0030] Within the same execution batch, multiple extraction tasks are scored according to their priority. Tasks are sorted in descending order, with higher-priority tasks receiving execution resources first. Simultaneously, a preemptive termination strategy is introduced within each batch to further improve execution efficiency. Specifically, when the highest-priority extraction task within the same batch completes first, and its extraction result's confidence score reaches a preset master threshold... If the target attributes for a batch have been extracted with sufficiently high reliability, then the remaining extraction tasks within that batch that have not yet started or are waiting in the queue are skipped, and no further execution resources are allocated to them. The core logic of this strategy is that if the optimal rule has already given a high-confidence result, the marginal benefit of continuing to execute low-priority rules is extremely limited, while the computational resources and time costs they consume cannot be ignored. Especially when facing a large-scale ship project document set containing hundreds of mapping records, the preemptive termination strategy can significantly reduce ineffective computational overhead.

[0031] It should be noted that the preemptive termination strategy is triggered only when the confidence score of the highest priority task meets the main threshold condition. If the highest priority task executes successfully but its confidence score is lower than the threshold, the preemptive termination strategy will not be triggered. If the condition is not met, termination will not be triggered, and subsequent tasks within the batch will be executed sequentially. The extraction results of each task will participate in the subsequent cross-domain mutual verification process to improve the reliability of the final result by integrating evidence from multiple sources. (Main Threshold) The value of the threshold needs to be balanced between ensuring accuracy and improving efficiency. It can be set uniformly in the global configuration layer of the mapping rule base, or a stricter primary threshold can be set for specific attribute types (such as safety-critical attributes) to ensure that the extraction quality of high-risk data fields is not compromised by efficiency optimization.

[0032] Based on the document's metadata and text features, a hierarchical classification process is performed to assign document type identifiers and generate standardized document objects, including: Perform the first layer of rule-based deterministic keyword matching: Extract the file name text and perform edit distance matching with the document type alias library. Scan the front area of ​​the document body to detect deterministic feature words. If at least a preset threshold of deterministic feature words are hit, directly output the document type identifier and set the classification confidence to a deterministic value. If the first-level matching fails, the second-level classification based on text vectorization and a linear classifier is executed: After preprocessing the document text by word segmentation and stop word removal, inverse document frequency vector features are extracted using character-level and word-level multi-granularity features. The vector features are input into a linear classifier, which outputs probability estimates for each candidate class using a one-to-many strategy. If the highest probability value exceeds the first probability threshold and the difference between the highest probability and the second highest probability is not lower than the preset difference threshold, then the candidate category with the highest probability will be used as the document type identifier. If the output conditions are not met in the second layer, the third layer, based on the pre-trained semantic model, is executed for classification. The filename and the truncated text portion are concatenated into an input sequence, and a pre-trained semantic model extracts semantic vectors.

[0033] The semantic vector is input into the classification output layer to calculate the probability distribution of each candidate category. The category with the highest probability is taken as the document type identifier. If the highest probability is lower than the second probability threshold, the document is marked as awaiting manual judgment.

[0034] Upon receiving the ship project documents, the documents need to be type-identified so that the corresponding extraction rules can be retrieved from the mapping rule base. The hierarchical classification processing adopts a three-layer architecture from simple to complex and from fast to slow. The upward propagation stops when the lower level can produce a deterministic output. The higher-level classifier is only activated when the lower level cannot meet the output conditions, thereby controlling the overall computational cost while ensuring classification accuracy.

[0035] The first layer of classification uses rule-based deterministic keyword matching, targeting ship project document types with significant document naming conventions or fixed characteristic words. The entire text content of the filename is extracted and edit distance matching is performed against a pre-maintained document type alias library. Edit distance matching tolerates common spelling variations, abbreviation differences, and version number appendices in filenames. The edit distance threshold can be configured individually for different types of alias entries; a smaller tolerance distance is set for shorter standard names, while a wider tolerance distance can be set for longer descriptive names. Simultaneously, the initial area of ​​the document text (usually the first few pages or paragraphs) is scanned, and a set of deterministic characteristic words predefined for each document type is detected within this area. Deterministic characteristic words refer to words or phrases that are highly specific to a particular document type and rarely appear in other document types, such as contract number format declarations specific to shipbuilding contracts, fixed heading phrases on technical specification covers, and fixed header terms in drawing catalogs. If the number of definitive feature words matched in the front region is not less than the preset threshold, the corresponding document type identifier is directly output, and the confidence score of this classification is set to a certain value (usually a fixed value of full score or close to full score, indicating that the result does not require further verification). The first layer of judgment logic is simple and fast. For project batches with well-defined document naming conventions or distinctive document header features, most documents can be classified at this layer.

[0036] If the first-level matching fails (i.e., the filename edit distances do not fall within the threshold range, and the number of deterministic feature words matched in the front region is insufficient to reach the preset threshold), then the second-level classification is activated. This layer is based on text vectorization and a linear classifier. Word segmentation is performed on the document body text, removing stop words (including auxiliary words, prepositions, and high-frequency common words without actual semantic meaning). Inverse document frequency vector features are extracted at both the character and word levels, i.e., character-level features are constructed simultaneously. The TF-IDF vectors of the character level and the word level are concatenated to form a complete multi-granularity feature vector. Character-level features can capture local character patterns and abbreviation rules of professional terms in the document, while word-level features reflect the overall vocabulary distribution characteristics of the document. The combination of the two can cover the actual situation of diverse terminology and inconsistent expression in ship project documents.

[0037] The concatenated multi-granularity feature vectors are input into a linear classifier. The linear classifier employs a one-vs-rest strategy, training an independent binary classifier for each candidate document type. Each classifier outputs a probability estimate of the current document belonging to its corresponding category, ultimately yielding the probability distribution of all candidate categories. If the highest probability value exceeds a first probability threshold... Furthermore, the difference between the highest probability value and the second highest probability value is not less than a preset difference threshold. The candidate category with the highest probability is then output as the document type identifier. A difference threshold is introduced. The purpose is to eliminate fuzzy judgments when the probabilities of the two types are similar, even if the highest probability exceeds... If the probability is too close to the second-highest probability, the current feature vector is considered insufficient to distinguish between the two candidate types and is not output at this layer. Instead, it is passed to the third layer for deeper semantic understanding. The second-layer classification is suitable for documents with irregular naming but statistically discriminative word distribution in the text. In shipbuilding project practice, various technical reports, calculation sheets, inspection records, and other document types often have relatively stable word distribution characteristics, and the second-layer classification can effectively handle such situations.

[0038] When the second layer fails to meet the output conditions, the third classification layer is activated. This layer achieves deeper contextual semantic understanding based on a pre-trained semantic model. The filename text and truncated portions of the document body (e.g., the first few sentences or summary paragraphs) are concatenated into a unified input sequence using a specific delimiter and fed into the pre-trained semantic model. The pre-trained semantic model performs deep encoding on the input sequence, extracting a comprehensive semantic vector that integrates the semantics of the filename and the content of the body. This semantic vector captures the deep semantic features of the document and has a stronger discriminative ability for document types with insignificant differences in word distribution but distinct semantic themes. The semantic vector is then fed into the classification output layer, which is typically a fully connected layer followed by a softmax activation function. The probability distribution of each candidate category is calculated, and the category with the highest probability is taken as the document type identifier. If the highest probability is lower than a second probability threshold... If the pre-trained semantic model is deemed unable to complete the classification with sufficient confidence, the document is marked as awaiting manual judgment. Human reviewers then intervene to confirm the document type, and the review results can be simultaneously fed back to the document type alias library or the deterministic feature word library to continuously improve the rule system.

[0039] After all three levels of classification are completed, standardized document objects are generated. These standardized document objects not only contain the original document content but also record metadata such as document type identifier, classification level (indicating which classifier layer output the type identifier), classification confidence score, and classification timestamp. For documents output from the third layer with the highest probability higher than [previous level], [further details are needed]. For documents output from the first layer, the confidence score is directly set to the highest probability value; for documents output from the second layer, the confidence score is also set to the highest probability value; for documents output from the first layer, the confidence score is set to a fixed value. This confidence score information will be used in subsequent cross-domain mutual verification and optimal matching selection stages to comprehensively evaluate the reliability of the extraction results. The standardized document object structure ensures the consistency of the interface between subsequent extraction task generation and parallel search stages. Regardless of the original document format, all documents enter the subsequent processing flow in the same structured form.

[0040] Perform parallel searches across three domains on standardized document objects according to the configured search strategy, including: Perform regular expression matching searches based on keywords and their variants in the text area, including: Retrieve the main keyword specified by the search strategy and the set of variant keywords derived from the thesaurus. Iterates through the document's main text blocks, performing exact matches against the primary keyword. When a match is found, the context window is captured, and regular expressions are applied sequentially for pattern capture. If a regular expression fails to capture the pattern, the window is expanded and captured again. If none of the main keywords are matched, initiate variant keyword matching and execute regular expression capture. Perform cell value extraction search within the table area based on semantic similarity of the table header and row / column intersection location, including: Traverse the document tables to extract candidate header areas, and perform semantic similarity calculations between the header text and the header keyword set specified by the search strategy to select the target table. After constructing the column name to column index mapping relationship of the target table, locate the target row and column according to the row matching pattern and column matching pattern specified by the search strategy, and extract the text value of the cross cell according to the extraction position strategy; Perform keyword-based location searches within the chapter area, constraining the search scope after matching chapter title levels, including: The document chapter structure tree is parsed to form a chapter-level index. The target chapter node is matched in the chapter tree according to the chapter matching pattern specified by the search strategy. Keyword matching and regular expression capture extraction are performed within the text range corresponding to the target chapter.

[0041] When performing a three-domain parallel search on a standardized document object, the search processes of the text area, table area and chapter area are independent and proceed synchronously. Each domain completes data location and extraction according to the pre-configured parameters and rules in the search strategy, and finally the results are aggregated for subsequent cross-domain mutual verification.

[0042] For searching the main text area, the search first reads the list of main keywords from the search strategy and expands each main keyword using a thesaurus to generate a set of variant keywords. The thesaurus stores mappings of professional terms in the field of marine engineering; for example, expressions like "main engine power," "main propulsion power," and "ME power" are grouped into the same semantic group, ensuring that variant keywords cover various writing styles arising from differences in expression habits within the document. When traversing the document's main text blocks, precise string matching is performed on each block using the main keywords. Once a match is found, a fixed-length context window is immediately extracted centered on the matched position. The window length is determined by the context window parameter in the search strategy, typically covering several characters before and after the matched word to ensure the target value or description falls completely within the window. Within the extracted context window, the regular expression list configured in the search strategy is applied sequentially for pattern capture. Regular expressions are prioritized, with higher-precision patterns being tried first. If all regular expressions within the current window fail to capture the pattern, the window is expanded by a certain number of characters at both ends before re-executing the regular expression capture. This expansion operation can be repeated until the maximum window length allowed by the strategy is reached. If the primary keyword is not matched in the entire text, the process switches to the variant keyword set. Matching is performed one by one according to the weight of the keywords in the set. Upon a match, the context window is captured and the regular expression capture process is executed until the target value is successfully extracted or the variant keyword set is completely traversed. After a successful regular expression capture, the captured group content is used as candidate attribute values, and the type of matched keyword (primary keyword or variant keyword) and its position are recorded for subsequent confidence assessment.

[0043] For table region searches, all identified table objects in the document are traversed, and candidate header regions are extracted for each table. The header region is typically the first row or first column of the table, determined based on the table structure parsing results. Semantic similarity is calculated between the text content of each cell in the candidate header region and the header keyword set specified in the search strategy. The semantic similarity is calculated based on the cosine similarity of the two text segments using a pre-trained text vector model. Let the vector representation of the current candidate header cell text be... The vector representation of the target header keywords is as follows: Then semantic similarity The calculation is as follows: When a table contains elements with a similarity exceeding a preset threshold... When the header cell is selected, the table is chosen as the target table. If multiple tables meet the threshold condition, the one with the highest similarity is selected. After selecting the target table, a mapping relationship between column names and column indices is constructed, that is, binding the header text of each column with the column number, forming a search structure that can quickly locate the column index by column name. Then, the target row is located according to the row matching mode configured in the search strategy. The row matching mode can be in various forms, such as exact string matching, regular expression matching, or direct specification of row number. The column matching mode is similar, supporting matching by column name or direct specification of column index. After determining the target row index and target column index, the text value is read from the cell where the row and column intersect according to the extraction position strategy. The extraction position strategy supports taking the value of the intersecting cell itself, as well as taking the value of the cell at the offset position of the row or column where the intersecting cell is located, to deal with the situation where the value and label are not in the same cell in some tables. The extracted cell text value, along with the position identifier of the target table in the document and the header similarity score, is recorded together for subsequent tracing and confidence assessment.

[0044] For searching chapter areas, the process begins by parsing the standardized document object to identify headings at all levels and their hierarchical relationships, constructing a chapter-level index tree. Each node in the chapter-level index tree stores the chapter title text, level depth, and the starting and ending character positions within the document. Leaf nodes correspond to the lowest-level chapter, and the text range of non-leaf nodes covers the text range of all their child nodes. Chapter structure parsing relies on heading style tags in the document format or text lines in the body text that conform to the hierarchical heading regular expression pattern. For multi-level numbered headings commonly found in shipbuilding project documents (such as the "3.2.1" format), the chapter depth is automatically inferred from the numbering level. After the chapter-level index is constructed, target chapter node matching is performed in the chapter tree according to the chapter matching pattern configured in the search strategy. The chapter matching pattern can specify precise title text, keyword inclusion matching, or regular expression matching, and can be supplemented with hierarchical constraints to limit the search to chapter nodes at a specific depth, thereby avoiding ambiguity caused by chapters with the same name at different levels. After matching the target chapter node, the start and end positions of that node are used as the search scope boundaries. Keyword matching and regular expression capture are performed on the text content within this scope. The specific process is the same as the exact matching and regular expression capture logic of the main text area, but the search space is strictly constrained to the target chapter text area, effectively eliminating interference from keywords with the same name in other chapters of the document. If the target chapter node has multiple levels of sub-chapters, the search scope can be selected to cover only the text directly under the current node (excluding sub-chapter content) or cover the entire subtree range, depending on the strategy configuration, to adapt to different document chapter organization methods. After the chapter area search is completed, the capture results, along with the title text and hierarchical path of the hit chapter node, are output together. The hierarchical path is represented by the sequence of chapter titles from the root node to the target node, such as "Technical Specifications > Propulsion System > Host Parameters", which facilitates accurate reconstruction of the chapter context from which the extraction originated during subsequent data tracing.

[0045] The search results from all three domains carry the source domain identifier, the hit location information, and their respective confidence-related indicators. After all parallel searches are completed, they are submitted to the cross-domain mutual verification stage. The mutual verification logic determines the semantic consistency of the results from the three domains, and then completes the confidence adjustment and optimal matching selection to ensure the accuracy and reliability of the final attribute values.

[0046] Perform cross-domain mutual verification on the search results from the three domains and adjust the confidence level accordingly to select the best match, including: Obtain the candidate extraction results and their initial confidence scores returned by the main text area, table area and chapter area respectively. Select two different search domains from the three search domains to form a domain pair combination. Extract standardized attribute values ​​from the candidate extraction results of each domain pair combination. Calculate the difference measure between the two standardized attribute values ​​as the inter-domain consistency measure. If the inter-domain consistency metric of the domain pair combination meets the preset consistency judgment condition, then the two search domains of the domain pair are determined to be in a mutually confirmed state, and a preset positive gain value is applied to the initial confidence score of the candidate extraction results in the mutually confirmed state. If the inter-domain consistency metric of the domain pair combination does not meet the preset consistency judgment condition, the two search domains of the domain pair are determined to be in conflict. After reducing the confidence score of the candidate results involved in the conflict state, the deep semantic arbitration process is triggered. Deep semantic arbitration backtracks the original context in the source document and extracts the extended context semantic vector for each candidate extraction result of the conflict state. The extended context semantic vector of each candidate extraction result is independently semantically compared with the semantic vector of the target data attribute description, and the semantic similarity is calculated. The candidate extraction result with the highest semantic similarity is selected as the final output.

[0047] After parallel extraction across the three search domains, each search domain returns a set of candidate extraction results, each carrying an initial confidence score generated by the domain's internal scoring mechanism. The initial confidence score for the main text region is derived from the positional weight of regular expression matching and keyword hit strength; the initial confidence score for the table region is derived from a comprehensive evaluation of the semantic similarity of the table header and the accuracy of cell positioning; and the initial confidence score for the chapter region is derived from the degree of matching of the chapter title hierarchy and the offset distance of keyword positioning. These three types of initial confidence scores remain independent and are not merged before entering the cross-domain mutual verification process.

[0048] The core mechanism of cross-domain mutual verification is to construct domain pair combinations and calculate the inter-domain consistency metric within each domain pair. Pairing the three search domains—text, table, and chapter—results in three domain pair combinations: text and table, text and chapter, and table and chapter. For each domain pair combination, the candidate extraction result with the highest current confidence is taken from each of the two search domains. The original extracted text is then standardized to obtain standardized attribute values. Standardization includes unit normalization, numerical format unification, and string denoising to eliminate surface inconsistencies caused by differences in document layout. After standardization, the difference metric between the two standardized attribute values ​​is calculated as the inter-domain consistency metric for that domain pair. ,in and These represent the two search domain numbers in the domain pair. For numeric attributes, Take the ratio of the absolute value of the difference between the two values ​​to the reference magnitude; for text attributes, Take the normalized edit distance or the semantic distance based on word vectors.

[0049] Obtaining inter-domain consistency metrics Then, it is compared with the preset consistency judgment condition. The preset consistency judgment condition consists of a consistency threshold. Definition: When When, it is determined that the two search domains of the domain pair are in a mutually confirmed state; when When this occurs, the two search domains of the pair are determined to be in a conflict state. Threshold Different configurations can be made for different attribute types, such as numeric attributes. A relative error tolerance of 5% is typically set for text-type attributes. The normalized edit distance is usually set to an upper limit of 0.15.

[0050] For domain pairs that are determined to be mutually confirmed, a preset positive gain value is applied to the initial confidence scores of the candidate extraction results returned by each of the two search domains in the domain pair. This allows candidate results that participate in the confirmation process to receive higher weight in subsequent rankings. Positive gain value. The magnitude can be increased based on the number of domain pairs involved in the confirmation: if a candidate result is confirmed by two different domain pairs simultaneously, then the cumulative effect is applied. The gain reflects the positive support effect of multi-domain consistency on the reliability of the results. In the scenario of ship project data extraction, key dimensional parameters such as the overall length of the ship and the design draft usually appear simultaneously in the main text description, parameter summary table and related chapters of the technical specifications. The three search domains are very likely to form a state of mutual confirmation between each other. At this time, the confidence score of the candidate results will be significantly higher than the results extracted by only a single domain after the cumulative gain, which effectively improves the reliability of the final output.

[0051] For domain pairs identified as conflicting, the confidence scores of the two candidate extraction results involved in the conflict are reduced by a preset penalty value. After the confidence level is lowered, if a candidate result is inferior in all conflict domain pairs and its adjusted confidence level is below the minimum confidence threshold, it can be directly removed from the candidate set; otherwise, the candidate result is retained and marked as being in a conflict pending arbitration state, triggering the deep semantic arbitration process. Conflict states are quite common in actual engineering scenarios. For example, inconsistencies may arise between the gross tonnage value in a shipbuilding contract and the registered data in the corresponding table due to version differences, or between the main engine power in the chapter description and the value marked in the technical specifications table due to unit conversion errors. These discrepancies will trigger conflict determination and enter the arbitration process.

[0052] The deep semantic arbitration process performs a deep comparison based on contextual semantics for the candidate extraction results of conflict states. For each conflict candidate, it traces back to its original position in the source document, and expands several sentences or lines before and after the candidate's position, extracting expanded contextual text fragments. The size of the expansion range is dynamically adjusted according to the document type: for clause-intensive contract documents, the expansion range is three sentences before and after; for table-dominated technical specification documents, the expansion range is the current line and two adjacent lines. The expanded contextual text fragments are input into a pre-trained semantic encoding model to generate expanded contextual semantic vectors. ,in This is the number of the candidate extraction result.

[0053] Simultaneously, the attribute definition text of the current target data attribute is read from the mapping rule base. This text contains the semantic description of the attribute, typical expression methods, and contextual feature keywords, and is also input into the semantic encoding model to generate a semantic vector describing the target attribute. Target attribute description semantic vector It remains fixed within the same arbitration process, serving as a unified reference benchmark for semantic comparison of all conflict candidate results.

[0054] For each conflict candidate, compute its extended context semantic vector. semantic vectors describing target attributes semantic similarity between The calculation method is the cosine similarity between the two vectors: Semantic similarity of each candidate result Calculations are performed independently, without interference. After calculation, select... The highest-scoring candidate extraction result is used as the final output of deep semantic arbitration. After the candidate result with the highest semantic similarity is determined, its corresponding original candidate extraction value will replace other conflicting candidate results as the final extraction value of that attribute in the current document.

[0055] After a complete process of cross-domain mutual verification and deep semantic arbitration, each target data attribute receives a unique final candidate extraction result and its adjusted confidence score. When all domain pairs are mutually confirmed, the candidate result with the highest confidence score is directly used as the optimal match output; if a conflict exists and is resolved through arbitration, the candidate result output by arbitration is used as the optimal match, and its confidence score is updated to the semantic similarity obtained from arbitration. The corresponding values ​​after normalization mapping are used to maintain the dimensional consistency of the confidence system. The final selected optimal matching result, along with its adjusted confidence score, is passed to the subsequent structured data table filling stage, completing the convergence process from multi-domain parallel extraction to a single reliable output.

[0056] The search strategy for mapping records in the mapping rule base is dynamically and adaptively updated according to the following mechanism: Monitor the extraction failure rate and average confidence score of each mapping record within the preset evaluation period. When the extraction failure rate of a mapping record exceeds the preset warning threshold or the average confidence score is lower than the preset lower limit threshold, trigger the adaptive adjustment process of the search strategy for that mapping record. The adaptive adjustment process was tried sequentially: After expanding the keyword variant derivation range of the mapping record search strategy to the preset expanded range, re-evaluate the indicator recovery status. If it is not recovered, downgrade the regular expression matching mode of the search strategy from the exact mode to the loose mode and re-evaluate. If it is still not recovered, expand the context window width of the search strategy to the preset expanded width and re-evaluate. After each adjustment, the adjusted search strategy is repeatedly verified with a preset number of verifications. If the evaluation metric returns to the normal range, the adjustment process is terminated and the adjusted parameters are persistently updated to the search strategy configuration of the mapping record. If the evaluation metric still does not return after all adjustment operations are performed in sequence, the mapping record is marked as pending manual revision and the administrator is notified to intervene in the revision.

[0057] Each mapping record in the mapping rule base continuously accumulates extraction result data during operation, which forms the basis for adaptive updates of the search strategy. Within a preset evaluation period, two core metrics are calculated for each mapping record: extraction failure rate and average confidence score. The extraction failure rate reflects the proportion of the mapping record that failed to successfully locate the target attribute value under the current search strategy configuration, while the average confidence score is the average final confidence score obtained after cross-domain mutual verification of the three domain search results. When the extraction failure rate of a mapping record exceeds a preset warning threshold, or its average confidence score falls below a preset lower limit threshold, it is determined that the current search strategy for that mapping record can no longer meet normal extraction requirements, triggering the adaptive adjustment process. Both the warning threshold and the lower limit threshold are uniformly configured by the administrator according to the project's accuracy requirements during the initialization phase of the mapping rule base and can be adjusted as needed during subsequent operation and maintenance. The triggering condition uses "OR" logic, meaning that triggering occurs if any metric exceeds its limit, ensuring early detection of search strategy degradation.

[0058] The adaptive adjustment process attempts three types of adjustment operations sequentially according to a fixed priority order. After each type of operation is completed, the metrics are immediately re-evaluated. Once the metrics return to the normal range, subsequent adjustments are terminated to avoid excessive modification of the search strategy configuration. The first type of adjustment operation targets the keyword variant derivation range. In the original search strategy, the derivation range of keyword variants is usually limited to a narrow range, such as synonym replacement and abbreviation expansion, to ensure matching accuracy. When the metrics exceed these limits, the derivation range is expanded to a preset expanded range. This includes introducing a wider thesaurus coverage, allowing cross-language equivalents (such as Chinese-English glossary entries), and incorporating non-standard expressions of industry terminology. The expanded variant set is then reapplied to regular expression matching searches in the main text area and keyword location searches in the chapter area. By expanding the coverage of candidate matching words, the recall rate is improved, thereby improving the extraction failure rate metric.

[0059] If, after expanding the keyword variant derivation range and re-evaluating, the extraction failure rate and average confidence score still do not return to the normal range, then the second type of adjustment operation will be implemented: downgrading the regular expression matching mode from exact mode to relaxed mode. In exact mode, the regular expression requires the target string to strictly match the pattern in terms of boundaries, format, and special characters, suitable for scenarios where document format specifications and numerical expressions are consistent. In relaxed mode, the regular expression relaxes the matching constraints on whitespace characters, punctuation marks, number formats (such as the presence or absence of thousands separators, decimal places), and unit expressions (such as the mixing of "kW" and "kilowatt"), allowing for the omission or change of order of some fields to accommodate the format diversity issues caused by version differences or inconsistent compilation standards in ship project documents. The downgrade operation only applies to the search strategy configuration of the current mapping record and does not affect other mapping records. After the downgrade is completed, a re-evaluation will also be performed to determine whether the indicators have recovered.

[0060] If the metrics do not recover after downgrading from precise mode to relaxed mode, the third adjustment operation is initiated: expanding the context window width to the preset expansion width. The context window width determines how many characters or lines are extended forward and backward as the candidate extraction range after locating the keyword in the main text area. The original configuration has a narrower window width to reduce noise interference; after expansion, the candidate extraction range is extended forward and backward to the number of characters or lines specified by the preset expansion width, ensuring that the target attribute value can be included in the candidate set even if it appears far from the keyword. This adjustment is particularly effective for situations where there are many inserted explanatory texts, footnotes, or cross-references in shipbuilding project documents, resulting in a large physical distance between the keyword and the attribute value. After all three adjustment operations are completed and re-evaluated in sequence, if the metrics still do not recover, the automatic adjustment process is terminated, and manual intervention is initiated.

[0061] After each adjustment operation, the adjusted search strategy is repeatedly validated with a preset number of validations, rather than relying solely on the result of a single evaluation. Repeated validation involves performing extractions on multiple historical document samples or newly arrived documents corresponding to the preset number of validations, accumulating and statistically analyzing the extraction failure rate and average confidence score to eliminate misjudgments caused by sample randomness in a single evaluation. Only when the repeated validation results consistently meet the recovery conditions is the indicator considered to have recovered to the normal range. After recovery is determined, the adjustment process is terminated, and the parameters modified in this adjustment—including the expanded set of keyword variants, the currently effective regular expression matching pattern identifier, and the currently effective context window width value—are persistently written to the search strategy configuration field of the mapping record, overwriting the original configuration. The persistence operation synchronously updates the most recent update timestamp of the mapping record, ensuring that the timeliness dimension in subsequent priority scoring calculations is aware of this adjustment.

[0062] If all three adjustment operations are executed sequentially, and the evaluation metrics still fail to recover after repeated verification, the status field of the mapping record will be marked as "Pending Manual Revision," and a notification will be sent to the administrator. The notification will include the mapping record's identifier, the current extraction failure rate, the current average confidence score, the attempted adjustment operations, and a summary of the verification results for each operation. The administrator will then manually analyze the mapping record based on the notification. Possible revision directions include redefining the keyword set, modifying data transformation rules, adjusting search domain priorities, or redesigning regular expression patterns. After manual revision, the administrator will write the revised search strategy configuration to the mapping record, restore the status field from "Pending Manual Revision" to normal, and reset the failure rate statistics for that evaluation period to zero, allowing subsequent automatic monitoring to start accumulating data from the newly configured baseline again.

[0063] In the design of the entire adaptive update mechanism, the order of the three types of adjustment operations reflects the gradual principle of expanding from local to global relaxation: the keyword variant derivation range expansion only affects the candidate word set, having the least impact on matching accuracy; the regular expression matching mode downgrade affects the matching constraint strength, with moderate side effects; and the context window width expansion affects the positioning range of candidate values, posing the greatest risk of introducing noise. This order ensures that while restoring extraction capabilities, the accuracy of the search strategy is maintained as much as possible, avoiding an increase in the false extraction rate due to excessive relaxation. The evaluation cycle, warning threshold, lower limit threshold, preset expansion range, preset expansion width, and preset verification count are all stored as configurable parameters in the global configuration area of ​​the mapping rule base, supporting separate settings according to project type or document category to meet the differentiated needs of different shipbuilding projects in terms of data quality requirements and document standardization.

[0064] A second aspect of this invention provides an automated data extraction system for ship projects based on document type and attribute mapping, comprising: The rule base unit is used to build a mapping rule base, which stores document type definitions and data attribute definitions, as well as mapping records that associate document type identifiers with target attribute identifiers and configure search strategies and data transformation rules; The document identification unit is used to receive ship project documents, extract the metadata and text features of the documents, perform hierarchical classification processing based on the metadata and text features of the documents to assign document type identification, and generate standardized document objects. The three-domain verification unit is used to retrieve associated mapping records from the mapping rule base based on document type identifiers to generate an extraction task list. It performs parallel searches in the three domains according to the search strategy for standardized document objects: in the text area, it performs regular expression matching search using keywords and variant keywords; in the table area, it performs cell value extraction search using semantic similarity of table headers and row and column cross-location; in the chapter area, it performs keyword location search after constraining the search range by matching chapter title levels; it performs cross-domain mutual verification on the search results of the three domains; it determines the mutual confirmation or conflict status based on the semantic consistency between the extraction results of each search domain, adjusts the confidence level accordingly, and selects the optimal match. The data traceability unit is used to populate the standardized attribute values ​​into the structured data table according to the attribute definition. For each attribute value, the source document identifier, original text location information, matching search domain identifier and extraction time are recorded to form a data traceability chain.

[0065] A third aspect of the present invention provides an electronic device, comprising: processor; Memory used to store processor-executable instructions; The processor is configured to invoke instructions stored in the memory to execute the aforementioned method.

[0066] A fourth aspect of the present invention provides a computer-readable storage medium having stored thereon computer program instructions that, when executed by a processor, implement the aforementioned method.

[0067] This invention can be a method, apparatus, system, and / or computer program product. The computer program product may include a computer-readable storage medium having computer-readable program instructions loaded thereon for performing various aspects of the invention.

[0068] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. An automated method for extracting ship project data based on document type and attribute mapping, characterized in that, include: Construct a mapping rule base that stores document type definitions and data attribute definitions, as well as mapping records that associate document type identifiers with target attribute identifiers and configure search strategies and data transformation rules; Receive ship project documents, extract the document's metadata and text features, perform hierarchical classification processing based on the document's metadata and text features to assign document type identifiers, and generate standardized document objects; Based on the document type identifier, the associated mapping records are retrieved from the mapping rule base to generate an extraction task list. For standardized document objects, parallel searches are performed in three domains according to the search strategy: in the text area, regular expression matching is performed using keywords and variant keywords; in the table area, cell value extraction is performed using semantic similarity of table headers and row and column cross-location; and in the chapter area, keyword location search is performed after constraining the search scope by matching chapter title levels. Cross-domain mutual verification is performed on the search results of the three domains. Based on the semantic consistency between the extraction results of each search domain, mutual confirmation or conflict status is determined, and the confidence is adjusted accordingly before selecting the optimal match. Each standardized attribute value is populated into a structured data table according to the attribute definition. For each attribute value, the source document identifier, original text location information, matching search domain identifier, and extraction time are recorded to form a data traceability chain.

2. The method according to claim 1, characterized in that, Building a mapping rule base includes: A priority score is calculated for each mapping record. The priority score is obtained by weighting and summing the document source authority score, historical extraction success rate score, and rule timeliness score according to preset weight coefficients. The source authority dimension score is determined according to the sorting position of the document type associated with the mapping record in the preset authority level sequence and the segmented mapping rules. The historical extraction success rate dimension score is the ratio of the cumulative number of successful extractions to the total number of extraction attempts for the mapping record. The timeliness dimension score is assigned according to the time interval between the most recent update time of the mapping record and the current time, based on a preset decay coefficient. Based on the attribute dependencies parsed between extraction tasks, a directed acyclic dependency graph is constructed. The extraction tasks are divided into multiple execution batches by topological sorting. Independent extraction tasks without attribute dependencies are assigned to the first batch for parallel execution, while extraction tasks that depend on previous results are assigned to subsequent batches for sequential execution after the dependency attribute values ​​are obtained. Extraction tasks within the same batch are arranged in descending order of priority score and a preemptive termination strategy is adopted: when the extraction task with the highest priority in the same batch is successfully executed and the confidence score reaches the preset main threshold, the remaining extraction tasks in the batch are skipped.

3. The method according to claim 1, characterized in that, Based on the document's metadata and text features, a hierarchical classification process is performed to assign document type identifiers and generate standardized document objects, including: Perform the first layer of rule-based deterministic keyword matching: extract the file name text and perform edit distance matching with the document type alias library, scan the front area of ​​the document text to detect deterministic feature words, and if at least a preset threshold of deterministic feature words are hit, directly output the document type identifier and set the classification confidence to a deterministic value; If the first-level matching fails, the second-level classification based on text vectorization and a linear classifier is executed: after preprocessing the document text by word segmentation and stop word removal, the inverse document frequency vector features of the word frequency are extracted according to multi-granularity features at the character level and word level. The vector features are input into a linear classifier, which outputs probability estimates for each candidate class using a one-to-many strategy. If the highest probability value exceeds the first probability threshold and the difference between the highest probability and the second highest probability is not lower than the preset difference threshold, then the candidate category with the highest probability will be used as the document type identifier. If the second layer does not meet the output conditions, the third layer is executed for classification based on a pre-trained semantic model: the filename and the truncated part of the text are concatenated into the input sequence, and the semantic vector is extracted by the pre-trained semantic model. The semantic vector is input into the classification output layer to calculate the probability distribution of each candidate category. The category with the highest probability is taken as the document type identifier. If the highest probability is lower than the second probability threshold, the document is marked as awaiting manual judgment.

4. The method according to claim 1, characterized in that, Perform parallel searches across three domains on standardized document objects according to the configured search strategy, including: Perform regular expression matching searches based on keywords and their variants in the text area, including: Retrieve the main keyword specified by the search strategy and the set of variant keywords derived from the thesaurus. Iterates through the document's main text blocks, performing exact matches against the primary keyword. When a match is found, the context window is captured, and regular expressions are applied sequentially for pattern capture. If a regular expression fails to capture the pattern, the window is expanded and captured again. If none of the main keywords are matched, initiate variant keyword matching and execute regular expression capture. Perform cell value extraction search within the table area based on semantic similarity of the table header and row / column intersection location, including: Traverse the document tables to extract candidate header areas, and perform semantic similarity calculations between the header text and the header keyword set specified by the search strategy to select the target table. After constructing the column name to column index mapping relationship of the target table, locate the target row and column according to the row matching pattern and column matching pattern specified by the search strategy, and extract the text value of the cross cell according to the extraction position strategy; Perform keyword-based location searches within the chapter area, constraining the search scope after matching chapter title levels, including: The document chapter structure tree is parsed to form a chapter-level index. The target chapter node is matched in the chapter tree according to the chapter matching pattern specified by the search strategy. Keyword matching and regular expression capture extraction are performed within the text range corresponding to the target chapter.

5. The method according to claim 1, characterized in that, Perform cross-domain mutual verification on the search results from the three domains and adjust the confidence level accordingly to select the best match, including: Obtain the candidate extraction results and their initial confidence scores returned by the main text area, table area and chapter area respectively. Select two different search domains from the three search domains to form a domain pair combination. Extract standardized attribute values ​​from the candidate extraction results of each domain pair combination. Calculate the difference measure between the two standardized attribute values ​​as the inter-domain consistency measure. If the inter-domain consistency metric of the domain pair combination meets the preset consistency judgment condition, then the two search domains of the domain pair are determined to be in a mutually confirmed state, and a preset positive gain value is applied to the initial confidence score of the candidate extraction results in the mutually confirmed state. If the inter-domain consistency metric of the domain pair combination does not meet the preset consistency judgment condition, the two search domains of the domain pair are determined to be in conflict. After reducing the confidence score of the candidate results involved in the conflict state, the deep semantic arbitration process is triggered. Deep semantic arbitration backtracks the original context in the source document and extracts the extended context semantic vector for each candidate extraction result of the conflict state. The extended context semantic vector of each candidate extraction result is independently semantically compared with the semantic vector of the target data attribute description, and the semantic similarity is calculated. The candidate extraction result with the highest semantic similarity is selected as the final output.

6. The method according to claim 1, characterized in that, The search strategy for mapping records in the mapping rule base is dynamically and adaptively updated according to the following mechanism: Monitor the extraction failure rate and average confidence score of each mapping record within the preset evaluation period. When the extraction failure rate of a mapping record exceeds the preset warning threshold or the average confidence score is lower than the preset lower limit threshold, trigger the adaptive adjustment process of the search strategy for that mapping record. The adaptive adjustment process attempts the following steps: after expanding the keyword variant derivation range of the mapping record search strategy to the preset expanded range, the indicator recovery status is re-evaluated. If the recovery is not achieved, the regular expression matching mode of the search strategy is downgraded from the exact mode to the loose mode and then re-evaluated. If the recovery is still not achieved, the context window width of the search strategy is expanded to the preset expanded width and then re-evaluated. After each adjustment, the adjusted search strategy is repeatedly verified with a preset number of verifications. If the evaluation metric returns to the normal range, the adjustment process is terminated and the adjusted parameters are persistently updated to the search strategy configuration of the mapping record. If the evaluation metric still does not return after all adjustment operations are performed in sequence, the mapping record is marked as pending manual revision and the administrator is notified to intervene in the revision.

7. An automated data extraction system for shipbuilding projects based on document type and attribute mapping, used to implement the method as described in any one of claims 1-6, characterized in that, include: The rule base unit is used to build a mapping rule base, which stores document type definitions and data attribute definitions, as well as mapping records that associate document type identifiers with target attribute identifiers and configure search strategies and data transformation rules; The document identification unit is used to receive ship project documents, extract the metadata and text features of the documents, perform hierarchical classification processing based on the metadata and text features of the documents to assign document type identification, and generate standardized document objects. The three-domain verification unit is used to retrieve associated mapping records from the mapping rule base based on document type identifiers to generate an extraction task list. It performs parallel searches in the three domains according to the search strategy for standardized document objects: in the text area, it performs regular expression matching search using keywords and variant keywords; in the table area, it performs cell value extraction search using semantic similarity of table headers and row and column cross-location; in the chapter area, it performs keyword location search after constraining the search range by matching chapter title levels; it performs cross-domain mutual verification on the search results of the three domains; it determines the mutual confirmation or conflict status based on the semantic consistency between the extraction results of each search domain, adjusts the confidence level accordingly, and selects the optimal match. The data traceability unit is used to populate the standardized attribute values ​​into the structured data table according to the attribute definition. For each attribute value, the source document identifier, original text location information, matching search domain identifier and extraction time are recorded to form a data traceability chain.

8. An electronic device, characterized in that, include: processor; Memory used to store processor-executable instructions; The processor is configured to invoke instructions stored in the memory to execute the method according to any one of claims 1 to 6.

9. A computer-readable storage medium having computer program instructions stored thereon, characterized in that, When the computer program instructions are executed by the processor, they implement the method described in any one of claims 1 to 6.

Images