Data fusion method and system of heterogeneous product data management system based on ship

Abstract

The invention provides a ship-based heterogeneous product data management system data fusion method and system. The method comprises the following steps: constructing a ship product unified data model, and carrying out structure standardization and semantic alignment on design system and management system data from different sources; establishing a heterogeneous data mapping rule, defining a field corresponding relation and type conversion logic, and realizing automatic data alignment; capturing a data change event of the source system through an event-driven mechanism, pushing the data change event to the target system through the message-oriented middleware, and executing field-level data fusion according to a mapping rule; when a plurality of events concurrently modify the same object, performing automatic conflict merging by adopting a conflict-free copy data type (CRDT) or operation transformation (OT) algorithm; the system further establishes a reverse event channel. Data standardization, automation and bidirectional consistent fusion between ship design and a management system are realized, and the real-time performance, the accuracy and the traceability of data integration are remarkably improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer data processing, and in particular to a data fusion method and system for a heterogeneous product data management system based on a ship. BACKGROUND

[0002] Modern ship product structures are extremely complex, and their design and manufacturing processes are highly dependent on various types of digital software systems. In a typical ship design process, the design department usually uses professional three-dimensional computer-aided design (CAD) software, such as AVEVA Marine (hereinafter referred to as AM) and the like, to perform detailed design of hulls, machinery, electrical systems, and other multi-specialty designs, thereby generating a large amount of model and drawing data. At the same time, the enterprise will use product data management (PDM) or product lifecycle management (PLM) systems, such as Tiger PDM and the like, to centrally manage engineering bill of materials (EBOM), manufacturing bill of materials (MBOM), process flow, and material quantity information generated during the design process.

[0003] However, three-dimensional design CAD software and PDM / PLM systems are often developed by different suppliers, and there are natural differences between the two in terms of underlying architecture, data model, object definition, and data storage format, which is referred to as "data heterogeneity". For example, the part model data in the AM system cannot be directly parsed and used by the Tiger PDM system. This heterogeneity results in a huge gap between ship design data and management data, forming a "data island".

[0004] In order to break through the data link, the existing technology usually adopts the following ways: Manual export and import: design personnel manually export data lists or model files from CAD software such as AM, and then manually enter them into the PDM / PLM system by process or management personnel. This method is extremely inefficient, and human errors are easily introduced during repeated data transcription, resulting in inconsistent data such as part information, material quantity information, and weld information, which affects subsequent production and procurement processes; Customized point-to-point development: special data conversion scripts or interface programs are developed for specific two software (such as AM and Tiger PDM). Although this method can achieve a certain degree of automation, it has obvious defects: first, the development and maintenance cost is high, and when the software version of either party is upgraded, the script or program may fail and needs to be redeveloped; second, it can usually only handle specific types of data, making it difficult to cover a wide variety of data such as drawings, principle information, three-dimensional lightweight models, nesting information, and painting area lists, and the completeness of data fusion is insufficient; finally, this "chimney-type" integration architecture has poor scalability and is difficult to adapt to the introduction of more new digital systems by the enterprise.

[0005] Therefore, we propose a ship-based heterogeneous product data management system data fusion method and system. SUMMARY

[0006] The purpose of the present application is to overcome the deficiencies of the prior art, provide a ship-based heterogeneous product data management system data fusion method and system, solve the technical problems mentioned in the background art.

[0007] To achieve the above purpose, the present application provides the following technical solutions: The ship-based heterogeneous product data management system data fusion method comprises the following steps: S1, analyze the part data structure of the source system and the target system, extract the common attribute field, establish a unified data model and register it to the metadata management module; S2, analyze the source system data structure, generate field mapping relationship and type conversion rule according to the unified data model, and complete consistency check and registration; S3, capture data change events in the source system, push them to the target system through the message middleware, complete field conversion and loading according to the mapping rule, and automatically merge concurrent modifications using conflict-free replication algorithm; S4, when the target system data changes, generate reverse event transmission to the source system, execute update and conflict resolution, record synchronization log and regularly check data consistency; S5, generate event processing log and consistency report, provide state query and automatic playback function, realize traceability and stable maintenance of data fusion process.

[0008] S1 is: analyzing the part data structure of the source heterogeneous system and the target data management system, collecting object type, field definition, data level relationship and file association method; extracting common object feature set, establishing standardized attribute field set, defining fields including node code, name, parent node code, part type, engineering number, secret level and file information. According to the attribute field set, a unified data model template is constructed, a machine-readable structure definition format is used to establish model level relationship and inheritance mechanism, and the model is registered in the metadata management module to generate a unique model identifier, which is used for subsequent mapping and verification.

[0009] S2 is: analyzing the source system data structure, extracting field name, type, level and semantic label, generating source data structure description file; according to the unified data model and source data structure description file, a field mapping table is established, the field corresponding relationship and type conversion rule are defined. Execute mapping consistency and integrity check, generate verified mapping rule file, and register it in the mapping rule management module to form a callable configuration.

[0010] S3 is specifically: deploying a change event capture module in the source system, monitoring object addition, modification, deletion or file replacement in real time, generating a structured event object; encapsulating the event into a unified format message and pushing it to the event queue of the message middleware, and the target data management system receives the message through the subscription mechanism and executes field-level data conversion and loading. When multiple events are detected to involve the same object, a strategy based on a conflict-free replication data type or operation transformation algorithm is used to perform automatic conflict merging; after merging, the final data object is written into the target database, and the playback mechanism is triggered in abnormal cases to ensure the integrity and traceability of data synchronization.

[0011] S4 is specifically: when the object in the target system is modified, approved or archived, a target event containing a direction identifier is generated and returned to the source system through the reverse channel; the source system parses the event message and performs the corresponding update operation, and calls a lock-free conflict merging algorithm for concurrent modified objects to generate a consistency confirmation event; after completing synchronization, a synchronization log is recorded, periodic consistency checking is performed, and the automatic repair or manual review mechanism is triggered for inconsistent data to maintain the end-to-end data consistency of the double-end data.

[0012] S5 is specifically: the processing results of all events are summarized to generate fusion logs and synchronization state snapshots, record event numbers, directions, states, conflict flags and verification summaries; periodic consistency detection and automatic playback repair are performed, a verification report is generated and archived, a state query interface and visual monitoring function are provided, a data fusion report is output, and time stamp signature and tamper-proof storage are performed to realize traceability and compliance verification of the fusion process.

[0013] The ship-based heterogeneous product data management system data fusion system comprises: A unified data modeling module is used for analyzing the part data structures of the source system and the target system, extracting general attribute fields, establishing a unified data model and registering it to a metadata management module; A mapping rule establishing module is used for analyzing the source system data structure, generating field mapping relationships and type conversion rules according to the unified data model; An event-driven fusion module is used for capturing data change events in the source system and pushing event messages to the target system through the message middleware; A bidirectional synchronization and consistency maintenance module is used for generating reverse event messages when the target system changes and transmitting them to the source system, performing update and conflict merging processing; A state monitoring and result output module is used for summarizing the processing results of fusion events, generating fusion logs and state snapshots, and recording event numbers, states and consistency verification values.

[0014] The beneficial effects of the present application are: This invention establishes a unified data model covering data objects from multiple systems, standardizes the field structure, hierarchical relationship, and file association of systems from different sources, and completely solves the data incompatibility problem caused by the difference in model structure between AM system and PDM / PLM system, enabling multi-disciplinary data such as hull, marine engineering, and electrical systems to interact and reuse under a unified semantic framework.

[0015] This invention achieves automatic field alignment and type conversion through field mapping rules and semantic matching algorithms, reducing manual intervention and repetitive data entry. In the data fusion process, a consistency verification and anomaly repair mechanism is introduced, which can effectively avoid data deviations caused by format differences, null values ​​or type conflicts, and ensure the integrity and reliability of data conversion results.

[0016] This invention achieves real-time perception and asynchronous transmission of cross-system data changes by deploying an event capture module in the source system and combining it with the publish / subscribe mechanism of the message middleware; it uses conflict-free replicated data type (CRDT) or operation transformation (OT) algorithm to automatically merge concurrent conflicts, so that the system can maintain eventual consistency and high throughput performance in high-concurrency, multi-node scenarios. Attached Figure Description

[0017] Figure 1 This is a schematic diagram of a data fusion method for a ship-based heterogeneous product data management system according to the present invention; Figure 2 This is a schematic diagram of the data fusion system framework for a ship-based heterogeneous product data management system according to the present invention. Detailed Implementation

[0018] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0019] Example 1: As Figure 1 As shown, this embodiment provides a data fusion method for a ship-based heterogeneous product data management system, including the following steps: S1. Construct a unified data model, analyze the component data structure of the source system and the target system, extract common attribute fields, establish a unified data model and register it to the metadata management module; S2. Establish data mapping rules, parse the source system data structure, generate field mapping relationships and type conversion rules according to the unified data model, and complete consistency verification and registration; S3, performing event-driven data fusion, capturing data change events in the source system, pushing to the target system through message middleware, completing field conversion and loading according to mapping rules, and automatically merging concurrent modifications using conflict-free replication algorithm; S4, realizing bidirectional synchronization and consistency maintenance, generating reverse events when the data of the target system changes and transmitting to the source system, performing update and conflict resolution, recording synchronization log and checking data consistency regularly; S5, outputting fusion results and state monitoring, generating event processing log and consistency report, providing state query and automatic playback function, realizing traceability and stable maintenance of data fusion process; S1 specifically includes the following sub-steps: S110, data collection and structure recognition: collecting the basic data structure of the source heterogeneous system and the target data management system, including part object type, field definition, data level relationship and file association method, providing complete input information basis for subsequent unified modeling.

[0020] Preferably, the collection process is performed through a standardized interface, and the interface types include database connectors (such as JDBC, ODBC), file parsers (such as JSON, XML parser) and system API calling modules; The system automatically generates a structure description file (StructureDescriptor) during collection, which records the field path, data type and level index, and is used for subsequent structure alignment and semantic analysis of the unified model.

[0021] S120, object feature extraction and standardized attribute set generation: analyzing the data organization characteristics of different systems, extracting a universal object feature set, such as node code, name, parent node code, part type, engineering number, security level and associated file information; the object feature set is used to form a standardized attribute set that can be reused across systems.

[0022] Preferably, the feature extraction uses a hybrid algorithm based on pattern matching and feature mapping rules: the system performs regular expression matching on field naming patterns to identify semantically similar fields; then calculates the semantic similarity of the fields through a keyword vectorization model to automatically generate candidate mapping relationships; finally, the field normalization is completed by an artificial confirmation module or a rule engine, thereby ensuring the consistency and reusability of attributes between different systems.

[0023] S130, unified data model definition and structure template construction: according to the object feature set, a unified data model structure template is constructed, various objects such as parts, cabin sections, trays, systems and regions are abstracted into standardized data objects, and the attribute type and constraint rules of each field are clearly defined; Preferably, the unified data model is defined in JSON Schema format, with each attribute containing field name, data type, length, mandatory condition, value range, and constraint condition, to ensure that the model structure can be machine-readable verified and has expansion compatibility.

[0024] The system automatically generates a Schema verification file (schema.json) in the model definition stage and verifies its syntax integrity and hierarchical consistency through a parser to ensure self-consistency in the model definition stage.

[0025] S140, hierarchical relationship and inheritance mechanism establishment: a hierarchical relationship and inheritance mechanism is established for the unified data model, so that the model can represent multi-level assembly relationship and its parent-child dependency chain to support hierarchical management of complex ship structure; The hierarchical relationship can be implemented in tree structure, foreign key association, or hierarchical index mode to ensure that the parent-child dependency between objects can be automatically parsed and tracked by the system.

[0026] Preferably, the tree structure is used to build the parent-child hierarchy, and the system automatically maintains a node path table (PathTable) and an index tree (IndexTree), which can realize traceable navigation of the assembly structure through path analysis.

[0027] S150, model registration and metadata index establishment: the unified data model is registered in the metadata management module of the data fusion service to form a model definition that can be used for subsequent data mapping, event analysis, and data verification; The model registration includes writing the model structure in the form of metadata entries into the database or configuration file, and generating a unique model identification, so that the subsequent modules can call, load, and verify the corresponding model definition based on the identification, to realize unified management and version control.

[0028] Preferably, the model identification (ModelID) is generated by the system according to the model content SHA-256 hash value or random UUID, to ensure that each model version has global uniqueness and tamper resistance; The system also establishes a model index table (ModelIndex) in the metadata management module, including model name, version number, creation time, and digest value, to support fast retrieval and version backtracking.

[0029] S2 specifically includes the following sub-steps: S210, source system data structure analysis and field description generation: analyze the self-owned data model of the source heterogeneous system, extract its field name, data type, field level, and object category, and form a source system data structure description file.

[0030] Preferably, the parsing process is performed by a "Structure Analyzer", and is achieved by the following steps: Automatic scanning of table structure and field definition based on system interface or database metadata; Identifying semantic tags of fields (such as PartID, ParentID, MaterialCode, etc.); Classifying and grouping fields to form object-level hierarchical description; Generating a structure description file (SourceModelDesc.json) to record field identification, field path, data type, whether mandatory, and corresponding unit information, which is used for subsequent field-level mapping; The system performs consistency verification on the file to ensure that the field path is unique and the type definition is complete, thereby providing a machine-readable semantic basis for subsequent mapping.

[0031] S220, field mapping relationship definition and semantic rule establishment: according to the unified data model and the source system data structure description file, a field mapping table is established to clearly define the corresponding relationship and conversion logic between each source field and target model field.

[0032] Preferably, the system uses a "Semantic Matcher" to automatically generate candidate mapping relationships, which performs the following operations: Identify candidate fields by name similarity calculation (Levenshtein distance); Calculate semantic correlation using field context semantic model (Word2Vec or BERT); According to the correlation threshold (default 0.85), the optimal mapping pair is selected; After manual or rule engine review and confirmation, the final mapping pair is generated; Form a configurable key-value mapping file (MappingRule.json), where each mapping item contains: sourceField (source field name), targetField (target field name), transformRule (conversion function), defaultValue (default value); through this structured definition, the visualization, editability and traceability of the mapping rule are realized.

[0033] S230, data type conversion and exception handling mechanism configuration: configure data type conversion strategy and exception handling mechanism for the mapping rule, when the source system field type and target field type are incompatible, perform standardized conversion according to the preset type mapping rule.

[0034] Preferably, the system has a built-in TypeMappingMatrix, which records common type correspondence rules, including but not limited to: string ↔ enumeration: regular matching strategy; numerical type ↔ floating point type: unified precision to three decimal places; Boolean ↔ integer: 0 / 1 ↔ True / False bidirectional mapping; Time date type: unified ISO-8601 standard format (YYYY-MM-DDTHH:mm:ssZ).

[0035] If an exception or null value is detected during the conversion process, the system automatically triggers an error interception mechanism, writes the exception to the mapping error log table, and records the field name, error type, timestamp, and processing result.

[0036] At the same time, the system provides a configurable "automatic repair strategy" that can complete the default value or enable the fallback rule according to the context when it finds a null value, to ensure data integrity and consistency.

[0037] S240, mapping rule consistency check and integrity verification: perform consistency and integrity verification on the mapping rule, check whether each target field has a corresponding source field or default value, and verify that the field hierarchy, data type, and value range are consistent with the unified data model.

[0038] Preferably, the verification module performs the following operations: compare the consistency of the total number of fields (source field quantity ≤ target field quantity); detect field type compatibility; verify that the mapping rate of mandatory fields ≥ 95%; check the value range and unit consistency; generate a hash digest (SHA-256) for the passed mapping table to ensure that the file has not been tampered with.

[0039] The system generates a verification report, records the pass rate, abnormal field list, and correction suggestions, and archives the results together with the mapping version number to support version tracing and comparison.

[0040] S250, mapping rule registration and index management: register the verified mapping rule in the data mapping rule management module to form a mapping configuration file that can be directly called by subsequent event-driven synchronization tasks.

[0041] The registration process includes the following steps: store the mapping rule file in the configuration database or version control repository (such as Git); generate a unique mapping rule identifier (MappingRuleID); establish an associated index with the corresponding unified data model identifier (ModelID); record the creation time, version number, digest value, and maintainer information; after registration, the system automatically performs a load verification in the background to ensure that the mapping file syntax is correct and the version is readable.

[0042] Preferably, the system adopts a HashIndex mechanism to support rule retrieval with O(1) time complexity. Through the above mechanism, the data fusion engine can quickly load the corresponding rule version according to the identifier when performing incremental synchronization tasks, realizing cross-system calls and dynamic expansion.

[0043] S3 specifically includes the following sub-steps: S310, source system change event capture and structured packaging: deploying a change event capture module in the source heterogeneous system to monitor the addition, modification, deletion or file replacement of objects in real time.

[0044] When the above operation is detected, the system generates a structured data change event (Event) and records the event identifier and timestamp.

[0045] Preferably, the event capture module includes the following sub-units: Listening unit: capturing change signals in real time through database triggers, file system hooks (FileHook) or API callback interfaces; Event analysis unit: type identification of captured data operations, extraction of object identifier (objectId), operation type (eventType), modification content (payload) and timestamp (timestamp); Encryption and signature unit: using the HMAC-SHA256 algorithm to generate a signature (signature) for the event content to prevent forgery and tampering; Structured packaging unit: convert event content to a unified format JSON object, fields include: { "eventId":"UUID", "objectId":"OBJ-001", "eventType":"UPDATE", "timestamp":"2025-10-05T14:20:30Z", "payload":{...}, "signature":"HMAC-SHA256-Value" } Event persistence unit: store the generated event in the event buffer (EventBuffer) and local log table to support breakpoint recovery and playback. This module ensures that the event generation delay is less than 10ms, and can support real-time response in high-concurrency scenarios.

[0046] S320, event message packaging and asynchronous distribution mechanism: the data change event is packaged as a message object (MessageObject), and pushed into the event queue of the message middleware (MessageBroker).

[0047] Preferably, the system implements event distribution in a publish / subscribe (Pub / Sub) mode, specifically including: Message formatting: event object is serialized into JSON or ProtocolBuffer format, and a uniform message header (Header) is added, including: repository code (repositoryCode); model identification (modelId); mapping rule identification (mappingRuleId); message priority (priorityLevel); and encryption checksum (checksum) Message transmission mechanism: message pushing is completed through an asynchronous communication channel, and ACK (acknowledgment mechanism) is used to ensure the reliability of message transmission; when message sending fails or no confirmation is received, the system automatically triggers a retry mechanism (default retry 3 times, interval increasing by 2 seconds). Message storage and offset control: the message middleware maintains an event offset (offset) and a transmission state table, which are used for breakpoint continuation and repeated consumption detection; preferably, Kafka or RabbitMQ is used as the message middleware to ensure high throughput and high reliable transmission.

[0048] Through the above design, the system can ensure that messages are not lost, not repeated, and traceable in the case of network fluctuations or node restarts.

[0049] S330, event analysis and field-level data conversion: The target data management system receives event messages in real time through a subscription mechanism, analyzes event content, and performs field-level conversion according to the mapping rules established in S210-S250.

[0050] Preferably, the system includes the following processing procedures: event decoding module: deserializes the serialized message into an event object structure; mapping engine call: calls the corresponding mapping file (MappingRule.json) based on mappingRuleId, and performs type conversion and data cleaning according to field rules; semantic completion and anomaly detection: if a field is missing or the value is abnormal, the automatic completion module (AutoCompletionEngine) is triggered, and historical versions or rule default values are used for repair; Data packaging and batch identification: the converted data is packaged into a unified DataPacket, containing structured fields and file reference paths; unique transaction number allocation: a unique transaction identification (transId) is assigned to each DataPacket, and the mapping version and timestamp are recorded for sequential verification and log tracking.

[0051] The data packet structure is as follows: { "transId":"TXN-20251005-0001", "objectId":"OBJ-001", "mappingRuleVersion":"v2.1", "data":{...}, "files":[" / path / to / drawing.stp"] } This step ensures field-level data consistency and enables batch traceability management.

[0052] S340, concurrent conflict detection and lock-free merging strategy (CRDT / OT) When multiple events are detected to involve the same object (i.e., the same objectId but different eventId), the system automatically calls the conflict-free replicated data type (CRDT) or operation transformation algorithm (OT) for automatic conflict merging. Preferably, the system adopts the following strategies according to the data type: Counting class fields (such as quantity, serial number): use G-Counter (incremental counter) algorithm to realize cumulative merging; attribute class fields (such as name, status): adopt LWW-Register strategy, with the value of the newer timestamp as the reference; set class fields (such as part list): adopt 2P-Set (Two-Phase Set) structure to support element addition and deletion merging; nested structure or file reference: adopt a causal relationship analysis algorithm based on VectorClock (VectorClock), to judge the sequence of events and merge operations; preferred implementation: the system records the version vector table (VersionVector) of each object, which is used to quickly determine the conflict state.

[0053] After merging, the system generates a unified final state data object (MergedObject) and writes it into the conflict resolution record table (ConflictLog), recording the eventId, objectId, conflict type, merging strategy, and final result for subsequent traceability and verification.

[0054] Through the above mechanism, the system can achieve eventual consistency in a distributed environment without a central lock, greatly improving concurrency throughput and fault tolerance.

[0055] S350, Data Persistence and Replay Mechanism: The final data object, after field transformation and conflict merging, is written to the target system database and persistence operation is performed.

[0056] Preferably, the system maintains the following sub-modules in the background: Status recording module: Generates a processing status record for each event, with fields including eventId, objectId, processStatus (Processing, Success, Fail), processTime, and errorMessage; Exception interception and replay queue: When an exception is detected, the system automatically writes the corresponding event into the replay queue and triggers an automatic replay task; Replay Interface: Provides a RESTful API ( / api / v1 / replay?eventId={eventId}), which supports replaying unsuccessful events by event number, object ID, or time range; Status confirmation and log synchronization: After each replay, the system updates the ReplayLog and processing results to ensure full traceability of the data fusion process; Optimal implementation: The database layer adopts a transaction log (WAL, Write-Ahead Logging) and an idempotent write strategy to prevent duplicate insertion. Through the above design, the system realizes a closed-loop process from event generation, transmission, parsing, conflict merging to persistence, ensuring that every data event can be automatically recovered in abnormal, network outage, or restart scenarios.

[0057] S4 specifically includes the following sub-steps: S410, Target Event Capture and Direction Identifier Generation: When a data object in the target data management system is modified, approved, updated, or archived, the target event capture module is triggered to generate the corresponding target change event.

[0058] Preferably, the module includes the following sub-units: Database monitoring unit: Monitors state changes of objects in the target system in real time through database triggers, change data capture (CDC) mechanisms, or message hooks; Event construction unit: Encapsulates change operations into structured event objects, with fields including eventId, objectId, eventType, timestamp, payload, signature, and direction; Direction identification generation unit: add a direction field to each event, which takes values including: "upstream": indicating the source system → target system direction; "downstream": indicating the target system → source system direction; Signature and verification unit: use JWT (JSON Web Token) combined with HMAC-SHA256 to sign the event message body, ensuring tamper-proofing and source authentication. The event structure is completely consistent with the source event to ensure that the event format is compatible and the semantics are aligned. After the event is generated, the system writes the event to the target event buffer (TargetEventBuffer) to support asynchronous sending and sequence control.

[0059] S420, event packaging and reverse transmission mechanism: the target end change event is packaged into a message object in a unified format and transmitted back to the source heterogeneous system via the reverse channel (ReverseQueue) of the message middleware. Preferably, the reverse transmission process includes the following steps: Message formatting and header packaging: add a message header (Header) to each message, including target system identification (targetSystemCode), model version number (modelVersion), signature field (signature), direction field (direction), and message priority (priorityLevel); encrypted transmission: reverse transmission uses HTTPS protocol channel, combined with TLS1.3 encryption standard and JWT authorization token mechanism, to realize end-to-end secure communication; reliable transmission control: the middleware maintains the reverse event channel (ReverseQueue) and offset index table (OffsetIndex), and ensures "Exactly Once Delivery" through transactional submission (TransactionalPublish); breakpoint resume mechanism: the system records the last confirmed offset (LastAckOffset), and when the transmission is interrupted or abnormally exited, the transmission task can be resumed from the breakpoint; replay protection: double verification through event number (eventId) and signature (signature) to prevent message duplication. This reverse transmission mechanism ensures the security, integrity and high availability of the reverse synchronization process.

[0060] S430, source system analysis and reverse update execution: the source system receives and analyzes the target end event message, and executes the corresponding update operation according to the event type.

[0061] Preferably, the analysis module executes the following process: Event decoding and signature verification: check event signature and direction field to ensure the message source is legitimate; field alignment and data conversion: according to the mapping rules of S210-S250 and the unified data model field definition of S110-S150, the attributes in the payload are aligned at the field level and standardized conversion is performed; Event type processing logic: if the event type is CREATE: add a new object record in the source system; if the event type is UPDATE: update the attribute fields of the corresponding object; if the event type is DELETE: perform soft delete, mark the deletion state without direct physical deletion; Operation log recording: all update operations are recorded to the source system change log table (SourceChangeLog), including fields such as eventId, direction, operationType, executeTime, operator, and signature; Transaction commit and rollback protection: the source system uses transaction mechanism to ensure atomicity of each event processing; if processing fails, it is automatically rolled back and written to the error table (ErrorLog). Through this step, the source system realizes real-time response and structure-level consistent update of the target system changes.

[0062] S440, bidirectional conflict detection and lock-free merging confirmation mechanism: when detecting that the source system and the target system modify the same object (i.e. the objectIds are the same and the eventTimestamps overlap), the system automatically starts the conflict detection module and calls the CRDT merging engine for lock-free conflict resolution.

[0063] Preferably, the system uses the following strategies according to field types: Scalar fields (such as status values): use LWW-Register (Latest Write Wins) strategy, with the latest timestamp value as the reference; Set fields (such as component lists): adopt Two-PhaseSet (2P-Set) structure to merge add and delete operations; Nested object and file reference fields: use VectorClock (Vector Clock) algorithm to determine the causal relationship; Approval status field: according to the predefined business priority table (PriorityTable), make directional judgment, and prefer to keep the results of the approval party system; Audit Consistency Verification: After conflict resolution is completed, the system generates a consistency verification event, writes it to the consistency log table, and records the merging result and reason. Preferably, the system performs a hash check (hashDigest = SHA-256(payload)) on the merging result and compares it with the records of both systems; when a difference is detected, the system automatically triggers a consistency verification task and replays the corresponding event. Through the above mechanism, the system realizes bidirectional lock-free conflict resolution and eventual consistency maintenance in a fully distributed and asynchronous environment.

[0064] S450, Synchronization Log Recording and Periodic Consistency Verification: After bidirectional synchronization is completed, the system records the synchronization log (SyncLog), including the following fields: eventId, direction, sourceSystemCode, targetSystemCode, status (Success / Fail), syncTime, retryCount, conflictFlag, signature.

[0065] The system implements the following enhancement strategies: Log Storage and Audit Synchronization: Log data is written to the log database (LogDB) and synchronized to the audit module (AuditModule), supporting queries by time, object, and direction; Periodic Consistency Task: The system scheduler (Scheduler) performs a consistency verification task every pre-set period (e.g., 10 minutes), calculating the source and target data digest values (hashDigest); Automatic Repair Mechanism: When a hashDigest inconsistency is detected, the system automatically replays the corresponding event (ReplayTask) until the two ends are consistent; Exception Alarm Mechanism: If the inconsistency persists after three replays, the system marks the task as "manual review required" and sends an alarm notification through the monitoring system; Data Tamper-Proofing Mechanism: All log entries have a timestamp signature for audit tracing and compliance verification. Through this mechanism, the system realizes verifiable consistency maintenance after bidirectional data synchronization, ensuring the correctness and traceability of the synchronization result.

[0066] S5 specifically includes the following sub-steps: S510, Event Fusion Log Generation and State Statistics: After bidirectional data synchronization is completed, the system starts the event fusion log generation module, which summarizes and structures the processing status of all events.

[0067] The system performs the following operations: Traverse all event processing records, extract eventId, objectId, direction, mappingRuleId, processStatus, processTime, errorCode, errorMessage, conflictFlag, retryCount, etc. Generate a fusion log file, each record corresponds to an event number, used to identify the processing status and execution result of the event. During log generation, the system automatically calculates key statistical indicators such as success rate, failure rate, and conflict rate. To prevent duplicate log generation, the system uses an idempotent mechanism based on transaction locks to ensure that each event generates only one corresponding log record. (5) After the log file is generated, it is immediately written to the fusion log database, and a digital signature field (signature = HMAC-SHA256 (allRecords)) is added.

[0068] The log record format is as follows: { "eventId":"EVT-20251005-0001", "objectId":"OBJ-0042", "direction":"upstream", "mappingRuleId":"MAP-V2.3", "processStatus":"Success", "conflictFlag":false, "retryCount":0, "timestamp":"2025-10-05T22:15:00Z", "signature":"SHA256-HashValue" } This step ensures that the processing status of each event in the system can be completely tracked, and its integrity and tamper resistance can be verified through digital signature.

[0069] S520, synchronization state snapshot generation and hash verification mechanism: Generate a synchronization state snapshot (Snapshot) for each fused object to record the data version number and consistency digest value of the source system and the target system.

[0070] The system performs the following steps: Read the key fields of source and target system objects (including node code, part type, quantity, update time, file path); Concatenate the fields in ascending order of field name to form a unified string (CanonicalString), ensuring consistent field order; Calculate the hash digest value (hashDigest = SHA-256(CanonicalString)) and store it in the snapshot record; Generate a snapshot file (Snapshot.json) for each object, including the fields: versionIdSource, versionIdTarget, updateTime, hashDigest, verificationResult (Pass / Fail); If the source and target digest values are detected to be inconsistent (hashDigestSource ≠ hashDigestTarget), the system automatically triggers a consistency repair task. This mechanism can complete object-level consistency verification with O(1) time complexity and support large-scale data verification with extremely low storage cost.

[0071] S530, status query interface and visual monitoring panel: the system provides a status query interface and visual monitoring panel (FusionDashboard) to display the data fusion status and abnormal distribution in real time.

[0072] Preferably: the interface adopts RESTful architecture, and the API specification is as follows: GET / api / v1 / fusion / status?eventId={eventId}, return parameters include eventId, objectId, processStatus, conflictFlag, hashDigestSource, hashDigestTarget, lastSyncTime, repairStatus; The interface supports paging query, state filtering (such as "only failed tasks"), and time interval filtering; The visual panel adopts a front-end and back-end separation architecture, with line charts, pie charts, and heat maps displayed on the front end to show task success rate, conflict frequency, and playback progress; The system supports exporting CSV, PDF, and Excel format reports for project management to conduct statistics and audits; preferably, the dashboard provides real-time alert functions (AlertPanel) that automatically highlight and alarm when the conflict rate exceeds the set threshold (default 5%). Through the above design, the system realizes real-time monitoring and visual decision support for fusion tasks.

[0073] S540, Periodic Consistency Check and Automatic Repair Mechanism: The system periodically performs consistency check and repair tasks to maintain data integrity in long-term operation. Preferably, the scheduler automatically performs the following tasks every set period (e.g., every 30 minutes): Compare the hash digest values of all objects (hashDigestSource vs. hashDigestTarget); When a difference is detected, generate a replay task and write it to the replay queue; The replay module reloads the event payload and re-executes the data fusion process according to the original mapping rules and conflict merging algorithm; If the replay fails three times, the system will mark the task as "manual review required" and write detailed error stack information (errorTrace) to the audit log; All consistency check tasks generate a validation report (ValidationReport) that records object number, difference field, repair status, and timestamp; Preferably, the system adopts a three-level protection architecture: the first layer is automatic detection; the second layer is automatic repair; and the third layer is manual review. This mechanism ensures that the system can maintain consistency and has self-healing ability in a large-scale heterogeneous environment.

[0074] S550, Archiving and Tamper-Proof Storage of Fusion Results: The system archives and saves the final fusion results, logs, and reports in a unified manner and outputs them as auditable multi-format files.

[0075] Preferably: The archived content includes: total number of tasks, success rate, failure rate, number of conflicts, average processing delay, number of replays, consistency pass rate, etc.; the system can export PDF, Excel, or JSON format reports according to user demand; all event numbers in the report have clickable hyperlinks that can jump to the corresponding log details page; all report files and log summaries are stored synchronously in the log database (LogDB) and backup server (BackupServer); the system generates a timestamp signature at the time of archiving and writes the signature information to the blockchain ledger (BlockchainLedger) to achieve tamper-proof and long-term verifiability of the fusion results; the system supports differential archiving mode, which only records data blocks that are different from the last archive to save storage space. Through the above steps, the system realizes a closed-loop system of "fusion-validation-repair-archiving-tracing" to ensure the judicial admissibility and traceability of the fusion process and results.

[0076] Example Two: As Figure 2As shown, the embodiment provides a ship-based heterogeneous product data management system data fusion system, comprising: A unified data modeling module is configured to analyze the part data structures of the source system and the target system, extract common attribute fields, establish a unified data model, and register the unified data model to a metadata management module, so as to realize standardized definition and semantic unification of the data structures of multiple systems. A mapping rule establishing module is configured to parse the data structure of the source system, generate a field mapping relationship and a type conversion rule according to the unified data model, perform consistency and integrity checking on the mapping result, and register the mapping rule to a mapping rule management module. An event-driven fusion module is configured to capture data change events in the source system, and push event messages to the target system through a message middleware; the target system performs field-level data conversion and loading according to the mapping rule, and when it is detected that multiple events involve the same object, invokes a conflict-free replicated data type (CRDT) or an operation transform (OT) algorithm to automatically merge conflicts. A bidirectional synchronization and consistency maintenance module is configured to generate reverse event messages and transmit the reverse event messages to the source system when the target system changes data, and perform update and conflict merging processing; the module is further configured to record synchronization logs, perform periodic consistency checking, and trigger automatic playback or manual review operations when data differences are detected. A state monitoring and result output module is configured to summarize the processing results of the fusion events, generate fusion logs and state snapshots, record event numbers, states, and consistency check values; the module is further configured to provide a state query interface and a visual monitoring function, output fusion reports, and perform timestamp signing and tamper-proof storage, so as to realize traceability and verification of the entire data fusion process.

[0077] The above formulas are dimensionless values calculated, the formulas are obtained by collecting a large amount of data to simulate a formula of the most recent real situation, and the preset parameters and threshold values in the formulas are set by a person skilled in the art according to actual conditions.

[0078] Those skilled in the art can realize that the modules and algorithm steps of each example described in combination with the embodiments disclosed herein can be realized by electronic hardware or a combination of computer software and electronic hardware. Whether the functions are realized in hardware or software depends on the specific application and design constraints of the technical solution. A person skilled in the art can use different methods to realize the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.

[0079] Those skilled in the art can clearly understand the specific working process of the system, device and module described above can refer to the corresponding process in the foregoing method embodiments for the convenience and brevity of description, and will not be repeated here.

[0080] In several embodiments provided in the present application, it should be understood that the disclosed system, device and method can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of the modules is only a logical function division, and actual implementation can have another division manner, for example, a plurality of modules or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the displayed or discussed each other can be indirect coupling or communication connection through some interfaces, devices or modules, and can be electrical, mechanical or other forms.

[0081] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto, any person skilled in the art can easily think of changes or replacements within the technical range disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

[0082] Finally: the above is only the preferred embodiment of the present application, and is not used to limit the present application, any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the protection scope of the present application.

Claims

1. A ship-based heterogeneous product data management system data fusion method, characterized in that, Comprise the following steps: S1, the source system and the target system of the parts data structure are analyzed, the general attribute field is extracted, the unified data model is established and registered to the metadata management module; S2, the source system data structure is parsed, the field mapping relationship and type conversion rule are generated according to the unified data model, and consistency check and registration are completed; S3, the data change event is captured in the source system, is pushed to the target system through the message middleware, the field conversion and loading are completed according to the mapping rule, and the non-conflict replication algorithm is used to automatically merge and concurrent modification; S4, when the target system data changes, the reverse event is generated and transmitted to the source system, the update and conflict resolution are executed, the synchronization log is recorded and the data consistency is checked periodically; S5, event processing log and consistency report are generated, state query and automatic playback function are provided, and the traceability and stable maintenance of data fusion process are realized.

2. The ship-based, heterogeneous product data management system data fusion method of claim 1, wherein, S1 is specifically: The source heterogeneous system and the parts data structure of the target data management system are analyzed, the object type, field definition, data level relationship and file association mode are collected; The general object feature set is extracted, the standardized attribute field set is established, and the fields including node code, name, parent node code, part type, engineering number, secret level and file information are defined.

3. The ship-based, heterogeneous product data management system data fusion method of claim 2, wherein, According to the attribute field set, a unified data model template is constructed, a machine-readable structure definition format is used, a model level relationship and inheritance mechanism are established, and the model is registered to the metadata management module to generate a unique model identifier, which is used for subsequent mapping and checking.

4. The ship-based, heterogeneous product data management system data fusion method of claim 1, wherein, S2 is specifically: The source system data structure is parsed, the field name, type, level and semantic label are extracted, and the source data structure description file is generated; According to the unified data model and the source data structure description file, a field mapping table is established, the field corresponding relationship and type conversion rule are defined.

5. The ship-based, heterogeneous product data management system data fusion method of claim 4, wherein, The mapping consistency and integrity check is executed, the verified mapping rule file is generated, and is registered to the mapping rule management module to form a callable configuration.

6. The ship-based, heterogeneous product data management system data fusion method of claim 1, wherein, S3 is specifically: The change event capture module is deployed in the source system, the object addition, modification, deletion or file replacement operation is monitored in real time, and the structured event object is generated. The event is encapsulated into a unified format message and pushed to the event queue of the message middleware, and the target data management system receives the message through the subscription mechanism and executes the field level data conversion and loading.

7. The ship-based, heterogeneous product data management system data fusion method of claim 6, wherein, When multiple events involve the same object, the strategy based on non-conflict replication data type or operation transformation algorithm is used to execute automatic conflict merging; After merging, the final data object is written into the target database, and the playback mechanism is triggered in the abnormal case to ensure the integrity and traceability of data synchronization.

8. The ship-based, heterogeneous product data management system data fusion method of claim 1, wherein, S4 is specifically: When the object in the target system is modified, approved or archived, the target event containing the direction identifier is generated and returned to the source system through the reverse channel; The source system parses the event message and executes the corresponding update operation, the non-lock conflict merging algorithm is called for concurrent modification of the object, and the consistency confirmation event is generated; After synchronization, the synchronization log is recorded, the periodic consistency check is executed, the automatic repair or manual review mechanism is triggered for the inconsistent data, so that the final consistency of the data in both ends is maintained.

9. The ship-based, heterogeneous product data management system data fusion method of claim 1, wherein, S5 is specifically: The processing results of all events are summarized to generate a fusion log and a synchronization state snapshot, record event number, direction, state, conflict flag and verification digest; Periodic consistency detection and automatic playback repair are performed to generate a verification report and archive it; A state query interface and visual monitoring function are provided to output a data fusion report and perform timestamp signing and tamper-proof storage to achieve traceability and compliance verification of the fusion process.

10. A ship-based heterogeneous product data management system data fusion system employing the ship-based heterogeneous product data management system data fusion method of any one of claims 1-9, characterized by, It includes: A unified data modeling module for analyzing the component data structures of source systems and target systems, extracting common attribute fields, establishing a unified data model and registering it with the metadata management module; A mapping rule establishment module for analyzing the data structure of the source system and generating field mapping relationships and type conversion rules based on the unified data model; An event-driven fusion module for capturing data change events in the source system and pushing event messages to the target system through a message middleware; A bidirectional synchronization and consistency maintenance module for generating reverse event messages when data changes occur in the target system and transmitting them to the source system to perform update and conflict merging processing; A state monitoring and result output module for summarizing the processing results of fusion events, generating a fusion log and a state snapshot, and recording event number, state and consistency verification value.

Images