A method and system for synchronous monitoring of a digital twin of a tubular string

Abstract

This invention discloses a digital twin synchronous monitoring method and system for tubing processing. It utilizes the Jinpeng tubing processing system and includes a multi-source data acquisition and access module, a raw data parsing and formatting module, an initial snapshot construction and basic state loading module, a real-time data temporary storage and time alignment module, an object identifier establishment and state association module, a unified state data construction module, a digital twin model synchronous update module, a queue consumption and real-time display control module, and a synchronous monitoring result output module. This invention fulfills the application requirements for digital twin synchronous monitoring of the tubing processing process.

Description

Technical Field

[0001] This invention relates to the field of drilling operation technology, and in particular to a method and system for synchronous monitoring of digital twins for tubing string processing. Background Technology

[0002] In the field of drilling operation technology, Jinpeng's tubing handling system can manage equipment, workstations, and related operational data during tubing handling, and can provide equipment operating status, object information, and processing-related data. Meanwhile, digital twin technology has been applied to scenarios such as industrial equipment operation monitoring, 3D visualization, and status synchronization. By mapping the physical equipment status to digital space, it enables equipment status display and process monitoring.

[0003] In existing technologies, digital models are typically updated by collecting PLC data, sensor data, equipment pose data, or operational status data, thereby enabling equipment action display, status demonstration, or partial process monitoring. In the context of column processing, related technologies can also achieve equipment action visualization, partial object display, and local status synchronization.

[0004] However, most existing solutions are geared towards monitoring general equipment or displaying single scenarios, mainly focusing on device-level status display and screen-level synchronization. They have not yet formed a digital twin synchronous monitoring mechanism that adapts to the interface structure, object organization method, and real-time update characteristics of the Jinpeng column processing system.

[0005] Existing technical solutions are prone to causing misalignment of the system's startup or reconnection phases, making it difficult to directly drive continuous updates of the digital twin model. They also lack a data mapping and differentiated update mechanism adapted to the Jinpeng column processing system, resulting in insufficient continuity and accuracy of monitoring results. Summary of the Invention

[0006] The purpose of this invention is to provide a digital twin synchronous monitoring method for tubing processing. Based on the Jinpeng tubing processing system, it overcomes the shortcomings of existing technologies, realizes unified state organization of data from different sources, and performs differentiated updates according to object type, thereby achieving synchronous monitoring of the digital twin of the tubing processing process.

[0007] To achieve the above technical objectives, this invention provides a digital twin synchronous monitoring method for tubing processing, which employs the Jinpeng tubing processing system. The method includes: S1 Multi-Source Data Acquisition and Access: Acquires multi-source data related to the tubing processing process through the Jinpeng tubing processing system; S2: Raw data parsing and formatting: The raw data obtained in S1 is parsed and formatted to form a standardized dataset that can be processed by the digital twin system; S3: Initialize snapshot construction and basic state loading, and establish an initial synchronization link when the digital twin system starts up; S4: Real-time data storage and time alignment processing. Before the initial snapshot is loaded, the incremental frames received in real time are temporarily stored. After the snapshot is loaded, the temporarily stored real-time frames are released in sequence and the system switches to real-time synchronization mode. In real-time synchronization mode, the snapshot data and real-time frame data are aligned under a unified time reference. S5: Object identification establishment and status association processing, establishing object-level association relationships with pipe column objects and equipment objects as the core; S6: Unified state data construction. After completing the object association, the state data of each object is organized in a unified manner to form a target state dataset used to drive the update of the digital twin model. S7: Synchronous update of the digital twin model: Based on the target state dataset obtained in S6, the digital twin model is synchronously updated. S8: Queue consumption and real-time display control. To ensure the real-time and continuous nature of monitoring results, the real-time frame queue is subject to restricted consumption. S9: Output synchronous monitoring results. After the digital twin model is updated, output the synchronous monitoring results corresponding to the current physical processing.

[0008] Accordingly, the present invention also provides a digital twin synchronous monitoring system for pipe processing, which has a Jinpeng pipe processing system and executes a corresponding digital twin synchronous monitoring method for pipe processing. It includes the following modules connected sequentially along the data information flow direction: a multi-source data acquisition and access module, a raw data parsing and formatting processing module, an initial snapshot construction and basic state loading module, a real-time data temporary storage and time alignment processing module, an object identification establishment and state association processing module, a unified state data construction module, a digital twin model synchronous update module, a queue consumption and real-time display control module, and a synchronous monitoring result output module.

[0009] The method and system of this invention overcome the shortcomings in initialization synchronization, unified state organization, and object mapping and real-time updates for the Jinpeng tubing processing system, and meet the application requirements of digital twin synchronous monitoring of the tubing processing process. Attached Figure Description

[0010] Figure 1 This is a flowchart of the method of the present invention.

[0011] Figure 2 This is a system architecture diagram of the present invention. Detailed Implementation

[0012] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0013] like Figure 1 As shown, this invention provides a digital twin synchronization monitoring method for pipe processing, which uses the Jinpeng pipe processing system. The method includes: S1 Multi-source data acquisition and access: acquiring multi-source data related to the pipe processing process through the Jinpeng pipe processing system; S2 Raw data parsing and formatting: parsing and formatting the raw data acquired in S1 to form a standardized dataset that can be processed by the digital twin system; S3 Initial snapshot construction and basic state loading: establishing an initial synchronization link when the digital twin system starts; S4 Real-time data temporary storage and time alignment processing: performing temporary storage processing on the incremental frames received in real time before the initial snapshot is loaded; after the snapshot is loaded, releasing the temporarily stored real-time frames in sequence and switching to real-time synchronization mode. In this formula, the snapshot data and real-time frame data are aligned under a unified time reference; S5: Object identification establishment and state association processing, establishing object-level association relationships with pipe objects and device objects as the core; S6: Unified state data construction, after completing the object association, the state data of each object are uniformly organized to form a target state dataset used to drive the update of the digital twin model; S7: Synchronous update of the digital twin model, according to the target state dataset obtained in S6, the digital twin model is synchronously updated; S8: Queue consumption and real-time display control, in order to ensure the real-time and continuity of monitoring results, the real-time frame queue is subject to restricted consumption processing; S9: Synchronous monitoring result output, after the digital twin model is updated, the synchronous monitoring result corresponding to the current physical processing is output.

[0014] As a further improvement, in S1, the multi-source data includes at least one of the following: real-time physical device status data output by the PLC, device pose matrix data calculated by the backend, entity list data for model definition, device status snapshot data for system initialization, drill pipe structure data for static distribution representation of the tubing, and real-time frame data for real-time incremental synchronization; the real-time physical device status data is received by the backend at high frequency via a UDP link, the backend parses the received messages, and uses them as the data basis for subsequent kinematic calculations and real-time status generation; the system simultaneously accesses the initialization data and incremental data through the HTTP snapshot interface and the WebSocket real-time stream interface.

[0015] As a further improvement, S2 obtains raw data from different sources and converts it into a unified and processable data structure, providing a foundation for subsequent time alignment, object association, and model updates. Specifically, it includes: S21: parsing the equipment operating parameters in the PLC message; S22: performing kinematic calculations in the backend to generate a pose matrix corresponding to the equipment object; S23: converting the pose result of the equipment object into a unified 4×4 homogeneous transformation matrix format; S24: outputting the matrix as a flat16 array in column-major order; S25: extracting the sequence number field, timestamp field, matrix field, drill rod in-situ field, and numerical status field from the real-time frame.

[0016] As a further improvement, S3 completes the initial state construction of the digital twin scenario, specifically including: S31: establishing a WebSocket real-time connection; S32: concurrently acquiring model definition data, device status snapshot data, and drill pipe structure data through the HTTP interface; S33: establishing a mapping relationship between the digital twin object and the Jinpeng system entity based on the model definition data; S34: loading the initial pose state of each entity at the system startup time based on the device status snapshot data; S35: loading the static distribution state of the pipe column object based on the drill pipe structure data; the model definition data includes at least: a unique entity identifier, a model file name, and a display strategy; the device status snapshot data includes at least the complete pose matrix of each entity; and the drill pipe structure data includes at least the static position coordinates corresponding to the drill pipe.

[0017] As a further improvement, S4 avoids the problem of misalignment in the initialization phase where the model has not finished loading but the real-time data has arrived in advance, and reduces the impact of historical backlog frames on the synchronization results. Specifically, it includes: S41: Identifying the order of data based on the sequence number field in the real-time frame; S42: Determining the generation time corresponding to the real-time state based on the timestamp field; S43: Using the snapshot state as the initial baseline state and the real-time frame as the subsequent incremental update state; S44: Performing same-stage merging processing on data from different sources and with different refresh cycles; S45: Adopting the latest frame overlay strategy to ensure that the system always uses the current physical state as the standard.

[0018] As a further improvement, in S5, the device object is associated with at least its pose matrix, display strategy, and numerical state, and the pipe string object is associated with at least its static position, in-situ state, and display state, thereby realizing the conversion from data frame to object state. Specifically, this includes: S51: Establishing an entity-level identifier for the device object based on the unique entity identifier returned by the model definition interface; S52: Establishing an object-level index relationship for the pipe string object based on the drill pipe structure data and real-time in-situ data; S53: Binding the device pose state, pipe string in-situ state, and real-time numerical state to the corresponding objects; S54: Continuously associating the data of the same object at different time points; S55: Forming an object-level state record result.

[0019] As a further improvement, in S6, the target state dataset includes at least one of the following information: object identification information, object pose information, object presence / disappearance information, object numerical state information, model or component information to which the object belongs, and time tag information corresponding to the object.

[0020] As a further improvement, S7 specifically includes: S71: Applying the corresponding flat16 pose matrix to the device object to update the device or component pose; S72: For objects supporting component-level control, updating their local pose according to the entity-component dual-layer structure; S73: Switching between visible and hidden states for the pipeline object based on the in-situ state field in the real-time frame; S74: For resident entities, maintaining the pose from the previous moment when the matrix is ​​missing in the real-time frame; S75: For conditional entities, performing hiding or removal processing when the corresponding matrix is ​​missing; The updating of the device object is completed using a matrix direct-driven method, and the updating of the pipeline object is completed using a combination of static position and in-situ state-driven method, thereby enabling the digital twin model to reflect the current state of the physical system in real time.

[0021] As a further improvement, S8 reduces the display trailing problem caused by the backlog of historical frames, making the monitoring results closer to the actual situation on site, and specifically includes: S81: setting a depth limit for the real-time frame queue; S82: automatically overwriting old frames when new frames arrive; S83: consuming real-time frames in the queue according to the time budget in the rendering loop; S84: completing the update of the current frame while ensuring the smoothness of the main thread; S85: presenting the updated object state in the digital twin scene in real time.

[0022] As a further improvement, in S9, the synchronous monitoring results include at least: equipment operating status, equipment pose status, pipe column object in-situ status, pipe column distribution status, visualization results in the digital twin scenario, and link activity status; when no valid real-time data is received within a preset time period, the system status can be marked as disconnected to indicate that the current data link is abnormal.

[0023] As a further improvement, in S2, the pose matrix is ​​directly generated by the backend after completing the kinematic calculations to reduce the computational burden on the frontend. In S4, initialization alignment is achieved by using snapshot initialization, real-time frame storage, and release switching. In S7, device objects are updated using matrix direct-drive updates, and column objects are updated using a combination of static position and in-situ state. In S8, the latest frame overlay strategy is adopted to ensure that the digital twin system always uses the current physical state as the standard.

[0024] Accordingly, the present invention provides a digital twin synchronous monitoring system for pipe processing, which has a Jinpeng pipe processing system and executes a digital twin synchronous monitoring method for pipe processing. It includes the following modules connected sequentially along the data information flow direction: a multi-source data acquisition and access module, a raw data parsing and formatting processing module, an initial snapshot construction and basic state loading module, a real-time data temporary storage and time alignment processing module, an object identification establishment and state association processing module, a unified state data construction module, a digital twin model synchronous update module, a queue consumption and real-time display control module, and a synchronous monitoring result output module.

[0025] This invention overcomes the shortcomings of existing technologies, such as the lack of a connection mechanism between initialization snapshots and real-time streaming data, which can easily lead to misalignment of the system startup or reconnection phases, different data structures and refresh rates from different sources, lack of a unified object state organization method, difficulty in directly driving continuous updates of the digital twin model, and lack of data mapping and differentiated update mechanisms adapted to the Jinpeng pipe column processing system for equipment objects, component objects, and pipe column objects, resulting in insufficient continuity and accuracy of monitoring results. This invention avoids the deficiencies in initialization synchronization, unified state organization, and object mapping and real-time updates for the Jinpeng pipe column processing system, and realizes and meets the application requirements of synchronous monitoring of digital twins in the pipe column processing process.

[0026] This invention addresses the data interface structure of the Jinpeng pipe column processing system by establishing a unified access and object mapping mechanism for model definition data, equipment snapshot data, pipe column structure data, and real-time frame data. This enables platform-side data to be converted into object state data within the digital twin system. The invention establishes a synchronous processing mechanism of "snapshot initialization + real-time frame temporary storage + release and switching." During system startup or reconnection, the initial snapshot state is loaded first, then the temporarily stored real-time data is released and switched to the real-time synchronous state, thereby reducing state misalignment and synchronization distortion during the initialization phase. Furthermore, this invention establishes a unified state construction and differentiated synchronous update mechanism for equipment objects, component objects, and pipe column objects. Equipment objects are updated using a pose matrix-driven approach, while pipe column objects are updated using a combination of static position and in-situ state, thereby improving the real-time performance, continuity, and accuracy of digital twin monitoring.

[0027] This invention proposes a method for synchronous monitoring of digital twins in the pipe column processing system based on the Jinpeng pipe column processing system. This method relies on the existing data interfaces of the Jinpeng pipe column processing system to acquire model definition data, equipment status snapshot data, pipe column structure data, and real-time frame data. It then constructs a digital twin object and its state representation corresponding to the physical system in the digital space. By connecting and processing initialization data with real-time data, unifying the state organization of data from different sources, and performing differentiated updates based on object type, synchronous monitoring of the digital twin process of pipe column processing is achieved.

[0028] In a preferred embodiment of the present invention, the main software process may include the following steps: Step S1: Multi-source data acquisition and access The Jinpeng tubing processing system collects multi-source data related to the tubing processing process, and the multi-source data includes at least: (1) Real-time status data of physical equipment output by PLC; (2) Device pose matrix data obtained from backend calculation; (3) Entity list data used for model definition; (4) Device status snapshot data used for system initialization; (5) Drill pipe structure data used for static distribution representation of the tubing string; (6) Real-time frame data used for real-time incremental synchronization.

[0029] The physical device status data is received at high frequency by the backend via a UDP link. The backend parses the received messages and uses them as the data basis for subsequent kinematic calculations and real-time status generation. The system also accesses initialization data and incremental data through HTTP snapshot interface and WebSocket real-time stream interface.

[0030] Step S2: Raw data parsing and formatting The raw data obtained in step S1 is parsed and formatted to form a standardized dataset that can be processed by the digital twin system, specifically including: (1) Parse the equipment operating parameters in the PLC message; (2) Perform kinematic calculations in the backend to generate the pose matrix corresponding to the device object; (3) Convert the pose results of the device object into a unified 4×4 homogeneous transformation matrix format; (4) Output the matrix as a flat16 array in column-major order; (5) Extract the sequence number field, timestamp field, matrix field, drill rod in-situ field and numerical status field from the real-time frame.

[0031] After this step, the raw data from different sources are transformed into a unified and processable data structure, providing a foundation for subsequent time alignment, object association, and model updates. Preferably, the pose matrix in step S2 is directly generated by the backend after completing the kinematic calculations, thereby reducing the computational burden on the frontend.

[0032] Step S3: Initialize snapshot building and load basic state When the digital twin system starts up, an initial synchronization link is established, which specifically includes: (1) Establish a real-time WebSocket connection; (2) Obtain model definition data, equipment status snapshot data and drill pipe structure data concurrently through the HTTP interface; (3) Establish the mapping relationship between digital twin objects and Jinpeng system entities based on the model definition data; (4) Load the initial pose state of each entity at the system startup time based on the equipment status snapshot data; (5) Load the static distribution state of the pipe string object according to the drill pipe structure data.

[0033] The model definition data includes at least a unique entity identifier, model file name, and display strategy; the device status snapshot data includes at least the complete pose matrix of each entity; and the drill pipe structure data includes at least the static position coordinates of the drill pipe. This step completes the initial state construction of the digital twin scenario.

[0034] Step S4: Real-time data storage and time alignment processing While the snapshot initialization is not yet complete, the incremental frames received in real time are temporarily stored; after the snapshot is loaded, the temporarily stored real-time frames are released in sequence, and the system switches to real-time synchronization mode.

[0035] In real-time synchronization mode, snapshot data and real-time frame data are aligned using a unified time base, specifically including: (1) Identify the order of data based on the sequence number field in the real-time frame; (2) Determine the generation time corresponding to the real-time status based on the timestamp field; (3) Use the snapshot state as the initial baseline state and the real-time frame as the subsequent incremental update state; (4) Data from different sources and with different refresh cycles are merged in the same stage; (5) Adopt the latest frame overlay strategy to ensure that the system always takes the current physical state as the standard.

[0036] This step avoids the state misalignment problem of "real-time data arriving before the model is fully loaded" during the initialization phase, and also reduces the impact of historical backlog frames on the synchronization results. Preferably, step S4 uses a "snapshot initialization + real-time frame temporary storage + release switching" method to achieve initialization alignment.

[0037] Step S5: Object Identification Establishment and State Association Processing Establish object-level relationships based on pipe string objects and equipment objects, specifically including: (1) Establish an entity-level identifier for the device object based on the unique entity identifier returned by the model definition interface; (2) Based on the drill pipe structure data and real-time in-situ data, establish an object-level index relationship for the tubing string object; (3) Bind the device pose status, the pipe column in-situ status and the real-time numerical status to the corresponding objects; (4) Continuously associate data of the same object at different time points; (5) Form object-level state record results.

[0038] In this step, the device object is associated with at least its pose matrix, display strategy, and numerical state, and the column object is associated with at least its static position, in-situ state, and display state, thereby realizing the transformation from "data frame" to "object state".

[0039] Step S6: Construction of Unified State Data After object association is completed, the state data of each object is organized uniformly to form a target state dataset used to drive the update of the digital twin model. The target state dataset includes at least: (1) Object identification information; (2) Object pose information; (3) Object presence / visibility information; (4) Object numerical status information; (5) Information about the model or component to which the object belongs; (6) The time stamp information corresponding to the object.

[0040] Specifically, for device objects, a relationship structure of "entity ID - pose matrix - numerical state" is formed; for column objects, a relationship structure of "object index - static position - in-situ state" is formed. This step unifies the previously scattered snapshot data, real-time frame data, and structural data into a state representation that can directly drive twin updates. The object mapping is illustrated in Table 1.

[0041] Table 1. Object Mapping Diagram

[0042] Step S7: Synchronous Update of Digital Twin Model Based on the target state dataset obtained in step S6, the digital twin model is synchronously updated, specifically including: (1) Apply the corresponding flat16 pose matrix to the device object to update the device or component pose; (2) For objects that support component-level control, update their local pose according to the entity-component two-layer structure; (3) Toggle the visibility of the column object based on the in-situ status field in the real-time frame; (4) For resident entities, maintain the pose from the previous time step when the matrix is ​​missing in the real-time frame; (5) For conditional entities, perform hiding or removal processing when the corresponding matrix is ​​missing.

[0043] In this process, the updating of device objects is accomplished using a matrix-driven direct approach, while the updating of pipe objects is accomplished using a combination of static position and in-situ state-driven approaches, thereby enabling the digital twin model to reflect the current state of the physical system in real time. Preferably, in step S7, device objects are updated using a matrix-driven direct approach, while pipe objects are updated using a combination of static position and in-situ state-driven approaches.

[0044] Step S8: Queue Consumption and Real-time Display Control To ensure the real-time and continuous nature of monitoring results, restricted consumption processing is applied to the real-time frame queue, specifically including: (1) Set a depth limit for the real-time frame queue; (2) When a new frame arrives, the old frame is automatically overwritten; (3) Consume real-time frames from the queue according to the time budget during the rendering loop; (4) Update the current frame while ensuring the smoothness of the main thread; (5) Present the updated object status in the digital twin scene in real time.

[0045] This step reduces display trailing issues caused by historical frame backlog, making the monitoring results closer to the actual on-site conditions. Preferably, step S8 employs a latest frame overlay strategy, ensuring the digital twin system always uses the current physical state as its reference.

[0046] Step S9: Output of synchronous monitoring results After the digital twin model is updated, a synchronization monitoring result corresponding to the current physical processing is output, and the synchronization monitoring result includes at least: (1) Equipment operating status; (2) Equipment position and orientation; (3) The in-situ status of the tubular object; (4) Distribution of tubing; (5) Visualization results in digital twin scenarios; (6) Link activity status.

[0047] When no valid real-time data is received within a preset time period, the system status can be marked as disconnected to indicate that the current data link is abnormal.

[0048] Example 1: System Startup and Initialization Synchronization Example This embodiment uses the process of the Jinpeng tubular processing system entering the digital twin synchronous monitoring state after startup as an example to illustrate the initialization synchronization method of the present invention.

[0049] The Jinpeng drill string processing system includes a PLC data source, a model definition interface, an equipment status snapshot interface, a drill pipe structure interface, and a real-time frame push interface. The backend system receives raw messages from the PLC, parses the messages, performs kinematic calculations, generates the pose matrix corresponding to the equipment object or component object, and constructs real-time frame data. The frontend digital twin system receives snapshot data and real-time frame data, and completes object loading, initial state establishment, and synchronous display.

[0050] In this embodiment, the system first establishes a WebSocket connection with the real-time frame interface. After establishing the real-time connection, it concurrently requests the model definition interface, the device state snapshot interface, and the drill pipe structure interface to obtain model definition data, device initial pose data, and drill string static distribution data, respectively. The model definition data is used to establish the mapping relationship between the digital twin object and the Jinpeng system entity, the device state snapshot data is used to load the initial pose of each device object when the system starts, and the drill pipe structure data is used to establish the static position distribution of the drill string object.

[0051] When a real-time frame arrives before the snapshot data has been fully loaded, the front-end system does not immediately perform a scene update. Instead, it temporarily stores the real-time frame. In one implementation, only the most recently arrived frame is retained as a heldframe. Once the model definition data and device state snapshot data are loaded, the heldframe is sent to the processing queue, and the system switches to real-time synchronization. This method establishes a connection between the initialization state and the real-time state, reducing object misalignment and state transitions during system startup.

[0052] In one specific implementation, the PLC message received by the backend is a fixed-length message, for example, 141 bytes. After parsing the valid message, the backend generates a real-time frame containing a sequence number s, a timestamp ts, a matrix dictionary m, a numerical status dictionary v, and a pipe in-situ array pr. After completing the initial loading, the frontend uses the device status snapshot as the initial baseline state and the real-time frame as the incremental update state to enter the continuous synchronous monitoring process.

[0053] Example 2: Real-time Operation and Object State Synchronization Example This embodiment uses the digital twin synchronization process after the system enters a stable operating state as an example to illustrate the object state construction and differentiated update method of the present invention.

[0054] After the system enters real-time synchronization mode, the backend continuously receives PLC messages and generates real-time frames. The matrix dictionary *m* in the real-time frame represents the pose state of equipment and component objects, the numerical state dictionary *v* represents the numerical states related to equipment operation, and the pipe column in-situ array *pr* represents the in-situ state of each pipe column point. Upon receiving the real-time frames, the frontend, based on the entity mapping relationship established in the model definition data, uniformly organizes the states of different objects to form a target state dataset. The target state dataset includes at least object identifiers, object poses, object numerical states, object in-situ states, and time stamps.

[0055] For equipment and component objects, the front end directly applies the corresponding flat16 pose matrix from the matrix dictionary m to the corresponding digital twin object to complete the object pose update. For pipe string objects, the front end determines their spatial position based on the static point coordinates returned by the drill pipe structure interface, and then controls the in-situ or visible state of the corresponding pipe string object based on the pipe string in-situ array pr. This forms a differentiated synchronous update method where "equipment objects are updated using pose matrix-driven updates, and pipe string objects are updated using a combination of static position and in-situ state updates".

[0056] In terms of object lifecycle management, one implementation classifies objects into resident entities and conditional entities. For resident entities, attitude updates are performed when the corresponding matrix exists in the current real-time frame, and the previous valid state is maintained when the corresponding matrix is ​​missing. For conditional entities, display and updates are performed when the corresponding matrix exists in the current real-time frame, and hiding is performed when the corresponding matrix is ​​missing. This approach improves the continuity of synchronous monitoring results in mixed scenarios.

[0057] In one specific implementation, the front end sets up a real-time frame processing queue, poseQueue, and sets a depth limit on the queue, for example, a queue depth of 3. When a new frame arrives and the queue is full, older frames are automatically discarded, and newer state data is retained first. In the rendering loop, the real-time frames in the queue are consumed according to a preset time budget, thereby reducing state trailing caused by the backlog of historical frames.

[0058] Example 3: Simultaneous Monitoring During the Tube String Grabbing Process This embodiment uses the process of the tube column being picked up from the storage location and transported to the target workstation as an example to illustrate the application of the present invention in a specific work scenario.

[0059] In this process, the drill pipe structure interface pre-provides the static point coordinates corresponding to each pipe string object. The front end establishes the initial position distribution of each pipe string object in the digital twin scenario based on these point coordinates. After the operation begins, the back end continuously pushes real-time frames. When the grasping device moves, the matrix dictionary m in the real-time frame changes to represent the attitude changes of the grasping device and related components. Simultaneously, the original storage state corresponding to the target pipe string in the pipe string in-situ array pr changes from in-situ to out-of-situ, while the position state corresponding to the target workstation changes from out-of-situ to in-situ. The front end updates the grasping device object and the target pipe string object synchronously based on the changes in the device matrix and the changes in the pipe string in-situ state, thereby reflecting the grasping and transfer process of the target pipe string in the digital twin scenario.

[0060] If no valid real-time frame is received within a preset time period, the system marks the link status as abnormal or disconnected and outputs a link status prompt on the digital twin monitoring interface. After the link is restored, synchronization updates continue to be performed based on subsequent real-time frames. This embodiment demonstrates the real-time synchronization monitoring effect of the present invention in specific pipe handling operations.

[0061] The technical effects and advantages of this invention are reflected in: 1) A mechanism for connecting snapshots and real-time streams during the initialization phase has been established.

[0062] When the digital twin system starts up or reconnects, this invention does not directly use real-time data to drive scene updates. Instead, it first establishes a real-time connection and then acquires model definition data, device status snapshot data, and column structure data. Before the snapshot data is loaded, the real-time frames are temporarily stored. After the snapshot data is loaded, the temporarily stored real-time frames are released and the system switches to real-time synchronization.

[0063] Compared to existing technologies that separate or directly overlay initialization data and real-time data, this invention establishes a seamless mechanism of "snapshot initialization + real-time frame temporary storage + release switching".

[0064] (2) A unified state construction mechanism for equipment objects, component objects and pipe column objects has been established.

[0065] This invention addresses the model definition data, equipment pose data, numerical state data, drill pipe structure data, and in-situ state data in the Jinpeng tubing processing system. It organizes these data according to object identifier, object pose, object numerical state, object in-situ state, and time label to form a target state dataset that can directly drive digital twin updates.

[0066] Compared to existing technologies that display or partially update data from different sources separately, this invention transforms the dispersed data processing results into a unified object-level state representation.

[0067] (3) A differentiated synchronous update mechanism for different object types has been established.

[0068] This invention employs different update methods for different object types: for device objects and component objects, the pose matrix is ​​used to directly drive the update; for pipe objects, the static position and real-time in-situ status are combined for the update; and for objects with different lifecycle attributes, differentiated maintenance, display, or hiding processing is adopted.

[0069] Compared to the general and single model update methods in existing technologies, this invention can adapt to scenarios where fixed equipment, movable parts and column objects coexist in the Jinpeng column processing system, thereby improving the continuity and accuracy of digital twin synchronous monitoring.

[0070] It should be understood that the scope of protection sought by this invention is not limited to the non-limiting embodiments, which are merely illustrative examples. The substantive scope of protection claimed in this application is further embodied in the scope provided by the independent claims and their dependent claims.

Claims

1. A method for synchronous monitoring of digital twins in tubing processing, which employs the Jinpeng tubing processing system, characterized in that: The method includes: S1 Multi-Source Data Acquisition and Access: Acquires multi-source data related to the tubing processing process through the Jinpeng tubing processing system; S2: Raw data parsing and formatting: The raw data obtained in S1 is parsed and formatted to form a standardized dataset that can be processed by the digital twin system; S3: Initialize snapshot construction and basic state loading, and establish an initial synchronization link when the digital twin system starts up; S4: Real-time data storage and time alignment processing. Before the initial snapshot is loaded, the incremental frames received in real time are temporarily stored. After the snapshot is loaded, the temporarily stored real-time frames are released in sequence and the system switches to real-time synchronization mode. In real-time synchronization mode, the snapshot data and real-time frame data are aligned under a unified time reference. S5: Object identification establishment and status association processing, establishing object-level association relationships with pipe column objects and equipment objects as the core; S6: Unified state data construction. After completing the object association, the state data of each object is organized in a unified manner to form a target state dataset used to drive the update of the digital twin model. S7: Synchronous update of the digital twin model: Based on the target state dataset obtained in S6, the digital twin model is synchronously updated. S8: Queue consumption and real-time display control. To ensure the real-time and continuous nature of monitoring results, the real-time frame queue is subject to restricted consumption. S9: Output synchronous monitoring results. After the digital twin model is updated, output the synchronous monitoring results corresponding to the current physical processing.

2. The method for synchronous monitoring of digital twins for tubing processing according to claim 1, characterized in that: In S1, the multi-source data includes at least one of the following: real-time status data of physical equipment output by PLC, equipment pose matrix data calculated by the backend, entity list data for model definition, equipment status snapshot data for system initialization, drill pipe structure data for static distribution representation of the pipe string, and real-time frame data for real-time incremental synchronization. The real-time status data of the physical device is received by the backend via a high-frequency UDP link. The backend parses the received messages and uses them as the data basis for subsequent kinematic calculations and real-time status generation. The system also accesses initialization data and incremental data through the HTTP snapshot interface and the WebSocket real-time stream interface.

3. The method for synchronous monitoring of digital twins for tubing processing according to claim 2, characterized in that: S2 obtains raw data from different sources, which is then transformed into a unified and processable data structure, providing a foundation for subsequent time alignment, object association, and model updates. Specifically, this includes: S21: Parse the device operating parameters in the PLC message; S22: Perform kinematic calculations in the backend to generate a pose matrix corresponding to the device object; S23: Convert the pose result of the device object into a uniform 4×4 homogeneous transformation matrix format; S24: Output the matrix as a flat16 array in column-major order; S25: Extract the sequence number field, timestamp field, matrix field, drill pipe in-situ field, and numerical status field from the real-time frame.

4. The method for synchronous monitoring of digital twins for tubing processing according to claim 3, characterized in that: S3 completes the initial state construction of the digital twin scenario, specifically including: S31: Establish a real-time WebSocket connection; S32: Concurrently obtain model definition data, equipment status snapshot data, and drill pipe structure data via HTTP interface; S33: Establish a mapping relationship between digital twin objects and Jinpeng system entities based on the model definition data; S34: Load the initial pose state of each entity at system startup based on the device status snapshot data; S35: Load the static distribution state of the tubing string object based on the drill pipe structure data; The model definition data includes at least: unique entity identifier, model file name and display strategy; the device status snapshot data includes at least the complete pose matrix of each entity; and the drill pipe structure data includes at least the static position coordinates of the drill pipe.

5. The method for synchronous monitoring of digital twins for tubing processing according to claim 4, characterized in that: The S4 method avoids the problem of real-time data arriving prematurely before the model is fully loaded during the initialization phase, thus preventing state misalignment and reducing the impact of historical backlog frames on the synchronization results. Specifically, it includes: S41: Identify the order of data based on the sequence number field in the real-time frame; S42: Determine the generation time corresponding to the real-time status based on the timestamp field; S43: Use the snapshot state as the initial baseline state and the real-time frame as the subsequent incremental update state; S44: Perform simultaneous merging of data from different sources and with different refresh cycles; S45: Employs the latest frame overlay strategy to ensure that the system always uses the current physical state as the standard.

6. The method for synchronous monitoring of digital twins for tubing processing according to claim 5, characterized in that: In S5, a device object is associated with at least its pose matrix, display strategy, and numerical state, and a column object is associated with at least its static position, in-situ state, and display state, thereby realizing the transformation from data frame to object state, specifically including: S51: Based on the unique entity identifier returned by the model definition interface, establish an entity-level identifier for the device object; S52: Establish object-level index relationships for the tubing string objects based on drill pipe structure data and real-time in-situ data; S53: Bind the device pose status, the pipe in-place status, and the real-time numerical status to the corresponding objects; S54: Continuously associate data of the same object at different time points; S55: Generate object-level state record results.

7. The method for synchronous monitoring of digital twins for tubing processing according to claim 6, characterized in that: In step S6, the target state dataset includes at least one of the following information: object identification information, object pose information, object presence / disappearance information, object numerical state information, model or component information to which the object belongs, and time tag information corresponding to the object.

8. The method for synchronous monitoring of digital twins for tubing processing according to claim 7, characterized in that: Specifically, S7 includes: S71: Apply the corresponding flat16 pose matrix to the device object to update the device or component pose. S72: For objects that support component-level control, update their local pose according to the entity-component two-layer structure; S73: Toggles the visibility of the column object based on the in-situ status field in the real-time frame; S74: For resident entities, maintain the pose from the previous time step when the matrix is ​​missing in the real-time frame; S75: For conditional entities, perform hiding or removal processing when the corresponding matrix is ​​missing; The device object is updated using a matrix direct drive method, while the column object is updated using a combination of static position and in-situ state drive method, so that the digital twin model reflects the current state of the physical system in real time.

9. The method for synchronous monitoring of digital twins for tubing processing according to claim 8, characterized in that: The S8 reduces the display trailing problem caused by historical frame backlog, making the monitoring results closer to the actual situation on site, and specifically includes: S81: Set a depth limit for the real-time frame queue; S82: Automatically overwrites old frames when a new frame arrives; S83: Consume real-time frames from the queue according to the time budget during the rendering loop; S84: Update the current frame while ensuring the smoothness of the main thread; S85: Present the updated object state in the digital twin scene in real time.

10. The method for synchronous monitoring of digital twins for tubing processing according to claim 9, characterized in that: In S9, the synchronous monitoring results include at least: equipment operating status, equipment pose status, pipe column object in-situ status, pipe column distribution status, visualization results in the digital twin scenario, and link activity status; when no valid real-time data is received within a preset time period, the system status can be marked as disconnected to indicate that the current data link is abnormal.

11. The method for synchronous monitoring of digital twins for tubing processing according to claim 10, characterized in that: In S2, the pose matrix is ​​directly generated by the backend after completing the kinematic calculations to reduce the computational burden on the frontend. In S4, initialization alignment is achieved by using snapshot initialization, real-time frame storage, and release switching. In S7, device objects are updated using matrix-driven direct updates, and column objects are updated using a combination of static position and in-situ state. In S8, the latest frame overwrite strategy is adopted to ensure that the digital twin system always uses the current physical state as the standard.

12. A digital twin synchronous monitoring system for tubing processing, comprising a Jinpeng tubing processing system and executing any one of the digital twin synchronous monitoring methods for tubing processing according to claims 1 to 11, characterized in that: It includes the following modules connected sequentially along the data flow direction: multi-source data acquisition and access module, raw data parsing and formatting processing module, initial snapshot construction and basic state loading module, real-time data temporary storage and time alignment processing module, object identification establishment and state association processing module, unified state data construction module, digital twin model synchronous update module, queue consumption and real-time display control module, and synchronous monitoring result output module.

Images