An oilfield station digital twin running state prediction method based on visual real-time perception

Abstract

The application discloses an oilfield station digital twin operation state prediction method based on visual real-time sensing, and belongs to the technical field of oilfield station intelligent monitoring and digital twin. The method is aimed at the problems that visual real-time sensing data and sensing timing operation data are difficult to be fused, and a digital twin model is difficult to be dynamically updated, acquires station sensing data, performs feature extraction and TCN-CNN cross-modal feature fusion, obtains an operation state prediction result, and updates the digital twin model based on the xyz coordinate offset original parameters of equipment and generates equipment fault alarm information, and is suitable for the fields of oilfield station intelligent monitoring, digital twin operation and maintenance and fault early warning.

Description

Technical Field

[0001] This invention belongs to the field of intelligent monitoring and digital twin technology for oilfield stations, specifically involving a digital twin operation status prediction technology for oilfield stations based on real-time visual perception. Background Technology

[0002] As crucial nodes in oil and gas gathering, processing, and storage, oilfield stations have been continuously evolving towards digitalization, visualization, and intelligent operation and maintenance in recent years. With the deepening application of sensor networks, video monitoring, edge computing, and digital twin technologies in oilfield scenarios, mapping and managing station equipment, process status, and operating environment using virtual models has become an important technological path to improve safety production and operational efficiency. Real-time monitoring of oilfield station operating status and early fault prediction are core components of oilfield safety production and intelligent management, and the implementation of digital twin technology has become a key carrier for the linkage between the physical and virtual worlds of oilfield stations.

[0003] Existing technologies already include digital twin centralized control solutions for oilfield stations. For example, CN116594365A discloses an intelligent digital twin centralized control method and system for unmanned oilfield stations. This method constructs a 3D simulation model, collects operational status data of process equipment and environmental data, and maps the relevant information to the centralized control model and the 3D simulation model to achieve visualized management and control of the station. This type of solution demonstrates that digital twin modeling and status mapping for oilfield stations has become an industry trend. However, its focus remains primarily on the mapping and display of equipment status, environmental information, and video information. A more in-depth integrated processing chain is still lacking regarding how to jointly model based on real-time visual perception data and time-series sensor data, how to further output operational status prediction results, and how to drive real-time updates of the digital twin model.

[0004] Meanwhile, multimodal inspection and anomaly identification technologies in oilfield scenarios are also developing. For example, CN120724335A discloses a multimodal intelligent inspection system and device for oilfield gathering and transportation stations. It collects various data such as vibration, temperature, pressure, flow rate, visible light, infrared thermal imaging, and depth camera data, and generates a unified format time-series health vector after spatiotemporal alignment to identify abnormal risks. CN117371220A discloses a method for predicting faults based on digital twins, reflecting the technological trend of combining digital twin technology with fault prediction. However, existing solutions typically process visual perception, sensor time-series analysis, state prediction, and digital twin updates separately, failing to form a unified data processing chain around the oilfield station scenario. Therefore, problems remain, such as the difficulty in effectively integrating real-time visual perception data and sensor time-series operational data, and the difficulty in timely driving digital twin model updates through operational state prediction results.

[0005] Therefore, how to achieve cross-modal fusion processing of real-time visual perception data and time-series sensor operation data, and dynamically update the center coordinates of the corresponding equipment in the digital twin model based on the xyz coordinate offset parameters of the equipment, while generating equipment fault alarm information, so as to improve the accuracy and real-time performance of oilfield station operation status prediction and digital twin dynamic mapping, has become a technical problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0006] To address the challenges in existing technologies such as the difficulty in fusing real-time visual perception data with temporal sensing data and the difficulty in dynamically updating digital twin models, this invention proposes the following solution: A method for predicting the operational status of an oilfield station based on real-time visual perception using a digital twin, the method comprising: S1. Acquire oilfield station perception data, which includes real-time visual perception data and sensor time-series operation data. S2. Extract features from the real-time visual perception data and the time-series sensing data respectively to obtain visual feature vectors and time-series feature vectors; S3. Input the visual feature vector and the temporal feature vector into the TCN-CNN cross-modal feature fusion module to perform cross-modal feature fusion and obtain cross-modal fused features; S4. Based on the cross-modal fusion features, the operational status prediction results of the sensing samples of the oilfield station to be tested are obtained; S5. Extract the original xyz coordinate offset parameters of the equipment from the real-time visual perception data corresponding to the sensing sample of the oilfield station to be tested. Scale the original xyz coordinate offset parameters using a preset mapping coefficient α to obtain the actual offset parameters. Update the center coordinates of the corresponding equipment in the digital twin model based on the actual offset parameters. Generate equipment fault alarm information based on the comparison result between the modulus of the actual offset parameters and the preset fault judgment threshold β. S6. Visualize and log the predicted operating status results, the equipment fault alarm information, and the operating status of the digital twin model.

[0007] Furthermore, the station perception data mentioned in S1 comes from synthetic data or real data, and the acquired station perception data is preprocessed to form a training set and a test set.

[0008] Furthermore, if the station perception data originates from synthetic data, the basic dimension of visual features is preset to 64 dimensions and the basic dimension of temporal features is preset to 16 dimensions. Based on the status labels, real-time visual perception data and sensor temporal operation data corresponding to three types of states—normal, equipment failure, and personnel violation—are generated. The sensor temporal operation data includes pressure data, temperature data, and flow data. If the station perception data originates from real data, visual-temporal perception data in CSV format is loaded.

[0009] Furthermore, the preprocessing includes: performing dimensionality verification and normalization on the real-time visual perception data, and performing sequence length normalization and standardization on the sensing time-series operation data.

[0010] Furthermore, S2 performs feature extraction on the real-time visual perception data, including: reshaping the one-dimensional visual perception features into 1×8×8 two-dimensional features, extracting local visual depth features through two layers of convolution and pooling operations, and then mapping the extracted visual features into a 128-dimensional visual feature vector through a fully connected layer.

[0011] Furthermore, S2 performs feature extraction on the sensing time-series data, including: constructing a time-series feature extraction backbone using three-order stacked TCN residual modules. Each TCN residual module adopts a causal convolution structure with dilation coefficients. The dilation coefficients of the three-order TCN residual modules are 1, 2, and 4 respectively. Each TCN residual module includes a double convolutional layer and skip connections. When the channel dimensions of the input features and the convolutional output features do not match, the input features are transformed by a 1×1 convolutional kernel. After completing the residual convolution, the time-series feature sequence is compressed into a 128-dimensional time-series feature vector by adaptive average pooling.

[0012] Furthermore, S3 performs cross-modal feature fusion on the visual feature vector and the temporal feature vector, including: concatenating the 128-dimensional visual feature vector and the 128-dimensional temporal feature vector to obtain a 256-dimensional concatenated feature; and performing a nonlinear transformation on the 256-dimensional concatenated feature through a fully connected layer and ReLU nonlinear activation to obtain a 256-dimensional cross-modal fused feature.

[0013] Furthermore, S4 trains and optimizes the operation status prediction model based on the cross-modal fusion features and state labels, and converts the classification prediction values ​​into prediction confidence scores for three states: normal, equipment failure, and personnel violation. The operation status prediction model is trained using cross-entropy loss as the supervision loss for state classification, and the Adam optimizer is combined with gradient descent to update the model weights.

[0014] Furthermore, the digital twin model mentioned in S5 is a basic digital twin model of an oilfield station built based on Open3D. The basic digital twin model includes the ground, storage tanks, and pipelines. When the modulus of the actual offset parameter exceeds the fault judgment threshold β, equipment fault alarm information is generated and the corresponding equipment is highlighted in red; otherwise, the original color of the corresponding equipment is maintained.

[0015] Based on the same inventive concept, the present invention also proposes a computer storage medium storing a computer program thereon, which executes the method described in the present invention when running on a processor.

[0016] Compared with the prior art, the present invention has the following beneficial effects: The oilfield station digital twin operation status prediction method based on real-time visual perception described in this invention acquires oilfield station perception data, including real-time visual perception data and time-series sensor operation data, and performs feature extraction and cross-modal feature fusion separately. This changes the situation in the prior art where real-time visual perception data and time-series sensor operation data are processed separately and are difficult to fuse effectively. Thus, the operation status prediction result can be obtained based on the fused cross-modal fusion features.

[0017] This invention achieves cross-modal feature fusion by inputting visual feature vectors and temporal feature vectors into the TCN-CNN cross-modal feature fusion module. This changes the existing technology's approach of separating station scene information from equipment dynamic operation information, thereby enabling operation status prediction to be based on the joint representation of real-time visual perception data and sensor temporal operation data.

[0018] This invention extracts features from the time-series operational data of sensors and processes the data using a three-order stacked TCN residual module, a causal convolutional structure with an expansion coefficient, a double convolutional layer, and skip connections. This addresses the shortcomings of existing technologies in fully mining the temporal correlation features and dynamic trends of pressure, temperature, and flow data, thereby enabling the formation of time-series feature vectors for subsequent operational status prediction.

[0019] This invention extracts the original xyz coordinate offset parameters of the equipment from the real-time visual perception data corresponding to the perception sample of the oilfield station to be tested, and updates the center coordinates of the corresponding equipment in the digital twin model after scaling by the mapping coefficient α. This changes the situation in the prior art where the digital twin model relies on static parameters or manual updates and is difficult to reflect changes in the field status in a timely manner, thereby realizing the dynamic updating of the position of the corresponding equipment in the digital twin model.

[0020] This invention generates equipment fault alarm information by comparing the modulus of the actual offset parameter with the fault judgment threshold β. This changes the situation in the prior art where the digital twin model and fault alarm are separated and difficult to link in a timely manner, thereby combining the updating of the digital twin model with the generation of equipment fault alarm information.

[0021] This invention visualizes and logs the operational status prediction results, equipment fault alarm information, and digital twin model operational status, thereby changing the previous situation where the operational status prediction results and digital twin model operational status were presented separately. This facilitates the unified output of oilfield station operational status prediction results, equipment fault alarm information, and digital twin model operational status.

[0022] This invention features cross-modal fusion processing of real-time visual perception data and sensor time-series operation data, dynamic updating of digital twin models based on the original parameters of xyz coordinate offset of equipment, and generation of equipment fault alarm information. It can realize the collaborative processing of oilfield station operation status prediction and dynamic mapping of digital twins, and is applicable to fields such as intelligent monitoring of oilfield stations, digital twin operation and maintenance, and fault early warning. Attached Figure Description

[0023] Figure 1 This is a flowchart of the oilfield station digital twin operation status prediction method described in this invention. Detailed Implementation

[0024] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of them. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0025] Implementation Method 1 like Figure 1 As shown, a method for predicting the operational status of an oilfield station based on real-time visual perception using a digital twin is presented. The method includes: S1. Acquire oilfield station perception data, which includes real-time visual perception data and sensor time-series operation data. S2. Extract features from the real-time visual perception data and the time-series sensing data respectively to obtain visual feature vectors and time-series feature vectors; S3. Input the visual feature vector and the temporal feature vector into the TCN-CNN cross-modal feature fusion module to perform cross-modal feature fusion and obtain cross-modal fused features; S4. Based on the cross-modal fusion features, the operational status prediction results of the sensing samples of the oilfield station to be tested are obtained; S5. Extract the original xyz coordinate offset parameters of the equipment from the real-time visual perception data corresponding to the sensing sample of the oilfield station to be tested. Scale the original xyz coordinate offset parameters using a preset mapping coefficient α to obtain the actual offset parameters. Update the center coordinates of the corresponding equipment in the digital twin model based on the actual offset parameters. Generate equipment fault alarm information based on the comparison result between the modulus of the actual offset parameters and the preset fault judgment threshold β. S6. Visualize and log the predicted operating status results, the equipment fault alarm information, and the operating status of the digital twin model.

[0026] In this embodiment, by simultaneously incorporating real-time visual perception data and time-series sensor operation data into the processing flow, and connecting cross-modal feature fusion, operation status prediction, digital twin model update, and equipment fault alarm information generation into a continuous processing chain, this approach differs from the existing technology where visual perception, time-series analysis, and digital twin update are separated. This enables the operation status prediction results to be linked with the operation status of the digital twin model, thereby achieving collaborative processing of oilfield station operation status prediction and dynamic mapping of the digital twin.

[0027] Preferably, the real-time visual perception data can be used to characterize the appearance status of equipment, the location status of equipment, or the activity status of on-site personnel in the oilfield station; the time-series sensing operation data can be used to characterize pressure changes, temperature changes, and flow rate changes in the oilfield station; and the operation status prediction results can correspond to any one of the three states: normal, equipment failure, and personnel violation.

[0028] Furthermore, the station perception data mentioned in S1 comes from synthetic data or real data, and the acquired station perception data is preprocessed to form a training set and a test set.

[0029] In this embodiment, by distinguishing between synthetic data and real data sources from the station perception data sources, and uniformly preprocessing and dividing the dataset after input, subsequent feature extraction, cross-modal feature fusion, and operational status prediction are based on data with relatively uniform format.

[0030] Preferably, the training set is used to train and optimize the running state prediction model, and the test set is used to test the trained and optimized running state prediction model.

[0031] Furthermore, if the station perception data originates from synthetic data, the basic dimension of visual features is preset to 64 dimensions and the basic dimension of temporal features is preset to 16 dimensions. Based on the status labels, real-time visual perception data and sensor temporal operation data corresponding to three types of states—normal, equipment failure, and personnel violation—are generated. The sensor temporal operation data includes pressure data, temperature data, and flow data. If the station perception data originates from real data, visual-temporal perception data in CSV format is loaded.

[0032] In this embodiment, by pre-setting the basic dimensions of visual features and the basic dimensions of temporal features for the synthetic data, and generating corresponding data according to three states: normal, equipment failure, and personnel violation, the sample generation process can be kept consistent with the subsequent classification target. At the same time, by using the CSV format to load real data, the process of accessing real data can be made more direct, so that the synthetic data and real data can jointly serve the training and testing of the operational status prediction model.

[0033] Preferably, the synthesized data can generate multiple sets of real-time visual perception data and multiple sets of time-series sensor operation data according to preset rules, and include corresponding status labels; the real data can be generated by collecting data on-site at oilfield stations and then processing it. The visual and time-series components of the visual-time-series perception data can be stored according to the same sample correspondence for subsequent joint processing.

[0034] Furthermore, the preprocessing includes: performing dimensionality verification and normalization on the real-time visual perception data, and performing sequence length normalization and standardization on the sensing time-series operation data.

[0035] In this embodiment, by performing dimensionality verification and normalization on the real-time visual perception data, the visual input can meet the requirements of subsequent feature extraction. By performing sequence length normalization and standardization on the sensing time-series running data, the differences in length and numerical scale between different samples can be reduced, thereby making the real-time visual perception data and the sensing time-series running data more suitable for subsequent feature extraction and cross-modal feature fusion.

[0036] Preferably, the dimension verification can be used to check whether the real-time visual perception data meets the preset input dimension requirements; the sequence length normalization can be achieved by completion or truncation; the standardization processing can be performed separately for pressure data, temperature data, and flow data.

[0037] Furthermore, S2 performs feature extraction on the real-time visual perception data, including: reshaping the one-dimensional visual perception features into 1×8×8 two-dimensional features, extracting local visual depth features through two layers of convolution and pooling operations, and then mapping the extracted visual features into a 128-dimensional visual feature vector through a fully connected layer.

[0038] In this embodiment, by reshaping one-dimensional visual perception features into 1×8×8 two-dimensional features, and extracting visual local depth features through two layers of convolution and pooling operations, the real-time visual perception data can be made more suitable for hierarchical feature expression. Then, by mapping it to a 128-dimensional visual feature vector through a fully connected layer, the visual features can meet the requirements of subsequent cross-modal feature fusion in terms of dimensionality, thereby providing a visual information basis for operational status prediction.

[0039] Preferably, the one-dimensional visual perception features can be formed from the real-time visual perception results in the oilfield station; the two-layer convolution and pooling operations can sequentially extract local features and compress feature representations to obtain a visual feature expression suitable for fusion.

[0040] Furthermore, S2 performs feature extraction on the sensing time-series data, including: constructing a time-series feature extraction backbone using three-order stacked TCN residual modules. Each TCN residual module adopts a causal convolution structure with dilation coefficients. The dilation coefficients of the three-order TCN residual modules are 1, 2, and 4 respectively. Each TCN residual module includes a double convolutional layer and skip connections. When the channel dimensions of the input features and the convolutional output features do not match, the input features are transformed by a 1×1 convolutional kernel. After completing the residual convolution, the time-series feature sequence is compressed into a 128-dimensional time-series feature vector by adaptive average pooling.

[0041] In this embodiment, by using a three-order stacked TCN residual module to process the time-series sensor data and setting the expansion coefficients to 1, 2, and 4 respectively, the variation characteristics of pressure, temperature, and flow data at different time scales can be extracted. Through the combined processing of dual convolutional layers, skip connections, and 1×1 convolutional kernels, the time-series feature extraction process can be kept coherent. By compressing the data into a 128-dimensional time-series feature vector through adaptive average pooling, the time-series features and visual features can form a correspondence in the subsequent fusion stage, thereby enabling the dynamic change trend of the time-series sensor data to participate in the prediction of the operating status.

[0042] Preferably, the causal convolutional structure can be used to maintain the sequential relationship of temporal data processing; the skip connection can be used to introduce the input features of the previous layer into the subsequent output, so that the temporal feature extraction backbone maintains feature transfer during multi-layer processing; the adaptive average pooling is used to compress temporal feature sequences of different lengths into a uniform dimensional representation.

[0043] Furthermore, S3 performs cross-modal feature fusion on the visual feature vector and the temporal feature vector, including: concatenating the 128-dimensional visual feature vector and the 128-dimensional temporal feature vector to obtain a 256-dimensional concatenated feature; and performing a nonlinear transformation on the 256-dimensional concatenated feature through a fully connected layer and ReLU nonlinear activation to obtain a 256-dimensional cross-modal fused feature.

[0044] In this embodiment, by concatenating the 128-dimensional visual feature vector with the 128-dimensional temporal feature vector, and further performing nonlinear transformation through a fully connected layer and ReLU nonlinear activation, the method of independent participation of visual features and temporal features in the analysis is changed. This allows the information corresponding to the real-time visual perception data and the information corresponding to the time-series sensor operation data to form a joint expression in a unified feature space, so that the obtained cross-modal fusion features can simultaneously reflect the on-site scene status and the equipment operation status.

[0045] Preferably, the 256-dimensional splicing feature can be regarded as a direct combination of visual features and temporal features, and the 256-dimensional cross-modal fusion feature can be regarded as a joint feature representation obtained after further transformation, which is used for subsequent operation state prediction.

[0046] Furthermore, S4 trains and optimizes the operation status prediction model based on the cross-modal fusion features and state labels, and converts the classification prediction values ​​into prediction confidence scores for three states: normal, equipment failure, and personnel violation. The operation status prediction model is trained using cross-entropy loss as the supervision loss for state classification, and the Adam optimizer is combined with gradient descent to update the model weights.

[0047] In this embodiment, the operational status prediction model is trained and optimized based on cross-modal fusion features and state labels. The predicted values ​​are then converted into prediction confidence levels for three states: normal, equipment failure, and personnel violation. This ensures that the operational status prediction results not only have category outputs but also corresponding confidence levels, facilitating the judgment of the state of the oilfield station's sensing samples. Cross-entropy loss and the Adam optimizer, combined with gradient descent, are used to update the model weights, enabling the operational status prediction model to be trained around the three-state classification objective.

[0048] Preferably, the state label is used to characterize the state category to which the sample belongs, and the prediction confidence is used to characterize the output degree of the oilfield station's perceived sample belonging to the corresponding state category; after training, the samples in the test set can be input into the trained and optimized operation state prediction model to obtain the operation state prediction results.

[0049] Furthermore, the digital twin model mentioned in S5 is a basic digital twin model of an oilfield station built based on Open3D. The basic digital twin model includes the ground, storage tanks, and pipelines. When the modulus of the actual offset parameter exceeds the fault judgment threshold β, equipment fault alarm information is generated and the corresponding equipment is highlighted in red; otherwise, the original color of the corresponding equipment is maintained.

[0050] In this embodiment, by adopting a digital twin basic model of an oilfield station built based on Open3D and updating the center coordinates of the corresponding equipment according to the actual offset parameters, the equipment status in the digital twin model can correspond to the real-time visual perception results. At the same time, by comparing the magnitude of the actual offset parameters with the fault judgment threshold β, and generating equipment fault alarm information and highlighting the corresponding equipment in red when the threshold is exceeded, the changes in equipment status in the digital twin model can be linked with the equipment fault alarm information. Thus, the digital twin model is not only used for scene display, but also for reflecting abnormal equipment status.

[0051] Preferably, the mapping coefficient α can be used to convert the original xyz coordinate offset parameters of the equipment into actual offset parameters that are compatible with the scale of the digital twin model; the fault judgment threshold β is used to judge whether the change in the current position of the equipment reaches the alarm condition; the ground, storage tank and pipeline are the basic components in the digital twin basic model, and in practical applications, other model objects related to the operating status of the oilfield station may also be included.

[0052] Implementation Method 2 This implementation describes the real-world application of a visual real-time perception-based digital twin operation status prediction method for oilfield stations. The method centers on real-time visual perception and dynamic mapping of the digital twin, supported by dual-source data processing, TCN temporal feature extraction, CNN visual feature extraction, cross-modal feature fusion, and deep learning prediction. It utilizes full-process visualization as the operational platform to construct a closed-loop processing flow for equipment fault detection, personnel violation identification, and operation status prediction in oilfield station scenarios.

[0053] In this embodiment, the overall system architecture adopts a 5-layer closed-loop architecture, enabling data communication and result feedback between each layer. The 5-layer closed-loop architecture includes: a visualization interaction layer, a digital twin mapping layer, an operational status prediction layer, a cross-modal feature fusion layer, and a visual-temporal feature extraction layer. Specifically, the visualization interaction layer integrates the entire process of data loading or generation, model training, status prediction, twin mapping, and result display; the digital twin mapping layer uses mapping parameters from real-time visual perception data to achieve real-time updates of equipment location and operational status in the oilfield station's digital twin model and visualizes fault status; the operational status prediction layer constructs a classification head based on cross-modal fusion features to predict three states: normal, equipment fault, and personnel violation, and outputs the prediction results and confidence levels; the cross-modal feature fusion layer achieves deep fusion of temporal and visual features based on feature splicing and nonlinear transformation; and the visual-temporal feature extraction layer is divided into a CNN visual feature extraction branch and a TCN temporal feature extraction branch, respectively extracting station visual features and sensor temporal operational features.

[0054] In this embodiment, the core modules include a dual-source data processing module, a CNN visual feature extraction module, a TCN temporal feature extraction module, a TCN-CNN cross-modal feature fusion module, an operational status prediction module, a digital twin real-time mapping module, and an integrated visualization interaction module. The dual-source data processing module employs a dual-source data supply mechanism of synthetic and real data. Synthetic data is used to generate real-time visual perception data and sensor temporal operational data matching three states: normal, equipment malfunction, and personnel violation, along with complete status labeling information. Real data supports standardized format visual-temporal perception data loading and parsing, enabling switching between the two data sources. For synthetic data, the preset basic dimensions of visual features are 64, and the basic dimensions of temporal features are 16. Based on the status labels, real-time visual perception data and sensor time-series operation data are generated. The sensor time-series operation data includes pressure data, temperature data, and flow data, simulating the dynamic change trend of time-series data and automatically labeling status labels. For real data, it supports loading visual-temporal perception data in CSV format, performs dimensionality verification and normalization on visual feature data, and normalizes the sequence length of sensor time-series operation data by completing or truncating it, and completes data standardization processing.

[0055] In this embodiment, the CNN visual feature extraction module constructs a visual feature extraction architecture based on a convolutional neural network, converting one-dimensional visual perception features into two-dimensional features to adapt to CNN convolution operations. Specifically, the 64-dimensional one-dimensional visual perception features are reshaped into 1×8×8 two-dimensional features, and local visual depth features are extracted through two layers of convolution and pooling operations. Then, the features are mapped to 128 dimensions through a fully connected layer to output a visual feature vector.

[0056] In this embodiment, the TCN temporal feature extraction module designs a feature extraction method based on a temporal convolutional network for pressure, temperature, and flow data. This module first performs feature-by-feature standardization on the original sensor temporal operation data to eliminate differences in dimensions and values. Then, it constructs the temporal feature extraction backbone using three-order stacked TCN residual modules. Each residual module employs a causal convolutional structure with dilation coefficients to ensure unidirectional transmission of temporal information and prevent future information leakage. Each dilated convolutional layer uses a 3×1 one-dimensional convolutional kernel with a stride of 1, and padding operations ensure that the lengths of the feature sequences before and after convolution are consistent. The dilation coefficients of the three-order residual modules are set sequentially to [1, 2, 4]. Each TCN residual module employs a dual convolutional layer and skip connections structure. The input features first undergo two levels of dilated convolution operations with weighted normalization and linear rectified function activation, then are added to the input features after identity mapping, and finally output module features after linear rectified function activation. When the channel dimensions of the input features and the convolution output features do not match, a 1×1 convolution kernel is used to transform the dimensions of the input features to ensure dimension matching of the residual connections. Finally, adaptive average pooling compresses the temporal feature sequence to a fixed dimension of 128, outputting a temporal feature vector.

[0057] In this embodiment, the formula for feature-by-feature standardization of the original sensing time-series data is as follows:

[0058] Where X is the original sensing time series operation data, μ is the mean of the corresponding feature of the sensing time series operation data, σ is the standard deviation of the corresponding feature of the sensing time series operation data, ε is the minimum value to prevent the denominator from being 0, and X′ is the standardized sensing time series operation data.

[0059] In this embodiment, the core calculation logic of a single TCN residual module is as follows:

[0060] in, Input features to the TCN module, This is a two-stage dilation convolution operation. For residual connectivity features, Output timing features for the TCN module.

[0061] In this embodiment, the TCN-CNN cross-modal feature fusion module constructs a cross-modal feature fusion mechanism based on feature concatenation and nonlinear transformation. The 128-dimensional temporal feature vector extracted by TCN and the 128-dimensional visual feature vector extracted by CNN are concatenated to obtain a 256-dimensional concatenated feature. Then, a nonlinear transformation of the concatenated feature is achieved through a fully connected layer and a linear rectified function activation to output the 256-dimensional fused feature.

[0062] In this embodiment, the feature splicing formula is:

[0063] Where T is a 128-dimensional temporal feature vector and V is a 128-dimensional visual feature vector. It features 256-dimensional splicing.

[0064] In this embodiment, the nonlinear fusion formula is:

[0065] Where W is the weight matrix of the fully connected layer, and b is the bias term. It is a 256-dimensional cross-modal fusion feature.

[0066] In this embodiment, the operation status prediction module constructs a station operation status classification head based on cross-modal fusion features. It maps the fusion features to the status classification dimension through a fully connected layer and random deactivation regularization, and calculates the prediction confidence of each status using a normalized exponential function, thus predicting three statuses: normal (0), equipment failure (1), and personnel violation (2). During training, this module uses cross-entropy loss as the supervision loss for status classification and employs an adaptive moment estimation optimizer combined with gradient descent to update model weights. This embodiment introduces random deactivation regularization to adapt to the small sample data scenario of oilfield stations and avoid model overfitting; it also uses a normalized exponential function to quantify the multi-class confidence output.

[0067] In this embodiment, the formula for mapping the cross-modal fusion features to the state classification dimension to obtain the classification prediction value Li is as follows:

[0068] in, This is the class head weight matrix. For the category head bias item, These correspond to the predicted values ​​for the three states, respectively.

[0069] In this embodiment, the classification prediction values ​​are converted into prediction confidence scores for each state using a normalized exponential function, as shown in the formula:

[0070] Where i=1, 2, and 3 correspond to the three states: normal, equipment malfunction, and personnel violation, respectively. Let be the prediction confidence level for the i-th class of states.

[0071] In this embodiment, the real-time digital twin mapping module constructs a basic digital twin model of the oilfield station based on Open3D. This basic model includes the ground, storage tanks, and pipelines. The module extracts the original xyz coordinate offset parameters of the equipment from real-time visual perception data, scales these parameters using a mapping coefficient α to obtain the actual offset parameters, and then updates the center position of the corresponding equipment in the digital twin model based on these actual offset parameters, completing the real-time update of the equipment positions in the digital twin model. Simultaneously, it calculates the modulus of the actual offset parameters and compares it with a preset threshold β. If the modulus exceeds the threshold, the equipment is determined to be in a fault state, and the corresponding equipment in the digital twin model is highlighted in red; if it does not exceed the threshold, the original color is maintained. The paper explicitly states that the mapping coefficient α is set to 0.1, which is based on the correspondence between the actual size and displacement of the physical equipment at a joint station in Daqing Oilfield and the model size and coordinate system of the corresponding equipment in the digital twin model. This was determined after establishing a linear mapping relationship through calibration experiments on 10 sets of equipment of different scales, ensuring that the displacement changes of the physical equipment are restored in a 1:10 ratio in the digital twin model. The fault judgment threshold β is set to 0.5, which was determined through statistical analysis of equipment displacement fault cases over the past two years and optimization through orthogonal experiments. The initial threshold range is [0.3, 0.8]. When β=0.5, the equipment fault detection accuracy can reach 98.2%, and the false alarm rate is less than 1.5%.

[0072] In this embodiment, the scaling formula for the mapping parameters is:

[0073]

[0074] in, The original parameters for the xyz coordinate offset of the device. α is the actual offset parameter, and α is the mapping coefficient.

[0075] In this embodiment, the formula for updating the device center coordinates is:

[0076] in, This is the current center position of the equipment. The updated device center location, This is the actual offset parameter.

[0077] In this embodiment, the fault status determination formula is:

[0078] when Greater than When the equipment is in a faulty state, it is determined that the equipment is in a faulty state; when Not greater than When this happens, the device is determined to be in normal condition.

[0079] In this embodiment, to verify the effectiveness of the proposed method in real-world scenarios, experimental verification and performance analysis were conducted. The experimental hardware environment consisted of an Intel Xeon W-2245 CPU, an NVIDIA RTX 3090Ti GPU (24GB VRAM), and 64GB of RAM. The software environment included Python 3.8, PyTorch 1.12, and CUDA 11.6. The experimental dataset used was a field-collected dataset from a joint station in Daqing Oilfield from 2023 to 2025, containing visual perception data and sensor time-series data for three states: normal operation, equipment failure, and personnel violations, totaling 12,800 samples. 80% of these samples were used as the training set, and 20% as the test set. Simultaneously, a small-sample scarce dataset was constructed, with training data proportions set at 10%, 30%, and 50% to simulate the real-world oilfield scenario where data is scarce. Evaluation metrics included classification accuracy, precision, recall, and single-frame inference time.

[0080] The above detailed description of the technical solution provided by the present invention is intended to highlight the advantages and benefits of the technical solution provided by the present invention. However, the above detailed embodiments are not intended to limit the scope of protection of the present invention. Any reasonable modifications and improvements to the present invention, recombination of embodiments, and equivalent substitutions based on the spirit and principles of the present invention should be included within the scope of protection of the present invention.

[0081] Those skilled in the art will understand that the above description is merely a preferred embodiment of the present invention, and the features described in the various embodiments and / or claims disclosed in the present invention can be combined or combined in various ways, even if such combinations or combinations are not explicitly described in the disclosure of the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or make equivalent substitutions for some of the technical features. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principle scope of the present invention should be considered to fall within the protection scope of the present invention.

[0082] Although preferred embodiments of the invention have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including both the preferred embodiments and all changes and modifications falling within the scope of the invention. Clearly, those skilled in the art can make various alterations and modifications to the invention without departing from its spirit and scope. Thus, if these modifications and modifications of the invention fall within the scope of the claims and their equivalents, the invention is also intended to include these modifications and modifications.

Claims

1. A method for predicting the operational status of an oilfield station using a digital twin based on real-time visual perception, characterized in that, The method includes: S1. Acquire oilfield station perception data, which includes real-time visual perception data and sensor time-series operation data. S2. Extract features from the real-time visual perception data and the time-series sensing data respectively to obtain visual feature vectors and time-series feature vectors; S3. Input the visual feature vector and the temporal feature vector into the TCN-CNN cross-modal feature fusion module to perform cross-modal feature fusion and obtain cross-modal fused features; S4. Based on the cross-modal fusion features, the operational status prediction results of the sensing samples of the oilfield station to be tested are obtained; S5. Extract the original xyz coordinate offset parameters of the equipment from the real-time visual perception data corresponding to the sensing sample of the oilfield station to be tested. Scale the original xyz coordinate offset parameters using a preset mapping coefficient α to obtain the actual offset parameters. Update the center coordinates of the corresponding equipment in the digital twin model based on the actual offset parameters. Generate equipment fault alarm information based on the comparison result between the modulus of the actual offset parameters and the preset fault judgment threshold β. S6. Visualize and log the predicted operating status results, the equipment fault alarm information, and the operating status of the digital twin model.

2. The method according to claim 1, characterized in that, The station perception data mentioned in S1 comes from synthetic data or real data, and the acquired station perception data is preprocessed to form training set and test set.

3. The method according to claim 2, characterized in that, If the station perception data originates from synthetic data, the basic dimensions of visual features are preset to 64 dimensions and the basic dimensions of temporal features are preset to 16 dimensions. Based on the status labels, real-time visual perception data and sensor temporal operation data corresponding to three types of states—normal, equipment failure, and personnel violation—are generated. The sensor temporal operation data includes pressure data, temperature data, and flow data. If the station perception data originates from real data, visual-temporal perception data in CSV format is loaded.

4. The method according to claim 2, characterized in that, The preprocessing includes: performing dimensionality verification and normalization on real-time visual perception data, and performing sequence length normalization and standardization on sensing time-series operation data.

5. The method according to claim 1, characterized in that, S2 performs feature extraction on the real-time visual perception data, including: reshaping the one-dimensional visual perception features into 1×8×8 two-dimensional features, extracting local visual depth features through two layers of convolution and pooling operations, and then mapping the extracted visual features into a 128-dimensional visual feature vector through a fully connected layer.

6. The method according to claim 1, characterized in that, S2 performs feature extraction on the sensing time-series running data, including: constructing a time-series feature extraction backbone using three-order stacked TCN residual modules. Each TCN residual module adopts a causal convolution structure with dilation coefficients. The dilation coefficients of the three-order TCN residual modules are 1, 2, and 4 respectively. Each TCN residual module includes a double convolutional layer and a skip connection. When the channel dimensions of the input features and the convolutional output features do not match, the input features are transformed by a 1×1 convolutional kernel. After completing the residual convolution, the time-series feature sequence is compressed into a 128-dimensional time-series feature vector by adaptive average pooling.

7. The method according to claim 1, characterized in that, S3 performs cross-modal feature fusion on the visual feature vector and the temporal feature vector, including: concatenating the 128-dimensional visual feature vector and the 128-dimensional temporal feature vector to obtain a 256-dimensional concatenated feature; and performing a nonlinear transformation on the 256-dimensional concatenated feature through a fully connected layer and ReLU nonlinear activation to obtain a 256-dimensional cross-modal fused feature.

8. The method according to claim 1, characterized in that, S4 trains and optimizes the operation status prediction model based on the cross-modal fusion features and state labels, and converts the classification prediction values ​​into prediction confidence scores for three states: normal, equipment failure, and personnel violation. The operation status prediction model is trained using cross-entropy loss as the supervision loss for state classification, and the Adam optimizer is combined with gradient descent to update the model weights.

9. The method according to claim 1, characterized in that, The digital twin model described in S5 is a basic digital twin model of an oilfield station built based on Open3D. The basic digital twin model includes the ground, storage tanks, and pipelines. When the modulus of the actual offset parameter exceeds the fault judgment threshold β, equipment fault alarm information is generated and the corresponding equipment is highlighted in red; otherwise, the original color of the corresponding equipment is maintained.

10. A computer storage medium having a computer program stored thereon, characterized in that, The computer program, when run on a processor, performs the method described in any one of claims 1-9.

Images