Offshore unmanned platform remote video inspection system and method

Abstract

The invention relates to the field of offshore platform monitoring and early warning, and discloses a remote video inspection system and method for an offshore unmanned platform, and the method comprises the steps: a multi-modal data fusion network implementation module obtains multi-modal sensing data; an edge computing node of the edge-cloud co-processing implementation module is responsible for receiving and caching multi-modal sensing data; the self-adaptive task scheduling mechanism module is used for monitoring load states of edge nodes and cloud resources in real time; judging that the task execution position is an edge end, a cloud end or a collaborative mode of edge coarse screening and cloud end fine judgment; when the network bandwidth is limited, the transmission of key alarm data and model updating data is guaranteed preferentially; the intelligent analysis module adopts an improved YOLOv8 target detection model and a multi-modal data fusion network to realize accurate identification and dynamic tracking of equipment abnormity and security risks; and edge end lightweight reasoning and cloud deep reinforcement learning are fused to form a closed-loop inspection flow integrating data acquisition, intelligent analysis and decision feedback.

Description

Technical Field

[0001] This invention relates to the field of offshore platform monitoring and early warning technology, and in particular to a remote video inspection system and method for offshore unmanned platforms, applicable to automated safety monitoring of unmanned facilities on oil and gas platforms. Background Technology

[0002] As marine resource development expands into deep-sea and distant areas, the safe operation and maintenance of unmanned offshore platforms faces increasingly severe challenges. These platforms operate in harsh environments characterized by high salt spray, strong corrosion, and extreme temperature and humidity, resulting in persistently high risks of equipment and pipeline corrosion, flammable gas leaks, and fires. According to statistics from the International Maritime Organization, approximately 68% of global offshore platform accidents in the past five years stemmed from equipment condition monitoring failures. Traditional manual inspection methods are limited by sea conditions, with an average effective operational window of less than 60 days per year, a single inspection cost exceeding $20,000, and a consistently high rate of missed detection for hidden risks, exceeding 15%.

[0003] To reduce reliance on manual labor, existing remote monitoring systems often employ a combination of visible light monitoring and infrared thermal imaging. For example, while pipeline leak detection systems can locate temperature anomalies through thermal imaging, the lack of cross-validation with gas concentration data results in a false alarm rate as high as 35%. Furthermore, cloud-based centralized processing architectures require 4K video streams to be transmitted via satellite links, resulting in an end-to-end latency of 5-8 seconds per stream, severely delaying fire warnings. An even more significant challenge lies in the demanding requirements placed on the dynamic adaptability of systems by complex offshore conditions: edge computing solutions with fixed threshold alarm mechanisms experience a false alarm rate exceeding 40% in low-light or salt spray environments at night.

[0004] In recent years, attempts to upgrade technology have consistently been constrained by the challenge of balancing hardware resources and algorithm performance. Multimodal fusion solutions, due to excessive computational load, generally have inference speeds below 2 FPS when deployed on edge devices. While lightweight models improve processing speed, they come at the cost of accuracy, with the accuracy of identifying device corrosion features decreasing by more than 15 percentage points compared to the baseline model. Fundamentally, existing technologies have not yet overcome three bottlenecks: the computational power requirements for real-time fusion of multi-source data are mismatched with the performance of edge hardware; low-bandwidth environments limit the efficiency of high-definition video transmission; and static analysis models struggle to adapt to the dynamic disturbances of the marine environment.

[0005] Therefore, there is an urgent need to build a closed-loop inspection system that integrates multispectral collaborative sensing, edge-cloud hierarchical computing, and dynamic optimization decision-making, so as to achieve a synergistic leap in risk identification accuracy and response time under limited resource constraints. Summary of the Invention

[0006] To address the aforementioned problems, the present invention aims to provide a remote video inspection system and method for unmanned offshore platforms. This system overcomes the difficulties of "unclear visibility, inability to transmit data, and inaccurate judgment" faced by offshore platforms, providing highly reliable intelligent support for deep-sea energy facilities, enabling all-weather intelligent inspection of unmanned offshore platforms, and effectively reducing the rate of missed inspections and the cost of manual intervention.

[0007] To achieve the above objectives, in a first aspect, the technical solution adopted by the present invention is as follows: a remote video inspection system for an unmanned offshore platform, comprising: a multimodal data fusion network implementation module deployed in key areas of the unmanned offshore platform to acquire multimodal perception data; the key areas include the process area, wellhead area, electrical room, and external transmission area; an edge-cloud collaborative processing implementation module deployed on-site or in nearby facilities, equipped with embedded lightweight computing hardware; edge computing nodes responsible for receiving and caching multimodal perception data; running a lightweight real-time inference model optimized and distributed from the cloud for preliminary target detection, anomaly identification, and alarm; executing low-latency critical tasks; and an adaptive task scheduling mechanism module as software middleware. It is used to monitor the load status of edge nodes and cloud resources in real time; make dynamic decisions based on task attributes to determine the task execution location as edge, cloud, or a collaborative mode of edge coarse screening plus cloud fine judgment; prioritize the transmission of key alarm data and model update data when network bandwidth is limited; the intelligent analysis module adopts an improved YOLOv8 target detection model and a multimodal data fusion network, which integrates image recognition, thermal imaging temperature analysis, and gas concentration monitoring to achieve accurate identification and dynamic tracking of equipment anomalies and safety risks; at the same time, it integrates lightweight inference at the edge and deep reinforcement learning in the cloud to form a closed-loop inspection process that integrates data collection, intelligent analysis, and decision feedback.

[0008] Furthermore, the multimodal data fusion network includes: Visible light image coding branch is used to extract device structural features; The thermal imaging temperature coding branch is used to extract thermodynamic anomaly features; The gas concentration coding branch is used to extract leakage diffusion characteristics; The fusion decoder receives high-order feature vectors from the three branch outputs and generates a fusion risk map through cross-modal attention weight allocation.

[0009] Furthermore, the visible light image coding branch uses an explosion-proof visible light camera, the thermal imaging temperature coding branch uses an infrared thermal imager, and the gas concentration coding branch uses an electrochemical gas sensor. All sensors are integrated through a protective housing to achieve salt spray environment adaptability, and the electrochemical gas sensors are deployed in a distributed manner. Each sensor is connected to the edge node via an industrial bus.

[0010] Furthermore, the improved YOLOv8 target detection model integrates visible light image thermal imaging temperature and gas concentration monitoring data through a multimodal data fusion network; the multimodal data fusion network adopts an attention mechanism fusion architecture to achieve dynamic tracking of equipment mechanical failure, gas leakage, and fire risks.

[0011] Furthermore, the edge-cloud collaborative processing implementation module adopts a layered collaborative processing architecture, including: Lightweight inference models are deployed at the edge, and computational complexity is reduced through deep compression technology to achieve real-time anomaly detection in harsh environments; The cloud-based system continuously optimizes decision-making strategies based on a reinforcement learning framework and dynamically adjusts detection parameters by setting a multi-dimensional reward function. A two-way parameter synchronization channel is established between the edge and the cloud. The cloud periodically distributes optimized models, while the edge uploads key scenario data to support strategy iteration. It has a built-in performance evaluation mechanism that triggers an adaptive update process when the detection performance fluctuates, and starts an edge offline mode to ensure basic inspection functions when the network is interrupted.

[0012] Furthermore, the cloud uses deep learning models to comprehensively assess the health status of equipment, generate a three-dimensional risk heat map, and establish a time-series analysis model to track the evolution trend of equipment status and predict potential failure risks.

[0013] Furthermore, the adaptive task scheduling mechanism module includes: The environmental adaptability assessment module quantifies the weight of environmental factors on task execution. The task priority dynamic mapping module adjusts the execution level and resource allocation ratio of tasks such as equipment fault detection, gas leak early warning, and fire tracking based on environmental adaptability.

[0014] Furthermore, the task priority dynamic mapping module is as follows: When the environmental adaptability is below the threshold, the execution level of the gas leak early warning task is upgraded to the highest priority; When thermal imaging temperature data is abnormal, resource preemption scheduling for fire tracking tasks is triggered.

[0015] Secondly, the technical solution adopted by this invention is as follows: a remote video inspection method for an unmanned marine platform, implemented based on the aforementioned remote video inspection system for an unmanned marine platform, comprising: synchronously acquiring the video stream temperature field and gas concentration matrix of marine facilities through a multimodal perception module; performing real-time target detection at the edge using a lightweight YOLOv8 model and outputting primary abnormal events; optimizing the detection threshold in the cloud based on historical abnormal data and reinforcement learning strategies, and sending updated model parameters to the edge; dynamically switching the data local processing or cloud backhaul path according to environmental visibility and network latency using an adaptive task scheduling mechanism; performing intelligent analysis based on the improved YOLOv8 target detection model and generating equipment status diagnosis results based on a multimodal risk identification network; and forwarding alarm videos through a streaming media service linked to an application management server and outputting equipment health reports.

[0016] Furthermore, intelligent analysis methods include: After preprocessing the multimodal data, a preliminary fused multimodal data packet is generated; the multimodal data packet includes aligned image frames, temperature matrix, and gas concentration vector. An improved YOLOv8 model is used to achieve multimodal feature fusion and target detection and recognition: a multimodal feature fusion layer is introduced after the standard YOLOv8 backbone network. This multimodal feature fusion layer fuses texture and shape features extracted from the visible light image branch, temperature distribution features extracted from the thermal imaging branch, and concentration feature vectors of gas concentration data after feature engineering or small network processing; a feature-level fusion strategy is adopted to enable the model to learn cross-modal correlations during the feature extraction stage for early warning. Deployment and execution strategy: Lightweight version models are deployed on edge nodes and optimized through pruning, quantization, and knowledge distillation techniques. They are responsible for real-time frame-level detection and output preliminary detection boxes, categories, confidence levels, and key anomaly alarms. The full-precision model is run in the cloud to verify the original data or high-confidence suspected data, perform fine-grained analysis including fault type subdivision, leakage rate estimation, and time-series correlation analysis to track the target's motion trajectory and state evolution. Deep reinforcement learning is used to optimize cloud-based decision-making and to classify and label tasks for adaptive task scheduling and execution.

[0017] The present invention has the following advantages due to the adoption of the above technical solutions: 1. This invention employs multimodal fusion for accurate identification: The improved YOLOv8 fusion network utilizes the complementarity of visible light, thermal, and gas information to significantly enhance the identification accuracy and robustness of combined risks of equipment failure, gas leakage, and fire, while reducing the impact of single sensor failure or environmental interference.

[0018] 2. This invention adopts an edge-cloud collaborative efficiency: edge processing ensures real-time response to critical alarms within seconds; cloud-based in-depth analysis and DRL optimization provide globally optimal decision-making; the collaborative mode balances real-time and accuracy requirements and reduces dependence on unstable satellite or microwave links.

[0019] 3. This invention adopts adaptive scheduling to optimize resources: dynamically allocate computing network resources, maximize the use of limited edge resources to process high-priority real-time tasks, ensure the stability and efficiency of the system under high load or network fluctuations, and extend the life of edge devices.

[0020] 4. This invention adopts intelligent closed-loop decision-making: DRL-driven strategy optimization realizes system self-learning and self-evolution, continuously improves inspection efficiency and risk identification capabilities, and forms a complete closed loop of perception, analysis, decision-making and feedback.

[0021] 5. This invention significantly reduces costs and increases efficiency: It enables unmanned automatic inspection around the clock, greatly reducing the frequency and risks of manual platform access and lowering maintenance costs; it accurately identifies risks early to prevent accidents from escalating and improves the platform's inherent safety level and operational efficiency. Attached Figure Description

[0022] Figure 1 This is a schematic diagram of the structure of the remote video inspection system based on an unmanned maritime platform in an embodiment of the present invention; Figure 2 This is a schematic diagram of the cross-modal feature fusion architecture in an embodiment of the present invention; Figure 3 This is a flowchart illustrating the use of the remote video inspection system based on an unmanned maritime platform in an embodiment of the present invention. Detailed Implementation

[0023] To address the challenges of existing systems in efficiently and synchronously acquiring and transmitting multimodal data, resulting in significant delays in cloud analysis and sluggish response to critical risk events; the insufficient reliability of traditional inspection hardware under conditions such as typhoons, salt spray, and strong electromagnetic interference, failing to guarantee continuous monitoring capabilities under extreme conditions, and the existence of blind spots in the platform's overall coverage due to the single-sensor deployment mode, it is necessary to improve the accuracy of identifying multi-dimensional risks such as equipment failures, gas leaks, and fires through algorithm improvements; and the limitations of existing technical architectures due to fluctuations in maritime communication bandwidth, environmental disturbances, and edge computing power constraints, which cannot simultaneously meet the requirements of low-latency video stream forwarding, highly robust multi-source risk perception, and accurate multi-task diagnosis, there is an urgent need to build an intelligent closed-loop inspection system that integrates "end-edge-cloud" collaboration, balancing real-time performance with decision optimization capabilities.

[0024] This invention addresses the shortcomings of existing technologies by proposing a remote video inspection system and method for unmanned offshore platforms. By integrating a multimodal perception module, an edge-cloud collaborative intelligent analysis platform, and an adaptive task scheduling mechanism, it achieves all-weather automated inspection of unmanned facilities on offshore oil and gas platforms. Key features include: constructing a multimodal data fusion network using an improved YOLOv8 target detection model to collaboratively process visible light images, thermal imaging temperature, and gas concentration monitoring data, enabling accurate identification and dynamic tracking of equipment mechanical faults, gas leaks, and fire risks; innovatively designing an edge-cloud dual-layer processing architecture, where lightweight real-time inference is performed at the edge, and the cloud optimizes decision-making strategies through deep reinforcement learning, forming a closed-loop inspection process from data acquisition and intelligent analysis to decision feedback; and dynamically allocating computing resources using an adaptive task scheduling mechanism. This invention significantly reduces reliance on manual labor and improves the real-time identification capability and operational efficiency of safety risks in harsh environments.

[0025] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. All other embodiments obtained by those skilled in the art based on the described embodiments of the present invention are within the scope of protection of the present invention.

[0026] It should be noted that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the scope of exemplary embodiments according to the invention. As used herein, the singular form is intended to include the plural form as well, unless the context clearly indicates otherwise. Furthermore, it should be understood that when the terms "comprising" and / or "including" are used in this specification, they indicate the presence of features, steps, operations, devices, components, and / or combinations thereof.

[0027] In one embodiment of the present invention, a remote video inspection system for an unmanned maritime platform is provided. In this embodiment, as... Figure 1 As shown, the system includes: a multimodal data fusion network implementation module, an edge-cloud collaborative processing implementation module, an adaptive task scheduling mechanism module, and an intelligent analysis module.

[0028] The multimodal data fusion network module is deployed in key areas of the offshore unmanned platform to acquire multimodal perception data; the key areas include the process area, wellhead area, electrical room and external transmission area.

[0029] The edge-cloud collaborative processing implementation module is deployed on-site or in nearby facilities, equipped with lightweight computing hardware such as embedded GPUs or AI accelerator cards; edge computing nodes are responsible for receiving and caching multimodal perception data; running lightweight real-time inference models optimized and distributed from the cloud for preliminary target detection, anomaly identification and alarms; and performing low-latency critical tasks.

[0030] The adaptive task scheduling mechanism module is a software middleware that spans the edge and cloud. It monitors the load status of edge nodes and cloud resources in real time, including CPU or GPU utilization, memory, bandwidth, and task queue length. Based on task attributes, it dynamically determines the task execution location: edge, cloud, or a collaborative model combining edge coarse screening and cloud fine-tuning. When network bandwidth is limited, it prioritizes the transmission of critical alarm data and model update data. Task attributes include real-time requirements, computational complexity, data volume, and risk level.

[0031] The intelligent analysis module adopts an improved YOLOv8 target detection model and a multimodal data fusion network, which integrates image recognition, thermal imaging temperature analysis, and gas concentration monitoring to achieve accurate identification and dynamic tracking of equipment anomalies and safety risks. At the same time, it integrates lightweight edge inference and cloud-based deep reinforcement learning to form a closed-loop inspection process that integrates data acquisition, intelligent analysis, and decision feedback, significantly improving the safety monitoring efficiency and intelligence level of unmanned maritime platforms.

[0032] In the above embodiments, the system also includes a cloud analysis center located in the land-based data center, which has powerful computing and storage capabilities. It is responsible for receiving raw data, preprocessed data and preliminary analysis results uploaded by edge nodes; running high-precision complex models for in-depth analysis, historical data mining and pattern learning; continuously optimizing decision-making strategies based on deep reinforcement learning (DRL) algorithms, including alarm threshold adjustment, inspection path planning and maintenance suggestion generation; and performing model training and update management, and pushing the optimized lightweight model to edge nodes.

[0033] In the above embodiments, such as Figure 2 As shown, the multimodal data fusion network includes: Visible light image coding branch is used to extract device structural features; The thermal imaging temperature coding branch is used to extract thermodynamic anomaly features; The gas concentration coding branch is used to extract leakage diffusion characteristics; The fusion decoder receives high-order feature vectors from the three branch outputs and generates a fusion risk map through cross-modal attention weight allocation.

[0034] Specifically, the visible light imaging branch uses a convolutional neural network to extract structural features of the equipment, focusing on capturing mechanical deformation and oil stains; the thermal imaging branch processes temperature field data to identify areas of abnormal temperature rise; and the gas concentration branch uses a temporal network to analyze diffusion trends. The high-order feature vectors output from each branch are input into the fusion decoder.

[0035] The fusion decoder employs a cross-modal attention mechanism to achieve feature collaboration. Visible light features serve as the query benchmark, thermal imaging features provide the key vector, and gas features constitute the value vector. Feature focusing is achieved through dynamic allocation of attention weights. This mechanism prioritizes strengthening features strongly correlated with environmental risk while suppressing interference from irrelevant information.

[0036] An improved target detection head, incorporating fusion feature inputs, generates a 3D risk map. The detection head features three dedicated output channels for mechanical faults, gas leaks, and fire risks, each adapted to the characteristics of targets at different scales. The output layer integrates spatiotemporal coordinates with risk probabilities to form equipment condition diagnostic results.

[0037] Optionally, the feature extraction stage retains the multi-scale pyramid structure to accommodate targets of different sizes; the attention mechanism employs multi-head parallel processing to improve fusion robustness; and the gas diffusion analysis introduces a sliding time window to capture the dynamic process.

[0038] In this embodiment, the visible light image encoding branch uses an explosion-proof visible light camera, the thermal imaging temperature encoding branch uses an infrared thermal imager, and the gas concentration encoding branch uses an electrochemical gas sensor. All sensors are integrated through a protective housing to achieve salt spray environment adaptability, and the electrochemical gas sensors are deployed in a distributed manner. Each sensor is connected to an edge node via an industrial bus to ensure synchronous acquisition of multi-source data.

[0039] Specifically, the multimodal data acquisition module synchronously collects equipment operating status data through a multi-source sensor network deployed on the offshore platform. High-definition visible light cameras are used to collect visual information about the equipment's appearance, instrument readings, and personnel activities; infrared thermal imagers are used to monitor abnormal temperature distributions in key equipment components such as motor bearings, valve flanges, and electrical connectors in real time, identifying potential overheating faults; a distributed array of gas concentration sensors is used to monitor the leakage concentrations of combustible methane and toxic H2S in real time; optional environmental sensors include anemometers and thermometers / hygrometers to assist in environmental condition perception; and a data preprocessing unit integrated into the sensing nodes or edge is responsible for preliminary filtering, calibration, and temporal and spatial synchronization alignment of the raw images, temperature field data, and gas concentration data. The visible light camera acquires a high-definition video stream at a fixed frame rate, the infrared thermal imager periodically scans the equipment's temperature field distribution, and the gas sensors monitor changes in hazardous gas concentrations in real time. Data from each sensor is aligned using timestamps to ensure spatiotemporal consistency of the multi-source data.

[0040] In the above embodiments, the improved YOLOv8 target detection model integrates visible light image thermal imaging temperature and gas concentration monitoring data through a multimodal data fusion network; the multimodal data fusion network adopts an attention mechanism fusion architecture to achieve dynamic tracking of equipment mechanical failure, gas leakage, and fire risks.

[0041] In the above embodiments, the edge-cloud collaborative processing implementation module adopts a layered collaborative processing architecture, including: Lightweight inference models are deployed at the edge, and computational complexity is reduced through deep compression technology to achieve real-time anomaly detection in harsh environments; The cloud-based system continuously optimizes decision-making strategies based on a reinforcement learning framework and dynamically adjusts detection parameters by setting a multi-dimensional reward function. A two-way parameter synchronization channel is established between the edge and the cloud. The cloud periodically distributes optimized models, while the edge uploads key scenario data to support strategy iteration. It has a built-in performance evaluation mechanism that triggers an adaptive update process when the detection performance fluctuates, and starts an edge offline mode to ensure basic inspection functions when the network is interrupted.

[0042] In this embodiment, the edge device employs an embedded AI processor equipped with a multi-core CPU and a high-performance GPU, enabling it to operate over a wide temperature range. The communication interface integrates a 4G and satellite dual-mode link, and incorporates vibration isolation and heat dissipation structures to adapt to harsh maritime conditions.

[0043] The lightweight analysis model runs at the edge to perform frame-by-frame inspection of the video stream. An improved target detection algorithm is used to identify abnormal equipment conditions, including features such as mechanical structural deformation and surface oil accumulation. Simultaneously, temperature field data is processed to mark areas of abnormal temperature rise and spatially matched with the video analysis results.

[0044] In this embodiment, the cloud uses a deep learning model to comprehensively assess the health status of the equipment and generate a three-dimensional risk heat map; and establishes a time series analysis model to track the evolution trend of the equipment status and predict potential failure risks.

[0045] The cloud server is deployed on a cloud platform, equipped with large-scale computing resources and GPU acceleration units, and runs a relational database cluster to store historical monitoring data. Data transmission uses an encrypted protocol, and a cross-regional disaster recovery mechanism is established to ensure system robustness.

[0046] In use, the cloud server receives preprocessed data from the edge and performs multimodal data fusion analysis. A deep learning model comprehensively assesses the health status of the equipment, generating a 3D risk heatmap. A time-series analysis model is established to track the evolution trend of the equipment's status and predict potential failure risks.

[0047] In this embodiment, the edge-cloud collaborative processing implementation module also includes an auxiliary system. The auxiliary system comprises an industrial-grade power supply system and network equipment, providing stable power support and data transmission channels for the hardware. All equipment is installed in a standard cabinet and fixed to the platform body via a load-bearing structure.

[0048] In the above embodiments, the adaptive task scheduling mechanism module includes: The environmental adaptability assessment module quantifies the weight of environmental factors on task execution. The task priority dynamic mapping module adjusts the execution level and resource allocation ratio of tasks such as equipment fault detection, gas leak early warning, and fire tracking based on environmental adaptability.

[0049] In this embodiment, the environmental adaptability assessment module enables dynamic task scheduling. This module collects multi-dimensional environmental parameters such as sea state, visibility, and wind speed in real time, generates a dynamic environmental adaptability index through weighted calculation, and establishes a quantitative correlation model between environmental status and task execution risk, providing a decision-making basis for subsequent task scheduling.

[0050] In this embodiment, the task priority dynamic mapping module is: When the environmental adaptability is below the threshold, the execution level of the gas leak early warning task is upgraded to the highest priority; When thermal imaging temperature data is abnormal, resource preemption scheduling for fire tracking tasks is triggered.

[0051] Specifically, the task priority dynamic mapping module automatically adjusts the task execution level based on environmental adaptability indicators. This module establishes a dynamic priority mapping mechanism for three core tasks: mechanical fault detection, gas leak early warning, and fire tracking. In low-adaptability scenarios, it automatically elevates gas safety monitoring to the highest priority, ensuring timely response to critical risks.

[0052] In this embodiment, the task priority dynamic mapping module further includes a resource elastic allocation module and a fault tolerance control module. Wherein: The resource elastic allocation module enables intelligent scheduling of computing resources. By monitoring the system status in real time, this module triggers a resource preemption mechanism when it detects emergency events such as sudden temperature changes. At the same time, it dynamically selects the data processing path of edge local processing or cloud backhaul analysis based on network latency, and allocates computing resources according to task priority.

[0053] The fault-tolerant control module ensures the reliability of the system under abnormal operating conditions. This module automatically activates task degradation contingency plans under extreme environmental conditions, executes a critical data caching mechanism when network communication is interrupted, and prioritizes the transmission of cached data after system recovery, ensuring the continuity of monitoring operations.

[0054] In the above embodiments, the system of the present invention further includes a decision support module and a feedback optimization module. Wherein: The decision support module automatically generates operation and maintenance suggestions based on the analysis results, including: equipment maintenance priority ranking; emergency shutdown suggestions; and preventive maintenance plans. The system displays the analysis results through a visual interface and supports remote expert consultation.

[0055] The feedback optimization module continuously collects feedback from operations and maintenance personnel to optimize the analysis model. A closed-loop learning mechanism is established to regularly update the analysis algorithms at both the edge and cloud levels, improving the system's recognition accuracy.

[0056] In the above embodiments, the target objects of the remote video inspection system for marine oil and gas unmanned platforms of the present invention cover four core categories: equipment and facility status, environmental safety risks, personnel behavior, and structural integrity. Specifically, this includes equipment and facility issues such as process pipeline leaks, structural displacement and deformation, and signs of mechanical failure; environmental safety issues such as oil spills, flammable gas leaks, and initial fires; personnel behavior issues such as lack of protective equipment and intrusion into dangerous areas; and structural integrity issues such as underwater pipeline vibration risks and cathodic protection anomalies. Based on a lightweight visual model and a multi-source feature fusion mechanism, the system achieves accurate identification of potential hazards such as equipment corrosion patches, pipeline micro-leaks, and structural deformations. Simultaneously, it combines underwater acoustic signatures and optical features to enhance small target detection capabilities. When a gas sensor or vibration monitoring network triggers an abnormal threshold, the system automatically schedules video streams from related perspectives for multimodal verification, forming a dynamic closed loop of "perception-verification-decision-execution." In terms of personnel safety supervision, the system uses behavior recognition and location data to capture violations such as not wearing protective equipment and entering high-risk areas in real time, and simultaneously generates evacuation routes. The necessity of personnel units: Although it is called an "unmanned platform," human intervention is still required for maintenance, repair, and emergency response. By integrating multimodal perception data and intelligent analysis algorithms, the system achieves accurate identification, risk assessment, and closed-loop management of the aforementioned targets, ultimately constructing a fully intelligent inspection system from anomaly detection to autonomous decision-making.

[0057] In the above embodiments, the multi-source heterogeneous data expansion of the sensing module is further included. The multimodal sensing module can further integrate underwater sonar sensors and millimeter-wave radar. The underwater sonar sensors are deployed at the platform support structure and subsea pipeline interfaces, and actively detect surface corrosion, marine organism attachment, and underwater displacement deformation through acoustic waves. Their frequency range is preferably 100kHz to 500kHz, with detection accuracy down to the centimeter level. An acoustic vibration sensor array is added to key equipment such as compressor and pump bases to collect vibration spectrum characteristics covering the 0 to 10kHz range, and identifies abnormal mechanical vibration modes through edge-end fast Fourier transform.

[0058] In the above embodiments, further, lightweight enhancements to the multimodal fusion network are also included. The improved YOLOv8 model can be optimized as follows: Neural Architecture Search (NAS) technology is introduced to automatically generate ultra-lightweight fusion network branches for edge hardware, compressing the model's computational load to less than 30% of the original structure while maintaining cross-modal correlation accuracy. An adaptive feature selection mechanism is adopted to dynamically adjust the fusion weights of visible light, thermal imaging, and gas data based on real-time environmental conditions such as light intensity and rain / fog levels. Specifically, the weights of visible light features are reduced under strong light, while the weights of thermal imaging and millimeter-wave features are enhanced in rainy / foggy conditions. An uncertainty quantification layer is added to generate confidence intervals while outputting detection results, triggering cloud-based verification when there are conflicts in the confidence levels of the multimodal evidence chain.

[0059] The above embodiments further include a federated learning mechanism that integrates edge and cloud computing. A federated learning framework is introduced during the model training phase: each edge node of the offshore platform trains a lightweight model using local data, encrypting only the model gradients before uploading them to the cloud. The cloud aggregates the gradients from multiple nodes to update the global model, which is then distributed to the edge after knowledge distillation. A differentiated privacy protection strategy is established: standard federated learning is used for ordinary equipment status data, while differential noise perturbation is added to data in sensitive areas such as wellhead areas to meet industrial data security compliance requirements.

[0060] In the above embodiments, a further step is to include digital twin-driven decision-making simulation. A digital twin of the platform is constructed in the cloud: integrating a 3D geographic information model, equipment topology relationships, and real-time sensor data streams to dynamically map the physical platform state. Deep reinforcement learning (DRL) decision-making strategies are first simulated in the digital twin environment to mimic gas diffusion paths under different wind speeds, verifying the effectiveness of the strategies before being deployed to the physical platform. Based on the simulation results, an optimal emergency response plan library is generated, including equipment shutdown sequences, evacuation route planning, and emergency resource scheduling schemes.

[0061] In the above embodiments, further, energy efficiency optimization of resource scheduling is also included. An adaptive task scheduling mechanism extends the green computing strategy: an edge device energy consumption performance model is established, and the computing mode is dynamically switched according to the platform's power supply status, such as the remaining power of the solar battery; a high-performance mode is activated when the power is high, and an ultra-low-power inference model is activated when the power is low. A tidal computing window is designed to utilize idle satellite communication bandwidth periods, such as low-rate windows at night, to batch upload non-urgent data, reducing communication costs. A task offloading game theory algorithm is implemented to form a computing resource cooperation alliance among multiple edge nodes, and high-load tasks are allocated through Nash equilibrium.

[0062] Furthermore, the above embodiments also include designs adapted to extreme environments. For environments with high salt spray and strong electromagnetic interference: the sensing device is encapsulated with a nano-hydrophobic coating to reduce lens condensation and salt crystal adhesion. The edge computing hardware utilizes conductive cooling and a sealed nitrogen-filled design, extending its operating temperature range to -40°C to +85°C. The communication link is configured with multi-path redundant transmission, automatically switching between 4G or 5G, microwave, and satellite links based on channel quality, and performing data packet fragmentation, verification, and retransmission.

[0063] In summary, as Figure 3 As shown, the working principle of the remote video inspection system for marine oil and gas unmanned platforms of the present invention is as follows: All-weather equipment monitoring is achieved using fixed high-definition cameras and various types of sensors. At the data acquisition layer, high-resolution cameras deployed in key areas of the platform (such as the process area, power module, and wellhead equipment) continuously capture the equipment's appearance, simultaneously working with infrared thermal imagers to monitor temperature field distribution, acoustic sensors to analyze mechanical vibration spectra, and gas detectors to track combustible gas leaks. All sensing devices are redundantly networked to cover the entire platform, forming a comprehensive monitoring network. A cross-validation mechanism between infrared and ultraviolet bands effectively suppresses interference such as wave reflections, and hardware synchronization between thermal imagers and visible light cameras ensures spatiotemporal data alignment, providing reliable input for multimodal analysis.

[0064] The system achieves real-time data transmission via a high-speed communication network. It employs a transmission architecture that integrates satellite links and microwave communication, coupled with local edge computing nodes on the platform to intelligently compress video streams and extract key frames, effectively reducing bandwidth requirements. An encrypted transmission protocol ensures the integrity and security of data during long-distance transmission back to the land-based control center, with communication latency strictly controlled to within 200 milliseconds.

[0065] At the data processing level, edge computing nodes perform noise reduction filtering and keyframe extraction on the raw video stream locally. A lightweight improved YOLOv8 model is used for initial screening of equipment corrosion, and an adaptive bitrate algorithm dynamically compresses the data volume—transmitting full-frame-rate video in high-bandwidth environments, and only pushing thermal imaging slices of key areas when satellite links fluctuate. The preprocessed feature data is uploaded to the cloud analysis center, where a spatiotemporal feature fusion layer correlates the spatiotemporal evolution of visible light texture, thermal imaging temperature gradient, and gas diffusion patterns. At this stage, the improved YOLOv8 multimodal network simultaneously diagnoses the equipment corrosion level, leak source coordinates, and fire risk probability. Combined with the root cause analysis engine, it traces the causal chain of "corrosion → sealing failure → leakage" to generate a quantitative risk assessment map.

[0066] like Figure 3As shown, the system maps the analysis results to the platform's digital twin model through a 3D visualization engine: corrosion areas are rendered using heatmap gradients, leakage paths are dynamically simulated by particle flow, and high-risk fire areas are marked with pulse warning boxes. Maintenance personnel can interactively operate the 3D model via a web client, clicking on equipment components to instantly retrieve real-time video streams to verify diagnostic conclusions. The system synchronously triggers dynamic resource scheduling strategies, automatically prioritizing emergency response team data streams during fire warnings.

[0067] like Figure 3 As shown, centralized management and control are ultimately achieved through a visualization platform at the land-based control center. The digital twin system integrates the real-time status of all equipment, marks anomaly points in the 3D model, and supports historical data review and trend analysis. The system has intelligent optimization capabilities, dynamically adjusting the priority of monitoring focus areas, automatically generating equipment health assessment reports, and linking with the maintenance management system to form a closed-loop management process from fault detection to repair verification. This system significantly improves the accuracy of equipment status perception and the speed of risk response, reducing the cost of manual inspections while ensuring production safety.

[0068] In one embodiment of the present invention, a remote video inspection method for a marine unmanned platform is provided, which is implemented based on the remote video inspection system for marine unmanned platforms described in the above embodiments. In this embodiment, the method includes the following steps: 1) Synchronously acquire video stream temperature field and gas concentration matrix of offshore facilities through multimodal sensing modules; 2) A lightweight YOLOv8 model is used at the edge to perform real-time object detection and output primary anomaly events; 3) The cloud optimizes the detection threshold based on historical anomaly data and reinforcement learning strategies, and sends the updated model parameters to the edge. 4) The adaptive task scheduling mechanism dynamically switches between local data processing and cloud backhaul paths based on environmental visibility and network latency. 5) Intelligent analysis is performed based on the improved YOLOv8 target detection model, and equipment status diagnosis results are generated based on the multimodal risk identification network; 6) The application management server is linked with the streaming media service to forward alarm videos and output device health reports.

[0069] In step 5) above, the intelligent analysis method includes the following steps: 5.1) After preprocessing the multimodal data, a preliminary fused multimodal data packet is generated; the multimodal data packet includes aligned image frames, temperature matrix, and gas concentration vector; Specifically, each sensor acquires data in real time; the data preprocessing unit performs spatiotemporal alignment operations, including registering visible light images and thermal images using timestamps and spatial coordinates, and associating gas sensor locations with image regions; and generates a preliminary fused multimodal data packet containing aligned image frames, temperature matrices, and gas concentration vectors.

[0070] 5.2) An improved YOLOv8 model is used to achieve multimodal feature fusion and target detection and recognition: A multimodal feature fusion layer, such as Concatenation or Attention-based Fusion, is introduced after the standard YOLOv8 backbone network; this multimodal feature fusion layer fuses texture and shape features extracted from the visible light image branch, temperature distribution features extracted from the thermal imaging branch, and concentration feature vectors of gas concentration data after feature engineering or small network processing; a feature-level fusion strategy is adopted to enable the model to learn cross-modal correlations during the feature extraction stage for early warning, such as high-temperature areas accompanied by changes in specific gas concentrations indicating leaks and fires; Specifically, the detection and identification targets include: equipment status identification such as pumps, compressors, and valve bodies; detection of mechanical fault characteristics including visible light-based detection of loose bolts, oil leaks, and misalignment, as well as thermal imaging-based detection of overheating anomalies; gas leak identification of leak sources such as flanges and welds, using a combination of visible light white haze characteristics, thermal imaging low-temperature or high-temperature plume characteristics, and gas concentration exceeding standards for judgment and dynamic tracking; fire risk identification through early identification of visible light flame characteristics, thermal imaging characteristics of high-temperature area spread, and characteristics of sudden increases in combustible gas concentration to construct a multimodal evidence chain for early and accurate alarm; and support for risk identification of personnel intrusion, marine organism attachment, and structural corrosion.

[0071] 5.3) Deployment and execution strategy: The lightweight version of the model is deployed on the edge nodes and optimized through pruning, quantization and knowledge distillation techniques. It is responsible for real-time frame-level detection and outputs preliminary detection boxes, categories, confidence levels and key anomaly alarms. The full-precision model is run in the cloud to verify the original data or high-confidence suspected data, and perform fine-grained analysis, including fault type subdivision, leakage rate estimation and time series correlation analysis to track the target's movement trajectory and state evolution.

[0072] 5.4) Optimize cloud-based decision-making based on deep reinforcement learning, and perform task classification and labeling to enable adaptive task scheduling and execution.

[0073] Specifically, cloud-based decision-making is optimized based on deep reinforcement learning, including the construction of a DRL model: the agent observes the state, including historical and current detection results (i.e., the number, location, and confidence of various anomalies), environmental parameters (i.e., wind speed and direction), platform operating status (i.e., start / stop load), and resource status (i.e., edge load and bandwidth); defines actions, including adjusting the edge model detection frequency, monitoring priority for specific areas, alarm thresholds (i.e., temperature and concentration), pushing new model versions to the edge, generating maintenance work order suggestions, and optimizing virtual inspection paths; designs a reward function with the overall system performance as the goal, rewarding accurate and timely identification of high-risk events, reducing false alarms and false negatives, and optimizing resource utilization (including reducing edge energy consumption and balancing load, and shortening response time), while penalizing missed critical alarms, resource overload, and frequent invalid alarms; the cloud continuously trains the DRL agent using massive historical data and online learning, employing PPO or DQN algorithms, and transforms the optimized decision-making strategy into parameter commands that are dynamically distributed to edge nodes and system management modules to form a closed-loop feedback.

[0074] Task Classification and Labeling: Predefined system tasks include real-time video stream analysis, periodic thermal imaging scanning, gas data batch processing, model inference, model updates, and alarm uploads. Key labeled attributes include latency sensitivity, computational intensity, data volume, and priority. Resource Monitoring: Real-time collection of edge node computing resource, storage, network bandwidth status, and cloud queue deep processing latency status. Dynamic Scheduling Decisions: The scheduler applies preset rules or load balancing algorithms based on task attributes and resource status. Execution strategies include: forced edge processing for high real-time, medium-computation tasks such as flame detection; pushing high-computation, non-real-time tasks such as historical data trend analysis to the cloud for processing; employing a collaborative mode of edge preprocessing to locate ROI and cloud-based fine analysis of ROI regions for high-computation, high-real-time tasks such as precise leak localization in complex scenarios; proactively uploading raw data or edge cache data to the cloud for backup and deep mining when the network is idle; reducing the frequency of non-critical tasks such as low-risk area inspections or delaying low-priority batch tasks when edge resources are scarce; prioritizing bandwidth resource allocation to push new models to the edge when the cloud detects a significant improvement in model performance.

[0075] The system forms a closed-loop inspection process: after real-time collection of sensing data, the edge lightweight real-time analysis generates an initial alarm; key or suspicious data is uploaded to the cloud for in-depth analysis and DRL decision optimization; the optimized strategy model or instructions are issued to the edge nodes; the edge nodes execute the new strategy or model update and start a new round of sensing analysis; the adaptive scheduling mechanism coordinates the data flow, task flow and resource allocation throughout the process to ensure efficient and stable operation.

[0076] The remote video inspection system for offshore oil and gas unmanned platforms targets four core categories: equipment and facility status, environmental safety risks, personnel behavior, and structural integrity. Specifically, this includes equipment and facility issues such as process pipeline leaks, structural displacement and deformation, and signs of mechanical failure; environmental safety issues such as oil spills, flammable gas leaks, and initial fires; personnel behavior issues such as lack of protective equipment and intrusion into dangerous areas; and structural integrity issues such as underwater pipeline vibration risks and cathodic protection anomalies. Based on a lightweight visual model and a multi-source feature fusion mechanism, the system accurately identifies potential hazards such as equipment corrosion patches, pipeline micro-leaks, and structural deformations. It also combines underwater acoustic signatures and optical features to enhance small target detection capabilities. When a gas sensor or vibration monitoring network triggers an anomaly threshold, the system automatically schedules related video streams for multimodal verification, forming a dynamic closed loop of "perception-verification-decision-execution." In terms of personnel safety supervision, the system uses behavioral recognition and location data to capture violations such as not wearing protective equipment and intrusion into high-risk areas in real time, simultaneously generating evacuation routes. The necessity of personnel units: Although it is called an "unmanned platform," personnel intervention is still required for maintenance, repair, and emergency response scenarios. By integrating multimodal sensing data with intelligent analysis algorithms, the system achieves accurate identification, risk assessment, and closed-loop management of the aforementioned targets, ultimately constructing a fully intelligent inspection system from anomaly detection to autonomous decision-making.

[0077] The method provided in this embodiment is based on the above system embodiments. For specific processes and details, please refer to the above embodiments, which will not be repeated here.

[0078] In summary, this invention constructs a multimodal collaborative perception and intelligent analysis closed-loop system to achieve all-weather automated inspection; improves the accuracy of identifying multi-dimensional risks such as equipment failure, gas leakage, and fire by refining algorithms; achieves 3D mapping, enabling the display of alarm information in a 3D model; and designs an edge-cloud dynamic collaborative mechanism that balances real-time performance with decision optimization capabilities. This system, through a three-in-one architecture of "multimodal perception - edge-cloud collaboration - closed-loop decision-making," overcomes the challenges of "unclear visibility, inability to transmit data, and inaccurate judgment" on offshore platforms, providing highly reliable intelligent protection for deep-sea energy facilities, enabling all-weather intelligent inspection of unmanned offshore platforms, and effectively reducing the missed detection rate and the cost of manual intervention.

[0079] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A remote video inspection system for an unmanned maritime platform, characterized in that, include: The multimodal data fusion network module is deployed in key areas of the unmanned maritime platform to acquire multimodal perception data; Key areas include the process area, wellhead area, electrical room, and external transmission area; The edge-cloud collaborative processing implementation module is deployed on-site or in nearby facilities and equipped with embedded lightweight computing hardware; the edge computing nodes are responsible for receiving and caching multimodal perception data; running a lightweight real-time inference model optimized and distributed from the cloud to perform preliminary target detection, anomaly identification and alarms; and executing low-latency critical tasks. The adaptive task scheduling mechanism module is a software middleware used to monitor the load status of edge nodes and cloud resources in real time; it makes dynamic decisions based on task attributes to determine whether the task execution location is at the edge, in the cloud, or in a collaborative mode of edge coarse screening plus cloud fine judgment; and it prioritizes the transmission of critical alarm data and model update data when network bandwidth is limited. The intelligent analysis module adopts an improved YOLOv8 target detection model and a multimodal data fusion network, which integrates image recognition, thermal imaging temperature analysis, and gas concentration monitoring to achieve accurate identification and dynamic tracking of equipment anomalies and safety risks. At the same time, it integrates lightweight edge inference and cloud-based deep reinforcement learning to form a closed-loop inspection process that integrates data collection, intelligent analysis, and decision feedback.

2. The remote video inspection system for unmanned maritime platforms as described in claim 1, characterized in that, Multimodal data fusion networks include: Visible light image coding branch is used to extract device structural features; The thermal imaging temperature coding branch is used to extract thermodynamic anomaly features; The gas concentration coding branch is used to extract leakage diffusion characteristics; The fusion decoder receives high-order feature vectors from the three branch outputs and generates a fusion risk map through cross-modal attention weight allocation.

3. The remote video inspection system for unmanned maritime platforms as described in claim 2, characterized in that, The visible light image coding branch uses an explosion-proof visible light camera, the thermal imaging temperature coding branch uses an infrared thermal imager, and the gas concentration coding branch uses an electrochemical gas sensor. All sensors are integrated through a protective housing to achieve salt spray environment adaptability, and the electrochemical gas sensors are deployed in a distributed manner. Each sensor is connected to the edge node via an industrial bus.

4. The remote video inspection system for unmanned maritime platforms as described in claim 1, characterized in that, The improved YOLOv8 target detection model integrates visible light image thermal imaging temperature and gas concentration monitoring data through a multimodal data fusion network. The multimodal data fusion network adopts an attention mechanism fusion architecture to achieve dynamic tracking of equipment mechanical failure, gas leakage, and fire risks.

5. The remote video inspection system for unmanned maritime platforms as described in claim 1, characterized in that, The edge-cloud collaborative processing implementation module adopts a layered collaborative processing architecture, including: Lightweight inference models are deployed at the edge, and computational complexity is reduced through deep compression technology to achieve real-time anomaly detection in harsh environments; The cloud-based system continuously optimizes decision-making strategies based on a reinforcement learning framework and dynamically adjusts detection parameters by setting a multi-dimensional reward function. A two-way parameter synchronization channel is established between the edge and the cloud. The cloud periodically distributes optimized models, while the edge uploads key scenario data to support strategy iteration. It has a built-in performance evaluation mechanism that triggers an adaptive update process when the detection performance fluctuates, and starts an edge offline mode to ensure basic inspection functions when the network is interrupted.

6. The remote video inspection system for unmanned maritime platforms as described in claim 5, characterized in that, The cloud-based system uses deep learning models to comprehensively assess the health status of equipment, generating a 3D risk heat map; and establishes a time-series analysis model to track the evolution trend of equipment status and predict potential failure risks.

7. The remote video inspection system for unmanned maritime platforms as described in claim 1, characterized in that, The adaptive task scheduling mechanism module includes: The environmental adaptability assessment module quantifies the weight of environmental factors on task execution. The task priority dynamic mapping module adjusts the execution level and resource allocation ratio of equipment fault detection, gas leak early warning, and fire tracking tasks according to environmental adaptability.

8. The remote video inspection system for unmanned maritime platforms as described in claim 7, characterized in that, The task priority dynamic mapping module is as follows: When the environmental adaptability is below the threshold, the execution level of the gas leak early warning task is upgraded to the highest priority; When thermal imaging temperature data is abnormal, resource preemption scheduling for fire tracking tasks is triggered.

9. A method for remote video inspection of an unmanned maritime platform, implemented based on the remote video inspection system for unmanned maritime platforms as described in any one of claims 1 to 8, characterized in that, include: The video stream temperature field and gas concentration matrix of the offshore facilities are collected synchronously through a multimodal sensing module. At the edge, a lightweight YOLOv8 model is used to perform real-time object detection and output primary anomaly events. The cloud-based system optimizes the detection threshold based on historical anomaly data and reinforcement learning strategies, and then sends the updated model parameters to the edge. The adaptive task scheduling mechanism dynamically switches between local data processing and cloud data transmission paths based on environmental visibility and network latency. Intelligent analysis is performed based on the improved YOLOv8 target detection model, and device status diagnosis results are generated based on the multimodal risk identification network. The application management server is linked with the streaming media service to forward alarm videos and output device health reports.

10. The remote video inspection method for an unmanned maritime platform as described in claim 9, characterized in that, Intelligent analysis methods include: After preprocessing the multimodal data, a preliminary fused multimodal data packet is generated; the multimodal data packet includes aligned image frames, temperature matrix, and gas concentration vector. An improved YOLOv8 model is used to achieve multimodal feature fusion and target detection and recognition: a multimodal feature fusion layer is introduced after the standard YOLOv8 backbone network. This multimodal feature fusion layer fuses texture and shape features extracted from the visible light image branch, temperature distribution features extracted from the thermal imaging branch, and concentration feature vectors of gas concentration data after feature engineering or small network processing; a feature-level fusion strategy is adopted to enable the model to learn cross-modal correlations during the feature extraction stage for early warning. Deployment and execution strategy: Lightweight version models are deployed on edge nodes and optimized through pruning, quantization, and knowledge distillation techniques. They are responsible for real-time frame-level detection and output preliminary detection boxes, categories, confidence levels, and key anomaly alarms. The full-precision model is run in the cloud to verify the original data or high-confidence suspected data, perform fine-grained analysis including fault type subdivision, leakage rate estimation, and time-series correlation analysis to track the target's motion trajectory and state evolution. Deep reinforcement learning is used to optimize cloud-based decision-making and to classify and label tasks for adaptive task scheduling and execution.

Images