Offshore platform anti-collision device and method based on multi-mode BEV perception

Abstract

The invention discloses an offshore platform anti-collision device and method based on multi-mode BEV perception, and the device comprises a three-dimensional fusion distance measurement module, a UWB distance measurement module, a ship navigation state monitoring module, a platform marine environment monitoring module, and a central processing unit. The three-dimensional fusion ranging module comprises a holder, a laser radar and a camera, and is used for performing 360-degree three-dimensional scanning on the periphery of the offshore platform; the UWB distance measuring modules are installed on the periphery of a ship and pile legs of an offshore platform. The ship navigation state monitoring module is used for acquiring ship position, navigational speed and course information in real time; the platform marine environment monitoring module collects wind wave data and weather data near the offshore platform and route information of passing ships through network communication. The invention discloses an offshore platform anti-collision device with real-time monitoring and collision risk assessment functions, the anti-collision safety of an offshore platform is improved, and the occurrence rate of collision accidents is reduced.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of intelligent monitoring and early warning, and in particular to a marine platform anti-collision device and method based on multi-modal BEV perception. BACKGROUND

[0002] In order to develop marine economy and protect marine ecological environment, it is necessary to further focus on promoting high-quality development of marine economy, accelerating construction of marine ecological civilization, and improving the level of marine scientific and technological innovation, so as to ensure the implementation of the strategy of building a strong marine country and promote the development of energy security. In the past two decades, most of the existing ship damage incidents have occurred due to collisions between ships and operating facilities or other ships. With the increase in the amount of marine ship operations and the alternation of key personnel on ships, the frequency of collision accidents has increased significantly in recent years. These accidents not only cause economic losses and environmental damage, but also threaten the life and property safety of offshore oilfield production.

[0003] The current early warning detection system on the ship mainly relies on long and short wave radars to scan the sea surface. The long and short wave radars mainly detect targets at a long distance in all-weather conditions, generate 2D planar image displays of target positions through scanning, and are suitable for long-distance obstacle warning and weather perception on the sea. However, this method has limited ability to identify small targets, and cannot provide intuitive reference during berthing operations. Key data is not highlighted. At the same time, due to the fact that the offshore platform cannot be as flexible as a ship to navigate and avoid collisions independently, collision avoidance can only be achieved by warning the approaching collision object when the collision risk occurs. SUMMARY

[0004] The purpose of the present application is to provide a marine platform anti-collision device and method based on multi-modal BEV perception to solve the problems raised in the background.

[0005] To achieve the above purpose, the present application provides the following technical solutions: The offshore platform anti-collision device based on multi-modal BEV perception comprises a three-dimensional fusion ranging module, a UWB ranging module, a ship navigation state monitoring module, a platform marine environment monitoring module and a central processor, the three-dimensional fusion ranging module is installed on the opposite corners of the offshore platform, the three-dimensional fusion ranging module comprises a holder, a laser radar and a camera, the laser radar and the camera are arranged on the holder, and the three-dimensional fusion ranging module is used for 360° three-dimensional scanning around the offshore platform; the UWB ranging module is installed at the ship circumferential and the pile leg of the offshore platform, and is used for acquiring the three-dimensional space physical position between the ship and the pile leg of the offshore platform in real time; the ship navigation state monitoring module is used for acquiring the ship position, speed and heading information in real time; the platform marine environment monitoring module collects the wind and wave data, weather data and passing ship route information near the offshore platform through network communication, and the three-dimensional fusion ranging module, the UWB ranging module, the ship navigation state monitoring module and the platform marine environment monitoring module are respectively in communication connection with the central processor.

[0006] Optionally, a safety airbag is further included, the safety airbag is installed on the side of the offshore platform, and the safety airbag is made of seawater-degradable material.

[0007] A method using the offshore platform anti-collision device based on multi-modal BEV perception comprises the following steps: S100, device deployment; S200, the data acquired by the three-dimensional fusion ranging module, the UWB ranging module, the ship navigation state monitoring module and the platform marine environment monitoring module are fused and calculated using the BEV perception technology, the camera image data and the laser radar point cloud data in the three-dimensional fusion ranging module and the UWB ranging data are converted into a coordinate system with the offshore platform as the center, time and space are aligned, integrated into a common bird's eye view representation space above the offshore platform, the detection data are synchronized to an intelligent perception background system, the distance calculation frequency and the early warning strength are dynamically adjusted, and an offshore platform operation knowledge base is constructed; S300, the camera image of the cyclic scanning is subjected to distortion correction, is converted into a bird's eye view of the offshore platform through a rephotographing transformation, is finally spliced and subjected to smoothing processing to obtain a top view of the offshore platform, a dynamic electronic fence of the offshore platform region is established, data with a high risk level are preferentially displayed on a front-end display page, point cloud and image features are extracted using the BEV perception technology, and staff on the offshore platform and the ship close to the offshore platform view the early warning and ask questions to the offshore platform operation knowledge base through a voice module of the perception system; S400, when the collision object still approaches the offshore platform after the early warning, the safety airbag of a specified region is automatically popped up according to the distance danger coefficient calculated by the BEV system; S500, a platform side collision object database is established by repeatedly scanning the target photographed by the three-dimensional fusion ranging module, weather data, wave data, collision object approaching time, moving away time, retention time, object position, collision object type, collision object image, collision object point cloud, UWB distance data and collision object three-dimensional model are recorded and statistically analyzed to construct a collision risk time and space distribution matrix, on this basis, personnel inspection work is adjusted, a marine platform-ship communication and risk early warning integrated platform is established, when the ship approaches within the platform visual distance, it is automatically associated with the platform equipment, automatic distance calculation warning and display of the digital twin picture consistent with the real world are carried out, and the data is pushed to the platform side and the ship side synchronously for two-way display and early warning.

[0008] Optionally, step S200 specifically comprises: Step S210: Establish a background data fusion system, open different ports and establish a connection with the hardware devices available on the offshore platform to collect data, obtain weather conditions, wave data, heading information of other ships within a 500-meter range of the offshore platform, calculate the dangerous weight coefficient of each weather and wave coefficient according to the number of risks occurring under different weather and wave conditions in history, and obtain the risk level weighting sum of the natural environment, and establish a cruise frequency mapping table for the weighted sum to automatically adjust the collection and fusion calculation frequency of each sensor device; Step S220: Perform three-dimensional target detection while mapping point cloud data to image data; before the offshore platform anti-collision system is formally put into use, the collected point cloud data and image data are sorted, labeled and audited, the image data is labeled by instance segmentation, and the point cloud data is labeled by segmentation, the same instance is labeled with the same class and ID during image and point cloud labeling, the fusion features of two-dimensional and three-dimensional are extracted, the objects detected in the image vision and three-dimensional point cloud are projected onto a unified plane centered on the platform according to the actual position relative to the offshore platform, and a group of BEV queries based on Transformer is designed to realize view transformation by combining the cross attention between position encoding and image features to map the perspective view of the image and point cloud to the top view; Step S230: When the pan-tilt rotates, the background perception system calculates the rotation and translation transformation matrix of the rotated laser radar point cloud and camera image coordinate system and the platform bird's eye view coordinate system respectively, applies the transformation matrix to the point cloud and image data to map the data to the platform bird's eye view coordinate system, calculates the three-dimensional perception coverage at this time, wakes up the ranging function of the UWB module in the uncovered area, and automatically pushes the UWB ranging result to the background perception system; Step S240: The multi-modal features collected by the camera, laser radar, UWB module and other hardware devices are fused through the BEV space feature, and the multi-modal feature data in each dimension of time and space is mapped to the BEV system. The BEV system aligns the multi-dimensional data in time and space according to the position of the sensor and the timestamp carried by each sensor data, and extracts and fuses the features. According to the object historical motion data in the bird's eye view, the trajectory tracking and prediction are carried out, and the trajectory of the surrounding ships is planned from the perspective of the offshore platform. The planned driving path is presented in the form of a bird's eye view in the BEV system and pushed to the passing ships. Step S250: The background perception system establishes a risk weight fusion matrix based on the received three-dimensional fusion ranging module data, UWB ranging data, weather data, wind wave coefficient, other ship routes, risk level results, and the operation and feedback of the platform site personnel to the perception system. A risk weight fusion matrix is established, and an offshore platform operation knowledge base is constructed. The weight of different data on the platform risk is calculated and updated in real time, and the distance calculation frequency and warning intensity are dynamically adjusted.

[0009] Optionally, step S300 specifically includes: Step S310: The overlapping regions in each adjacent frame image region during the rotation of the holder are fused and projected using a projection matrix, and the overlapping regions in the image are extracted for gray processing and threshold setting for binarization. Noise points are removed using Gaussian filtering to obtain an overlapping region mask image. The overlapping polygon contour is detected, and the color of the image in the region is adjusted to achieve a smooth splicing effect. Step S320: The distance of the object to the platform is calculated, and the hierarchical electronic fence parameters are established according to the distance. The platform is divided into three levels of control zones, and the warning zone triggers an audible and light alarm and pushes AIS warning information. The intervention zone automatically sends collision avoidance instructions to the ship terminal. The emergency zone links the platform emergency system, shuts down, and evacuates personnel. The fence dynamically adjusts itself according to real-time sea conditions and ship types, and uses historical collision data to train a risk prediction model to optimize the fence threshold. Step S330: The camera video installed on the offshore platform recognizes the actions of the crew, evaluates the captain's berthing ability in combination with AIS data, objectively evaluates the driving of the offshore ship from the perspective of the platform, and pushes the risk report to the management platform in real time. Step S340: The BEV system establishes a fixed-length queue for storing the current frame and the multi-frame multi-sensor data of the recent history. Whenever the system newly collects a frame of real-time data, the BEV system automatically deletes the oldest frame of fused data in the queue, and adds the new multi-sensor data to the multi-modal fused data in the BEV space for feature fusion and extraction. The real-time image, distance, and collision indication are dynamically displayed in the panoramic overhead view, and the fluctuation of the historical data is displayed to help the offshore workers understand the trend of the platform surrounding environment; Step S350: The worker asks questions to the offshore platform operation knowledge base through the voice module of the perception system.

[0010] Optionally, step S500 specifically comprises: Step S510: For the standardization storage of the laser radar, camera, and UWP multi-modal detection data, a MySQL database table structure is designed, and a collision risk matrix is constructed. Step S520: A multi-protocol data fusion transmission is constructed, and weather data, wave data, collision object approaching time, away time, retention time, object direction, collision object type, collision object image, collision object point cloud, UWB distance data, and collision object three-dimensional model are input to the background database through network communication for statistical analysis, and a collision risk time and space distribution matrix is constructed, on the basis of which the personnel inspection work is adjusted. Step S530: A two-way collision warning mechanism and closed-loop management system are established, and the platform end and the ship end display the electronic fence state synchronously, the ship's bridge integrates a head-up display, the obstacle direction and collision avoidance suggestions are superimposed in real time, the effectiveness index of collision avoidance is defined, the system reliability is verified through Monte Carlo simulation and real ship test, and a plan-execution-check-treatment cycle report is generated. Step S540: Two-way distance detection and collision warning are realized, the platform equipment is automatically associated when the ship approaches within the platform's field of view distance, distance calculation and warning are automatically performed, the digital twin picture consistent with the real world is displayed, the data is synchronously pushed to the platform side and the ship side, and the data is pushed to the land management platform.

[0011] In summary, the technical effects and advantages of the present application are as follows: (1) multi-modal perception fusion: laser radar (millimeter level precision) and video (semantic information) complement each other, solving the problem of missing small targets (such as buoys) in traditional radar; (2) construction of an offshore platform operation knowledge base, which can directly ask the platform surrounding situation by voice; (3) dynamic intelligent electronic fence: combined with the environment and ship behavior, self-adaptive adjustment reduces false alarms by 70%; (4) risk prediction data-driven: preventive maintenance based on time and space matrix, improving the inspection efficiency by more than 40%. BRIEF DESCRIPTION OF DRAWINGS

[0012] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiments or prior art description. Obviously, the drawings in the following description only constitute some embodiments of the present application, and for those skilled in the art, other drawings can also be obtained from these drawings without creative labor.

[0013] Figure 1 Structure diagram of a three-dimensional fusion ranging module in an embodiment of the present application; Figure 2 Installation diagram of a three-dimensional fusion ranging module in an embodiment of the present application; Figure 3 Flow chart of UWB signal processing in an embodiment of the present application; Figure 4 Division diagram of three-level control zones of a three-dimensional electronic fence in an embodiment of the present application; Figure 5 System topology diagram of a collision avoidance system in an embodiment of the present application.

[0014] In the figure, 1 is a gimbal, 2 is a laser radar, and 3 is a camera. DETAILED DESCRIPTION

[0015] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments only constitute some of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of the present application.

[0016] In addition, the technical features involved in different embodiments of the present application described below can be combined with each other as long as there is no conflict.

[0017] To solve the current situation of ship collision at sea, enhance the ability of ship safety management technology means, and analyze the causes of ship collision in depth, this embodiment proposes to deploy corresponding technical systems from the ship side (improve personnel behavior management, enhance equipment assisted driving) and the sea platform end to break through data interaction, and realize the construction of ship anti-collision early warning system. By deploying three-dimensional fusion ranging modules based on three-dimensional laser radar and video image fusion, UWB ranging equipment, weather, and wave perception sensors around the sea platform, real-time three-dimensional scanning and visualization of the platform perimeter are realized, a multi-sensor BEV fusion perception system of the sea platform is established, a safety airbag is added on the side of the platform to automatically pop up the safety airbag according to the BEV system danger level when encountering an unavoidable collision scene, and at the same time, data is synchronized to the berthing ship bridge through wireless transmission.

[0018] This embodiment proposes a collision avoidance device for offshore platforms based on multimodal BEV perception, including a 3D fusion ranging module, a UWB ranging module, a ship navigation status monitoring module, a platform marine environment monitoring module, and a central processing unit, establishing a comprehensive multimodal fusion intelligent perception system for the ship and offshore platform. The 3D fusion ranging module is installed diagonally on the offshore platform, such as... Figure 1 As shown, the 3D fusion ranging module includes a gimbal, a lidar, and a camera. The lidar and camera are mounted on the gimbal. By controlling the movement of the gimbal, the 3D fusion ranging module can perform a 360° 3D scan around the offshore platform. The UWB ranging module is installed around the ship and at the platform's legs. The UWB ranging module is used to acquire the 3D spatial physical position between the ship and the platform's legs in real time. When the gimbal rotates, the UWB ranging module can also cover areas not scanned. It determines the distance between the ship and the platform based on the flight time of the pulse signals from both ends of the device, further enhancing protection. The ship navigation status monitoring module is used to acquire the ship's position, speed, and heading information in real time. The ship navigation status monitoring module consists of Beidou positioning sensors, inertial navigation sensors, etc. The platform marine environment monitoring module collects wind and wave data, weather data, and passing ship route information near the offshore platform through network communication. The 3D fusion ranging module, UWB ranging module, ship navigation status monitoring module, and platform marine environment monitoring module are all connected to the central processing unit.

[0019] Optionally, it also includes airbags installed around the perimeter of the offshore platform, the airbags being made of seawater-degradable materials.

[0020] This embodiment also proposes a method for a collision avoidance device for offshore platforms based on multimodal BEV perception, including: Step S100: Equipment Deployment; The device consists of a 3D fusion ranging module, a UWB ranging module, a ship navigation status monitoring module, and a platform marine environment monitoring module, establishing a comprehensive multimodal fusion intelligent sensing system for the ship and offshore platform. The 3D fusion ranging module consists of a gimbal, lidar, and a camera. The camera has a high-definition lens and is installed diagonally on the offshore platform. By controlling the movement of the intelligent gimbal, it performs a 360° 3D scan around the platform. The UWB ranging module consists of multiple low-power UWB sensors, installed around the ship and at the platform's legs, to acquire the 3D spatial physical position between the ship and the platform's legs in real time. When the gimbal rotates, it can also cover areas not scanned, further enhancing protection. The ship navigation status monitoring module consists of BeiDou positioning sensors, inertial navigation sensors, etc., to acquire information such as the ship's position, speed, and heading in real time. The platform marine environment monitoring module collects wind and wave data, weather data, and information on possible passing ship routes near the offshore platform through network communication.

[0021] Step S110: design a three-dimensional fusion ranging module integrating an intelligent gimbal, a laser radar, and a camera as shown in Figure 1 The laser point cloud acquisition sensor is combined with the visible light camera to develop a special three-dimensional fusion ranging module for offshore platform collision prevention. The integrated perception hardware has the characteristics of corrosion resistance, shock absorption, waterproofness, and heat dissipation. The device is carried on a two-axis gimbal. The device main body is composed of a main shell, a waterproof panel, a silica gel sealing ring, a rain shield, and a vision assembly. The vision assembly includes a laser radar and a camera. The vision assembly is fixed on the pressure rivet nut in the main shell by screws. The silica gel sealing ring is placed between the main shell and the waterproof panel, and the waterproof and dustproof effect is achieved by the pressure of screw fastening. The rain shield is installed above the main shell and extends at the front end to achieve the rain-shielding effect on the visual surface. The three-dimensional fusion ranging module is installed at the upper left corner and the lower right corner of the offshore platform as shown in Figure 2 In order to maximize the use of laser radar point cloud data and network camera image data, the radar and camera installed on the same side are calibrated and calibrated, and the data collected are pre-fused. On the basis of not discarding effective perception data, feature superposition is performed. The fusion and calculation process is accelerated by CUDA operator to ensure the accuracy and real-time performance of distance calculation. Step S120: the central processing unit receives the point cloud and image data of the three-dimensional fusion ranging module installed at the two corner points of the platform. Time synchronization and spatial synchronization are performed according to the time stamp, device id, and other information of each frame of data. The transformation matrix of adjacent frame point clouds is calculated by the icp (Iterative Closest Point) algorithm. The adjacent frame point clouds collected during the gimbal rotation are three-dimensionally spliced. The three-dimensional point cloud transformation matrix is reduced in dimension and applied to the adjacent image frames collected by the camera for splicing. The spliced point cloud and image data are pre-fused, and the relatively sparse point cloud is mapped to the image data with rich texture information. Step S130: investigate the offshore platform in the field. The time and specific location of the collision accident in the historical berthing data are analyzed and analyzed for the specific offshore platform. The UWB ranging module is installed in the accident-prone corner point area of the platform. After the UWB ranging module is calibrated, the UWB ranging data is collected in real time in the background database for monitoring the risk situation near the area that cannot be scanned during the gimbal rotation, further strengthening the detection of the risk area, and reducing the probability of risk omission. Step S140: optimizing the device deployment scheme, selecting a diagonal deployment of two sets of three-dimensional fusion ranging modules (device spacing ≥ 80% of the length of the platform diagonal), adjusting the pitch angle (15°-30°) and horizontal deflection angle (±45°) to ensure 360° coverage, and verifying the point cloud density (≥100 points / ㎡) and video overlap rate (≥20%) to achieve three-dimensional protection. When no other objects are detected around the platform, the reference origin is rotated clockwise by 180° back to the origin every minute, and then counterclockwise by 180° back to the origin, and the two angle points are scanned simultaneously. The scanned images and point cloud data are sent to the central processor for processing. Secondly, when the three-dimensional fusion ranging module detects a collision object, the intelligent gimbal will automatically adjust the gimbal rotation rate and angle according to the algorithm to continuously track the object that may have a collision risk. At the same time, in order to reduce system power consumption and reduce data redundancy, the background system will modify the detection frequency of different devices according to the current risk level (for example, when the three-dimensional fusion ranging module is rotating with the gimbal, the UWB ranging module in the area scanned by the three-dimensional fusion ranging module will be in a dormant state, and when the gimbal turns away from the area, the UWB ranging module will automatically wake up); Step S200, using BEV (Bird's Eye View, bird's eye view) perception technology to fuse and calculate the data obtained by the three-dimensional fusion ranging module, UWB ranging module, ship navigation state monitoring module, and platform ocean environment monitoring module, convert the multi-source data such as camera image data, laser radar point cloud data, and UWB ranging data in the three-dimensional fusion ranging module into a coordinate system centered on the offshore platform, perform time and space alignment, and integrate into a bird's eye view representation space above the offshore platform. The detection data is synchronized to the background perception system. Input all the information obtained each time into the offshore platform sea state model, establish an offshore platform operation knowledge base, dynamically adjust the distance calculation frequency and warning intensity, and realize multiple safeguards for the safety of the offshore platform. In this embodiment, the background system is all the logical processing parts in the entire project except the front-end display page. The background system includes a background perception system, a background data fusion system, and a BEV system. The background perception system is responsible for collecting multi-source data such as camera image data, laser radar point cloud data, and UWB ranging data in the three-dimensional fusion ranging module. The background data fusion system is responsible for calculating and fusing the data collected by the background perception system. The BEV system is mainly responsible for extracting and fusing the features of the data calculated and fused by the background data fusion system according to the BEV bird's eye view standard, and displaying the corresponding information to the bird's eye view page on the front end.

[0022] Step S210: Establish a background data fusion system, open different ports and establish connections with the available hardware devices on the offshore platform to collect data, such as through the offshore weather information center, wave measuring sensors, and sea navigation ship route data to obtain weather conditions, wave data, and other ship heading information within a 500-meter range of the offshore platform. According to the number of times of risks occurring under different weather and wave conditions, the dangerous weight coefficients of each weather and wave coefficient are calculated, and the risk level weighting sum of the natural environment is obtained. A cruise frequency mapping table is established for the weighted sum, and the collection and fusion calculation frequency of each sensor device is automatically adjusted; Step S220: The three-dimensional fusion ranging module has completed calibration and calibration in step S100. In order to achieve more accurate fusion calculation effect, while mapping the point cloud data to the image, three-dimensional target detection is performed using OpenPCDNect and mmdetection3d open source frameworks. Before the offshore platform anti-collision system is formally put into use, the point cloud and image data collected by the three-dimensional fusion ranging module at sea are sorted, labeled and audited. xanylabeling is used for instance segmentation labeling of image data, and semantic-segmentation-editor is used for segmentation labeling of point cloud data. The same instance is labeled with the same class and ID when labeling the image and point cloud. The accuracy of offshore object recognition is further improved through two-dimensional and three-dimensional fusion feature extraction. The objects detected in the image vision and three-dimensional point cloud are projected to a unified plane centered on the platform according to the actual position relative to the offshore platform, eliminating the differences between laser radar and camera perspective and image data and point cloud data formats. Based on the three-dimensional coordinate information displayed by the laser radar point cloud and the rich texture information of the camera image, the precise three-dimensional environment around the offshore platform is obtained in the form of an overhead view. By designing a group of BEV queries (based on sparse queries) based on Transformer, cross-attention between position encoding and image features is realized to realize view transformation to map the perspective view of the image and point cloud to the overhead view. Due to the addition of point cloud coordinate information in the image data, the spatial relationship perception ability of the Transformer network is further enhanced, achieving more accurate and rapid integration of multi-source heterogeneous sensor data into a unified overhead view. Step S230: When the three-dimensional fusion ranging module rotates with the gimbal, the position of the acquired laser radar point cloud data and camera image data relative to the coordinate system of the offshore platform also changes. The BEV algorithm determines a unified coordinate system reference (hereinafter referred to as the platform bird's eye view coordinate system) centered on the offshore platform by designing a global and unified bird's eye view perspective. The background perception system automatically acquires the timestamp and angle of the gimbal rotation. According to the offshore platform CAD drawing, the gimbal installation position, the scanning range of the intelligent three-dimensional perception device, and the gimbal angle value, the rotation translation transformation matrix of the rotated laser radar point cloud and camera image coordinate system relative to the platform bird's eye view coordinate system is calculated. The transformation matrix is applied to the point cloud and image data to map the data to the platform bird's eye view coordinate system. The three-dimensional perception coverage range at this time is calculated. The ranging function of the UWB module in the uncovered area is awakened. The UWB device transmits data through pulse signals. The received signals are adjusted and decoded using digital signal processing to extract useful data. The data is preprocessed by denoising and filtering. The object position is accurately measured by time difference. The UWP signal processing flowchart is shown in Figure 3 The UWB ranging results are actively pushed to the background perception system to reduce the risk of missed detection and further improve the safety of platform operations. Step S240: The multi-modal features collected by the camera, laser radar, UWB module and other hardware devices are fused by the BEV spatial feature fusion algorithm. The multi-modal feature data in each dimension of time and space is mapped to the BEV system. The BEV system aligns the multi-dimensional data in time and space according to the position of the sensor installation and the timestamp carried by each sensor data, and extracts and fuses the features. This way, without losing any sensor data, the multi-source data is pre-fused, effectively avoiding the missing, missing and missing problems that may exist in single source data, and achieving complementarity and mutual verification between different modal data. Since the BEV algorithm converts the objects around the offshore platform measured by different sensors into the same platform bird's eye view coordinate system, the object trajectory has continuity. Therefore, the object trajectory can be tracked and predicted according to the historical motion data in the bird's eye view, and the trajectory of the surrounding ships can be planned from the perspective of the offshore platform. The planned driving path is presented in the form of a bird's eye view in the BEV system and pushed to the passing ships, providing driving suggestions from the platform perspective to reduce the risk of platform collision.

[0023] Step S250: The background perception system establishes a risk weight fusion matrix for the received three-dimensional fusion ranging module data, UWB ranging data, meteorological data, wind wave coefficient, other ship routes, risk level results, and the operation and feedback of platform field personnel to the perception system, and inputs them into the offshore platform sea state large model, establishes an offshore platform operation knowledge base, calculates and updates the weight of different data on the platform risk in real time, dynamically adjusts the distance calculation frequency and early warning intensity, and realizes multiple safeguards for offshore platform safety.

[0024] Step S300, to further improve the visual and audio alarm effect of intelligent perception data visualization, the image of the circular scan is corrected for distortion, and then converted into an aerial view of the offshore platform through rephotography transformation. Finally, the aerial view of the offshore platform is obtained by splicing and smoothing, a dynamic electronic fence of the offshore platform area is established, data with high risk levels are preferentially displayed on the front-end display page, the BEV (Bird's Eye View) perception technology is used to extract point cloud and image features to provide more comprehensive scene information, making it convenient for workers on the platform and nearby ships to check and warn, and workers can also ask questions to the offshore platform operation knowledge base through the voice module of the perception system, without watching the screen, they can also understand the latest situation of the platform; Step S310: Since the point cloud image mapping relationship between the laser radar point cloud data and the network camera image data has been established in steps S100 and S120, the overlapping area in each adjacent frame image region during the rotation of the pan-tilt is fused and simultaneously projected using a projection matrix. The projection transformation relationship can be changed through the BEVFormer method in step S220. The overlapping area in the image is extracted for grayscale processing and binarization by setting a threshold. Noise points are removed using Gaussian filtering to obtain an overlapping area mask image. The overlapping polygon contour is detected, and the color of the image in the area is adjusted to achieve a smooth splicing effect. The BEV algorithm integrates the overlapping areas of multiple sensors into a unique and accurate overlapping view data through the perspective of the offshore platform aerial view coordinate system. The panoramic projection equation of the image is also applied to the point cloud to obtain the effect of simultaneously containing image and point cloud in the panoramic overhead view. At the same time, three-dimensional point cloud can be used to accurately calculate the distance of obstacles around the offshore platform. According to the distance, an electronic fence is set up, and offshore platform workers can clearly and intuitively observe the distance of objects near the offshore platform from the platform on the interface; Step S320: Since the BEV algorithm realizes a unified aerial view coordinate system, the distance of objects near the platform to the platform can be intuitively observed through the overhead view. The distance of the objects to the side of the platform is calculated, and hierarchical electronic fence parameters are established according to the distance. The platform is divided into three levels of control areas, as follows Figure 4The shown: ① early warning zone (500-1000 meters): trigger sound and light alarm and push AIS warning information; ② intervention zone (200-500 meters): automatically send collision avoidance instructions to ship terminal; ③ emergency zone (<200 meters): linkage platform emergency system (shutdown, personnel evacuation). At the same time, the fence dynamically adjusts itself, adjusts the fence radius (±10%) according to real-time sea conditions (wave height, visibility) and ship type (tanker / fishing boat), and uses historical collision data to train a risk prediction model (LSTM network) to optimize the fence threshold; Step S330: Using the high-definition camera and long-range detection laser radar installed diagonally on the offshore platform, the ship crew behavior intelligent analysis function can be added to the offshore platform to assess the ship crew's performance. The camera video recognizes the crew's actions (such as not wearing a life jacket, violating regulations), and the AIS (Automatic Identification System) data (speed, turning rate) is used to evaluate the captain's docking ability (scoring model). The platform's perspective objectively assesses the ship's movement and pushes real-time risk reports to the management platform. Step S340: In order to make the visualization of the offshore platform anti-collision system more intuitive and smooth, the BEV system will establish a queue with a fixed length (e.g. 30) to store the current frame and the recent history of multiple frames of multi-sensor data. Whenever the system collects a new real-time data, the BEV background system will automatically delete the oldest frame of fusion data in the queue and add the new multi-sensor data to the BEV space multi-modal fusion data after feature fusion and extraction. Since the queue length is fixed and the system resources are not much, it can achieve fast calculation and fusion, improving the real-time performance of system fusion and calculation while ensuring data accuracy. By dynamically displaying real-time images, distances, and collision diagrams in the panoramic view (bird's eye view), the system also shows the fluctuation of the last 30 data to help offshore workers understand the trend of the platform's surrounding environment. Step S350: Workers can ask questions to the offshore platform knowledge base through the perception system's voice module, without needing to watch the screen to understand the latest platform information. For example, workers can wake up the background perception system's voice module by keywords, and the background perception system can operate various sensor devices, including initializing, waking up, and obtaining data from sensors. The worker can ask what objects are near the platform and how far each object is from the platform. If the object is a ship, the worker can directly ask for specific information about the ship, such as its heading and speed, which helps workers make risk predictions and take appropriate plans.

[0025] Step S400, install a special offshore version of the safety airbag on the side of the platform, when the collision object is still approaching the offshore platform after the warning, to avoid the risk of collision and cause more danger and loss, the background system will automatically pop up the safety airbag in the specified area according to the distance risk coefficient calculated by the BEV system to slow down the harm of collision, and at the same time, considering the marine environment and environmental protection, the airbag after being popped up is composed of seawater degradable ingredients, which can be dissolved in seawater and will not generate harmful substances to marine life: Step S410: Collect the collision cases of offshore platforms in domestic sea areas, mark the corresponding positions on the offshore platform model by analyzing the positions of different platforms where the collision occurs, and perform matrix transformation on the same type of offshore platform model to align the rotation and translation, and form a three-dimensional offshore platform collision heat map by accumulating the collision points at the corresponding positions. The 80% of the positions with the highest collision risk in the heat map are sampled and recorded; Step S420: Install the offshore platform dedicated safety airbag at the multiple collision-prone points calculated in step S410. The design principle of the airbag is borrowed from the design principle of the airbag in the car. The difference between the two is that the airbag in the car needs to receive a collision signal of a certain strength before it can pop up, but for the platform, if a collision signal of a certain strength has been generated, it means that an object has collided with the platform. At this time, it is too late to pop up the safety airbag. Therefore, the control of the airbag on the offshore platform is transferred to the BEV system and the manual control of the offshore platform staff. These two ways can automatically pop up or manually pop up before the collision occurs, which can minimize the risk of collision and form an active defense mechanism for offshore platform collision prevention.

[0026] Step S430: Considering the marine environment and environmental protection, since the platform airbag needs to protect a larger area, the volume and surface material surface area of the airbag are also relatively large. It is also difficult to handle the airbag after being popped up in the marine operating environment. Therefore, a new material composed of seawater degradable ingredients that can be dissolved in seawater and will not generate harmful substances to marine life is used as the airbag composition material. It will automatically start to decompose after being soaked in seawater for more than 12 hours.

[0027] Step S500, a platform peripheral collision object database is established by repeatedly scanning the target photographed by the three-dimensional fusion ranging module, weather data, wave data, collision object approaching time, moving away time, residence time, object position, collision object type, collision object image, collision object point cloud, UWB distance data, and collision object three-dimensional model are recorded and statistically analyzed to construct a collision risk time and space distribution matrix, on the basis of which personnel inspection work is adjusted, an offshore platform-ship communication and risk early warning integrated platform is established, two-way distance detection and collision warning are realized, a closed-loop management anti-collision system is realized, when a ship approaches within the visual distance of the platform, it is automatically associated with the platform equipment, automatically calculates the distance and warns, and a digital twin picture consistent with the real world is displayed, and the data is pushed to the platform side and the ship side to realize two-way display and warning: Step S510: Standardized storage of laser radar, network camera, and UWP multi-modal detection data: design MySQL database table structure, fields include: timestamp (UTC standard), GPS coordinates (WGS-84), target type (ship / float / iceberg), point cloud data (LAS format), three-dimensional model (OBJ format), video segment (H.265 encoding), collision risk matrix construction: based on residence time (T), distance (D), and target size (S), calculate risk index: Risk=0.4×T / Tmax + 0.3×Dmin / D + 0.3×S / Smax generate space-time heat map, mark high-risk period (such as night, heavy fog) and area (platform southeast side), personnel inspection optimization: dynamically adjust the inspection route and frequency according to the risk matrix, deploy AR inspection system (Microsoft HoloLens) in high-risk areas, automatically superimpose virtual warning lines and historical collision markers; Step S520: Construct multi-protocol data fusion transmission: design a hybrid communication protocol based on 5G NR (frequency band 3.5GHz) + LoRa (spreading factor SF10) to ensure reliable transmission of point cloud data (bandwidth ≥ 50Mbps) and video stream (delay ≤ 200ms), the system topology is as follows Figure 5 As shown, weather data, wave data, collision object approaching time, moving away time, residence time, object position, collision object type, collision object image, collision object point cloud, UWB distance data, and collision object three-dimensional model are recorded into the background database through network communication for statistical analysis to construct a collision risk time and space distribution matrix, on the basis of which personnel inspection work is adjusted; the collision risk time and space distribution matrix is the same as the collision risk matrix in S510. The background database refers to the platform peripheral collision object database in S500, which is responsible for storing sensor data and also storing system calculation data such as the distance between a collision object and the platform at a certain time and the danger coefficient.

[0028] Step S530: Establish a two-way collision warning mechanism and closed-loop management system verification: The platform and the ship end display the electronic fence state synchronously, the ship's cockpit integrates the HUD (head-up display), and the real-time superposition of the obstacle direction (azimuth angle ± 1° accuracy) and the collision avoidance suggestion (turning angle, deceleration value) is realized. Define the anti-collision effectiveness index (such as alarm accuracy ≥ 95%, false alarm rate ≤ 3%), verify the system reliability through Monte Carlo simulation and real ship test, and generate PDCA (plan-do-check-act) cycle report; Step S540: Realize two-way distance detection and collision warning, realize closed-loop management anti-collision system, when the ship approaches the platform within the visual distance, it is automatically associated with the platform equipment, automatically calculates the distance and warns, and displays the digital twin picture consistent with the real world, and synchronously pushes the data to the platform side and the ship side to realize two-way display and warning, at the same time, the data can be pushed to the land management platform, which is convenient for land personnel to understand the offshore platform environment and safety data.

[0029] In summary, the beneficial effects of the present embodiment: the present embodiment develops a set of real-time monitoring, collision risk assessment of offshore platform anti-collision early warning system. Improve the safety of offshore platform anti-collision, reduce the incidence of collision accident. ① Design a three-dimensional perception hardware device with multi-modal perception fusion, based on laser radar point cloud data and high-definition camera image data to calculate the distance of other objects that may collide around the platform, use the fine texture information of image vision to solve the problem of missing small targets (such as buoys) in traditional radar, and integrate UWB ranging technology and wireless communication technology to obtain offshore weather, wave coefficient and other data, and real-time detection of the relative position of the platform around the other ships is used to compensate for the detection of the blind area of the camera and radar vision and further reduce the risk of missing detection. ② Improve the visual and audio alert effect of intelligent perception data, learn from the way of vehicle panoramic surround view system, correct the distortion of the image, and then convert it into a bird's eye view of the offshore platform through the photography transformation, and finally splice and smooth to get the overhead view of the offshore platform, which is convenient for the staff on the platform and the ships near the platform to check and warn, and the staff can also ask questions to the offshore platform knowledge base through the voice module of the perception system without watching the screen. ③ Dynamic intelligent electronic fence: map the distance information of other objects calculated by point cloud to the panoramic overhead view, and give an audible and visual alarm when the distance is less than a specified threshold to alert the staff on the platform and the ships near the platform not to approach further. ④ Install a special version of safety airbag on the platform, when the warning is over and the collision object still approaches the offshore platform, to avoid more danger and loss caused by collision risk, the background system will automatically pop up the safety airbag in the specified area according to the distance risk coefficient calculated by the BEV system to slow down the collision damage, and considering the marine environment and environmental protection, the airbag after being popped up is composed of seawater degradable components, which can dissolve in seawater and will not generate harmful substances to marine life. ⑤ Risk prediction data driven: establish an offshore platform-ship communication and risk warning integrated platform, realize two-way distance detection and collision warning, realize closed-loop management of anti-collision system, when the ship approaches the platform within the visual distance, it is automatically associated with the platform equipment, automatically calculates the distance and warns, and displays the digital twin picture consistent with the real world.

[0030] For ease of description, spatial relative terms such as "upper", "lower", "outer", "inner", etc. are used in the embodiments to describe the relationship of one element or feature to another element or feature shown in the drawings. It should be understood that, in addition to the orientation shown in the drawings, the spatial terms are intended to include different orientations of the device in use or operation. Therefore, the exemplary term "lower" can include both the upper and lower orientations.

[0031] Although the preferred embodiments of the present application have been described above with reference to the accompanying drawings, the present application is not limited to the above-described specific embodiments, and the above-described specific embodiments are merely illustrative and not restrictive. A person of ordinary skill in the art can make many specific changes to the present application without departing from the spirit of the present application and the scope of protection of the claims, and these are all within the scope of protection of the present application.

Claims

1. A collision avoidance device for offshore platforms based on multimodal BEV perception, characterized in that, The system includes a 3D fusion ranging module, a UWB ranging module, a ship navigation status monitoring module, a platform marine environment monitoring module, and a central processing unit. The 3D fusion ranging module is installed diagonally on the offshore platform and includes a gimbal, a lidar, and a camera. The lidar and camera are mounted on the gimbal. The 3D fusion ranging module is used to perform 360° 3D scanning around the offshore platform. The UWB ranging module is installed around the ship and at the platform's legs. The UWB ranging module is used to acquire the 3D spatial physical position between the ship and the platform's legs in real time. The ship navigation status monitoring module is used to acquire the ship's position, speed, and heading information in real time. The platform marine environment monitoring module collects wind and wave data, weather data, and passing ship route information near the offshore platform through network communication. The 3D fusion ranging module, UWB ranging module, ship navigation status monitoring module, and platform marine environment monitoring module are all communicatively connected to the central processing unit.

2. The marine platform collision avoidance device based on multimodal BEV perception according to claim 1, characterized in that, It also includes airbags, which are installed around the perimeter of the offshore platform and are made of seawater-degradable materials.

3. A method for using a multimodal BEV-based collision avoidance device for offshore platforms as described in claim 1 or 2, characterized in that, Includes the following steps: S100, Equipment Deployment; The S200 uses BEV sensing technology to perform fusion calculations on the data acquired by the 3D fusion ranging module, UWB ranging module, ship navigation status monitoring module, and platform marine environment monitoring module. It converts the camera image data and lidar point cloud data in the 3D fusion ranging module and the UWB ranging data into a coordinate system centered on the offshore platform, performs time and space alignment and integration into a common bird's-eye view representation space above the offshore platform, and synchronizes the detection data to the intelligent sensing backend system to dynamically adjust the distance calculation frequency and early warning intensity, and build an offshore platform operation knowledge base. The S300 performs distortion correction on the cyclically scanned camera images, then converts them into a bird's-eye view of the offshore platform under phototransformation. Finally, it stitches and smooths the images to obtain a top view of the offshore platform. A dynamic electronic fence is established for the offshore platform area. Data with high risk levels is displayed first on the front-end display page. Point cloud and image features are extracted using BEV perception technology. Personnel on the offshore platform and on ships near the offshore platform can view and receive warnings. At the same time, personnel can ask questions to the offshore platform operation knowledge base through the voice module of the perception system. S400: When the collision object continues to approach the offshore platform after the warning, the background system will automatically deploy the airbags in the designated area based on the distance hazard factor calculated by the BEV system. The S500 uses repeated scanning to establish a collision object database around the platform for targets captured by the 3D fusion ranging module. It records weather data, wind and wave data, collision object approach time, distance time, dwell time, object location, collision object type, collision object image, collision object point cloud, UWB distance data, and collision object 3D model for statistical analysis. It constructs a collision risk temporal and spatial distribution matrix. Based on this, it adjusts personnel inspection work and establishes an integrated platform for communication and risk warning between the offshore platform and the ship. When the ship approaches within the platform's field of vision, it automatically associates with the platform equipment, automatically calculates the distance and issues a warning, and displays a digital twin image consistent with the real world. It also pushes the data synchronously to both the platform and the ship for two-way display and warning.

4. The method for a collision avoidance device for offshore platforms based on multimodal BEV perception according to claim 3, characterized in that, Step S200 specifically includes: Step S210: Establish a background data fusion system, open different ports and establish connections with available hardware devices on the offshore platform to collect data, obtain weather conditions, wind and wave data, and heading information of other ships within a 500-meter range of the offshore platform, calculate the danger weight coefficient of each weather and wind and wave coefficient based on the number of times the risk occurred under different historical weather and wind and wave conditions, and obtain the risk level weighted sum of the natural environment. Establish a cruise frequency mapping table for the weighted sum and automatically adjust the collection and fusion calculation frequency of each sensor device. Step S220: Simultaneously perform 3D target detection while mapping point cloud data onto image data; before the offshore platform anti-collision system is officially put into use, organize, label, and review the collected point cloud data and image data, perform instance segmentation and labeling on the image data, perform segmentation and labeling on the point cloud data, label the same instance with the same category and ID when labeling the image and point cloud, extract the fusion features of 2D and 3D, project the objects detected in the image vision and 3D point cloud onto a unified plane centered on the offshore platform according to their actual position relative to the offshore platform, and achieve view transformation by designing a set of BEV queries based on Transformer and combining the cross attention between position encoding and image features to map the perspective view of the image and point cloud to the top view; Step S230: When the gimbal rotates, the background perception system calculates the rotation and translation transformation matrices of the LiDAR point cloud, camera image coordinate system, and platform bird's-eye view coordinate system after rotation. The transformation matrices are applied to the point cloud and image data to map the data to the platform bird's-eye view coordinate system. The three-dimensional perception coverage area is calculated at this time. The ranging function of the UWB module in the uncovered area is activated, and the UWB ranging result is automatically pushed to the background perception system. Step S240: Based on the multimodal features collected by hardware devices such as cameras, lidar, and UWB modules, multimodal feature fusion is performed through BEV spatial feature unification, and the modal feature data in time and space of each dimension are mapped to the BEV system. The BEV system performs time and space alignment of multidimensional data and performs feature extraction and fusion based on the sensor installation location and the timestamps carried by each sensor data. Based on the historical motion data of objects in the bird's-eye view, trajectory tracking and prediction are performed, and the trajectories of surrounding ships are planned from the perspective of the offshore platform. The planned driving path is presented in the BEV system in the form of a bird's-eye view and pushed to the passing ships. Step S250: The background perception system establishes a risk weight fusion matrix based on the received 3D fusion ranging module data, UWB ranging data, meteorological data, wind and wave coefficient, other ship routes, risk level results, and the operation and feedback of the perception system by the platform personnel on site. It also constructs a knowledge base for offshore platform operations, calculates and updates the weight of different data on platform risks in real time, and dynamically adjusts the distance calculation frequency and early warning intensity.

5. The method for a collision avoidance device for offshore platforms based on multimodal BEV perception according to claim 4, characterized in that, Step S300 specifically includes: Step S310: The overlapping areas in each adjacent frame image region during the gimbal rotation process are fused and projection transformation is performed using a projection matrix. The overlapping areas in the image are extracted, grayscale processing is performed, and a threshold is set for binarization. Gaussian filtering is used to remove noise points to obtain the overlapping area mask image. The overlapping polygonal contours are detected, and the color of the image in the region is adjusted to achieve a smooth stitching effect. Step S320: Calculate the distance from the object to the perimeter of the platform and establish hierarchical electronic fence parameters based on the distance. Divide the platform into three control zones: Early Warning Zone: Trigger audible and visual alarms and push AIS warning information; Intervention Zone: Automatically send collision avoidance commands to the ship terminal; Emergency Zone: Link the platform's emergency system to shut down and evacuate personnel. The fence is dynamically and adaptively adjusted according to real-time sea conditions and ship type, and the risk prediction model is trained using historical collision data to optimize the fence threshold. Step S330: The crew's actions are identified by the video of the cameras installed on the offshore platform, and the captain's berthing ability is assessed by combining AIS data. The ship's operation at sea is objectively assessed from the platform's perspective, and risk reports are pushed to the management platform in real time. Step S340: The BEV system establishes a queue of fixed length to store the current frame and the most recent historical multi-frame multi-sensor data. Whenever the system acquires a new frame of real-time data, the BEV system automatically deletes the oldest frame of fused data in the queue and adds the new multi-sensor data to the multimodal fused data in the BEV space after feature fusion and extraction. While dynamically displaying real-time images, distances, and collision illustrations in the panoramic top view, it also displays the fluctuation of historical data to help offshore personnel understand the changing trends of the platform's surrounding environment. Step S350: Staff members ask questions to the offshore platform operation knowledge base through the voice module of the perception system.

6. The method for a collision avoidance device for offshore platforms based on multimodal BEV perception according to claim 3, characterized in that, Step S500 specifically includes: Step S510: For standardized storage of multimodal detection data from LiDAR, camera, and UWP, design the MySQL database table structure and construct the collision risk matrix; Step S520: Construct a multi-protocol data fusion transmission system. Through network communication, input weather data, wind and wave data, collision object approach time, distance time, dwell time, object location, collision object type, collision object image, collision object point cloud, UWB distance data, and collision object 3D model into the background database for statistical analysis. Construct a collision risk time and space distribution matrix, and adjust personnel inspection work based on this matrix. Step S530: Establish a two-way collision warning mechanism and verify the closed-loop management system: The electronic fence status is displayed synchronously on the platform and the ship. The ship's cockpit integrates a head-up display to overlay obstacle positions and collision avoidance suggestions in real time; define collision avoidance effectiveness indicators, verify the system reliability through Monte Carlo simulation and real ship testing, and generate a plan-execution-check-process cycle report; Step S540: Achieve two-way distance detection and collision alarm. When the ship approaches within the platform's field of vision, it automatically associates with the platform equipment, automatically calculates the distance and issues an early warning, and displays a digital twin image consistent with the real world. The data is also pushed synchronously to the platform and the ship, and simultaneously to the land management platform.

Images