AI-based electric tug safety navigation intelligent monitoring system and method

Abstract

This invention relates to the field of intelligent port management technology, specifically to an AI-based intelligent monitoring system and method for the safe navigation of electric tugboats. Through the coordinated operation of a monitoring system, a visual digital twin system, and an AI intelligent recognition algorithm system, it effectively addresses the problems of existing port monitoring systems, such as monitoring blind spots, inability to promptly detect and identify internal malfunctions of electric tugboats, unsafe crew behavior, and obstacles at sea like ice floes and floating objects. It achieves intelligent monitoring of electric tugboats and their internal equipment, intelligent identification of crew behavior and obstacles at sea, and real-time, intuitive display of navigation information, obstacle information, crew behavior, and internal equipment monitoring information, as well as precise target location positioning, thereby improving the level of intelligent monitoring for the safe navigation of electric tugboats.

Description

Technical Field

[0001] This invention relates to the field of intelligent port management technology, specifically to an AI-based intelligent monitoring system and method for the safe navigation of electric tugboats. Background Technology

[0002] Tugs have long played a vital role in port operations, assisting various types of electric tugs in unberthing, shifting, and escorting. Currently, most tugs still rely on diesel engines as their main power source. While diesel engines offer advantages such as high power density, fewer intermediate components, and high reliability, they also release significant amounts of carbon dioxide during operation.

[0003] In recent years, green electric tugboat technology has developed rapidly. Pure electric tugboats, with their advantages of zero emissions and low energy consumption, have become important equipment in port towing and near-shore operations. However, when electric tugboats are operating in busy port areas, obstacles such as floating ice and debris pose a threat to their navigation safety. Furthermore, malfunctions in critical equipment such as motors, lithium batteries, transformers, and frequency converters, as well as unsafe acts by crew members, can also endanger their navigation safety. Therefore, it is necessary to monitor the tugboat's own navigation status in real time and to monitor the operational status of critical equipment and unsafe acts by crew members. Current electric tugboat technology cannot monitor the surrounding waters, equipment, and crew behavior in a timely and intelligent manner, resulting in blind spots that seriously affect the efficiency of electric tugboat supervision and the identification of safety hazards. Therefore, improvements are needed. Summary of the Invention

[0004] The purpose of this invention is to provide an AI-based intelligent monitoring system and method for the safe navigation of electric tugboats, in order to overcome the problems existing in the prior art.

[0005] To achieve the above objectives, the present invention provides an AI-based intelligent monitoring system for the safe navigation of electric tugboats, including a monitoring system, a visual digital twin system, and an AI intelligent recognition algorithm system; The monitoring system is used to monitor the waters surrounding the electric tugboat, its internal equipment, and crew. It includes a camera, an AIS device, a radar scanner, multiple sensors, an infrared thermal imager, a drone, and a monitoring information buffer module. The camera acquires video surveillance data in real time, including multiple optical cameras installed on the drone, the exterior of the electric tugboat, and inside the tugboat. The AIS device acquires real-time status information of the electric tugboat and tracks its coordinates. The radar scanner acquires real-time three-dimensional spatial information of the waters surrounding the electric tugboat and transmits the scanning results. The infrared thermal imager collects infrared imaging information of the waters surrounding the electric tugboat. Sensors collect status information of the internal equipment of the electric tugboat. The visualization digital twin system utilizes infrared thermal imager imaging information, radar scanning data, and monitoring video data, combined with digital twin and 3D simulation technology, to build and simulate electric tugboats. It intelligently monitors and fully records the actual operation of electric tugboats in a digital visualization manner, and maps the operation of electric tugboats in real time in virtual space, enhancing the ability of data visualization. The visualized digital twin system includes a 3D simulation module, a display device, a data caching module, and a control module. The 3D simulation module is used for digital visualization, simulating the waters surrounding the electric tugboat, its internal equipment, and crew, enabling virtual monitoring and analysis of the real-world scenario, while simultaneously recording monitoring information for different time periods. The display device is used to display data, the data caching module is used to cache data, and the control module is used to control the transmission of equipment status information, achieving interconnection and interoperability of data among the monitoring system, the visualized digital twin system, the electric tugboat, and the AI ​​intelligent recognition algorithm system. The AI ​​intelligent recognition algorithm system includes a crew member detection and behavior tracking model, a marine obstacle detection and tracking model, an equipment fault diagnosis and prediction model, a data storage module, and a computing unit, which are used for marine obstacle recognition, electric tugboat crew behavior tracking, and internal equipment detection data analysis.

[0006] Based on the above technical solution, the present invention can be further improved as follows: As a further improvement to the above technical solution, the shooting device is linked to the monitoring information cache module through a video capture card. The shooting device is combined with an infrared thermal imager, which uses infrared thermal imaging technology to ensure the monitoring effect of maritime obstacles at night or in bad weather conditions. The infrared thermal imager is connected to an AI intelligent recognition algorithm system, and maritime obstacle recognition is performed using a maritime obstacle detection and tracking model and a calculation unit.

[0007] As a further improvement to the above technical solution, the multiple sensors collect the operating parameters of the internal equipment of the electric tugboat in real time and map them into the three-dimensional simulation module. The three-dimensional simulation module can dynamically reflect the changes in the state of the internal equipment, realizing the digital modeling of the internal equipment of the electric tugboat and the online twinning of the internal equipment data.

[0008] As a further improvement to the above technical solution, the multiple sensors are interconnected with the AI ​​intelligent recognition algorithm system. The sensors detect the status of the internal equipment of the tugboat in real time and use AI algorithms to perform health assessments. By collecting equipment operation data through the sensors and combining it with the equipment fault diagnosis and prediction model, potential hazards can be detected in advance and maintenance suggestions can be generated to improve navigation safety.

[0009] As a further improvement to the above technical solution, the multiple sensor detection signals are connected to the AI ​​intelligent recognition algorithm system to conduct health assessment of equipment operation status, use the equipment fault diagnosis prediction model and calculation unit to perform health analysis, obtain equipment fault information through equipment fault management, associate the causes of the fault and solutions, and perform early warning diagnosis of equipment status and risk prediction. Various devices that receive signal access on the electric tugboat include propulsion motors, lithium batteries, rudder propellers, power distribution systems and auxiliary equipment.

[0010] As a further improvement to the above technical solution, the multiple sensors include a vibration sensor. The vibration sensor detects the vibration signal during the operation of the electric tug propulsion motor. The equipment fault diagnosis and prediction model and calculation unit use big data to diagnose the vibration signal of the propulsion motor, extract abnormal values ​​for analysis, and realize early warning of the equipment.

[0011] As a further improvement to the above technical solution, the multiple sensors include a current sensor, a voltage sensor, and a temperature sensor. The current sensor detects the charging and discharging current of the lithium battery, the voltage sensor detects the charging and discharging voltage of the lithium battery, and the temperature sensor detects the temperature of the lithium battery, the distribution board, and the transformer windings. The equipment fault diagnosis prediction model and calculation unit use big data to diagnose and analyze parameters such as lithium battery temperature, overcurrent during charging and discharging, excessive total voltage, and the temperature of the distribution board and transformer windings, which facilitates fault diagnosis and problem tracing.

[0012] This invention also provides an AI-based intelligent monitoring method for the safe navigation of electric tugboats implemented according to the above system, comprising the following steps: Step 1: The monitoring system transmits the real-time collected information on the waters surrounding the electric tugboat, its internal equipment, and crew to the visualization digital twin system. The system uses a 3D simulation module to build and simulate the waters surrounding the electric tugboat, its internal equipment, and crew, and then displays the simulation in real time through a display device. Step 2: The monitoring system transmits video surveillance data, real-time status and location information of the electric tugboat, and internal equipment status information to the AI ​​intelligent recognition algorithm system; Step 3: The AI ​​intelligent recognition algorithm system starts the computing unit and, based on the crew detection and behavior tracking model, the marine obstacle detection and tracking model, and the equipment fault diagnosis and prediction model, performs real-time detection, analysis, and diagnosis of the waters surrounding the electric tugboat and its internal equipment. It identifies the behavior of the electric tugboat crew, analyzes different types of data, and stores the detection results of the electric tugboat's internal equipment, crew behavior tracking results, real-time status data of the electric tugboat, time, and other information in the data storage module. The above analysis and judgment results are then sent to the display device of the visualization digital twin system for display. Step 4: The visual digital twin system provides feedback to the user on anomalies based on the data analysis results received from the AI ​​intelligent recognition algorithm system. It promptly detects and identifies internal faults of the electric tugboat, unsafe behaviors of the crew, and obstacles at sea such as ice floes and floating objects, so that the user can schedule and maintain the electric tugboat.

[0013] As a further improvement to the above technical solution, the training method for the crew member detection and behavior tracking model includes the following steps: S1. The camera captures video and image data of the crew and performs enhancement processing, which is achieved by rotating, blurring and occluding the images. S2. Establish a crew dataset, annotate and preprocess each image data and store it in the crew dataset; S3. Establish a target detection model in the AI ​​intelligent recognition algorithm system, train the target detection model using the crew dataset, and obtain a crew detection and behavior tracking model; S4. Based on the crew detection and behavior tracking model, the computing unit in the AI ​​intelligent recognition algorithm system detects crew members in the video data collected by the shooting device, calculates the changes in the crew members' positions within the image to achieve crew tracking, and calculates the changes in the crew members' positions within the image through IoU (Intersection over Union), that is, the degree of overlap of the target detection boxes in two consecutive frames to achieve crew tracking.

[0014] As a further improvement to the above technical solution, the training method for the maritime obstacle detection and tracking model includes the following steps: S1. Infrared thermal imager, camera device, drone and radar scanning device collect image data of maritime obstacles and targets, establish maritime target dataset in AI intelligent recognition algorithm system, and label and preprocess the maritime obstacle and target data in the images and store them in the maritime target dataset. S2. Establish a maritime target detection model in the AI ​​intelligent recognition algorithm system, and train the maritime target detection model using a maritime target dataset. The training steps are the same as those for the crew target detection model mentioned above. Use the computing unit in the AI ​​intelligent recognition algorithm system to detect the collected maritime obstacle targets and calculate the changes in the position of the maritime obstacles within the image to achieve maritime obstacle tracking. S3. Train the ReID (Person Re-identification) model to extract target appearance features, and fine-tune it using a marine target dataset; input the detection and tracking results of the above marine obstacles into the DeepSORT (Multi-Target Tracking) model, predict the target position through Kalman filtering, and match it with the ReID model features; S4. Verify the above detection and tracking effects using MAP (Mean Average Precision) and MOTA (Multi-Target Tracking Accuracy) metrics.

[0015] The beneficial effects of this invention are as follows: The above-mentioned AI-based intelligent monitoring system and method for safe navigation of electric tugboats, through the coordinated operation of a monitoring system, a visual digital twin system, and an AI intelligent recognition algorithm system, can effectively solve the problems of existing port monitoring systems having monitoring blind spots, being unable to detect and identify internal faults of electric tugboats, unsafe behaviors of crew members, and obstacles at sea such as floating ice and debris in a timely manner. It realizes intelligent monitoring of electric tugboats and their internal equipment, intelligent identification of crew behavior and obstacles at sea, and real-time intuitive display of navigation information, maritime obstacles, crew behavior, and internal equipment monitoring information of electric tugboats, as well as precise positioning of target locations, thereby improving the level of intelligent monitoring of safe navigation of electric tugboats. Attached Figure Description

[0016] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0017] Figure 1 This is a structural diagram of the AI-based intelligent monitoring system for safe navigation of electric tugboats provided in a preferred embodiment of the present invention; Figure 2 These are monitoring images displayed by the visual digital twin system of this invention; Figure 3 and Figure 4 This is the user interface of the visual digital twin system of this invention. Detailed Implementation

[0018] The present invention will now be described in further detail with reference to the accompanying drawings and embodiments. These drawings are simplified schematic diagrams, which are only used to illustrate the basic structure of the present invention and therefore only show the components relevant to the present invention.

[0019] like Figures 1 to 4 As shown, a preferred embodiment of the present invention provides an AI-based intelligent monitoring system for the safe navigation of electric tugboats, comprising a monitoring system, a visual digital twin system, and an AI intelligent recognition algorithm system.

[0020] The monitoring system is used to monitor the waters surrounding the electric tugboat, its internal equipment, and crew. It includes a camera, an AIS device, a radar scanner, multiple sensors, an infrared thermal imager, a drone, and a monitoring information cache module. The camera connects to the monitoring information cache module via a video capture card, acquiring real-time video monitoring data. This includes multiple optical cameras installed on the drone, around the electric tugboat, and inside the tugboat. The AIS device acquires real-time status information of the electric tugboat and tracks its coordinates. The radar scanner acquires real-time three-dimensional spatial information of the waters surrounding the electric tugboat and transmits the scanning results. The infrared thermal imager collects infrared imaging information of the waters surrounding the electric tugboat. Sensors collect status information of the internal equipment of the electric tugboat.

[0021] The visualized digital twin system utilizes infrared thermal imager imaging information, radar scanning data, and monitoring video data, combined with digital twin and 3D simulation technologies, to build and simulate electric tugboats. It intelligently monitors and fully records the actual operation of electric tugboats in a digitally visualized manner, and maps the operation of electric tugboats in real time in virtual space, enhancing the ability of data visualization.

[0022] The visualized digital twin system includes a 3D simulation module, a display device, a data caching module, and a control module. The 3D simulation module is used for digital visualization, simulating the waters surrounding the electric tugboat, its internal equipment, and crew, enabling virtual monitoring and analysis of the real-world scenario, while simultaneously recording monitoring information for different time periods. The display device displays the data, the data caching module caches the data, and the control module controls the transmission of equipment status information, achieving interconnectivity between the monitoring system, the visualized digital twin system, the electric tugboat, and the AI ​​intelligent recognition algorithm system.

[0023] The AI ​​intelligent recognition algorithm system includes a crew member detection and behavior tracking model, a marine obstacle detection and tracking model, an equipment fault diagnosis and prediction model, a data storage module, and a computing unit, which are used for marine obstacle recognition, electric tugboat crew behavior tracking, and internal equipment detection data analysis.

[0024] The multiple sensors collect the operating parameters of the internal equipment of the electric tugboat in real time and map them onto the 3D simulation module. The 3D simulation module can dynamically reflect the changes in the state of the internal equipment, realizing the digital modeling of the internal equipment of the electric tugboat and the online twin of the internal equipment data.

[0025] The multiple sensors are interconnected with the AI ​​intelligent recognition algorithm system. The sensors detect the status of the equipment inside the tugboat in real time and use AI algorithms to perform health assessments. By collecting equipment operation data through the sensors and combining it with the equipment fault diagnosis and prediction model, potential problems can be detected in advance and maintenance suggestions can be generated to improve navigation safety.

[0026] The signals detected by the aforementioned multiple sensors are integrated into an AI intelligent recognition algorithm system for equipment operational health assessment. Health analysis is performed using an equipment fault diagnosis and prediction model and computational unit. Through equipment fault management, equipment fault information is obtained, the causes and solutions for these faults are identified, and early warning diagnosis and risk prediction are conducted for the equipment status. Various devices on the electric tugboat that receive signal input include the propulsion motor, lithium battery, propeller, power distribution system, and important auxiliary equipment. The computational unit can utilize an existing computer central processing unit (CPU), and the monitoring information cache module, data cache module, and data storage module can utilize existing hard disks or other storage devices. The equipment fault diagnosis and prediction model can employ existing AI models such as LSTM and Transformer.

[0027] Specifically, the multiple sensors include vibration sensors, which detect vibration signals during the operation of the electric tug propulsion motor. The equipment fault diagnosis and prediction model and calculation unit use big data to diagnose the vibration signals of the propulsion motor, extract abnormal values ​​for analysis, and realize early warning of the equipment.

[0028] Specifically, the multiple sensors include a current sensor, a voltage sensor, and a temperature sensor. The current sensor detects the charging and discharging current of the lithium battery, the voltage sensor detects the charging and discharging voltage of the lithium battery, and the temperature sensor detects the temperature of the lithium battery, the distribution board, and the transformer windings. The equipment fault diagnosis prediction model and calculation unit use big data to diagnose and analyze parameters such as lithium battery temperature, overcurrent during charging and discharging, excessive total voltage, and the temperature of the distribution board and transformer windings, which facilitates fault diagnosis and problem tracing.

[0029] The imaging device is combined with an infrared thermal imager, which uses infrared thermal imaging technology to ensure effective monitoring of obstacles at sea at night or in adverse weather conditions. The infrared thermal imager is connected to an AI intelligent recognition algorithm system, which uses a marine obstacle detection and tracking model and computing unit to identify marine obstacles.

[0030] The visual digital twin system is connected to the monitoring system. It receives video data, real-time status data of the electric tugboat, and three-dimensional spatial information from radar scans provided by the monitoring system. At the same time, the visual digital twin system is connected to the AI ​​intelligent recognition algorithm system to obtain detection data of internal equipment of the electric tugboat, crew behavior tracking data, marine obstacle detection data, and data analysis results.

[0031] Based on video data and information such as the status of electric tugboats, the AI ​​intelligent recognition algorithm system uses AI algorithms to achieve intelligent detection and tracking of electric tugboats and the surrounding waters, intelligent recognition of the behavior of the electric tugboat crew, and intelligent assessment of the health status of the internal equipment of the electric tugboat, displaying the results on a display device. The AI ​​algorithm can process two-dimensional data such as video information in real time, and perform fast and accurate target detection, recognition, and data analysis using models trained on large-scale target data.

[0032] This invention also provides an AI-based intelligent monitoring method for the safe navigation of electric tugboats implemented by the above-mentioned system, comprising the following steps: Step 1: The monitoring system transmits the real-time collected information on the waters surrounding the electric tugboat, its internal equipment, and crew to the visualization digital twin system. The system uses a 3D simulation module to build and simulate the waters surrounding the electric tugboat, its internal equipment, and crew, and then displays the simulation in real time through a display device. Specifically, the monitoring information caching module of the monitoring system sends data such as the real-time status data of the electric tugboat, the monitoring video data captured by the camera, and the scanning results of the radar scanning device to the data caching module of the visualization digital twin system. Then, the data caching module distributes the monitoring video data, the real-time status of the electric tugboat, and the radar scanning results to the 3D simulation module for model building and simulation, and transmits the simulation results to the display device for display in real time.

[0033] Step 2: The monitoring system transmits video surveillance data, real-time status and location information of the electric tugboat, and internal equipment status information to the AI ​​intelligent recognition algorithm system; Step 3: The AI ​​intelligent recognition algorithm system starts the computing unit and, based on the crew detection and behavior tracking model, the marine obstacle detection and tracking model, and the equipment fault diagnosis and prediction model, performs real-time detection, analysis, and diagnosis of the waters surrounding the electric tugboat and its internal equipment. It identifies the behavior of the electric tugboat crew, analyzes different types of data, and stores the detection results of the electric tugboat's internal equipment, crew behavior tracking results, real-time status data of the electric tugboat, time, and other information in the data storage module. The above analysis and judgment results are then sent to the display device of the visualization digital twin system for display. Step 4: The visual digital twin system provides feedback to the user on anomalies based on the data analysis results received from the AI ​​intelligent recognition algorithm system. It promptly detects and identifies internal faults of the electric tugboat, unsafe behaviors of the crew, and obstacles at sea such as ice floes and floating objects, so that the user can schedule and maintain the electric tugboat.

[0034] Specifically, the crew detection and behavior tracking model uses the computing unit in the AI ​​intelligent recognition algorithm system to detect the crew members of the electric tugboat in the video data collected by the camera device, calculate the changes in the crew members' positions within the image, and achieve behavior tracking of the electric tugboat crew members. The crew detection and behavior tracking model adopts the YOLO V8 algorithm model, which analyzes the images of electric tugboat crew members operating the camera device in real time, and identifies unsafe behaviors such as sleeping, leaving their posts, using mobile phones, and not wearing safety helmets.

[0035] Specifically, when a crew member is seen lying face down on a table or leaning against a seat for an extended period of time in video surveillance, it is determined to be suspected of sleeping on duty, and the video frame is captured to generate an alarm; when the number of personnel in the work area is below a specified threshold and the duration exceeds the specified threshold in video surveillance, it is determined to be suspected of leaving the post, and the video frame is captured to generate an alarm; when a crew member is seen holding a mobile phone in a position that reaches a threshold time in video surveillance, it is determined to be suspected of using a mobile phone, and the video frame is captured to generate an alarm; when the system determines that a crew member in a specified area in video surveillance is suspected of not wearing a safety helmet, the video frame is captured to generate an alarm.

[0036] The training method for the crew member detection and behavior tracking model includes the following steps: S1. The camera captures video and image data of the crew and performs enhancement processing, which is achieved by rotating, blurring and occluding the images. S2. Establish a crew dataset, annotate and preprocess each image data and store it in the crew dataset; S3. Establish a target detection model in the AI ​​intelligent recognition algorithm system, train the target detection model using the crew dataset, and obtain a crew detection and behavior tracking model; S4. Based on the crew detection and behavior tracking model, the computing unit in the AI ​​intelligent recognition algorithm system detects crew members in the video data collected by the shooting device and calculates the changes in the crew members' positions within the image to achieve crew tracking. For the same target, since the object's movement is smooth and slow, the crew members' positions within the image are calculated by using IoU (Intersection over Union), which is the degree of overlap of the target detection boxes in two consecutive frames, to achieve crew tracking.

[0037] In step S2, the preprocessing tool can use label Img, yolo mark, Vatic, or VoTT to annotate the image data one by one, forming an XML file in the dataset. This file mainly includes location parameters (the horizontal and vertical coordinates of the center of the labeled target and the size of the target area to be labeled).

[0038] In step S3, the crew member detection and behavior tracking model is based on the YOLO V8 algorithm model, and the target detection model training steps include: a. Preprocessing: noise reduction (filtering algorithm), image enhancement, and scaling the image to a size suitable for processing; b. Feature extraction: The image data in the crew dataset is input into the backbone network of YOLO V8 for convolution calculation to finally obtain the feature layer; c. Feature fusion: The feature layer is input into the feature pyramid network for feature fusion. d. Once the model training is complete, the fused features are processed by the next network step to obtain the predicted parameters. The predicted parameters are compared with the actual parameters, and the error is calculated. When the error converges, the model training is complete.

[0039] In step d, the metric for measuring error is IoU, which is the overlap ratio between the generated candidate bounding box and the original ground truth bounding box, i.e., the ratio of their intersection to their union. Model performance is evaluated quantitatively using error rate and accuracy. The error rate is calculated using formula (1), and the accuracy is calculated using formula (2). (1) (2) The training method for the maritime obstacle detection and tracking model includes the following steps: S1. Infrared thermal imager, camera device, drone and radar scanning device collect image data of maritime obstacles and targets, establish maritime target dataset in AI intelligent recognition algorithm system, and label and preprocess the maritime obstacle and target data in the images and store them in the maritime target dataset. S2. Establish a maritime target detection model in the AI ​​intelligent recognition algorithm system, and train the maritime target detection model using a maritime target dataset. The training steps are the same as those for the crew target detection model mentioned above. Use the computing unit in the AI ​​intelligent recognition algorithm system to detect the collected maritime obstacle targets and calculate the changes in the position of the maritime obstacles within the image to achieve maritime obstacle tracking. S3. Train the ReID (Person Re-identification) model to extract target appearance features, and fine-tune it using a marine target dataset; input the detection and tracking results of the above marine obstacles into the DeepSORT (Multi-Target Tracking) model, predict the target position through Kalman filtering, and match it with the ReID model features; S4. Verify the above detection and tracking effects using MAP (Mean Average Precision) and MOTA (Multi-Target Tracking Accuracy) metrics.

[0040] In step S2, the data collected by the infrared thermal imager, camera device, UAV and radar scanning device are synchronized and registered. The preprocessing tool can use label Img, yolo mark, Vatic, VoTT to annotate the image data one by one. The marine target detection model is an algorithm model based on YOLO V8.

[0041] Any descriptions not covered in the above specific embodiments of the present invention are known technologies in the field and can be implemented with reference to such known technologies.

[0042] Based on the preferred embodiments of the present invention described above, those skilled in the art can make various changes and modifications without departing from the inventive concept. The technical scope of this invention is not limited to the contents of the specification, but must be determined according to the scope of the claims.

Claims

1. An AI-based intelligent monitoring system for the safe navigation of electric tugboats, characterized in that: This includes monitoring systems, visual digital twin systems, and AI intelligent recognition algorithm systems; The monitoring system is used to monitor the waters surrounding the electric tugboat, its internal equipment, and crew. It includes a camera, an AIS device, a radar scanning device, multiple sensors, an infrared thermal imager, a drone, and a monitoring information buffer module. The camera acquires video monitoring data in real time, including multiple optical cameras installed on the drone, the exterior of the electric tugboat, and inside the tugboat. The AIS device acquires the status information of the electric tugboat in real time and collects and tracks the coordinate position of the electric tugboat. The radar scanning device acquires real-time three-dimensional spatial information of the water area surrounding the electric tugboat and transmits the scanning results; the infrared thermal imager is used to collect infrared imaging information of the water area surrounding the electric tugboat; and the sensors are used to collect status information of the internal equipment of the electric tugboat. The visualization digital twin system utilizes infrared thermal imager imaging information, radar scanning data, and monitoring video data, combined with digital twin and 3D simulation technology, to build and simulate electric tugboats. It intelligently monitors and fully records the actual operation of electric tugboats in a digital visualization manner, and maps the operation of electric tugboats in real time in virtual space, enhancing the ability of data visualization. The visualized digital twin system includes a 3D simulation module, a display device, a data caching module, and a control module. The 3D simulation module is used for digital visualization, simulating the waters surrounding the electric tugboat, its internal equipment, and crew, enabling virtual monitoring and analysis of the real-world scenario, while simultaneously recording monitoring information for different time periods. The display device is used to display data, the data caching module is used to cache data, and the control module is used to control the transmission of equipment status information, achieving interconnection and interoperability of data among the monitoring system, the visualized digital twin system, the electric tugboat, and the AI ​​intelligent recognition algorithm system. The AI ​​intelligent recognition algorithm system includes a crew detection and behavior tracking model, a marine obstacle detection and tracking model, an equipment fault diagnosis and prediction model, a data storage module, and a computing unit. It is used for electric tugboat detection and tracking, marine obstacle identification, electric tugboat crew behavior tracking, and internal equipment detection data analysis.

2. The AI-based intelligent monitoring system for safe navigation of electric tugboats according to claim 1, characterized in that: The shooting device is linked to the monitoring information cache module via a video capture card. The shooting device is combined with an infrared thermal imager, which uses infrared thermal imaging technology to ensure the monitoring effect of maritime obstacles at night or in bad weather conditions. The infrared thermal imager is connected to an AI intelligent recognition algorithm system, which uses a maritime obstacle detection and tracking model and a computing unit to identify maritime obstacles.

3. The AI-based intelligent monitoring system for safe navigation of electric tugboats according to claim 1, characterized in that: The multiple sensors collect the operating parameters of the internal equipment of the electric tugboat in real time and map them onto the 3D simulation module. The 3D simulation module can dynamically reflect the changes in the state of the internal equipment, realizing the digital modeling of the internal equipment of the electric tugboat and the online twin of the internal equipment data.

4. The AI-based intelligent monitoring system for safe navigation of electric tugboats according to claim 1, characterized in that: The multiple sensors are interconnected with the AI ​​intelligent recognition algorithm system. The sensors detect the status of the equipment inside the tugboat in real time and use AI algorithms to perform health assessments. By collecting equipment operation data through the sensors and combining it with the equipment fault diagnosis and prediction model, potential problems can be detected in advance and maintenance suggestions can be generated to improve navigation safety.

5. The AI-based intelligent monitoring system for safe navigation of electric tugboats according to claim 4, characterized in that: The signals detected by the multiple sensors are connected to the AI ​​intelligent recognition algorithm system for health assessment of equipment operation status. The system uses the equipment fault diagnosis prediction model and calculation unit to perform health analysis. Through equipment fault management, the system obtains equipment fault information, identifies the causes and solutions to the faults, and provides early warning diagnosis of equipment status and risk prediction. The various devices that receive signals on the electric tugboat include the propulsion motor, lithium battery, rudder propeller, power distribution system and auxiliary equipment.

6. The AI-based intelligent monitoring system for safe navigation of electric tugboats according to claim 5, characterized in that: The multiple sensors include vibration sensors, which detect vibration signals during the operation of the electric tug's propulsion motor. The equipment fault diagnosis and prediction model and calculation unit use big data to diagnose the vibration signals of the propulsion motor, extract abnormal values ​​for analysis, and realize early warning of the equipment.

7. The AI-based intelligent monitoring system for safe navigation of electric tugboats according to claim 6, characterized in that: The multiple sensors include a current sensor, a voltage sensor, and a temperature sensor. The current sensor detects the charging and discharging current of the lithium battery, the voltage sensor detects the charging and discharging voltage of the lithium battery, and the temperature sensor detects the temperature of the lithium battery, the distribution board, and the transformer windings. The equipment fault diagnosis prediction model and calculation unit use big data to diagnose and analyze parameters such as lithium battery temperature, overcurrent during charging and discharging, excessive total voltage, and the temperature of the distribution board and transformer windings, which facilitates fault diagnosis and problem tracing.

8. A method for intelligent monitoring of safe navigation of electric tugboats based on AI, as described in any one of claims 1-7, comprising the following steps: Step 1: The monitoring system transmits the real-time collected information on the waters surrounding the electric tugboat, its internal equipment, and crew to the visualization digital twin system. The system uses a 3D simulation module to build and simulate the waters surrounding the electric tugboat, its internal equipment, and crew, and then displays the simulation in real time through a display device. Step 2: The monitoring system transmits video surveillance data, real-time status and location information of the electric tugboat, and internal equipment status information to the AI ​​intelligent recognition algorithm system; Step 3: The AI ​​intelligent recognition algorithm system starts the computing unit and, based on the crew detection and behavior tracking model, the marine obstacle detection and tracking model, and the equipment fault diagnosis and prediction model, performs real-time detection, analysis, and diagnosis of the waters surrounding the electric tugboat and its internal equipment. It identifies the behavior of the electric tugboat crew, analyzes different types of data, and stores the detection results of the electric tugboat's internal equipment, crew behavior tracking results, real-time status data of the electric tugboat, time, and other information in the data storage module. The above analysis and judgment results are then sent to the display device of the visualization digital twin system for display. Step 4: The visual digital twin system provides feedback to the user on anomalies based on the data analysis results received from the AI ​​intelligent recognition algorithm system. It promptly detects and identifies internal faults of the electric tugboat, unsafe behaviors of the crew, and obstacles at sea such as ice floes and floating objects, so that the user can schedule and maintain the electric tugboat.

9. The AI-based intelligent monitoring method for safe navigation of electric tugboats according to claim 8, characterized in that: The training method for the crew member detection and behavior tracking model includes the following steps: S1. The camera captures video and image data of the crew and performs enhancement processing. The data enhancement is achieved by rotating, blurring, and occluding the images. S2. Establish a crew dataset, annotate and preprocess each image data and store it in the crew dataset; S3. Establish a target detection model in the AI ​​intelligent recognition algorithm system, train the target detection model using the crew dataset, and obtain a crew detection and behavior tracking model; S4. Based on the crew detection and behavior tracking model, the computing unit in the AI ​​intelligent recognition algorithm system detects crew members in the video data collected by the shooting device, calculates the changes in the crew members' positions within the image to achieve crew tracking, and calculates the changes in the crew members' positions within the image through IoU (Intersection over Union), that is, the degree of overlap of the target detection boxes in two consecutive frames to achieve crew tracking.

10. The AI-based intelligent monitoring method for safe navigation of electric tugboats according to claim 8, characterized in that: The training method for the maritime obstacle detection and tracking model includes the following steps: S1. Infrared thermal imager, camera device, drone and radar scanning device collect image data of maritime obstacles and targets, establish maritime target dataset in AI intelligent recognition algorithm system, and label and preprocess the maritime obstacle and target data in the images and store them in the maritime target dataset. S2. Establish a maritime target detection model in the AI ​​intelligent recognition algorithm system, and train the maritime target detection model using a maritime target dataset. The training steps are the same as those for the crew target detection model mentioned above. Use the computing unit in the AI ​​intelligent recognition algorithm system to detect the collected maritime obstacle targets and calculate the changes in the position of the maritime obstacles within the image to achieve maritime obstacle tracking. S3. Train the ReID (Person Re-identification) model to extract target appearance features, and fine-tune it using a marine target dataset; input the detection and tracking results of the above marine obstacles into the DeepSORT (Multi-Target Tracking) model, predict the target position through Kalman filtering, and match it with the ReID model features; S4. Verify the above detection and tracking effects using MAP (Mean Average Precision) and MOTA (Multi-Target Tracking Accuracy) metrics.

Images