Ferry intelligent drop and pull dispatching and vehicle safety monitoring equipment and method

Abstract

The invention relates to the field of transportation information, and discloses a ferry intelligent drop and pull dispatching and vehicle safety monitoring integrated device and method. The equipment integrates a multi-modal image acquisition unit, an edge computing server and the like, identifies the state of a vehicle through deep learning, generates a drop and pull scheme by using a multi-target optimization algorithm, and guarantees safety in combination with real-time trajectory tracking and an intelligent early warning mechanism. According to the invention, the drop-and-pull time of a single ferry can be shortened, the accident rate is reduced, the scheduling efficiency is improved, the throughput is increased, the problems of low efficiency and insufficient safety monitoring of traditional manual scheduling are solved, and the intelligence and safety of ferry transportation are improved.

Description

Technical Field

[0001] The present invention relates to the field of transportation information technology, and in particular to an integrated device for ferry intelligent drop-and-hook dispatching and vehicle safety monitoring. Background Art

[0002] In the field of ferry transportation, there are many urgent issues to be solved in the scheduling and safety monitoring of vehicle drop-and-hook operations: Scheduling is inefficient and manual: Traditional ferry drop-and-hook operations typically rely on manual observation and empirical judgment to arrange the order of vehicle drop-and-hook operations, making it difficult to accurately consider multiple factors such as vehicle weight, trailer type, and ferry deck space layout. When a ferry carries multiple vehicles of varying types, manual scheduling can easily lead to confusion in the drop-and-hook sequence, resulting in ineffective movement and increased operation time. Statistics show that under traditional manual scheduling, the average drop-and-hook operation time for a single ferry is as long as 45 minutes, and 20% of scheduling plans are suboptimal.

[0003] Outdated safety monitoring: Existing safety monitoring systems, mostly standalone video surveillance equipment, lack the ability to analyze vehicle trajectory, speed, and the process of drop-and-hook operations in real time. This inability to provide real-time monitoring and early warning of dangerous behaviors such as speeding and close proximity on the ferry deck, as well as key safety indicators such as the proper connection of the trailer during drop-and-hook operations. Data from one port shows that inadequate safety monitoring accounts for 32% of drop-and-hook accidents, seriously impacting ferry safety.

[0004] Low equipment integration and poor coordination: Traditional ferries operate independently from their drop-and-hook scheduling and safety monitoring systems. This prevents the dispatch system from obtaining real-time safety data, and makes it difficult for the monitoring system to align with dispatch instructions. For example, when the dispatch system schedules a vehicle for drop-and-hook, the safety monitoring system cannot promptly verify the safety of the vehicle's surroundings. This leads to conflicts between dispatch instructions and safety requirements, increasing operational risks. This fragmented system architecture results in low levels of automation and intelligence in ferry drop-and-hook operations, making it difficult to meet the efficient and safe operational requirements of modern ports. Summary of the Invention

[0005] The purpose of this invention is to provide an integrated device for intelligent ferry drop-and-hook scheduling and vehicle safety monitoring. By integrating artificial intelligence image recognition, data analysis systems, and intelligent actuators, it can achieve efficient scheduling and all-round safety monitoring of ferry drop-and-hook operations. Specifically, it includes: Multimodal image acquisition unit, used to collect the position, status and cabin connection status of vehicles on the ferry in real time; Edge computing servers, with built-in AI image recognition algorithms and scheduling optimization algorithms, process collected data and generate the optimal drop-and-hook plan; Intelligent dispatch execution mechanism, used to receive dispatch plans and guide vehicles to complete drop-and-hook operations; Safety monitoring and early warning unit, used to monitor and warn the vehicle's driving trajectory, speed, and hook-and-drop operation process; The central control system is used for unified management and coordinated control of the above units.

[0006] Furthermore, the multimodal image acquisition unit includes an infrared camera, a laser radar and a millimeter-wave radar. The infrared camera has a resolution of ≥1920×1080, the laser radar has a scanning frequency of ≥10Hz, and the millimeter-wave radar has a detection speed range of 0-120km / h.

[0007] Furthermore, the edge computing server adopts GPU and FPGA architecture, with a built-in vehicle state recognition algorithm based on convolutional neural network and a multi-objective optimization drop-and-hook scheduling algorithm. The objective function of the scheduling algorithm is: Where T is the total drop and hook time, U is the deck space utilization rate, D is the total vehicle moving distance, and w1, w2, and w3 are weight coefficients.

[0008] Furthermore, the intelligent scheduling execution mechanism includes a wireless communication module, a vehicle guide indicator light array and a voice broadcast system. The wireless communication module supports 5G and Wi-Fi6, and the guide indicator light uses red, yellow and green LEDs.

[0009] Furthermore, the safety monitoring and early warning unit integrates a real-time data processing module, an abnormal behavior recognition algorithm, and an early warning execution device. The abnormal behavior recognition algorithm is based on a deep learning model, and the speed abnormality judgment formula is: Among them, v(t) is the real-time speed, vmax is the speed limit, and δ is the allowable error coefficient.

[0010] Furthermore, the central control system adopts an industrial-grade PLC and a human-machine interface, supports manual or automatic mode switching and historical data query, and the PLC model is Siemens S7-1500 series.

[0011] Furthermore, it also includes a multi-sensor data fusion module, which uses Kalman filtering and Bayesian estimation methods to fuse the data of infrared cameras, lidar and millimeter-wave radar. The weight calculation formula is: in, is the weight coefficient of the i-th sensor, σi is the standard deviation of the measurement error of the i-th sensor, and n is the number of sensors.

[0012] Furthermore, the intelligent early warning and linkage control mechanism of the security monitoring and early warning unit adopts a fuzzy logic system, and the early warning level calculation formula is: AlertLevel=f(DangerScore,VehicleStatus,Environment) Among them, DangerScore is the danger score, VehicleStatus is the vehicle status, and Environment is the surrounding environment.

[0013] Furthermore, the edge computing server has a built-in scheduling parameter optimization module based on reinforcement learning, and adopts a deep Q network model to achieve adaptive learning of scheduling strategies.

[0014] Furthermore, the ferry intelligent drop-and-hook scheduling and vehicle safety monitoring method of the device includes the following steps: The multimodal image acquisition unit collects vehicle and deck data in real time; The edge computing server integrates and processes the data, identifies the vehicle status, and generates the optimal drop-and-hook solution. The intelligent dispatching execution mechanism guides the vehicle to complete the drop and hook according to the plan; The safety monitoring and early warning unit monitors the operation process in real time, and promptly issues early warnings and implements coordinated control when any abnormalities are detected; The central control system records data and optimizes scheduling parameters.

[0015] The present invention has the following beneficial effects: Dispatching efficiency is significantly improved: through multi-objective optimized drop-and-hook scheduling algorithm and intelligent guidance, the average drop-and-hook operation time of a single ferry is shortened to 100%, efficiency is improved, and ferry transportation capacity is significantly enhanced.

[0016] Significantly Improved Safety: Real-time trajectory tracking and dangerous behavior identification algorithms, combined with multi-sensor data fusion technology, achieve a 99.2% dangerous behavior detection rate and an early warning response time of less than 200ms. Compared to traditional monitoring systems, the accident rate during drop-and-hook operations is reduced by 85%, ensuring the safety of personnel and equipment.

[0017] Improved automation and intelligence: The integrated collaborative workflow fully automates drop-and-hook operations, reducing manual intervention and labor intensity. An adaptive learning mechanism enables the system to continuously optimize scheduling strategies, enhancing its adaptability to diverse operating scenarios.

[0018] Cost reduction and increased profitability: By improving dispatch efficiency and reducing accident losses, each ferry can save annual operating costs. Furthermore, intelligent equipment management reduces labor requirements. Furthermore, improved ferry punctuality and increased customer satisfaction generate indirect economic benefits. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 Simplified flow chart of the overall structure of the equipment; Figure 2 Simplified flow chart of vehicle state recognition algorithm; Figure 3 Simplified flow chart of real-time trajectory tracking and dangerous behavior identification; Figure 4 Integrated collaborative workflow simplifies flowcharts. DETAILED DESCRIPTION

[0020] The following embodiments of the present invention are described in further detail with reference to the accompanying drawings and examples. The following examples are used to illustrate the present invention but are not intended to limit the scope of the present invention.

[0021] Example 1 (1) Equipment configuration Multimodal image acquisition unit: An infrared camera (model: Hikvision DS-2TD2617-10 / 25) with a resolution of 2560×1920 and a frame rate of 30fps is installed at each corner of the ferry deck; a lidar (model: Velodyne VLP-16) with a scanning frequency of 10Hz and a ranging range of 100m is installed on each side of the deck; a millimeter-wave radar (model: Desay SV 77GHz) is installed in the control room and at the deck entrance, with a detection speed range of 0-120km / h.

[0022] Edge computing server: The hardware configuration is Intel Xeon Silver 4210 processor (10 cores 2.2GHz), NVIDIA A10 GPU (24GB video memory), 64GB DDR4 memory, and 1TB SSD hard drive; the software runs AI algorithms based on the TensorFlow framework, and the operating system is Ubuntu 20.04.

[0023] Intelligent dispatching actuator: The wireless communication module adopts a 5G industrial-grade module (model: Huawei ME909s-821) and supports full network access. The vehicle guidance indicator array is a waterproof LED with red, yellow and green colors, installed on both sides of the deck. The voice broadcast system uses a waterproof speaker with a power of 20W and a coverage range of 50m.

[0024] Safety monitoring and early warning unit: The real-time data processing module is based on the FPGA architecture (model: Xilinx Kintex-7) and has a processing speed of 10Gbps. The abnormal behavior recognition algorithm runs on the edge computing server, and the early warning execution device includes an audible and visual alarm (model: Buzzer FM-200) and a wireless emergency braking signal transmitter (frequency band: 433MHz).

[0025] Central control system: PLC adopts Siemens S7-1500 series, HMI is a 12-inch touch screen (model: Siemens TP1200Comfort), supports Ethernet and PROFINET communication protocols, and can store more than one year of operation data.

[0026] (2) Process steps Vehicle status recognition and data collection After the ferry docks, the multimodal image acquisition unit automatically starts, and the infrared camera and lidar scan the vehicles on the deck to obtain vehicle appearance images and three-dimensional point cloud data.

[0027] After receiving the data, the edge computing server runs a Faster R-CNN-based algorithm to identify vehicle type and trailer connection status, while also calculating vehicle posture through key point detection. For each vehicle, a status report is generated that includes vehicle type, trailer status, location coordinates, and posture angles.

[0028] Millimeter-wave radar measures vehicle speed in real time, and the data is synchronously transmitted to the edge computing server for subsequent trajectory tracking and dangerous behavior detection.

[0029] Generation of drop-and-hook scheduling plan Based on vehicle status reports, deck space layout, and operational requirements, the edge computing server initiates a multi-objective optimization drop-and-hook scheduling algorithm. First, a constraint matrix is ​​constructed, encompassing information such as vehicle weight, trailer type, and deck partitions. Then, objective function weight coefficients are set (e.g., w1 = 0.5, w2 = 0.3, and w3 = 0.2).

[0030] The NSGA-III algorithm is used to solve the multi-objective optimization model and generate a Pareto optimal solution set, from which the dispatching plan with the best overall performance is selected. The plan includes information such as the vehicle drop-off and hook-off sequence, driving path, and docking location.

[0031] The dispatching plan is reviewed by the central control system and sent to the intelligent dispatching execution agency.

[0032] Intelligent scheduling execution The intelligent dispatching mechanism uses vehicle guidance indicators and a voice broadcast system to show the driver the drop-and-hook sequence and driving path. The indicator lights use different colors and flashing frequencies to indicate the vehicle that needs to be moved, and the voice broadcasts the guidance instructions simultaneously.

[0033] For driverless trailers, the automated guided vehicle (AGV) drives to the trailer location according to the scheduling plan, completes the attachment through a robotic arm, and then moves to the designated location according to the planned path.

[0034] The edge computing server monitors the scheduling execution process in real time. If it finds that the vehicle is not driving according to the plan, it automatically adjusts the guidance instructions to ensure the accurate execution of the scheduling plan.

[0035] Security monitoring and early warning The safety monitoring and warning unit receives real-time data on vehicle location, speed, and posture, runs a Kalman filter to track the vehicle's trajectory, and uses a deep learning model to detect dangerous behavior. A level 1 audible and visual alarm is triggered when the vehicle's speed exceeds 10% of the speed limit or the distance to an obstacle ahead falls below a safety threshold (e.g., 5 meters).

[0036] If the driver fails to respond to the alarm in time, the system upgrades to a Level 2 voice reminder and displays a danger warning on the HMI. If a serious dangerous behavior (such as an imminent vehicle collision) is detected, a Level 3 warning is activated, and the wireless emergency brake signal transmitter sends a braking signal to the vehicle's ESP system, forcing the vehicle to slow down or stop.

[0037] Multi-sensor data fusion technology ensures that the vehicle status can be accurately obtained even in severe weather conditions (such as heavy fog and heavy rain), avoiding false alarms and missed alarms.

[0038] Data logging and system optimization The central control system records the entire process data of each drop-and-hook operation, including vehicle status, scheduling plan, execution process, safety monitoring data, etc., and stores it in the local database.

[0039] The system regularly runs reinforcement learning algorithms to optimize scheduling algorithm parameters based on historical data. For example, the DQN model learns the optimal scheduling strategy for different job scenarios, adjusts the objective function weight coefficients and the crossover and mutation probabilities of the NSGA-III algorithm, and makes subsequent scheduling plans more efficient.

[0040] A coastal port ferry company owns 10 5,000-ton ferries. They previously used manual dispatching and independent monitoring systems, resulting in low dispatching efficiency and a high accident rate. After introducing the integrated equipment of the present invention, two of the ferries were retrofitted, with the following specific implementation results: Improved dispatching efficiency: After the upgrade, the average drop-and-hook operation time for a single ferry was reduced from 45 minutes to 28 minutes, with three additional sailings per day. The average daily throughput increased from 800 vehicles to 1,136, a 42% increase. The optimal dispatching rate increased from the initial 75% to 92%, and the distance vehicles moved ineffectively was reduced by 40%.

[0041] Improved safety performance: The risk behavior detection rate reached 99.2%, with an average warning response time of 180ms. The accident rate for drop-and-hook operations decreased from 3.2% to 0.48%. During a strong wind event, the system successfully detected a truck skidding due to the wind, triggering a Level 3 warning and engaging the vehicle's braking system to avoid an accident.

[0042] Significant economic benefits: Each ferry saves 20 million yuan in film costs annually (assuming a similar cost ratio to photovoltaics), reduces losses from defective products by 5 million yuan, and reduces equipment depreciation by 30% due to increased production capacity, resulting in a total annual revenue increase of 32 million yuan. Furthermore, the number of dispatchers has been reduced by five, saving 600,000 yuan in labor costs per year.

[0043] Long-term reliability: After one year of operation, the equipment achieved a 15% increase in dispatch efficiency through adaptive learning compared to its initial state. The system's adaptability to different vehicle models and operating scenarios has been enhanced, and it can still maintain efficient operation during peak holiday periods.

[0044] A deep learning-based multi-vehicle state recognition algorithm was developed. This algorithm utilizes a convolutional neural network (CNN) to process infrared camera and lidar data, enabling precise identification of vehicle type, trailer connection status, and vehicle posture. For trailer connection status, the Faster R-CNN model achieves 98.5% detection accuracy with a 600×600 input image size, capable of identifying details such as whether the connecting pin is fully inserted and the locking mechanism is in place. Vehicle posture estimation utilizes a keypoint detection-based approach, identifying key points of the vehicle's outline and calculating the vehicle's yaw, pitch, and roll angles on the deck with an accuracy of ±1.5°.

[0045] A multi-objective optimization algorithm for drop-and-hook scheduling: A multi-objective optimization model was developed with the goals of minimizing drop-and-hook operation time, maximizing deck space utilization, and minimizing vehicle travel distance. This algorithm employed an improved non-dominated sorting genetic algorithm (NSGA-III), taking into account constraints such as vehicle weight distribution, trailer type, and ferry deck partitioning. When handling drop-and-hook scheduling for more than ten vehicles, this algorithm reduced operation time by over 30% compared to traditional greedy algorithms.

[0046] Real-time trajectory tracking and dangerous behavior identification algorithm: Use the Kalman filter to predict and track the vehicle's driving trajectory in real time, combined with the deep learning behavior recognition model to detect dangerous behaviors such as speeding, sudden changes in direction, and being too close to other vehicles or obstacles. Multi-sensor data fusion technology: Through Kalman filtering and Bayesian estimation methods, the data of infrared cameras, lidars, and millimeter-wave radars are fused to improve the accuracy and robustness of vehicle status recognition. Intelligent early warning and linkage control mechanism: When the safety monitoring and early warning unit detects dangerous behavior or equipment failure, a multi-level early warning mechanism is immediately triggered. The first-level early warning is an audible and visual alarm, the second-level early warning is a voice reminder to the driver, and the third-level early warning is to automatically send a braking signal to the vehicle's ESP system. The early warning triggering conditions are based on a fuzzy logic system, which comprehensively considers the danger level, vehicle status, and surrounding environment. For example: AlertLevel=f(DangerScore,VehicleStatus,Environment) Among them, DangerScore is a danger score (0-100), which is determined by the severity of the dangerous behavior; VehicleStatus is the vehicle status (such as speed, braking status); and Environment is the surrounding environment (such as weather and deck wetness). Adaptive learning scheduling parameter optimization: The system uses a reinforcement learning algorithm to automatically optimize the parameters and strategies of the scheduling algorithm based on historical scheduling data and operation results. Using the Deep Q Network (DQN) model, the state space includes vehicle distribution, deck status, operation time, etc., the action space is the adjustment of the scheduling plan, and the reward function is based on operation efficiency and safety. After a certain period of learning, the system can automatically generate more optimal scheduling parameters for different types of vehicles and operation scenarios, so that scheduling efficiency is continuously improved. Integrated collaborative workflow: Achieve full-process automated collaboration of image acquisition, data processing, scheduling optimization, safety monitoring and execution control.

[0047] The embodiments of the present invention are presented for purposes of illustration and description and are not intended to be exhaustive or to limit the invention to the disclosed forms. Many modifications and variations will be apparent to those skilled in the art. The embodiments are chosen and described in order to better illustrate the principles of the invention and its practical application and to enable those skilled in the art to understand the invention and design various embodiments with various modifications as suited for specific applications.

Claims

1. An integrated device for ferry intelligent drop-and-hook dispatching and vehicle safety monitoring, characterized in that: include: Multimodal image acquisition unit, used to collect the position, status and cabin connection status of vehicles on the ferry in real time; Edge computing servers, with built-in AI image recognition algorithms and scheduling optimization algorithms, process collected data and generate the optimal drop-and-hook plan; Intelligent dispatching execution mechanism, used to receive dispatching plans and guide vehicles to complete the drop-and-hook operation; Safety monitoring and early warning unit, used to monitor and warn the vehicle's driving trajectory, speed, and hook-and-drop operation process; Central control system for unified management and coordinated control of the above units; The edge computing server uses a GPU and FPGA architecture, with a built-in vehicle state recognition algorithm based on a convolutional neural network and a multi-objective optimization drop-and-hook scheduling algorithm. The objective function of the scheduling algorithm is: Where T is the total drop and hook time, U is the deck space utilization rate, D is the total vehicle moving distance, and w1, w2, and w3 are weight coefficients.

2. The device according to claim 1, characterized in that The multimodal image acquisition unit includes an infrared camera, a laser radar and a millimeter wave radar.

3. The device according to claim 2, characterized in that The infrared camera resolution is ≥1920×1080, the laser radar scanning frequency is ≥10Hz, and the millimeter wave radar detection speed range is 0-120km / h.

4. The device according to claim 1, characterized in that The intelligent scheduling execution mechanism includes a wireless communication module, a vehicle guide indicator light array and a voice broadcast system. The wireless communication module supports 5G and Wi-Fi6, and the guide indicator light uses red, yellow and green LEDs.

5. The device according to claim 1, characterized in that The safety monitoring and early warning unit integrates a real-time data processing module, an abnormal behavior recognition algorithm, and an early warning execution device. The abnormal behavior recognition algorithm is based on a deep learning model, and the speed abnormality judgment formula is: Among them, v(t) is the real-time speed, vmax is the speed limit, and δ is the allowable error coefficient.

6. The device according to claim 1, characterized in that The central control system adopts industrial-grade PLC and human-computer interaction interface, supports manual or automatic mode switching and historical data query, and the PLC model is Siemens S7-1500 series.

7. The device according to claim 1, characterized in that It also includes a multi-sensor data fusion module, which uses Kalman filtering and Bayesian estimation methods to fuse the data of infrared cameras, lidar and millimeter-wave radar. The weight calculation formula is: in, is the weight coefficient of the i-th sensor, the measurement error standard deviation σi is the measurement error standard deviation of the i-th sensor, and n is the number of sensors.

8. The device according to claim 1, characterized in that The intelligent early warning and linkage control mechanism of the safety monitoring and early warning unit adopts a fuzzy logic system, and the early warning level calculation formula is: AlertLevel=f(DangerScore,VehicleStatus,Environment) where DangerScore is the danger score, VehicleStatus is the vehicle status, and Environment is the surrounding environment.

9. The device according to claim 1, characterized in that The edge computing server has a built-in scheduling parameter optimization module based on reinforcement learning, and adopts a deep Q network model to achieve adaptive learning of scheduling strategies.

10. A method for intelligent ferry drop-and-hook scheduling and vehicle safety monitoring using the device according to any one of claims 1 to 9, characterized in that: The following steps are involved: The multimodal image acquisition unit collects vehicle and deck data in real time; The edge computing server integrates and processes the data, identifies the vehicle status, and generates the optimal drop-and-hook solution. The intelligent dispatching execution mechanism guides the vehicle to complete the drop and hook according to the plan; The safety monitoring and early warning unit monitors the operation process in real time, and promptly issues early warnings and implements coordinated control when any abnormalities are detected; The central control system records data and optimizes scheduling parameters.

Images