Bulk cargo terminal operation area man-vehicle position monitoring system based on multi-source data fusion

Abstract

The invention relates to a bulk cargo terminal operation area man-vehicle position monitoring system based on multi-source data fusion. The system comprises a heterogeneous sensing layer, a reliable transmission layer, an intelligent fusion and decision-making layer and a panoramic application layer. The heterogeneous sensing layer collects multivariate heterogeneous data of human and vehicle targets through a UWB positioning unit, a GNSS / INS combined positioning unit, an AI visual unit and an area sensing unit. The reliable transmission layer transmits the multivariate heterogeneous data to the processing center through the heterogeneous fusion network in a high-reliability and low-delay manner; the intelligent fusion and decision-making layer comprises a space-time unification gateway, an adaptive confidence evaluation and fusion engine, a dynamic digital twin model and a predictive early warning engine; the predictive early warning engine is used for carrying out risk early warning; and the panoramic application layer is used for realizing visualization, interaction and decision support of a monitoring scene. According to the invention, a full-scene, high-precision, high-reliability and predictable man-vehicle cooperative safety monitoring system can be realized, and the active safety prevention and control capability and the operation efficiency of a wharf are essentially improved.

Description

Technical Field

[0001] This invention relates to the technical field of industrial Internet of Things and smart safety monitoring, and in particular to a personnel and vehicle location monitoring system for bulk cargo terminal operation areas based on multi-source data fusion. Background Technology

[0002] Bulk cargo terminals are critical nodes in the logistics chain, and their operational areas exhibit four main characteristics: large area (encompassing storage yards, roads, frontage areas, silos, etc.), diverse elements (mobile machinery, transport vehicles, personnel, and fixed facilities intermingled), highly dynamic (continuously changing loading and unloading processes, and real-time updates to cargo stack formations), and numerous blind spots (visual blind spots, signal obstruction areas, and areas affected by severe weather). Traditional safety management models face severe challenges.

[0003] Passive response: mainly relies on fixed video surveillance with manual monitoring and walkie-talkie communication. It can only be traced after an accident occurs, and the prevention capability is weak.

[0004] Data fragmentation: The vehicle dispatching system, access control system, and video system operate independently, forming "information silos" that make it impossible to conduct a comprehensive assessment of the risks associated with the interaction between people, vehicles, and the environment.

[0005] Positioning technology bottlenecks: GNSS suffers from multipath effects due to cargo stacking obstructions in storage yards, causing accuracy to plummet to over 5 meters; it is completely ineffective indoors (such as in transfer stations). While UWB offers high accuracy, it is susceptible to multipath interference in open metallic environments (such as flat storage yards); full coverage deployment is prohibitively expensive. Visual recognition is significantly affected by lighting (glare, nighttime), weather (rain, fog, dust), and obstructions, and cannot provide precise absolute coordinates. RFID / ZigBee only provides area-based presence detection and cannot meet dynamic tracking requirements.

[0006] Therefore, there is an urgent need for an intelligent monitoring solution that can adapt to environmental changes, make comprehensive use of the advantages of multi-source information, and achieve a balance between cost and performance. Summary of the Invention

[0007] The present invention aims to address the shortcomings of the prior art by providing a personnel and vehicle location monitoring system for bulk cargo terminal operation areas based on multi-source data fusion.

[0008] To achieve the above objectives, the present invention adopts the following technical solution:

[0009] A multi-source data fusion-based monitoring system for the location of personnel and vehicles in a bulk cargo terminal operation area includes a heterogeneous perception layer, a reliable transmission layer, an intelligent fusion and decision-making layer, and a panoramic application layer.

[0010] The heterogeneous perception layer collects diverse and heterogeneous data on human and vehicle targets through UWB positioning units, GNSS / INS combined positioning units, AI vision units, and regional perception units.

[0011] The reliable transport layer transmits diverse and heterogeneous data to the processing center with high reliability and low latency through a heterogeneous converged network;

[0012] The intelligent fusion and decision-making layer includes a unified spatiotemporal gateway, an adaptive confidence assessment and fusion engine, a dynamic digital twin model, and a predictive early warning engine. The predictive early warning engine is used to assess the confidence of data sources, perform multi-layer data fusion, drive twin updates, and provide risk warnings based on trajectory prediction.

[0013] The panoramic application layer is used to realize the visualization, interaction and decision support of monitoring scenarios.

[0014] UWB positioning units include UWB positioning base stations deployed at key risk points, UWB personnel positioning tags worn by personnel, and UWB vehicle positioning units installed on mobile vehicles.

[0015] The GNSS / INS integrated positioning unit includes a multi-frequency GNSS receiver module and a six-axis inertial measurement unit installed on a moving vehicle.

[0016] AI vision units include smart network cameras and / or thermal imaging cameras with built-in AI computing cards.

[0017] Area sensing units include RFID deployed at entrances and exits of enclosed spaces, as well as electronic fence sensing cables or lidar beam sensors deployed in hazardous areas.

[0018] The reliable transmission layer adopts a redundant architecture that combines fiber optic ring network, 5G private network slicing and industrial Wi-Fi 6 network, and is equipped with an intelligent network manager to enable mobile terminals to roam seamlessly between heterogeneous networks.

[0019] The adaptive confidence assessment and fusion engine is configured to: calculate a dynamic confidence score for each data source in real time, the confidence score being determined based on the signal quality parameters of the data source itself, environmental interference factors, and the degree of consistency with other independent data sources; and use an iterative estimation algorithm based on confidence weighting to output the optimal state estimate of the target.

[0020] The dynamic digital twin model not only includes static geographic information, but also creates a dynamic twin with location, velocity, status and confidence attributes for each monitoring target, and supports real-time spatial relationship calculation and visualization between twins.

[0021] The predictive early warning engine is configured to: establish a short-term motion trajectory prediction model for each moving target; calculate the nearest encounter distance (CPA) and the time to nearest point (TCPA) between any two predicted trajectories of targets; dynamically set safety thresholds based on target type and operational scenario; and trigger tiered early warnings based on the comparison of CPA and TCPA with the safety thresholds.

[0022] The tiered early warning system includes: sending warning information to the vehicle-mounted or personnel terminals of dangerous targets; sending alarm information to the monitoring center; and, when an emergency risk is determined, sending deceleration suggestion commands to the power control system or auxiliary braking system of the target vehicle through a predefined interface.

[0023] The beneficial effects of this invention are: this invention can realize a human-vehicle collaborative safety monitoring system with full-scene coverage, high precision, high reliability, and predictability, which fundamentally improves the proactive safety control capabilities and operational efficiency of the port. Attached Figure Description

[0024] Figure 1 This is a system block diagram of the present invention;

[0025] The following will describe in detail, with reference to the accompanying drawings, embodiments of the present invention. Detailed Implementation

[0026] The principles and features of the present invention are described below with reference to the accompanying drawings. The embodiments given are for illustrative purposes only and are not intended to limit the scope of the invention. The invention is described more specifically in the following paragraphs by way of example with reference to the accompanying drawings. The advantages and features of the invention will become clearer from the following description. It should be noted that the drawings are in a very simplified form and use non-precise proportions, and are only used to facilitate and clarify the illustration of the embodiments of the invention.

[0027] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. The terminology used herein in the description of the invention is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. The term "and / or" as used herein includes any and all combinations of one or more of the associated listed items.

[0028] The present invention will be further described below with reference to the accompanying drawings and embodiments:

[0029] A monitoring system for the location of personnel and vehicles in the bulk cargo terminal operation area based on multi-source data fusion, such as Figure 1 As shown, it includes a heterogeneous perception layer, a reliable transmission layer, an intelligent fusion and decision-making layer, and a panoramic application layer.

[0030] 1. Heterogeneous perception layer.

[0031] The heterogeneous perception layer collects diverse and heterogeneous data on human and vehicle targets through UWB positioning units, GNSS / INS combined positioning units, AI vision units, and regional perception units.

[0032] UWB positioning units, or high-precision micro-coverage units, include UWB positioning base stations deployed at key risk points, UWB personnel positioning tags worn by personnel, and UWB vehicle-mounted positioning units installed on mobile vehicles.

[0033] Key risk areas include the 15-meter radius area below the ship loader chute, the area around the reclaimer's rotation center, the tippler room pit, and the main road intersection.

[0034] UWB personnel locator tags include explosion-proof and impact-resistant UWB-tagged helmets worn on the head.

[0035] Mobile vehicles include not only ordinary vehicles but also mobile machinery.

[0036] The UWB positioning unit provides a positioning accuracy of 10-30 cm in this area, serving as a "precision benchmark" for the system.

[0037] The GNSS / INS integrated positioning unit, also known as the wide-area macro coverage unit, includes a multi-frequency GNSS receiver module and a six-axis inertial measurement unit installed on a moving vehicle. It provides sub-meter level positioning in open areas. When entering areas with signal obstruction, the tightly coupled INS module initiates dead reckoning and suppresses error accumulation through zero-velocity correction (ZUPT) and vehicle dynamics constraint models, providing short-term, reliable, continuous position output.

[0038] AI vision units, also known as semantic perception and verification units, include intelligent network cameras and / or thermal imaging cameras with built-in AI computing cards.

[0039] Function 1 (Assisted Positioning and Identity Binding): The key point camera incorporates a lightweight deep learning model to identify vehicle license plates, machine serial numbers, and personnel safety helmet colors / numbers in real time. The identification results are then spatiotemporally correlated with the positioning data stream for verification. When brief jumps or conflicts occur in UWB / GNSS data, visual positioning bounding boxes are used for cross-validation and smoothing correction.

[0040] Function 2 (Untagged Target Detection): Detects temporary personnel or external vehicles without location tags, maps their 2D pixel coordinates to an approximate area of ​​the digital twin model using camera calibration parameters, and triggers an "Untagged Target Alarm" to alert management personnel.

[0041] The area sensing unit includes RFID deployed at the entrances and exits of enclosed spaces to record personnel entry and exit, and electronic fence sensing cables or lidar beam sensors deployed in hazardous areas (such as water edges or high-voltage electrical boxes) as independent sensing backups at the physical level.

[0042] 2. Reliable transmission layer (industrial-grade neural network).

[0043] The reliable transport layer transmits diverse and heterogeneous data to the processing center with high reliability and low latency through a heterogeneous converged network.

[0044] The reliable transmission layer adopts a redundant architecture that combines fiber optic ring network, 5G private network slicing and industrial Wi-Fi 6 network, and is equipped with an intelligent network manager to enable mobile terminals to roam seamlessly between heterogeneous networks.

[0045] It adopts a three-layer heterogeneous network architecture of "fiber optic backbone ring network + 5G private network slicing + Wi-Fi 6 redundant access".

[0046] Fixed equipment (base stations, cameras, RFID readers) are connected via an industrial fiber optic ring network to ensure high bandwidth and high reliability.

[0047] Mobile terminal (personnel, vehicle) data is preferentially transmitted back via the dock's dedicated 5G network (uRLLC low-latency high-reliability slicing). In areas with localized 5G signal blind spots, it automatically switches to a pre-set industrial Wi-Fi 6 Mesh network. The network manager enables seamless roaming and load balancing.

[0048] 3. Intelligent integration and decision-making layer.

[0049] The intelligent fusion and decision-making layer includes a unified spatiotemporal gateway, an adaptive confidence assessment and fusion engine, a dynamic digital twin model, and a predictive early warning engine. The predictive early warning engine is used to assess the confidence of data sources, perform multi-layer data fusion, drive twin updates, and provide risk warnings based on trajectory prediction.

[0050] Spatiotemporal unified access gateway: It adds a high-precision BeiDou network timestamp (PTP protocol) to all incoming data and transforms all spatial coordinates into an independent plane coordinate system with the dock measurement control point as the origin, eliminating reference differences.

[0051] The adaptive confidence assessment and fusion engine is configured to: calculate a dynamic confidence score for each data source in real time, the confidence score being determined based on the signal quality parameters of the data source itself, environmental interference factors, and the degree of consistency with other independent data sources; and use an iterative estimation algorithm based on confidence weighting to output the optimal state estimate of the target.

[0052] Confidence dynamic quantification model: Calculates a 0-1 confidence score C_i(t) in real time for each data source (e.g., the UWB data stream of a vehicle).

[0053] For UWB: C_uwb=f(signal-to-noise ratio, number of base stations involved in the calculation, multipath factor).

[0054] For GNSS: C_gnss=f(number of satellites, HDOP value, carrier phase continuity).

[0055] For vision: C_vision = f(object detection confidence, illumination score, occlusion degree).

[0056] For INS: C_ins = g(estimated time, inertial device error estimate), which decays over time.

[0057] Multi-layer fusion strategy:

[0058] Data-level fusion: When there are multiple location estimates for the same target at the same time, the confidence-weighted maximum expectation (EM) algorithm is used to iteratively solve for the optimal location estimate (x,y).

[0059] Feature-level fusion: It associates the "behavioral features" of visual recognition (such as people running, vehicles reversing) with the "motion features" of positioning trajectories (such as sudden speed changes, sudden direction changes) to determine high-risk combination events such as "people suddenly running towards a moving loader".

[0060] The dynamic digital twin model not only includes static geographic information, but also creates a dynamic twin with location, velocity, status and confidence attributes for each monitoring target, and supports real-time spatial relationship calculation and visualization between twins.

[0061] It not only includes static 3D geographic information models (GIS+BIM), but more importantly, it creates corresponding "dynamic twins" for each person, vehicle, and key equipment. These twins synchronize their location, speed, status (working / idle / faulty), and real-time confidence levels output by the fusion engine in real time. Spatial relationship calculations (distance, orientation, relative speed) can be performed between the twins.

[0062] The predictive early warning engine is configured to: establish a short-term motion trajectory prediction model for each moving target; calculate the nearest encounter distance (CPA) and the time to nearest point (TCPA) between any two predicted trajectories of targets; dynamically set safety thresholds based on target type and operational scenario; and trigger tiered early warnings based on the comparison of CPA and TCPA with the safety thresholds.

[0063] Short-term trajectory prediction based on kinematic model: For each moving twin, using its current state (position, velocity, acceleration, heading angle) and combined with dock road topology constraints (e.g., road boundaries, one-way streets), its trajectory Traj_pred for the next 3-5 seconds is predicted through an improved Kalman filter or linear quadratic estimation.

[0064] Collision detection algorithm: Periodically calculate the closest point of approach (CPA) between the predicted trajectories of any two moving twins and the time to CPA (TCPA). Set dynamic safety thresholds (e.g., 3 meters for vehicle - vehicle and 5 meters for vehicle - pedestrian). If CPA < threshold and TCPA < set time (e.g., 4 seconds), it is determined that there is a collision risk.

[0065] Hierarchical early warning includes: Sending warning messages to the on - vehicle or personnel terminals of dangerous targets; Sending alarm messages to the monitoring center; And when it is determined as an emergency risk, sending deceleration suggestion commands to the power control system or auxiliary braking system of the target vehicle through a predefined interface.

[0066] Hierarchical early warning and linkage control:

[0067] Level 1 early warning (prompt): TCPA > 3 seconds, low risk. Send a visual prompt "There is a crossing risk ahead" to the on - vehicle terminal of the relevant vehicle.

[0068] Level 2 early warning (warning): 2 seconds < TCPA ≤ 3 seconds. Trigger the audible and visual alarm of the on - vehicle terminal and send a warning message to the dispatching center.

[0069] Level 3 early warning (emergency intervention): TCPA ≤ 2 seconds, or a person is detected to intrude into the dynamic operation radius of the mobile machinery. The system automatically sends a deceleration suggestion command to the dangerous vehicle (through the on - vehicle system interface), and simultaneously sends a strong warning to the audible and visual alarm posts in the adjacent area. When necessary, send a "紧急制动" (emergency braking) suggestion to the machinery operator.

[0070] 4. Panoramic application layer

[0071] The panoramic application layer is used to achieve visualization, interaction, and decision - making support for the monitoring scenario.

[0072] Global situation awareness command screen: Based on game engine technology, achieve high - fidelity and low - latency real - time rendering. It can switch the "positioning accuracy view" (displaying the current positioning confidence of each area in the form of a heat map), "risk heat map", and "operation efficiency view" with one key.

[0073] Intelligent dispatching assistance: Bind the high - precision real - time position with the operation task list to achieve refined guidance of "task to vehicle, vehicle to position". Provide the dispatcher with the optimal dispatching suggestions based on the position.

[0074] Panoramic event backtracking and analysis: Support "digital twin scene replay" for any time period, and can analyze the whole process of the event from multiple perspectives, pause, and slow - play. The system automatically generates an accident report framework that meets safety standards.

[0075] This invention significantly improves accuracy and reliability: through confidence-driven multi-source fusion, the overall positioning accuracy is stabilized within sub-meter level in 95% of scenarios (reaching centimeter level in key areas), and the signal loss or false alarm caused by single-point failure is significantly reduced.

[0076] This invention enables true all-weather monitoring: the fusion solution ensures that even when GNSS fails or visibility is limited (nighttime, dust, fog), the system still has backup data sources such as INS, UWB, or RFID to support it, thus guaranteeing monitoring continuity.

[0077] This invention moves from "perception" to "cognition": the system can not only "see" the location, but also "understand" behavioral intentions (such as loitering, speeding, and illegal crossing) through multi-feature fusion, thus achieving intention-level safety warnings.

[0078] This invention empowers lean operations: high-precision location data becomes a digital asset, used to analyze operational bottlenecks, optimize routes, and calculate equipment utilization, directly improving terminal throughput and economic benefits.

[0079] The invention enhances system resilience: the layered heterogeneous architecture design ensures that a local failure in any subsystem will not lead to a global failure, meeting the high availability requirements of industrial systems.

[0080] Example 1:

[0081] The present invention will be described in detail for a coal export terminal with an annual throughput of 50 million tons.

[0082] Step 1: System Deployment

[0083] UWB Network: Six dustproof and waterproof base stations are deployed under the cantilever of each of the two ship loaders to form two high-precision positioning "bubbles"; four base stations are deployed at the intersection of each of the three main roads.

[0084] Terminal equipment: Install multi-mode vehicle terminals integrating "GNSS+INS+UWB+5G" on 120 mobile machines (loaders, rakes, etc.); equip 300 permanent workers with smart safety helmets with built-in UWB tags and low-power Bluetooth.

[0085] Visual Upgrade: Upgrade 40 of the 80 existing high-definition network cameras at the dock with AI, and deploy a lightweight license plate / helmet recognition model.

[0086] Network construction: A fiber optic ring network was laid along the main road of the wharf, and three 5G private network micro base stations were deployed to achieve 99% wireless signal coverage in the work area.

[0087] Step Two: Data Fusion Process (Taking a loader in operation as an example):

[0088] The loader was retrieving materials from the yard. At this time, the GNSS signal was good (C_gnss=0.9), and the license plate was successfully recognized visually (C_vision=0.8). The fusion engine primarily used GNSS and secondarily used vision, outputting a position accuracy of 0.8 meters.

[0089] The loader drove under the ship loader to transfer the cargo. GNSS signal was lost (C_gnss=0.1), UWB signal was perfectly connected (C_uwb=0.98), and the vision system could not recognize the license plate due to the elevation angle, but detected the vehicle (C_vision=0.6). The engine instantly switched to UWB data as the absolute primary signal, outputting a position accuracy of 0.2 meters.

[0090] Throughout this process, the INS remains operational, providing continuous attitude and short-term position references for motion consistency verification during fusion.

[0091] Step 3: Predictive Warning Example (Collision Avoidance): A dump truck A travels at 20 km / h along the main road, its predicted trajectory is Traj_A. An electric inspection vehicle B exits from the side yard entrance, its predicted trajectory is Traj_B.

[0092] At time T0: The warning engine calculates CPA = 2.5 meters and TCPA = 3.2 seconds. A level-two warning is triggered.

[0093] Action: A voice alarm sounded in the cab of truck A saying "Passing on the left, be careful" and the screen flashed; electric vehicle B emitted a "beep beep" warning sound; a red warning line appeared between the icons of the two vehicles on the dispatch center's large screen.

[0094] Result: Both drivers took evasive action in advance, and the risk was averted. The system recorded the warning and the process of its resolution.

[0095] Step Four: Management Closed Loop

[0096] In one instance, the system issued a Level 3 warning that "personnel are approaching the operating material handling machine." The command center confirmed the situation via a video-linked pop-up window and remotely intervened via the broadcast system to stop them. Afterward, the entire incident was reconstructed using the panoramic review function for safety education and accountability analysis.

[0097] The present invention has been described above by way of example with reference to the accompanying drawings. Obviously, the specific implementation of the present invention is not limited to the above-described manner. Any improvements made using the inventive concept and technical solution of the present invention, or direct application to other occasions without modification, are all within the protection scope of the present invention.

Claims

1. A personnel and vehicle location monitoring system for bulk cargo terminal operation areas based on multi-source data fusion, characterized in that, It includes a heterogeneous perception layer, a reliable transmission layer, an intelligent fusion and decision-making layer, and a panoramic application layer; The heterogeneous perception layer collects diverse and heterogeneous data on human and vehicle targets through UWB positioning units, GNSS / INS combined positioning units, AI vision units, and regional perception units. The reliable transport layer transmits diverse and heterogeneous data to the processing center with high reliability and low latency through a heterogeneous converged network; The intelligent fusion and decision-making layer includes a unified spatiotemporal gateway, an adaptive confidence assessment and fusion engine, a dynamic digital twin model, and a predictive early warning engine. The predictive early warning engine is used to assess the confidence of data sources, perform multi-layer data fusion, drive twin updates, and provide risk warnings based on trajectory prediction. The panoramic application layer is used to realize the visualization, interaction and decision support of monitoring scenarios.

2. The bulk cargo terminal operation area personnel and vehicle location monitoring system based on multi-source data fusion according to claim 1, characterized in that, UWB positioning units include UWB positioning base stations deployed at key risk locations, UWB personnel positioning tags worn by personnel, and UWB vehicle positioning units installed on mobile vehicles.

3. The bulk cargo terminal operation area personnel and vehicle location monitoring system based on multi-source data fusion according to claim 2, characterized in that, The GNSS / INS integrated positioning unit includes a multi-frequency GNSS receiver module and a six-axis inertial measurement unit installed on a moving vehicle.

4. The bulk cargo terminal operation area personnel and vehicle location monitoring system based on multi-source data fusion according to claim 3, characterized in that, AI vision units include smart network cameras and / or thermal imaging cameras with built-in AI computing cards.

5. The bulk cargo terminal operation area personnel and vehicle location monitoring system based on multi-source data fusion according to claim 4, characterized in that, Area sensing units include RFID deployed at entrances and exits of enclosed spaces, as well as electronic fence sensing cables or lidar beam sensors deployed in hazardous areas.

6. The bulk cargo terminal operation area personnel and vehicle location monitoring system based on multi-source data fusion according to claim 1, characterized in that, The reliable transmission layer adopts a redundant architecture that combines fiber optic ring network, 5G private network slicing and industrial Wi-Fi 6 network, and is equipped with an intelligent network manager to enable mobile terminals to roam seamlessly between heterogeneous networks.

7. The bulk cargo terminal operation area personnel and vehicle location monitoring system based on multi-source data fusion according to claim 1, characterized in that, The adaptive confidence assessment and fusion engine is configured to: calculate a dynamic confidence score for each data source in real time, the confidence score being determined based on the signal quality parameters of the data source itself, environmental interference factors, and the degree of consistency with other independent data sources; and use an iterative estimation algorithm based on confidence weighting to output the optimal state estimate of the target.

8. The bulk cargo terminal operation area personnel and vehicle location monitoring system based on multi-source data fusion according to claim 7, characterized in that, The dynamic digital twin model not only includes static geographic information, but also creates a dynamic twin with location, velocity, status and confidence attributes for each monitoring target, and supports real-time spatial relationship calculation and visualization between twins.

9. The bulk cargo terminal operation area personnel and vehicle location monitoring system based on multi-source data fusion according to claim 8, characterized in that, The predictive early warning engine is configured to: establish a short-term motion trajectory prediction model for each moving target; calculate the nearest encounter distance (CPA) and the time to nearest point (TCPA) between any two predicted trajectories of targets; dynamically set safety thresholds based on target type and operational scenario; and trigger tiered early warnings based on the comparison of CPA and TCPA with the safety thresholds.

10. The bulk cargo terminal operation area personnel and vehicle location monitoring system based on multi-source data fusion according to claim 9, characterized in that, The tiered early warning system includes: sending warning information to the vehicle-mounted or personnel terminals of dangerous targets; sending alarm information to the monitoring center; and, when an emergency risk is determined, sending deceleration suggestion commands to the power control system or auxiliary braking system of the target vehicle through a predefined interface.

Images