A wharf yard superhigh monitoring system

Abstract

The utility model relates to a kind of wharf yard superhigh monitoring system, including detection radar, industrial computer, supervision terminal, monitoring camera, audible-visual alarm, detection radar frame height is set in the middle of wharf yard and / or the periphery of wharf yard, monitoring camera is installed in the middle of wharf yard and / or the periphery of wharf yard, supervision terminal is placed in the dispatching room of wharf yard, audible-visual alarm is installed in the dispatching room of wharf yard;Industrial computer is connected with detection radar communication, supervision terminal is connected with industrial computer communication, monitoring camera, audible-visual alarm is connected with supervision terminal communication.The beneficial effect is: staff only needs to receive audible-visual alarm alarm signal in room after, supervision is carried out through supervision terminal, staff does not need to stare at goods storage in outdoor for a long time, avoid staff eye fatigue, work in outdoor for a long time.

Description

Technical Field

[0001] This utility model relates to the field of dock management technology, and in particular to a dock yard over-height monitoring system. Background Technology

[0002] A wharf is a structure located by the sea or river that provides a place for ships or ferries to dock, allowing passengers to embark and disembark, and for cargo to be loaded and unloaded. Some wharves are used for unloading bulk cargo for extended periods. Because there is a risk of cargo collapsing and ground subsidence due to overloading when cargo is stacked to a certain height, it is necessary to strictly control the height of cargo stacking.

[0003] Currently, the supervision of cargo stacking can only be controlled by staff visually observing the stacks on-site. This requires staff to keep their eyes on the stacks for a long time, which can easily cause eye fatigue. In addition, staff are exposed to the elements for extended periods of time outdoors. Utility Model Content

[0004] The purpose of this invention is to overcome the above-mentioned problems in the existing technology and provide a monitoring system for excessive height in wharf yards.

[0005] To achieve the above-mentioned technical objectives and effects, this utility model is implemented through the following technical solution:

[0006] A terminal yard over-height monitoring system includes:

[0007] A detection radar is used to detect three-dimensional data of cargo piles in a terminal yard. The detection radar is mounted at an elevation in the middle of the terminal yard and / or on the outer perimeter of the terminal yard. The detection range of the detection radar covers the cargo piles in the terminal yard.

[0008] An industrial control computer is used to receive three-dimensional data output from the detection radar and analyze it to obtain the height parameters of the cargo pile. The industrial control computer is communicatively connected to the detection radar.

[0009] A monitoring terminal, which is connected to an industrial control computer, is installed in the dispatch room of the dock yard;

[0010] The surveillance camera is connected to the monitoring terminal and is installed in the middle of the dock yard and / or on the periphery of the dock yard. The monitoring range of the surveillance camera covers the entire dock yard.

[0011] An audible and visual alarm device is provided, which is connected to a monitoring terminal and is installed in the dispatch room of the dock yard.

[0012] The detection radar includes at least one lidar.

[0013] The lidar is mounted on a column, which is installed in the middle of the dock yard and / or on the outer perimeter of the dock yard.

[0014] The lidar is installed at a height greater than the cargo height limit in the dock yard, and the lidar probe is tilted downward at an angle equal to half of the lidar's vertical field of view.

[0015] The monitoring terminal is a computer.

[0016] As a preferred embodiment, the detection radar also includes millimeter-wave radar.

[0017] As another preferred approach, it also includes inertial measurement sensors and GPS locators installed on mobile devices in the dockyard, which are wirelessly connected to an industrial control computer.

[0018] The beneficial effects of this utility model are as follows: It uses detection radar to detect the stockpiles in the yard, and the host analyzes the height of each stockpile based on three-dimensional data to achieve real-time and accurate monitoring of each stockpile. When the height exceeds the limit, the monitoring terminal will promptly notify the staff through an audible and visual alarm. The staff can verify the height by retrieving the real-time footage from the surveillance cameras around the stockpile that triggered the alarm and then notify the gantry crane operator to adjust the stockpile position. The staff only needs to receive the alarm signal from the audible and visual alarm indoors and monitor the stockpile through the monitoring terminal. This eliminates the need for staff to spend long periods of time outdoors watching the stockpiles, thus avoiding eye fatigue and prolonged outdoor work. Attached Figure Description

[0019] The accompanying drawings, which are included to provide a further understanding of the present invention and form part of this application, illustrate exemplary embodiments of the present invention and, together with the description thereof, serve to explain the present invention and do not constitute an undue limitation thereof. In the drawings:

[0020] Figure 1 This is a schematic diagram of the layout of the lidar and monitoring camera on the dock yard in Embodiment 1 of this utility model;

[0021] Figure 2 This is a schematic diagram of the structure of the over-height monitoring system for the wharf yard in Embodiment 1 of this utility model;

[0022] Figure 3 This is a schematic diagram of a three-dimensional model in Embodiment 1 of this utility model;

[0023] Figure 4 This is a schematic diagram of the layout of the lidar, monitoring camera and millimeter-wave radar on the dock yard in Embodiment 2 of this utility model;

[0024] Figure 5 This is a schematic diagram of the structure of the over-height monitoring system for the wharf yard in Embodiment 2 of this utility model;

[0025] Figure 6 This is a schematic diagram of the structure of the over-height monitoring system for the wharf yard in Embodiment 3 of this utility model;

[0026] Figure 7 This is a schematic diagram of the layout of the lidar and monitoring camera on the dock yard in Embodiment 4 of this utility model;

[0027] Figure 8 This is a schematic diagram of the layout of the lidar and monitoring camera on the dock yard in Embodiment 5 of this utility model;

[0028] The labels in the diagram are as follows: 1. Dockyard; 2. Cargo pile; 3. Column; 4. LiDAR; 5. Surveillance camera; 6. Millimeter-wave radar. Detailed Implementation

[0029] The present invention will now be described in detail with reference to the accompanying drawings and embodiments.

[0030] like Figure 1 and Figure 2 The first embodiment shown is a terminal yard over-height monitoring system, which includes a detection radar, an industrial control computer, a monitoring terminal, a monitoring camera, and an audible and visual alarm. The detection radar is used to detect the three-dimensional data of the cargo pile 2 in the terminal yard 1. The industrial control computer is used to receive the three-dimensional data output by the detection radar and analyze it to obtain the height parameters of the cargo pile. That is, the data analysis mainly relies on the industrial control computer with built-in three-dimensional data analysis software. The monitoring terminal is mainly responsible for loading the corresponding computer program and interacting with the management personnel.

[0031] The detection radar is mounted on one side of the outer perimeter of the dock yard, and its detection range covers the cargo piles in the dock yard; the monitoring camera is installed on one side of the outer perimeter of the dock yard, and its monitoring range covers the entire dock yard; the monitoring terminal is placed in the dispatch room of the dock yard; the audible and visual alarm is also installed in the dispatch room of the dock yard.

[0032] The industrial control computer communicates with the detection radar; the monitoring terminal communicates with the industrial control computer, and the monitoring camera and audible / visual alarm are respectively connected to the monitoring terminal. The monitoring terminal is a computer, specifically a desktop computer.

[0033] The detection radar includes a lidar 4, which is mounted on a pillar 3 located on one side of the perimeter of the dock yard. To save on installation costs, a surveillance camera 5 is also mounted on the pillar 3. The lidar 4 is externally housed in a protective cover mounted on the pillar, but the lidar's detection end is exposed outside the cover.

[0034] The lidar is installed at a height greater than the height limit of the cargo in the dock yard. The lidar probe is tilted downwards at an angle equal to half of the lidar's vertical field of view to fully detect the outline of the cargo stack and reduce blind spots.

[0035] LiDAR (LiDAR) generates high-precision 3D point cloud data by emitting laser pulses and receiving the reflected signals, calculating the distance to the target object based on the time. In a dockyard environment, LiDAR is used to accurately capture the contour information of the cargo stack surface.

[0036] In this embodiment, the Livox Avia lidar is used, with a point cloud data rate of 240,000 points / second. The horizontal field of view is 70.4°, the vertical field of view is 77.2°, and the data port is a 100 Mbps Ethernet interface. The lidar incorporates a BMI088 inertial measurement unit with a push frequency of 200 Hz. It is accompanied by the user-friendly Livox Viewer software, which can display, record, play back, and analyze 3D point clouds in real time, and supports advanced functions such as product settings and extrinsic parameter adjustment. The Livox Viewer software's simple interface makes it easy for users to get started. Users can select appropriate scanning patterns according to their needs.

[0037] The surveillance camera in this embodiment has two working modes:

[0038] The first type is the daily monitoring mode, which performs video monitoring from a set perspective according to the monitoring needs of the terminal yard, helping managers to strengthen the financial security of the yard, etc.

[0039] The second method, in conjunction with the height measurement mode of the terminal yard's over-height monitoring system, establishes a one-to-one correspondence between each monitoring camera and the cargo pile location. There will be multiple cargo stack locations in the terminal yard, i.e., multiple cargo piles. Each cargo pile location is numbered or encoded to generate location information composed of numbers, letters, etc. This location information can be plotted in the terminal yard for image recognition, or it can be directly marked on the pixel area of ​​the monitoring image for record lookup and identification. The ID of each monitoring camera and the matching relationship between each monitoring camera and its corresponding cargo pile location information are stored in the industrial control computer's database. The height measurement status parameters of the monitoring cameras (including the camera's tilt angle and horizontal angle) are set to ensure that the monitoring cameras can include the corresponding cargo pile in their field of view under these height measurement status parameter conditions.

[0040] Based on the above, the monitoring terminal is configured to execute an alert process and / or a linkage process if the height parameter of the cargo stack exceeds the preset height limit threshold.

[0041] The warning process includes controlling the audible and visual alarm to perform alarm actions, namely, controlling the audible and visual alarm to turn on, turn on the lights, and emit an alarm sound.

[0042] The linkage process includes: S11, identifying the location information of the cargo pile that triggered the alarm; S12, searching the preset database based on the location information of the cargo pile to obtain the ID of the monitoring camera matching the monitoring field of view; S13, searching the database based on the ID of the corresponding monitoring camera to obtain the height measurement status parameter information of the matching monitoring camera, and controlling the monitoring camera matching the monitoring field of view to adjust according to the corresponding height measurement status parameter information; S14, calling the video data of the monitoring camera matching the monitoring field of view for output (e.g., displaying it in a pop-up window on the monitoring terminal).

[0043] Based on the above settings, in the implementation of this embodiment, the outline of the cargo pile in the yard can be detected by lidar to form three-dimensional data. The host analyzes the height of the cargo pile based on the three-dimensional data to achieve real-time and accurate monitoring of the cargo pile height. When the height exceeds the limit, the monitoring terminal will promptly notify the staff through an audible and visual alarm. By adjusting the status of the monitoring cameras in the dock yard, a real-time image of the cargo pile that triggered the alarm can be provided, so as to manage the stacking height of the dock yard more efficiently and conveniently.

[0044] In this embodiment, given that the detection radar includes lidar, the industrial control computer is further configured to: acquire point cloud data from the lidar and perform visualization processing on the point cloud data to generate a three-dimensional model of the cargo stack.

[0045] The visualization process can be achieved through VTK, an open-source software toolkit for 3D graphics, image processing, and visualization. By using VTK to visualize the processed point cloud data and intuitively display the 3D model of the cargo stack, it is convenient for staff to check and adjust the processing results. On the other hand, it can also be used for the intuitive display of the final volume calculation results.

[0046] Based on the above, the warning process of the monitoring terminal also includes: visually differentiating the cargo piles that trigger alarms based on the 3D model of the cargo piles, and outputting the processed 3D model of the cargo piles that trigger alarms.

[0047] like Figure 3 The example shown illustrates how the area of ​​the cargo pile that triggered the alarm is colored on the 3D model. Normally, it's white or green, but the alarm area is highlighted in red. The effect is displayed as a pop-up window on a computer screen, allowing for quick viewing of the target in conjunction with the audible and visual alarm. Colors are not displayed because the attached image cannot be a color image.

[0048] In this embodiment, the specific process of obtaining the height parameter of the cargo pile by analyzing the three-dimensional data output from the receiving detection radar is as follows:

[0049] S21. Receive the 3D data output by the detection radar, i.e., the point cloud data generated by the lidar, and perform preprocessing; the preprocessing includes boundary filtering and point cloud density compensation, specifically:

[0050] S211. Read the point cloud data file generated by the lidar and retain the historical records for subsequent analysis;

[0051] S212. Based on the given geographical range (e.g., the boundary range of the wharf yard) or features (e.g., the boundary fence features of the wharf yard), filter out the point cloud data that belongs to the wharf yard range, i.e. the boundary filtering mentioned above.

[0052] S213. Considering the non-repeating scanning characteristics of lidar, the point cloud needs to be properly preprocessed to compensate for the non-uniformity of data density, i.e., point cloud density compensation.

[0053] S22. Based on the preprocessed point cloud data, further processing and analysis are performed, i.e., processing it into the aforementioned 3D model, and analyzing the height of each cargo stack. Specifically:

[0054] S221. Clustering and grouping, which includes segmenting point cloud data through clustering algorithms to obtain multiple data groups, that is, applying clustering algorithms to cluster and group point cloud data to divide the stockpile into different groups, such as the K-means algorithm.

[0055] S222, radius filtering, which includes removing noise points and outliers from each data group according to a preset radius threshold (the area used in the algorithm), that is, removing noise points and outliers by manually setting a radius threshold to ensure the cleanliness of the dataset;

[0056] S223, Mean Sampling Descent, which includes the mean sampling method to reduce the number of point clouds in the radius-filtered data. That is, by calculating the average value of the coordinates (or other attributes) of points in a local area of ​​the point cloud, and using this average value to represent the entire area, the number of point clouds is reduced, reducing CPU load while maintaining sufficient accuracy.

[0057] S224, Triangular Reconstruction, which includes reconstructing a 3D surface model based on data processed by mean sampling descent: that is, reconstructing a 3D surface model using tools such as VTK as described above, in order to distinguish different objects, such as distinguishing sand piles and cranes.

[0058] S225, gantry crane identification, which includes identifying and removing point cloud data representing gantry cranes to obtain target cargo stack data. Gantry crane identification can be based on feature recognition of images corresponding to 3D surface models, or on recognition and segmentation based on semantic segmentation models of deep learning. However, in the case of point cloud data, it can also be based on point cloud data for differentiation.

[0059] S226, Highest point detection, which includes three-dimensional vertex height measurement based on target cargo stack data, taking any point at the highest position from the side view and any point on the ground of the dock yard to calculate the vertical height between the two, the solution can be three-dimensional coordinate calculation, pixel height calculation based on image and then proportional conversion, etc.

[0060] In this embodiment, point cloud density compensation includes: reading point cloud data from multiple consecutive frames on the time axis and performing overlay or completion processing. Since the gantry crane's robotic arm or support structure may obstruct the target in the storage yard, forming point cloud holes or causing uneven point cloud density due to scanning angle issues, this embodiment uses a multi-frame point cloud overlay or completion algorithm (such as DeepPC) to repair the obstructed areas, thereby reducing the impact of holes on subsequent target segmentation.

[0061] Example of gantry crane recognition: Gantry cranes typically have a tall, vertical structure (height > 20m), a robotic arm outline, and a fixed base. Their geometric features (such as curvature and normal vector direction) may be similar to some areas of yard targets (such as container stacks and vehicles), leading to missegmentation. Therefore, multi-feature fusion can be used for recognition and segmentation. Specifically, multi-dimensional feature extraction can be performed based on one or more of the following: curvature features, normal vector direction, height threshold, and reflectivity, to improve the distinguishability between gantry cranes and yard targets.

[0062] It is understandable that, in the above example of gate machine recognition, the staff should provide the standard range and feature range in advance for the determination of parameters such as height threshold and reflectivity; while the feature range of curvature and normal vector can be predicted by a pre-trained machine model.

[0063] Based on the above, this application can reduce the impact of gantry crane interference, obtain a relatively accurate three-dimensional model of the cargo stack, and measure its height.

[0064] Considering that the metal parts of the gantry crane may produce "ghosting" or "expansion" phenomena due to high reflectivity, resulting in abnormal diffusion of the point cloud and obscuring the true structure; while the non-metallic parts of the quay crane (such as rubber tires and coatings) may be missed due to low reflectivity, forming incomplete point cloud data, the following further settings are made: the point cloud data is segmented based on the echo characteristic distribution data of the laser pulse of the preset lidar when it encounters different media, the point cloud data of high / low reflectivity regions are separated, and the data are filtered, completed, and superimposed respectively.

[0065] like Figure 4 and Figure 5The illustrated embodiment two, based on embodiment one, considers environmental noise. Weather conditions such as rain, snow, and fog can increase point cloud noise (e.g., short-range anomalies, scattering interference), interfering with edge detection of the gantry crane model. Therefore, the detection radar also includes a millimeter-wave radar 6, whose detection range also covers the cargo piles in the dock yard. The ability to perceive high / low reflectivity areas is enhanced through multi-modal fusion of lidar and millimeter-wave radar.

[0066] The column 3 is installed on one side of the outer perimeter of the dock yard, and the lidar 4, monitoring camera 5, and millimeter-wave radar 6 are all installed on the column 3.

[0067] like Figure 6 In the third embodiment shown, mobile devices (such as gantry cranes and forklifts) in the dockyard may temporarily and dynamically obscure the cargo stack structure, resulting in incomplete cargo stack features in the point cloud data. Therefore, point cloud density compensation also includes applying motion compensation algorithms to process the point cloud data. This requires inertial measurement sensors and GPS locators on mobile devices in the dockyard, and wireless communication connections between the inertial measurement sensors and GPS locators and the industrial control computer.

[0068] Example of motion compensation algorithm: Inertial measurement sensor calculates the relative displacement of an object through its detection data, which is existing technology and will not be elaborated further; GPS locator obtains the absolute position of the object through satellite positioning data. The absolute position and relative displacement are mutually verified, or the position is periodically reset by verifying the absolute position, so as to lock the position of the dynamic target and obtain the motion trajectory; According to the motion trajectory, the position of the dynamic target relative to the pile of goods can be known, thereby determining which parts of the point cloud data are dynamic interferences, or avoiding dynamic interferences in time sequence, etc.

[0069] In addition to the three implementation methods mentioned above, the following points will also be considered during the specific implementation process:

[0070] 1. Considering motion blur, i.e., high-speed moving objects (such as vehicles) may produce blur in the point cloud, interfering with the boundary identification of the cargo pile point cloud data, the preprocessing also includes: distinguishing the static features of the cargo pile from the dynamic target interference data based on the temporal consistency of static multi-frame point cloud, and filtering the temporary occlusion data of dynamic targets. Specifically: in continuous multi-frame lidar point cloud data, the position and shape of static targets remain relatively stable, while the position of dynamic targets changes over time; by comparing and analyzing the position, shape and reflection intensity of targets in multi-frame point clouds, stationary targets are identified for differentiation and filtering.

[0071] 2. Considering the differences in gantry crane models and installation methods at different docks, which cause performance degradation of deep learning-based semantic segmentation models (such as PointNet++) in unseen scenarios, resulting in inaccurate gantry crane recognition, transfer learning is used to adjust the model to adapt to the point cloud features of different gantry crane models, or self-supervised learning is used to adjust the model to extract general geometric features.

[0072] 3. Considering the high real-time requirements of the terminal yard ultra-high monitoring system: Port scenarios need to process large-scale point cloud data in real time (such as millions of points per second). Traditional algorithms (such as Euclidean clustering) have high computational overhead and are difficult to meet the deployment requirements of edge devices. Therefore, lightweight (algorithm) models such as MobileNet-PointNet are adopted, or the (algorithm) model volume is compressed through knowledge distillation to improve the inference speed of edge devices.

[0073] like Figure 7 The difference between Embodiment 4 and Embodiment 2 is that the column 3 is installed in the middle of the dock yard, and the lidar 4, the monitoring camera 5, and the millimeter-wave radar 6 are all installed on the column 3.

[0074] like Figure 8 The fifth embodiment shown differs from the second embodiment in that there are three pillars: one pillar 3 is installed in the middle of the dock yard, and the other two pillars are installed on opposite sides of the outer perimeter of the dock yard. The pillar in the middle of the dock yard is equipped with mirror-symmetrically arranged lidar 4, mirror-symmetrically arranged surveillance cameras 5, and mirror-symmetrically arranged millimeter-wave radar 6. The pillars on the outer perimeter of the dock yard are equipped with lidar 4, surveillance cameras 5, and millimeter-wave radar 6 on the side closest to the middle of the dock yard.

[0075] In this utility model, unless otherwise explicitly specified and limited, the terms "installation," "connection," "joining," and "fixing," etc., should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral part; they can refer to a mechanical connection, an electrical connection, or a connection that allows communication between them; they can refer to a direct connection or an indirect connection through an intermediate medium; they can refer to the internal communication of two components or the interaction between two components, unless otherwise explicitly limited. Those skilled in the art can understand the specific meaning of the above terms in this utility model according to the specific circumstances.

[0076] The foregoing has shown and described the basic principles, main features, and advantages of this utility model. Those skilled in the art should understand that this utility model is not limited to the above embodiments. The embodiments and descriptions in the specification are merely illustrative of the principles of this utility model. Various changes and modifications can be made to this utility model without departing from its spirit and scope, and all such changes and modifications fall within the scope of the claimed utility model.

Claims

1. A monitoring system for excessive height at a wharf yard, characterized in that, include: A detection radar is used to detect three-dimensional data of cargo piles in a terminal yard. The detection radar is mounted at an elevation in the middle of the terminal yard and / or on the outer perimeter of the terminal yard. The detection range of the detection radar covers the cargo piles in the terminal yard. An industrial control computer is used to receive three-dimensional data output from the detection radar and analyze it to obtain the height parameters of the cargo pile. The industrial control computer is communicatively connected to the detection radar. A monitoring terminal, which is connected to an industrial control computer, is installed in the dispatch room of the dock yard; The surveillance camera is connected to the monitoring terminal and is installed in the middle of the dock yard and / or on the periphery of the dock yard. The monitoring range of the surveillance camera covers the entire dock yard. An audible and visual alarm device is provided, which is connected to a monitoring terminal and is installed in the dispatch room of the dock yard.

2. The over-height monitoring system for wharf yards according to claim 1, characterized in that: The detection radar includes at least one lidar.

3. The over-height monitoring system for wharf yards according to claim 2, characterized in that: The lidar is mounted on a column, which is installed in the middle of the wharf yard and / or on the outer perimeter of the wharf yard.

4. The over-height monitoring system for wharf yards according to claim 2, characterized in that: The installation height of the lidar is greater than the height limit of the cargo in the dock yard, and the lidar probe is tilted downwards at an angle that is half of the lidar's vertical field of view.

5. The over-height monitoring system for wharf yards according to claim 1, characterized in that: The monitoring terminal is a computer.

6. The over-height monitoring system for wharf yards according to claim 1, characterized in that: The detection radar also includes millimeter-wave radar.

7. The over-height monitoring system for wharf yards according to claim 1, characterized in that: It also includes inertial measurement sensors and GPS locators installed on mobile devices in the dockyard, which are wirelessly connected to the industrial control computer.

Images