An intelligent water traffic law enforcement system based on IoT perception

By installing the Internet of Things sensing system of float modules, lidar and video vision perception modules on the waterway, combined with SLAM and AIS systems, the problem of poor detection effect of traditional water traffic monitoring in complex water surface environments is solved, and high accuracy is achieved. Ship monitoring and safety improvement.

CN118658334BActive Publication Date: 2024-11-08USTC SINOVATE SOFTWARE
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411134037.3
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-08-19
Publication Date
2024-11-08
Estimated Expiration
2044-08-19

AI Technical Summary

Technical Problem

Traditional water traffic monitoring systems are difficult to achieve real-time monitoring of long-distance ships under complex water surface meteorological conditions, and the existing technology cannot effectively overcome environmental impacts such as heavy fog, resulting in poor detection results.

Method used

It adopts an intelligent water traffic law enforcement analysis system based on IoT perception, including a buoy module, a lidar sensing module, a video vision sensing module and an emergency warning module. It is fixedly connected to the bottom of the water through the buoy module, and uses a lidar and a video camera to identify ship behavior. And integrate data into the SLAM system and AIS system to construct and locate the waterways, and monitor and warn of illegal activities in real time.

Benefits of technology

It improves the detection accuracy of ship monitoring, reduces the difficulty and cost of installation of fixed piles, enhances the safety of ships in the waterway, and can effectively monitor and identify ship behavior in complex water surface environments.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118658334B_ABST
    Figure CN118658334B_ABST
Patent Text Reader

Abstract

The present invention discloses an intelligent analysis and judgment system for water traffic law enforcement based on Internet of Things perception, the analysis and judgment system comprises a buoy module distributed along the water flow direction of the channel, a laser radar perception module, a video vision perception module, a ship behavior recognition module, and an emergency warning module; by installing the buoy module on the waterway line along the direction of the water flow, when the buoy module deviates from the route, the analysis and judgment system will control the adjustment pendulum to swing in the direction of deviation from the waterway line until it returns to the specified position of the waterway line; by fixing one end of the buoy module to the bottom of the water, the difficulty of installing the monitoring mark with fixed piles can be reduced, saving costs; arranging the buoy module, the laser radar perception module and the video vision perception module on the waterway line can overcome the problem in the existing solution that the monitoring equipment is on the shore, and the monitoring data is easily affected by the water surface meteorological environment due to the distance between the waterway and the shore, thereby improving the detection accuracy of the analysis and judgment system.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention belongs to the technical field of traffic monitoring systems, and in particular relates to an intelligent analysis system for water traffic law enforcement based on Internet of Things perception. Background Art

[0002] Water transportation is the same as land transportation. For some busy canal waterways, there are more and more water transportation participants. Traditional supervision is to monitor the behavior of ships in the river by installing long-focal-length cameras on the river bank. When this method is used in some vast waters, it is often ineffective or even impossible to achieve due to the complex hydrological and meteorological conditions on the river surface.

[0003] The invention patent with application number 201410310649.3 for an inland navigable ship over-height detection and warning system and its working method is based on computer vision theory, with inland navigable ships as the application background, and uses AIS equipment to obtain location information of the navigation environment. It can only judge the problem of ship over-height; it cannot overcome the complex meteorological conditions on the water surface. For example, when there is local fog in the waterway, the existing telephoto camera cannot monitor the distant ships in real time. Therefore, an intelligent judgment system for water traffic law enforcement based on Internet of Things perception is needed. Summary of the invention

[0004] In order to solve at least one of the problems raised in the above-mentioned background technology, the present invention provides an intelligent analysis system for water traffic law enforcement based on Internet of Things perception.

[0005] To achieve the above-mentioned purpose, the present invention provides the following technical solutions: an intelligent judgment system for water traffic law enforcement based on IoT perception, the judgment system comprising a buoy module distributed along the water flow direction of the waterway, a laser radar perception module, a video vision perception module, a ship behavior recognition module, and an emergency warning module;

[0006] Preferably, the buoy module includes an anchor point prompt mark, a first monitoring mark, a second monitoring mark and a third monitoring mark; a positioning QR code is sprayed on the side wall of the first monitoring mark facing the second monitoring mark, and a positioning QR code is sprayed on the side wall of the second monitoring mark facing the third monitoring mark;

[0007] Preferably, the laser radar sensing module includes a laser radar arranged at the upper end of the first monitoring mark;

[0008] Preferably, the video visual perception module includes a front camera arranged at the upper end of the second monitoring mark and a rear camera arranged at the upper end of the third monitoring mark;

[0009] Preferably, the ship behavior recognition module recognizes the behavior of the ship during navigation through the data obtained by the laser radar perception module and the video vision perception module; determines whether the recorded information of the ship is consistent with the information of the sailing ship, whether the ship has overloaded transportation behavior, and whether the ship is on the waterway, and whether the ship deviates and goes in reverse;

[0010] Preferably, the emergency warning module is a voice broadcast of illegal behavior by law enforcement personnel to the ship where the illegal behavior is committed via radio.

[0011] Preferably, the anchor point reminder marker, the first monitoring marker, the second monitoring marker and the third monitoring marker are distributed on the waterway line along the direction of water flow in the waterway, and are connected by a traction connection line arranged therebetween; an anchor hook is arranged at the lower end of the anchor point reminder marker, and the anchor point reminder marker is fixedly connected to the bottom of the water through the anchor hook; the anchor point reminder marker, the first monitoring marker, the second monitoring marker and the third monitoring marker and their connected accessories are each provided in two groups, distributed along the waterway line on both sides of the waterway.

[0012] Preferably, the working steps of the analysis and judgment system are as follows:

[0013] S1, arrange multiple monitoring marks on the waterway and initialize the front camera, rear camera and lidar;

[0014] S2, the first monitoring mark, the second monitoring mark and the third monitoring mark perform self-checking, initialization, determine their own positions, correct the viewing angle, and the first monitoring mark and the anchor prompt mark form a chain connection;

[0015] S3, the first monitoring mark, the second monitoring mark and the third monitoring mark use the SLAM system to locate the QR code on the side of the waterway and map the waterway to obtain static information of the waterway;

[0016] S4, uses the IoT system’s mapping point cloud to fuse with the AIS system information to obtain channel information; the front camera of the upper-level node in the chain network restores the motion of the lower-level node while restoring its own motion; each monitoring beacon acts as an independent unit to map and locate the channel and its own position;

[0017] S5 integrates the positioning information of the AIS system and the chain network to update the position of the buoy itself in the system in real time; it displays and updates the ship information and the ship's trajectory in real time in the central control system, making it easier for regulatory authorities to monitor and warn.

[0018] Preferably, in S4, the mapping step is as follows:

[0019] A, the analysis system uses the mask R-CNN instance segmentation network to identify and segment the ships detected in the buoy, and label different ships;

[0020] B, perform 3D reconstruction on the feature points and pixel information of each segmented ship;

[0021] C. Determine whether the recorded information of the ship is consistent with the information of the sailing ship, and whether the ship has any over-limit transportation behavior.

[0022] Preferably, in said S4, the positioning step is as follows: S41, using DiMP multi-target tracking to track the running tracks and directions of different ships, and using the IOU ratio to obtain the matching of the ship number with the DiMP network monitoring result;

[0023] S42, using the chain network to restore the track of each ship;

[0024] S43, judging whether the ship is in the waterway and whether the ship is veering off course or going in the opposite direction.

[0025] Preferably, the traction connection line is also used for chain connection communication between the front camera, the rear camera and the laser radar.

[0026] Preferably, the lower ends of the first monitoring mark, the second monitoring mark and the third monitoring mark are all provided with an adjustment disc and an adjustment pendulum, the adjustment disc is fixedly connected to the motor output shaft inside the monitoring mark, and the adjustment pendulum is fixedly connected to the adjustment disc; the swing direction of the adjustment pendulum is controlled by the analysis and judgment system.

[0027] Preferably, the swing direction of the adjusting flap is the same as the direction in which the monitoring beacon above it deviates from the route.

[0028] Compared with the prior art, the present invention has the following beneficial effects:

[0029] The present invention installs a buoy module on the waterway line along the direction of the water flow. When the buoy module deviates from the route, the analysis and judgment system will control the adjustment pendulum to swing in the direction of deviation from the waterway line until it returns to the specified position of the waterway line. By fixing one end of the buoy module to the bottom of the water, the difficulty of installing the monitoring mark with fixed piles can be reduced, saving costs; and the safety of ships sailing in the waterway can also be improved.

[0030] The present invention arranges buoy modules, lidar sensing modules and video vision sensing modules on the waterway line, thereby overcoming the problem in the existing solution that the monitoring equipment is on the shore and the monitoring data is easily affected by the water surface meteorological environment due to the distance of the waterway from the shore, thereby improving the detection accuracy of the analysis and judgment system. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] The accompanying drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation of the present invention. In the accompanying drawings:

[0032] Figure 1 is a system block diagram of the present invention;

[0033] Figure 2 It is a work flow chart of the present invention;

[0034] Figure 3 It is a schematic diagram of the three-dimensional structure of the buoy module of the present invention;

[0035] Figure 4 It is a schematic diagram of the buoy module and the water flow direction of the present invention;

[0036] Figure 5 A top view of a buoy module of the present invention;

[0037] Figure 6 This is a schematic diagram of the buoy module correction state of the present invention;

[0038] Figure 7 This is a schematic diagram of the structure of the third monitoring mark of the present invention;

[0039] In the figure: 1. Anchor point prompt mark; 2. LiDAR; 3. Front camera; 4. Rear camera; 5. Traction connecting line; 6. Adjustment disc; 7. Adjustment pendulum; 8. First monitoring mark; 9. Second monitoring mark; 10. Third monitoring mark; 11. Anchor hook. DETAILED DESCRIPTION

[0040] The following will be combined with the embodiments of the present invention to clearly and completely describe the technical solution of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0041] First, refer to Figures 1 to 7 The embodiment of the present invention relates to an intelligent water traffic law enforcement judgment system based on IoT perception, the judgment system includes a buoy module distributed along the water flow direction of the channel, a laser radar perception module, a video vision perception module, a ship behavior recognition module, and an emergency warning module;

[0042] The buoy module includes an anchor point reminder mark 1, a first monitoring mark 8, a second monitoring mark 9 and a third monitoring mark 10; a positioning QR code is sprayed on the side wall of the first monitoring mark 8 facing the second monitoring mark 9, and a positioning QR code is sprayed on the side wall of the second monitoring mark 9 facing the third monitoring mark 10;

[0043] The laser radar perception module includes a laser radar 2 disposed on the upper end of the first monitoring mark 8; the video visual perception module includes a front camera 3 disposed on the upper end of the second monitoring mark 9 and a rear camera 4 disposed on the upper end of the third monitoring mark 10;

[0044] The ship behavior recognition module uses the data obtained by the laser radar perception module and the video vision perception module to identify the behavior of the ship during navigation; it determines whether the recorded information of the ship is consistent with the information of the ship in motion, whether the ship has overloaded transportation behavior, whether the ship is on the waterway, and whether the ship is deviating from the course and going in reverse;

[0045] The emergency warning module allows law enforcement officers to broadcast illegal behaviors to ships via radio.

[0046] The anchor point reminder mark 1, the first monitoring mark 8, the second monitoring mark 9 and the third monitoring mark 10 are distributed on the waterway line along the water flow direction of the waterway, and are connected by a traction connection line 5 arranged therebetween; an anchor hook 11 is arranged at the lower end of the anchor point reminder mark 1, and the anchor point reminder mark 1 is fixedly connected to the bottom of the water through the anchor hook 11, and the anchor point reminder mark 1, the first monitoring mark 8, the second monitoring mark 9 and the third monitoring mark 10 and their connected accessories are arranged in two groups, distributed along the waterway line on both sides of the waterway.

[0047] The traction connection line 5 is also used for chain connection communication between the front camera 3, the rear camera 4 and the laser radar 2.

[0048] The lower ends of the first monitoring mark 8, the second monitoring mark 9 and the third monitoring mark 10 are all provided with an adjusting disc 6 and an adjusting pendulum 7. The adjusting disc 6 is fixedly connected to the motor output shaft inside the monitoring mark, and the adjusting pendulum 7 is fixedly connected to the adjusting disc 6. The swing direction of the adjusting pendulum 7 is controlled by the analysis and judgment system. The swing direction of the adjusting pendulum 7 is the same as the direction in which the monitoring mark above it deviates from the route.

[0049] Working principle: Anchor point reminder mark 1, first monitoring mark 8, second monitoring mark 9, and third monitoring mark 10 are installed in sequence on the channel line on one side of the channel; wherein the anchor point reminder mark 1 is fixedly connected to the bottom of the water through an anchor hook 11; since the installation direction of the anchor point reminder mark 1, the first monitoring mark 8, the second monitoring mark 9, and the third monitoring mark 10 is the direction of the water flow, and the lower ends of the first monitoring mark 8, the second monitoring mark 9, and the third monitoring mark 10 are all provided with an adjustment disc 6 and an adjustment pendulum 7 for deviation correction, since the first monitoring mark 8, the second monitoring mark 9, and the third monitoring mark 10 are connected in series through a traction connection line 5, when the first monitoring mark 8, the second monitoring mark 9, and the third monitoring mark 10 deviate from the route, the judgment system will control the adjustment pendulum 7 to swing in the direction of deviation from the channel line until it returns to the specified position of the channel line; by fixing one end of the buoy module with the bottom of the water, the difficulty of installing the monitoring mark with a fixed pile can be reduced, saving costs; it can also improve the safety of ships sailing in the channel;

[0050] The traction connection line 5 is also used for chain connection communication between the front camera 3, the rear camera 4 and the laser radar 2, which can improve the communication efficiency and communication quality between the devices.

[0051] The working steps of the analysis system are as follows:

[0052] S1, arrange multiple monitoring marks on the waterway line, and initialize the front camera 3, rear camera 4 and laser radar 2;

[0053] S2, the first monitoring mark 8, the second monitoring mark 9 and the third monitoring mark 10 perform self-checking, initialization, determine their own positions, and correct the viewing angle. The first monitoring mark 8 and the anchor point prompt mark 1 form a chain connection;

[0054] S3, the first monitoring mark 8, the second monitoring mark 9 and the third monitoring mark 10 use the SLAM system to locate the QR code on the side of the waterway and map the waterway to obtain static information of the waterway;

[0055] S4, uses the IoT system’s mapping point cloud to fuse with the AIS system information to obtain the waterway information; the front camera 3 of the upper-level node in the chain network restores the motion of the lower-level node while restoring its own motion; each monitoring beacon acts as an independent unit to map and locate the waterway and its own position;

[0056] The steps of building the map are as follows:

[0057] A,recognition system uses the mask R-CNN instance segmentation network to identify and segment the ships detected in the buoy, and label different ships;

[0058] B, perform 3D reconstruction on the feature points and pixel information of each segmented ship;

[0059] C. Determine whether the recorded information of the ship is consistent with the information of the sailing ship, and whether the ship has any over-limit transportation behavior.

[0060] The positioning steps are as follows:

[0061] S41, using DiMP multi-target tracking to track the trajectory and orientation of different ships, and using the IOU ratio to match the ship number with the DiMP network monitoring results;

[0062] S42, using the chain network to restore the track of each ship;

[0063] S43, determining whether the ship is in the waterway and whether the ship is yawed and reversed;

[0064] S5 integrates the positioning information of the AIS system and the chain network to update the position of the buoy itself in the system in real time; it displays and updates the ship information and the ship's trajectory in real time in the central control system, making it easier for regulatory authorities to monitor and warn.

[0065] Although the embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the claims and their equivalents.

Claims

1. An intelligent water traffic law enforcement system based on IoT perception, characterized by: The analysis and judgment system comprises a buoy module distributed along the water flow direction of the channel, a laser radar perception module, a video vision perception module, a ship behavior recognition module, and an emergency warning module; the buoy module comprises an anchor point prompt mark (1), a first monitoring mark (8), a second monitoring mark (9) and a third monitoring mark (10); a positioning two-dimensional code is sprayed on the side wall of the first monitoring mark (8) facing the second monitoring mark (9), and a positioning two-dimensional code is sprayed on the side wall of the second monitoring mark (9) facing the third monitoring mark (10); the laser radar perception module comprises a laser radar ( 2); the video visual perception module comprises a front camera (3) arranged on the upper end of the second monitoring mark (9) and a rear camera (4) arranged on the upper end of the third monitoring mark (10); the ship behavior recognition module recognizes the behavior of the ship during navigation through the data obtained by the laser radar perception module and the video visual perception module; determines whether the recorded information of the ship is consistent with the information of the ship in motion, whether the ship has any overload transportation behavior, and determines whether the ship is on the waterway route and whether the ship is deviating from the course and going in reverse; the emergency warning module is for law enforcement personnel to broadcast the illegal behavior of the ship to the illegal behavior through radio.

2. According to claim 1, the intelligent judgment system for water traffic law enforcement based on IoT perception is characterized by: The anchor point reminder mark (1), the first monitoring mark (8), the second monitoring mark (9) and the third monitoring mark (10) are distributed on the waterway line along the water flow direction of the waterway and are connected by a traction connection line (5) arranged therebetween; an anchor hook (11) is arranged at the lower end of the anchor point reminder mark (1), and the anchor point reminder mark (1) is fixedly connected to the bottom of the water via the anchor hook (11); the anchor point reminder mark (1), the first monitoring mark (8), the second monitoring mark (9) and the third monitoring mark (10) and their connected appendages are each arranged in two groups and distributed along the waterway line on both sides of the waterway.

3. According to claim 2, the intelligent water traffic law enforcement judgment system based on IoT perception is characterized by: The working steps of the analysis and judgment system are as follows: S1, arranging multiple monitoring marks on the waterway line, initializing the front camera (3), the rear camera (4) and the laser radar (2); S2, the first monitoring mark (8), the second monitoring mark (9) and the third monitoring mark (10) perform self-checking, initialization, determine their own positions, and correct the viewing angle, and the first monitoring mark (8) and the anchor point prompt mark (1) form a chain connection; S3, the first monitoring mark (8), the second monitoring mark (9) and the third monitoring mark (10) use the SLAM system to map the waterway using their own cameras and the QR code on the side of the waterway to obtain static information of the waterway; S4, uses the IoT system’s mapping point cloud to fuse with the AIS system information to obtain channel information; the front camera (3) of the upper node in the chain network restores the motion of the lower node while restoring its own motion; each monitoring beacon acts as an independent unit to map and locate the channel and its own position; S5 integrates the positioning information of the AIS system and the chain network to update the position of the buoy itself in the system in real time; it displays and updates the ship information and the ship's trajectory in real time in the central control system, making it easier for regulatory authorities to monitor and warn.

4. According to claim 3, the intelligent water traffic law enforcement judgment system based on IoT perception is characterized by: In S4, the mapping steps are as follows: A. The analysis system uses the Mask R-CNN instance segmentation network to identify and segment the ships detected in the buoy and label different ships; B, perform 3D reconstruction on the feature points and pixel information of each segmented ship; C. Determine whether the recorded information of the ship is consistent with the information of the sailing ship, and whether the ship has any over-limit transportation behavior.

5. According to claim 4, the intelligent water traffic law enforcement judgment system based on IoT perception is characterized by: In S4, the positioning steps are as follows: S41, using DiMP multi-target tracking to track the running tracks and directions of different ships, and using the IOU ratio to match the ship number with the DiMP network monitoring result; S42, using the chain network to restore the track of each ship; S43, judging whether the ship is in the waterway and whether the ship is veering off course or going in the opposite direction.

6. According to claim 5, the intelligent water traffic law enforcement judgment system based on IoT perception is characterized by: The traction connection line (5) is also used for chain connection communication between the front camera (3), the rear camera (4) and the laser radar (2).

7. According to claim 6, the intelligent water traffic law enforcement judgment system based on IoT perception is characterized by: The lower ends of the first monitoring mark (8), the second monitoring mark (9) and the third monitoring mark (10) are all provided with an adjusting disc (6) and an adjusting pendulum (7); the adjusting disc (6) is fixedly connected to the output shaft of the motor inside the monitoring mark, and the adjusting pendulum (7) is fixedly connected to the adjusting disc (6); the swinging direction of the adjusting pendulum (7) is controlled by the analysis and judgment system.

8. According to claim 7, the intelligent water traffic law enforcement judgment system based on IoT perception is characterized by: The swing direction of the adjusting swing piece (7) is the same as the direction in which the monitoring mark above it deviates from the route.