A ship lock accompanying type safety monitoring system based on mobile monitoring

Abstract

This invention discloses a lock-based accompanying safety monitoring system based on mobile monitoring, belonging to the field of intelligent shipping technology. The invention includes a perception layer, a processing layer, and an application layer. The perception layer includes a mobile camera for acquiring first vessel information; an accompanying detection system for generating camera position adjustment signals based on the first vessel information, identifying and locating vessels entering the lock using a Gaussian mixture background model, allocating cameras, stitching the acquired images together using an image stitching method, and generating camera position adjustment signals based on the difference between the stitched image and the overall vessel image; a camera movement system for adjusting the camera position according to the camera position adjustment signals, ensuring to a certain extent that there are no blind spots in the lock chamber monitoring; the data acquired by the perception layer is transmitted to the processing layer for data storage and processing, and the processed data is transmitted to the application layer for real-time monitoring, intelligent decision-making, and safety assessment of the lock.

Description

Technical Field

[0001] This invention relates to the field of intelligent shipping technology, specifically to a lock-based safety monitoring system based on mobile monitoring. Background Technology

[0002] As one of the most widely used navigation structures, locks effectively improve navigation conditions and promote waterway transportation and local economic development. From a developmental perspective, lock safety management is a perpetual theme in lock operation management and high-quality development; from an inherent safety perspective, strengthening lock safety management is crucial to people's well-being, is the foundation of people's livelihood and development, and is an important part of the waterway workers' implementation of the national strategy of building a strong transportation network.

[0003] In recent years, with the continuous development of video surveillance technology and the rapid popularization of high-definition video products, the high-definition market is gradually maturing and gaining recognition from various industries. High-definition video systems are also gradually being applied in the field of ship locks. Among them, the operation and scheduling of ship locks in Jiangsu Province generally uses the "eight-step" method of valve control: "open the upper valve - open the upper gate - close the upper valve - close the upper gate - open the lower valve - open the lower gate - close the lower valve - close the lower gate". According to the "eight-step" method, the staff needs to deploy monitoring equipment at multiple locations such as "upstream approach channel - upstream gate - upstream water level - upstream lock chamber - downstream lock chamber - downstream water level - downstream gate - downstream approach channel". The staff needs to pay attention to multiple monitoring screens based on these monitoring devices. However, the number of monitoring devices is limited, and blind spots are easy to appear in the monitoring angle. To reduce blind spots, it is necessary to increase the monitoring facilities, which increases the investment in ship lock hardware. Summary of the Invention

[0004] The purpose of this invention is to provide a lock-following safety monitoring system based on mobile monitoring. It applies mobile intelligent camera technology to the safe operation and management of locks. The equipment follows the movement of ships, and staff only need to focus on one screen. They can also manually switch between any monitoring screen at any time as needed, ensuring no blind spots in the lock chamber monitoring. This ensures the safe operation of the lock, guarantees the safe and rapid passage of ships, reduces the labor intensity of staff, saves on the investment in lock hardware, and improves the level of lock safety management.

[0005] This invention provides the following technical solution: a lock-based accompanying safety monitoring system based on mobile monitoring, comprising:

[0006] The perception layer is used to collect ship information and lock information in real time, and the ship information includes first ship information and second ship information.

[0007] The processing layer is used to calculate the processing results based on the ship information and lock information, including a data monitoring system, a data integration system, and a modeling and simulation system;

[0008] The application layer is used for safety monitoring, intelligent decision-making, and safety assessment based on the processing results, including safety early warning systems, ship identification systems, gate control systems, data monitoring systems, data query systems, and information sharing systems.

[0009] The sensors include water level sensors, gate sensors, and piezometers; the safety early warning system is used for safety monitoring and early warning during ship operation.

[0010] The ship identification system is used to identify the ship's name, registration number, and type.

[0011] The gate control system is used to control the opening and closing of the gate; the data monitoring system is used to monitor the entire process of ships entering and leaving the gate, including video monitoring and sensor monitoring.

[0012] The data query system is used to query all information about ships entering and exiting the lock, including ship type, ship name and number, number of ships, and time of entry and exit.

[0013] The information sharing system is used to share information such as the time of entry and exit of the lock gates and the type of vessel.

[0014] According to the above technical solution, the sensing layer includes:

[0015] Cameras are used to acquire initial ship information;

[0016] A camera movement system used to adjust the position of the camera based on a camera position adjustment signal;

[0017] The accompanying detection system is used to generate camera position adjustment signals based on the first ship information;

[0018] A sensor monitoring system is used to acquire information about a second vessel.

[0019] The first ship information refers to the acquired ship image information, such as the ship's shape and length; the second ship information refers to information such as the ship's draft, whether it is carrying mooring lines, and whether it has been in a collision.

[0020] According to the above technical solution, the camera moving system includes: a track device, a moving component, and a camera;

[0021] The track device is installed on the upper and lower channel walls and the lock chamber walls, the moving device is set on the track device, and the camera is installed above the moving component;

[0022] The track device includes a first circular track and a second circular track. The first circular track is located on one side of the city wall of the upper and lower navigation channels and the other side of the city wall of the lock chamber. The second circular track is located on the other side of the city wall of the upper and lower navigation channels and the other side of the city wall of the lock chamber. The monitoring directions of the cameras on the first and second circular tracks are opposite.

[0023] The first annular track includes a first upper annular track and a first lower annular track, which are connected end-to-end. The first upper annular track is used for mobile monitoring accompanying the ship, and the first lower annular track is used to transport cameras to the first track.

[0024] The second annular track includes a second upper annular track and a second lower annular track, which are connected end-to-end. The second upper annular track is used for mobile monitoring accompanying the ship, and the second lower annular track is used to transport cameras to the second track.

[0025] The moving assembly includes a moving device, a moving plate, a first rotating rod, a second rotating rod, and a mounting plate; the moving device is connected to the moving plate and located below the moving plate; the moving plate is fixedly connected to the first rotating rod and the second rotating rod respectively and located above the moving plate; the first rotating rod is located to the left of the second rotating rod; the first rotating rod and the second rotating rod are rotatably connected to the mounting plate respectively; the mounting plate is located above the first rotating rod and the second rotating rod; and the camera is mounted above the mounting plate.

[0026] The movable plate can move back and forth on the moving device; the first rotating rod and the second rotating rod can extend and retract vertically, either synchronously or asynchronously. The mounting plate is located above the first rotating rod and the second rotating rod, making the camera installation more secure and allowing the camera angle to be adjusted.

[0027] According to the above technical solution, the accompanying detection system includes:

[0028] The image target detection module uses a Gaussian mixture background model to identify and locate the first ship information, and determine the ships waiting to enter the lock. It can quickly identify the ships to enter the lock, avoid the influence of other factors within the camera range on ship tracking and monitoring, and locate the ships, so as to accurately locate the ships entering and exiting the lock.

[0029] The camera allocation module determines the number of cameras to be transported from the upper track to the lower track based on the ships waiting to enter the gate; it can rationally allocate cameras, improve camera utilization, and at the same time reduce the investment in monitoring equipment to a certain extent.

[0030] The camera adjustment module uses image stitching to process the images acquired by the cameras on the upper track and generates camera position adjustment signals. As the camera follows the ship's movement, it continuously adjusts the camera's position, reducing blind spots to a certain extent.

[0031] According to the above technical solution, the Gaussian mixture background model is as follows:

[0032]

[0033] P r(x) It is a Gaussian mixture background model, π k It is the weighting factor, k represents k Gaussian distributions, N(x; u) k ∑k) represents the probability density function of a multidimensional variable X when it follows a Gaussian distribution, u k Let and ∑k represent the mean vector and covariance matrix of the k-th Gaussian distribution, respectively. Using a Gaussian mixture background model is more in line with the actual situation.

[0034] According to the above technical solution, the camera position adjustment is as follows: when the ship enters the monitoring range of the camera on the first track, the edge information of the images acquired by all cameras on the first and second ring tracks is extracted using the Prewitt operator, and then Harris feature point detection is performed on the partitions. Then, the NCC algorithm and RANSAC algorithm are combined to achieve accurate matching of feature points between images. Next, image fusion is performed using image stitching. Finally, the fused panoramic image is cropped to complete image compensation, and a camera position adjustment signal is generated based on the difference between the stitched image and the overall image of the ship.

[0035] According to the above technical solution, the position of the camera is the distance between each adjacent camera on the first circular upper track and the distance between each adjacent camera on the second circular upper track; as the camera moves with the ship, the position of the camera is adjusted at any time according to the camera position adjustment signal.

[0036] According to the above technical solution, the image stitching method is as follows:

[0037]

[0038] Where I1(x,y) and I2(x,y) are the pixel values ​​at (x,y) in the images to be stitched, respectively; R1 and R2 are the coordinate sets of the images to be stitched; I(x,y) is the fused pixel value; w1 and w2 are weight coefficients, satisfying w1 + w2 = 1, and both w1 and w2 are greater than 0; the weights are... x2 and x1 are the x-coordinates of the boundary of the overlapping region, where x1≤x≤x2.

[0039] The image stitching process involves the following steps: First, the Prewitt operator is used to extract the edge information of the image, and then Harris feature point detection is performed on the partitioned image. Next, the NCC algorithm and RANSAC algorithm are combined to achieve accurate matching of feature points between images. Then, the image stitching method is used to fuse the images. Finally, the fused panoramic image is cropped to complete image compensation and output a stable video sequence. This method can effectively eliminate video jitter and output a stable video sequence.

[0040] According to the above technical solution, the camera allocation steps are as follows:

[0041] When a vessel enters the monitoring range of the first or second camera, the vessel is identified using a Gaussian mixture background model. The identified vessel waiting to enter the gate is then matched with information in the vessel reservation database to obtain the vessel length.

[0042] The number N of cameras that move with the ship is determined based on the ship's length and the camera's monitoring range.

[0043] The first annular lower track conveys to the first annular upper track A camera is conveyed from the second lower circular track to the second upper circular track. One camera.

[0044] According to the above technical solution, when all the cameras on the first ring upper track move to the first ring lower track and all the cameras on the second ring upper track move to the second ring lower track, the first camera is transported from the first ring lower track to the first ring upper track, and the second camera is transported from the second ring lower track to the second ring upper track.

[0045] The first camera is located at the junction of the first upper circular track and the first lower circular track; the second camera is located at the junction of the second upper circular track and the second lower circular track.

[0046] When the cameras finish following the ship's movement, all the cameras on the first ring upper track move to the first ring lower track, and at the same time, all the cameras on the second ring upper track move to the second ring lower track.

[0047] Compared with existing technologies, the beneficial effects achieved by this invention are as follows: By deploying relevant sensing devices and developing corresponding information systems, information management of daily work is realized; this system applies mobile intelligent camera technology to the safe operation management of locks, using a Gaussian mixture background model to identify and locate ships entering the lock, so that the camera moves along with the located ship, and staff only need to focus on one image to complete the monitoring of the ship's entry and exit from the lock; by determining the number of cameras transported from the second track to the first track based on the actual situation of the ships waiting to enter the lock, the cameras can be rationally allocated, improving the utilization rate of the cameras, and also reducing the investment in monitoring equipment to a certain extent; by using image stitching to adjust the position of the camera in real time during the movement of the ship, the monitoring inside the lock chamber is guaranteed to be without blind spots to a certain extent, thereby ensuring the safe operation of the lock, ensuring the safe and rapid passage of ships, reducing the labor intensity of staff, saving investment in lock hardware equipment, and improving the level of lock safety management. Attached Figure Description

[0048] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings:

[0049] Figure 1 This is an architecture diagram of a lock-based accompanying safety monitoring system based on mobile monitoring, according to the present invention.

[0050] Figure 2 This is a conceptual diagram of the lock chamber layout for a smart camera system that accompanies the operation of the lock.

[0051] Figure 3 This is a schematic diagram of the accompanying detection system.

[0052] Figure 4 This is a schematic diagram of the camera movement system structure;

[0053] In the diagram: 1. Track device; 2. Moving device; 3. Moving plate; 4. Second rotating rod; 5. First rotating rod; 6. Mounting plate; 7. Camera. Detailed Implementation

[0054] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present invention.

[0055] Please see Figures 1-4The present invention provides a technical solution: a lock-based accompanying safety monitoring system based on mobile monitoring, comprising:

[0056] The perception layer is used to collect ship information and lock information in real time, and the ship information includes first ship information and second ship information.

[0057] The accompanying detection system acquires ship images by using a camera to accompany the ship's movement, and detects ship information and ship motion status information.

[0058] The processing layer is used to calculate the processing result based on the ship information and lock information.

[0059] The processing layer is used to calculate the processing results based on the ship information and lock information, including a data monitoring system, a data integration system, and a modeling and simulation system;

[0060] The sensing layer includes:

[0061] Cameras are used to acquire initial ship information;

[0062] A camera movement system is used to adjust the position of a camera according to a camera position adjustment signal. Specifically, the camera movement system includes: a track device, a movement component, and a camera.

[0063] The track device is installed on the upper and lower channel walls and the lock chamber walls, the moving device is set on the track device, and the camera is installed above the moving component;

[0064] The track device includes a first circular track and a second circular track. The first circular track is located on one side of the city wall of the upper and lower navigation channels and the other side of the city wall of the lock chamber. The second circular track is located on the other side of the city wall of the upper and lower navigation channels and the other side of the city wall of the lock chamber. The monitoring directions of the cameras on the first and second circular tracks are opposite.

[0065] The first annular track includes a first upper annular track and a first lower annular track, which are connected end-to-end. The first upper annular track is used for mobile monitoring accompanying the ship, and the first lower annular track is used to transport cameras to the first track.

[0066] The second annular track includes a second upper annular track and a second lower annular track, which are connected end-to-end. The second upper annular track is used for mobile monitoring accompanying the ship, and the second lower annular track is used to transport cameras to the second track.

[0067] The moving assembly includes a moving device, a moving plate, a first rotating rod, a second rotating rod, and a mounting plate; the moving device is connected to the moving plate and located below the moving plate; the moving plate is fixedly connected to the first rotating rod and the second rotating rod respectively and located above the moving plate; the first rotating rod is located to the left of the second rotating rod; the first rotating rod and the second rotating rod are rotatably connected to the mounting plate respectively; the mounting plate is located above the first rotating rod and the second rotating rod; and the camera is mounted above the mounting plate.

[0068] The accompanying detection system generates a camera position adjustment signal based on first vessel information. It includes an image target detection module that uses a Gaussian mixture background model to identify and locate the first vessel information, determining the vessel waiting to enter the gate. The Gaussian mixture background model is:

[0069]

[0070] P r(x) It is a Gaussian mixture background model, π k It is the weighting factor, k represents k Gaussian distributions, N(x; u) k ∑k) represents the probability density function of a multidimensional variable X when it follows a Gaussian distribution, u k Let and ∑k represent the mean vector and covariance matrix of the k-th Gaussian distribution, respectively.

[0071] The camera allocation module determines the number of cameras to be transported from the upper track to the lower track based on the ships waiting to enter the gate. Specifically, when a ship enters the monitoring range of the first or second camera, the Gaussian mixture background model is used to identify the ship, and the identified ships waiting to enter the gate are matched with the information in the ship reservation database to obtain the ship length.

[0072] The number N of cameras that move with the ship is determined based on the ship's length and the camera's monitoring range.

[0073] The first annular lower track conveys to the first annular upper track A camera is conveyed from the second lower circular track to the second upper circular track. One camera.

[0074] When all the cameras on the first ring upper track move to the first ring lower track and all the cameras on the second ring upper track move to the second ring lower track, the first camera is transported from the first ring lower track to the first ring upper track, and the second camera is transported from the second ring lower track to the second ring upper track.

[0075] The first camera is located at the junction of the first upper circular track and the first lower circular track; the second camera is located at the junction of the second upper circular track and the second lower circular track.

[0076] When the cameras finish following the ship's movement, all the cameras on the first ring upper track move to the first ring lower track, and at the same time, all the cameras on the second ring upper track move to the second ring lower track.

[0077] The camera adjustment module uses image stitching to process the images acquired by the camera on the upper track and generates a camera position adjustment signal.

[0078] The camera position adjustment is achieved by stitching together the images acquired by all cameras on the first and second circular upper tracks using an image stitching method, and generating a camera position adjustment signal based on the difference between the stitched image and the overall image of the ship.

[0079] A sensor monitoring system is used to acquire information about a second vessel.

[0080] The application layer is used for safety monitoring, intelligent decision-making, and safety assessment based on the processing results, including a safety early warning system, a ship identification system, a gate control system, a data monitoring system, a data query system, and an information sharing system.

[0081] The sensors include water level sensors, gate sensors, etc.; the safety early warning system is used for safety monitoring and early warning during ship operation.

[0082] The ship identification system is used to identify the ship's name, registration number, and type.

[0083] The gate control system is used to control the opening and closing of the gate; the data monitoring system is used to monitor the entire process of ships entering and leaving the gate, including video monitoring and sensor monitoring.

[0084] The data query system is used to query all information about ships entering and exiting the lock, including ship type, ship name and number, number of ships, and time of entry and exit.

[0085] The information sharing system is used to share information such as the time of entry and exit of the lock gates and the type of vessel.

[0086] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus.

[0087] Finally, it should be noted that the above descriptions are merely preferred embodiments of the present invention and are not intended to limit the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or make equivalent substitutions for some of the technical features. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A lock attendant type safety monitoring system based on mobile monitoring, characterized by, include: The perception layer is used to collect ship information and lock information in real time, the ship information including first ship information and second ship information; The sensing layer includes: Cameras are used to acquire initial ship information; A camera movement system used to adjust the position of the camera based on a camera position adjustment signal; The accompanying detection system is used to generate camera position adjustment signals based on the first ship information; A sensor monitoring system is used to acquire information about a second vessel; The accompanying detection system includes: The image target detection module uses a Gaussian mixture background model to identify and locate the first ship information, and determine the ship waiting to enter the lock gate; The camera distribution module determines the number of cameras to be transported from the upper track to the lower track based on the ships waiting to enter the gate. The camera adjustment module processes the images acquired by the cameras on the upper track using an image stitching method to generate a camera position adjustment signal; the position of the camera is the distance between each adjacent camera on the first circular upper track and the distance between each adjacent camera on the second circular upper track. The processing layer is used to calculate the processing result based on the ship information and lock information; The application layer is used for security monitoring, intelligent decision-making, and security assessment based on the processing results. The camera movement system includes: a track device and a movement component; The track device is installed on the water-blocking wall and the lock chamber wall of the upstream and downstream navigation channels, the moving component is set on the track device, and the camera is installed above the moving component; The track device includes a first circular track and a second circular track. The first circular track is installed on the water-blocking wall of the upstream and downstream approach channels and on one side wall of the lock chamber. The second circular track is installed on the water-blocking wall on the other side of the upstream and downstream channels and on the other side wall of the lock chamber. The monitoring directions of the cameras on the first and second circular tracks are opposite. The first annular track includes a first upper annular track and a first lower annular track, which are connected end-to-end. The first upper annular track is used for mobile monitoring accompanying the ship, and the first lower annular track is used to transport cameras to the first upper annular track. The second annular track includes a second upper annular track and a second lower annular track, which are connected end-to-end. The second upper annular track is used for mobile monitoring accompanying the ship, and the second lower annular track is used to transport cameras to the second upper annular track. The moving component includes a moving device, a moving plate, a first rotating rod, a second rotating rod, and a mounting plate; the moving device is connected to the moving plate and located below the moving plate; the first rotating rod and the second rotating rod are fixedly connected to the moving plate and located above the moving plate; the first rotating rod is located to the left of the second rotating rod; the first rotating rod and the second rotating rod are rotatably connected to the mounting plate; the mounting plate is located above the first rotating rod and the second rotating rod; and the camera is mounted above the mounting plate.

2. A ship lock attendant type safety monitoring system based on mobile monitoring according to claim 1, characterized in that: The Gaussian mixture background model is as follows: ； is a Gaussian mixture background model, is a weight factor, denotes the probability density function of a multivariate variable X following a Gaussian distribution, and denote the mean vector and the covariance matrix of the th Gaussian distribution, respectively.

3. A ship lock attendant type safety monitoring system based on mobile monitoring according to claim 1, characterized in that: The camera position adjustment is performed by stitching together the images acquired by all cameras on the first and second ring tracks using an image stitching method, and generating a camera position adjustment signal based on the difference between the stitched image and the overall image of the ship.

4. A ship lock attendant type safety monitoring system based on mobile monitoring according to claim 3, characterized in that: The position of the camera is the distance between each adjacent camera on the first ring track and the distance between each adjacent camera on the second ring track; the position of the camera is adjusted in real time according to the camera position adjustment signal as the ship moves.

5. A mobile monitoring based lock attendant safety monitoring system according to claim 3, characterized in that: The image stitching method is as follows: ； wherein, and are pixel values at the image to be stitched , is a coordinate set of the image to be stitched; is a fused pixel value; , is a weight coefficient, satisfying , and are all greater than 0; the weight is , , is the horizontal coordinate of the boundary of the overlapping region, .

6. A lock-based accompanying safety monitoring system based on mobile monitoring according to claim 4, characterized in that: The steps for assigning the camera are as follows: When a vessel enters the monitoring range of the first or second camera, the vessel is identified using a Gaussian mixture background model. The identified vessel waiting to enter the gate is then matched with information in the vessel reservation database to obtain the vessel length. The number N of cameras that move with the ship is determined based on the ship's length and the camera's monitoring range. The first annular lower track conveys to the first annular upper track. A camera is transported from the second lower circular track to the second upper circular track. One camera.

7. A lock-based accompanying safety monitoring system based on mobile monitoring according to claim 6, characterized in that: When all the cameras on the first ring upper track move to the first ring lower track and all the cameras on the second ring upper track move to the second ring lower track, the first ring lower track transports the first camera to the first ring upper track, and the second ring lower track transports the second camera to the second ring upper track. The first camera is located at the connection between the first upper circular track and the first lower circular track; The second camera is located at the junction of the second upper circular track and the second lower circular track; When the cameras finish following the ship's movement, all the cameras on the first upper ring track move to the first lower ring track, and at the same time, all the cameras on the second upper ring track move to the second lower ring track.

Images