A video monitoring method based on ship lock usage scenario

A multi-camera and image recognition system for ship locks addresses the limitations of existing monitoring by providing panoramic views and real-time detection of abnormal behaviors, improving efficiency and accuracy.

CN118781561BActive Publication Date: 2024-12-03ANHUI JINHAI DEER INFORMATION TECHNOLOGY CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411266630.3
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-09-11
Publication Date
2024-12-03
Estimated Expiration
2044-09-11

AI Technical Summary

Technical Problem

Existing ship lock monitoring systems lack the ability to identify abnormal behaviors and predict impending dangers, require constant human supervision, and suffer from scattered monitoring areas and inefficient video data management.

Method used

Deploy multiple cameras with overlapping views and positioning signs, use a YOLOV3 image recognition model for real-time detection, register and fuse images into a panoramic view, and calculate a comprehensive evaluation value for real-time warnings.

Benefits of technology

Provides panoramic monitoring with no blind spots, automatically identifies abnormal behaviors, and issues timely alarms, enhancing monitoring efficiency and accuracy.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118781561B_ABST
    Figure CN118781561B_ABST
Patent Text Reader

Abstract

The present invention provides a video monitoring method based on a ship lock use scenario, and relates to the technical field of ship lock video monitoring. The present invention collects image data of a ship lock area to be monitored, decomposes the video by uploading it to a cloud server, obtains each frame of the video, detects targets, abnormal behaviors and positioning signs on the image by a trained YOLOV3 image recognition model, splices and fuses two images according to the recognition result of the positioning sign to form a panoramic image of the ship lock area, synthesizes a panoramic video using the panoramic image, and issues a real-time alarm according to the result of the abnormal behavior recognition. In the process of entering and exiting the ship lock, a real-time warning is issued by comprehensively evaluating the detection targets, and the comprehensive evaluation value is displayed, and the synthesized panoramic video and danger information are sent to a monitoring device for real-time monitoring.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of ship lock video monitoring, and in particular to a video monitoring method based on a ship lock use scenario. Background Art

[0002] Locks allow ships to pass through areas with different water levels, thereby increasing shipping capacity, shortening voyages, and reducing transportation costs. However, since too many ships need to pass through locks, collisions and accidents such as people falling into the water are unavoidable, so it is necessary to strengthen the supervision of locks through various methods.

[0003] In the prior art, the monitoring method for ship locks is mostly to install cameras at key positions of the ship locks, such as the entrance, exit, inside and outside the ship lock, and other key positions, and then monitor the captured images through the display screen in the machine room. This method can monitor to a certain extent, but these monitoring areas are relatively scattered, and staff who are familiar with the ship locks are required to accurately find the monitoring area of ​​the ship lock monitoring screen among many monitoring screens.

[0004] Although existing technology can provide monitoring to a certain extent, it is unable to identify abnormal behaviors such as whether the ship in the video has committed dangerous accidents. Staff are still required to keep a close eye on the surveillance video at all times to observe whether any danger has occurred. It is also unable to predict impending dangers. At the same time, a large amount of video is stored in the computer room, which will also affect the operating speed of the monitoring equipment.

[0005] The above information disclosed in this Background section is only for enhancement of understanding of the background of the present disclosure and therefore it may contain information that does not constitute the prior art that is already known to one of ordinary skill in the art. Summary of the invention

[0006] The purpose of the present invention is to provide a video monitoring method based on a ship lock usage scenario to solve the problems raised in the above background technology.

[0007] To achieve the above object, the present invention provides the following technical solutions:

[0008] A video monitoring method based on a ship lock usage scenario, the specific steps comprising:

[0009] Step 1: Deploy multiple cameras of the same model in the lock area to be monitored, so that the view areas of each adjacent camera overlap a part, and place multiple positioning signs within the boundary of the view frame of each camera;

[0010] Step 2: Build a YOLOV3 image recognition model, obtain historical monitoring images of the lock area as the initial training image set, increase the number of initial training image sets through preprocessing, and label the detection targets in the images to form a training image set;

[0011] Step 3: Train the model using the training image set and improve the accuracy and precision of target detection by setting the loss function;

[0012] Step 4: Decompose the surveillance video collected by the camera into time-series frame images, input the time-series frame images into the trained YOLOV3 image recognition model, output the label of the detection target, and issue a real-time alarm when abnormal behavior is detected;

[0013] Step 5: Based on the recognition results of the positioning sign, the same time frame images collected by two adjacent cameras are registered and converted into images in the same pixel coordinate system. The overlapping parts of the two converted images are fused to form a panoramic image of the ship lock monitoring area.

[0014] Step 6: Based on the detection label of the registered detection target, a real-time warning is performed by calculating the comprehensive evaluation value of the detection target and displaying the comprehensive evaluation value;

[0015] Step 7: The panoramic image of each frame of the ship lock monitoring area is synthesized into a real-time video according to the timestamp, and the alarm information and comprehensive evaluation value of the identified abnormal behavior are displayed in the video.

[0016] Furthermore, the camera is deployed in the following manner:

[0017] The cameras are installed in a vertically spaced arrangement on a horizontal line at the same height and evenly distributed in the lock monitoring area.

[0018] Furthermore, the positioning signs are signs of different shapes, at least four positioning signs are placed in the overlapping area of ​​the same camera viewfinder, and the positions of any three positioning signs are not in a straight line.

[0019] Furthermore, the specific method of increasing the number of the initial training image set by preprocessing and marking the detection targets in the images to form the training image set is:

[0020] The method of increasing the number of initial training image sets by preprocessing is: increasing the number of initial training image sets by flipping, cropping and rotating the initial training image sets, and marking the detection targets to be identified in the images with bounding boxes and labels;

[0021] The detection targets include positioning signs, ship status, personnel and abnormal behavior. The ship status includes the ship type and the ship waterline. The abnormal behavior includes personnel falling into the water and ship collision.

[0022] The label type includes the pixel coordinates of the center point of the bounding box, the bounding box coordinates, and the name of the detection target selected by the bounding box.

[0023] Furthermore, the loss function of the model is:

[0024]

[0025] in, is the intersection-and-union ratio, It is the minimum enclosed area of ​​the real box and the predicted box; is the union area of ​​the real box and the predicted box, is the intersection area of ​​the real box and the predicted box.

[0026] Furthermore, the method for registering the two graphics according to the coordinate position of the center point of the positioning sign is:

[0027] Select one of the images as the reference image and the other as the image to be registered, substitute the position coordinates of the center points of the positioning signs of the same shape identified in the two images into the registration model, calculate the parameters of the model, and then register the pixel coordinates of the image to be registered according to the registration model with confirmed parameters. The formula of the registration model is:

[0028]

[0029] in, is the position coordinate of the center point of the positioning sign in the reference image, To locate the center coordinates of the sign in the image that needs to be converted, is the homography matrix;

[0030] The method of fusing the overlapping parts to form a panoramic image is as follows:

[0031]

[0032] in, The image pixels in the overlapping area, is the pixel of an image to be stitched, is the pixel of the other image to be stitched, and They represent the weighting coefficients corresponding to the pixels in the overlapping area, where , .

[0033] Furthermore, the specific steps of providing real-time warning by comprehensive evaluation of the detection target are as follows;

[0034] The comprehensive assessment is calculated by calculating the safety score of the ship in the lock and the safety estimate when entering and leaving the lock respectively;

[0035] The safety score of the ship in the lock is based on the inspection results of the ship type, the probability of accidents for each type of ship is statistically analyzed through the historical data of the lock, and the value and weight of the ship are estimated according to the ship type and the waterline of the ship. The safe stopping distance of the ship is calculated based on the accident probability, estimated value and weight, and the safety score is calculated based on the distance between the stopped ships;

[0036] The calculation formula for safe stopping distance is:

[0037]

[0038] in, For safe stopping distance, is the distance conversion constant, is the weight conversion factor, For the waterline, is the probability of ship accident, is the value conversion coefficient;

[0039] The calculation formula of the safety score is:

[0040]

[0041] in, is the safety score, , Respectively The vessel and the distance between the vessels;

[0042] Safety estimation: The safety estimation of the ship is made based on its movement trajectory when entering and leaving the lock;

[0043] The calculation formula for the moving speed of the ship target is;

[0044]

[0045] in, For the The detected objects are in the pixel coordinate system Axis moving speed, For the The detected objects are in the pixel coordinate system Axis moving speed, Respectively The detection target is in The coordinates in the frame image, Respectively The detection target is in the Frame Image Difference The coordinates of the frame in the image, For the difference The time between images of the frame;

[0046] The calculation formula for the moving direction of the detected target is:

[0047]

[0048] in, For the Detection target The coordinate position at time For the Detection target The coordinate position at time is a positive number, For the The detected objects are in the pixel coordinate system Axis moving speed, For the The detected objects are in the pixel coordinate system Axis moving speed;

[0049] By solving the following equation for time , determine whether the detection target will collide;

[0050]

[0051] in, For the Detection target The coordinate position at time For the Detection target The coordinate position at time is a positive integer, and ;

[0052] If there is no time , then The detection target and The detection targets will not collide;

[0053] If there is time , then The detection target and The detection target is If there is a collision, an immediate warning should be given;

[0054] The calculation method of safety estimate is:

[0055]

[0056] in, For safety estimates, To avoid safety time;

[0057] The calculation formula for the comprehensive evaluation value is:

[0058]

[0059] in, is the comprehensive evaluation value, , are the weights of the safety score and safety estimate respectively. ;

[0060] when When there is a safety risk, an early warning will be issued immediately;

[0061] when There is no safety risk and no warning is required.

[0062] Compared with the prior art, the beneficial effects of the present invention are as follows: the present invention collects image data of the monitored area of ​​the ship lock, decomposes the video by uploading it to the cloud server, obtains each frame of the video, detects targets, abnormal behaviors and positioning signs in the image through the trained YOLOV3 image recognition model, splices and fuses the two images according to the recognition result of the positioning sign to form a panoramic image of the ship lock area, synthesizes a panoramic video using the panoramic image, and issues a real-time alarm based on the result of the abnormal behavior recognition. In the process of entering and exiting the ship lock, a real-time warning is issued by calculating the comprehensive evaluation value of the detection target, and the comprehensive evaluation value is displayed, and the synthesized panoramic video and danger information are sent to the monitoring equipment for real-time monitoring.

[0063] The present invention adopts multi-camera splicing technology to achieve panoramic monitoring of the lock without blind spots, which is helpful for staff to supervise the overall monitoring area of ​​the lock. Image recognition technology is used to perform real-time analysis of monitoring videos, automatically identify abnormal behaviors such as ship collisions, people falling into the water, etc., and issue alarms in time. This intelligent analysis system can greatly improve the efficiency and accuracy of monitoring. The present invention also uses image recognition technology to analyze the movement direction and speed of the detection target throughout the entire stage of entering and exiting the lock based on the recognition results of the detection target, and warns of the movement trajectory and parking status of the detection target. BRIEF DESCRIPTION OF THE DRAWINGS

[0064] Figure 1It is a schematic diagram of the overall method flow of the present invention. DETAILED DESCRIPTION

[0065] In order to make the objectives, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with specific embodiments.

[0066] It should be noted that, unless otherwise defined, the technical terms or scientific terms used in the present invention should be understood by people with ordinary skills in the field to which the present invention belongs. The words "first", "second" and similar words used in the present invention do not indicate any order, quantity or importance, but are only used to distinguish different components. "Include" or "comprise" and similar words mean that the elements or objects appearing before the word include the elements or objects listed after the word and their equivalents, without excluding other elements or objects. "Connect" or "connected" and similar words are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. "Up", "down", "left", "right" and the like are only used to indicate relative positional relationships. When the absolute position of the described object changes, the relative positional relationship may also change accordingly.

[0067] Example:

[0068] See also Figure 1 , the present invention provides a technical solution:

[0069] A video monitoring method based on a ship lock usage scenario, the specific steps comprising:

[0070] Step 1: Deploy multiple cameras of the same model in the lock area that needs to be monitored, so that the view areas of each adjacent camera overlap a part, and place multiple positioning signs within the boundary of each camera's view frame.

[0071] As a complex engineering system, the ship lock involves multiple key areas and equipment, such as the lock chamber, entrances and exits, water level monitoring points, etc. Traditional videos cannot capture the full picture of these key areas at one time, but stitching videos shot by multiple cameras into a panoramic video can provide a broader monitoring field of view, covering a wider area, and helping monitoring personnel to fully understand the operating status of the ship lock.

[0072] Furthermore, the cameras are deployed in a manner of vertically spaced arrangement, with the cameras installed on a horizontal straight line at the same height and evenly distributed in the ship lock monitoring area.

[0073] Video image acquisition is the first step in completing the video image stitching process. The quality of the acquired video images will affect the quality of the later video stitching. With the development of science and technology and the improvement of video quality requirements, the placement of cameras is also particularly important. The straight line where the central axis of the camera lens is located is used as the reference. The central axis of each camera is in the same plane and the distance between them is the same, ensuring that the video size shot by each camera is the same, reducing the calculation of image registration in the later stitching process. The overlapping part of the area shot by each adjacent camera accounts for one-fifth of the shooting area. The overlapping area helps to detect the positioning sign, thereby facilitating image stitching.

[0074] Furthermore, the positioning signs are signs of different shapes, at least four positioning signs are placed in the overlapping area of ​​the same camera viewfinder, and the positions of any three positioning signs are not in a straight line.

[0075] Step 2: Build a YOLOV3 image recognition model, obtain historical monitoring images of the lock area as the initial training image set, increase the number of initial training image sets through preprocessing, and label the detection targets in the images to form a training image set.

[0076] YOLOv3 uses a fully convolutional network to extract global feature information from the entire image, which helps to improve the accuracy of target detection. It also introduces multi-scale feature processing and detects objects of different sizes through feature maps at different levels. Therefore, it is very suitable for detecting ships and personnel in lock video surveillance.

[0077] Flipping the initial training image set includes flipping the image horizontally or vertically. The features and categories are maintained after horizontal or vertical flipping, so that the model can learn the performance of objects in different directions. Rotation is to randomly rotate the image by a small angle, usually between 0 and ±10 degrees, which can help the model adapt to the performance at different angles. Cropping is to crop a fixed-size area from the image as a training sample, which can help the model learn the performance of objects in different positions. In short, the role of data augmentation techniques such as flipping and rotation in target detection tasks is to increase data diversity, improve model accuracy and generalization ability, reduce overfitting, and enhance training efficiency.

[0078] Furthermore, the specific method of increasing the number of the initial training image set by preprocessing and marking the detection targets in the images to form the training image set is:

[0079] The method of increasing the number of initial training image sets by preprocessing is: increasing the number of initial training image sets by flipping, cropping and rotating the initial training image sets, and marking the detection targets to be identified in the images with bounding boxes and labels;

[0080] The detection targets include positioning signs, ship status, personnel and abnormal behavior. The ship status includes the ship type and the ship waterline. The abnormal behavior includes personnel falling into the water and ship collision.

[0081] The label type includes the pixel coordinates of the center point of the bounding box, the bounding box coordinates, and the name of the detection target selected by the bounding box.

[0082] Step 3: Train the model using the training image set and set the loss function to improve the accuracy and precision of target detection.

[0083] Furthermore, the loss function of the model is;

[0084]

[0085] in, is the intersection-and-union ratio, It is the minimum enclosed area of ​​the real box and the predicted box; is the union area of ​​the real box and the predicted box, is the intersection area of ​​the real box and the predicted box.

[0086] Intersection and Union It is used to measure the overlap between the bounding box predicted by the model and the bounding box of the true label. The higher the ratio, the more accurate the detection result. During the training process, the model generates multiple bounding boxes and predicts the confidence score and category of each bounding box. In order to determine which predicted bounding boxes are correct, It is used to evaluate the matching degree between the predicted bounding box and the true label bounding box. When it is greater than the set threshold, which is usually 0.5, the predicted bounding box is considered to have correctly detected the target.

[0087] Step 4: Decompose the surveillance video captured by the camera into time-series frame images, input the time-series frame images into the trained YOLOV3 image recognition model, output the label of the detection target, and issue a real-time alarm when abnormal behavior is detected.

[0088] When a camera is shooting video, there is vignetting in the lens. Vignetting is caused by the manufacturing of the lens and other optical factors, which causes the video image captured by the lens to appear bright in the middle and dark at the edges. In order to keep the brightness of the picture basically consistent, the image brightness can be unified to eliminate the impact of vignetting. It is necessary to perform the image brightness unification operation on the images collected by multiple cameras separately.

[0089] Step 5: Based on the recognition results of the positioning sign, the same time frame images collected by two adjacent cameras are registered and converted into images in the same pixel coordinate system. The overlapping parts of the two converted images are fused to form a panoramic image of the ship lock monitoring area.

[0090] Image registration is the process of aligning and matching the content and geometric features of multiple images. Its main purpose is to integrate the information of multiple images into the same reference coordinate system so that they correspond or overlap in space.

[0091] Furthermore, the method for registering the two graphics according to the coordinate position of the center point of the positioning sign is:

[0092] Select one of the images as the reference image and the other as the image to be registered, substitute the position coordinates of the center points of the positioning signs of the same shape identified in the two images into the registration model, calculate the parameters of the model, and then register the pixel coordinates of the image to be registered according to the registration model with confirmed parameters. The formula of the registration model is:

[0093]

[0094] in, is the position coordinate of the center point of the positioning sign in the reference image, To locate the center coordinates of the sign in the image that needs to be converted, is the homography matrix.

[0095] The position coordinates of the center point of the positioning sign in the two images , Substitute into the registration model to obtain;

[0096]

[0097] That is, a corresponding center point of a positioning sign can generate two constrained equations. Since the degree of freedom of the homography matrix is ​​8, at least 4 pairs of corresponding points are required, and any three points are not collinear. That is, the coordinate positions of at least 4 positioning signs of different shapes placed in the overlapping area of ​​the camera viewfinder can solve the homography matrix.

[0098] The homography matrix is ​​a transformation model. In the process of image registration, the two images can be aligned in the same coordinate system by matching the feature points in the two images and calculating the homography function between them.

[0099] The overlapping parts are fused to form a panoramic image by using a gradual-in-fade-out weighted mean fusion algorithm. The formula of the gradual-in-fade-out weighted mean fusion algorithm is as follows:

[0100]

[0101] in, The image pixels in the overlapping area, is the pixel of an image to be stitched, is the pixel of the other image to be stitched, and They represent the weighting coefficients corresponding to the pixels in the overlapping area, where , .

[0102] The fade-in and fade-out weighted mean fusion algorithm is not just a direct splicing of the pixel values ​​of an image, but a comprehensive consideration of the positional relationship of each pixel in the overlapping area, and the gradual change of the pixel weights is set according to the relative position, so that there is a transition process for the fusion of excessive overlapping areas, making the image of the overlapping area after fusion more natural.

[0103] Step 6: Based on the detection label of the registered detection target, a real-time warning is issued by calculating the comprehensive evaluation value of the detection target and displaying the comprehensive evaluation value.

[0104] Furthermore, the specific steps of providing real-time warning by comprehensive evaluation of the detection target are as follows;

[0105] By calculating the safety score of the ship in the lock and the safety estimate when entering and leaving the lock, a comprehensive assessment is calculated;

[0106] The safety score of a ship in a lock is based on the inspection results of the ship type. The accident probability of each type of ship is statistically analyzed through the historical data of the lock, and the value and weight of the ship are estimated based on the ship type and waterline. The safe stopping distance of the ship is calculated based on the accident probability, estimated value and weight, and the safety score is calculated based on the distance between the stopped ships.

[0107] When calculating safe stopping distances within locks, the probability of an accident occurring to a vessel, the estimated value, and the weight of the vessel are considered in order to ensure the safety and effectiveness of the operation. The probability of an accident occurring to a vessel directly affects the required safety distance. A high probability of an accident means that more safety space is required to reduce the risk to the vessel and the lock in the event of an accident. The consideration of estimated value is to protect high-value vessels from losses due to insufficient space and to ensure that high-value vessels have adequate protection measures when docked. The weight of the vessel also affects the safety distance, as heavier vessels require more space to operate stably and safely when moving and docking. Combining these factors allows for an accurate assessment of the safe stopping distance, ensuring that the locks are able to effectively prevent accidents and losses when handling different types and sizes of vessels, thereby improving the safety and efficiency of the overall operation.

[0108] The calculation formula for safe stopping distance is:

[0109]

[0110] in, For safe stopping distance, is the distance conversion constant, is the weight conversion factor, For the waterline, is the probability of ship accident, is the value conversion coefficient;

[0111] The calculation formula of the safety score is:

[0112]

[0113] in, is the safety score, , Respectively The vessel and the distance between the vessels;

[0114] Safety estimation: The safety estimation of the ship is made based on its movement trajectory when entering and leaving the lock;

[0115] The moving speed of the ship target is calculated by the frame difference method, which is a common technology in the field of video processing and computer vision to detect and track the movement of objects. It identifies the movement of the object by comparing the difference between two or more consecutive frames of images, and calculates the speed of the object based on the time difference between the frames.

[0116] The calculation formula for the moving speed of the ship target is;

[0117]

[0118] in, For the The detected objects are in the pixel coordinate system Axis moving speed, For the The detected objects are in the pixel coordinate system Axis moving speed, Respectively The detection target is in The coordinates in the frame image, Respectively The detection target is in the Frame Image Difference The coordinates of the frame in the image, For the difference The time between images of the frame;

[0119] The calculation formula for the moving direction of the detected target is:

[0120]

[0121] in, For the Detection target The coordinate position at time For the Detection target The coordinate position at time is a positive number, For the The detected objects are in the pixel coordinate system Axis moving speed, For the The detected objects are in the pixel coordinate system Axis moving speed;

[0122] By solving the following equation for time , determine whether the detection target will collide;

[0123]

[0124] in, For the Detection target The coordinate position at time For the Detection target The coordinate position at time is a positive integer, and ;

[0125] If there is no time , then The detection target and The detection targets will not collide;

[0126] If there is time , then The detection target and The detection target is If there is a collision, an immediate warning should be given;

[0127] The calculation method of safety estimate is:

[0128]

[0129] in, For safety estimates, To avoid safety time;

[0130] The calculation formula for the comprehensive evaluation value is:

[0131]

[0132] in, is the comprehensive evaluation value, , are the weights of the safety score and safety estimate respectively. ;

[0133] when When there is a safety risk, an early warning will be issued immediately;

[0134] when There is no safety risk and no warning is required.

[0135] Step 7: The panoramic image of each frame of the ship lock monitoring area is synthesized into a real-time video according to the timestamp, and the alarm information and comprehensive evaluation value of the identified abnormal behavior are displayed in the video.

[0136] The above formulas are all dimensionless and numerical calculations. The formula is a formula for the most recent real situation obtained by collecting a large amount of data and performing software simulation. The preset parameters in the formula are set by technicians in this field according to actual conditions.

[0137] The above embodiments may be implemented in whole or in part by software, hardware, firmware or any other combination thereof. When implemented by software, the above embodiments may be implemented in whole or in part in the form of a computer program product. Those skilled in the art may appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein may be implemented by electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed by hardware or software methods depends on the specific application and design constraints of the technical solution.

[0138] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, and may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0139] The above description is only a specific implementation manner of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application.

Claims

1. A video monitoring method based on a ship lock usage scenario, characterized in that: The specific steps include: Step 1: Deploy multiple cameras of the same model in the lock area to be monitored, so that the view areas of each adjacent camera overlap a part, and place multiple positioning signs within the boundary of the view frame of each camera; Step 2: Build a YOLOV3 image recognition model, obtain historical monitoring images of the lock area as the initial training image set, increase the number of initial training image sets through preprocessing, and label the detection targets in the images to form a training image set; Step 3: Train the model using the training image set and improve the accuracy and precision of target detection by setting the loss function; Step 4: Decompose the surveillance video collected by the camera into time-series frame images, input the time-series frame images into the trained YOLOV3 image recognition model, output the label of the detection target, and issue a real-time alarm when abnormal behavior is detected; Step 5: Based on the recognition results of the positioning sign, the same time frame images collected by two adjacent cameras are registered and converted into images in the same pixel coordinate system. The overlapping parts of the two converted images are fused to form a panoramic image of the ship lock monitoring area. Step 6: According to the label of the detected target after registration, a real-time warning is performed by calculating the comprehensive evaluation value of the detected target and displaying the comprehensive evaluation value; Step 7: The panoramic image of each frame of the ship lock monitoring area is synthesized into a real-time video according to the timestamp, and the alarm information and comprehensive evaluation value of the identified abnormal behavior are displayed in the video; The specific steps of performing real-time early warning by calculating the comprehensive evaluation value of the detection target are: The comprehensive assessment is calculated by calculating the safety score of the ship in the lock and the safety estimate when entering and leaving the lock respectively; The safety score of the ship in the lock is based on the inspection results of the ship type, the probability of accidents for each type of ship is statistically analyzed through the historical data of the lock, and the value and weight of the ship are estimated according to the ship type and the waterline of the ship. The safe stopping distance of the ship is calculated based on the accident probability, estimated value and weight, and the safety score is calculated based on the distance between the stopped ships; The calculation formula for safe stopping distance is: in, For safe stopping distance, is the distance conversion constant, is the weight conversion factor, For the waterline, is the probability of ship accident, is the value conversion coefficient; The calculation formula of the safety score is: in, is the safety score, , Respectively The vessel and the distance between the vessels; Safety estimation: The safety estimation of the ship is made based on its movement trajectory when entering and leaving the lock; The calculation formula for the moving speed of the ship target is; in, For the The detected objects are in the pixel coordinate system Axis moving speed, For the The detected objects are in the pixel coordinate system Axis moving speed, Respectively The detection target is in The coordinates in the frame image, Respectively The detection target is in the Frame Image Difference The coordinates of the frame in the image, For the difference The time between images of the frame; The calculation formula for the moving direction of the detected target is: in, For the Detection target The coordinate position at time For the Detection target The coordinate position at time is a positive number, For the The detected objects are in the pixel coordinate system Axis moving speed, For the The detected objects are in the pixel coordinate system Axis moving speed; By solving the following equation , determine whether the detection target will collide; in, For the Detection target The coordinate position at time For the Detection target The coordinate position at time is a positive integer, and ; If there is no time , then The detection target and The detection targets will not collide; If there is time , then The detection target and The detection target is If there is a collision, an immediate warning should be given; The calculation method of safety estimate is: in, For safety estimates, To avoid safety time; The calculation formula for the comprehensive evaluation value is: in, is the comprehensive evaluation value, , are the weights of the safety score and safety estimate respectively. ; when When there is a safety risk, an early warning will be issued immediately; when There is no safety risk and no warning is required.

2. A video monitoring method based on a ship lock usage scenario according to claim 1, characterized in that: The camera is deployed in the following manner: The cameras are installed in a vertically spaced arrangement on a horizontal line at the same height and evenly distributed in the lock monitoring area.

3. The video monitoring method based on the ship lock usage scenario according to claim 1 is characterized in that: The positioning signs are signs of different shapes. At least four positioning signs are placed in the overlapping area of ​​the same camera viewfinder, and the positions of any three positioning signs are not on a straight line.

4. The video monitoring method based on the ship lock usage scenario according to claim 1 is characterized in that: The specific method of increasing the number of the initial training image set by preprocessing and marking the detection targets in the images to form the training image set is as follows: The method of increasing the number of initial training image sets by preprocessing is: increasing the number of initial training image sets by flipping, cropping and rotating the initial training image sets, and marking the detection targets to be identified in the images with bounding boxes and labels; The detection targets include positioning signs, ship status, personnel and abnormal behavior. The ship status includes the ship type and the ship waterline. The abnormal behavior includes personnel falling into the water and ship collision. The label type includes the pixel coordinates of the center point of the bounding box, the bounding box coordinates, and the name of the detection target selected by the bounding box.

5. The video monitoring method based on the ship lock usage scenario according to claim 1 is characterized in that: The loss function of the model is: in, is the intersection-and-union ratio, It is the minimum enclosed area of ​​the real box and the predicted box; is the union area of ​​the real box and the predicted box, is the intersection area of ​​the real box and the predicted box.

6. The video monitoring method based on the ship lock usage scenario according to claim 1 is characterized in that: The method for registering two images according to the coordinate position of the center point of the positioning sign is: Select one of the images as the reference image and the other as the image to be registered, substitute the position coordinates of the center point of the positioning sign of the same shape identified in the two images into the registration model, calculate the parameters of the model, and then register the pixel coordinates of the image to be registered according to the registration model with confirmed parameters. The formula of the registration model is: in, is the position coordinate of the center point of the positioning sign in the reference image, To locate the center coordinates of the sign in the image that needs to be converted, is the homography matrix; The method of fusing the overlapping parts to form a panoramic image is as follows: in, The image pixels in the overlapping area, is the pixel of an image to be stitched, is the pixel of the other image to be stitched, and They represent the weighting coefficients corresponding to the pixels in the overlapping area, where , .