Water surface detection and identification method for water supervision and law enforcement security guards

Abstract

The invention discloses a water surface detection and identification method for water supervision and law enforcement security guards, and relates to a water surface detection and identification method. The invention aims to solve the problems that an existing detection and recognition method is single in function, various illegal regulations of a shipping ship cannot be well supervised, and illegal regulations of the ship cannot be processed in time due to the fact that the acquisition speed of a ship image is delayed. The method comprises the following steps: step 1, constructing a data set; step 2, establishing an image video stitching algorithm; step 3, training the data set in the step 1 in batches, and training the identification network by adopting different grouping methods to improve the accuracy and reliability of the identification network; and step 4, carrying out image identification on the spliced video stream. The invention belongs to the technical field of overwater safety supervision and identification.

Description

Technical Field

[0001] The invention relates to a water surface detection and identification method, belonging to the technical field of water safety supervision and identification. Background Art

[0002] During the shipping process, ships often have problems such as speeding and overdraft; in addition, some ships also have illegal dumping, entering restricted areas, illegal mooring and other illegal behaviors. These problems have affected the safety of the normal operation of the waterway to a certain extent, but the current manual recognition methods have great limitations; machine vision recognition methods based on machine learning and other methods have problems such as low recognition accuracy and susceptibility to environmental influences. For example: the publication number is CN118429913A, and the name of the invention is a ship recognition method based on multi-source remote sensing images, a server and storage medium, which compares and recognizes the recognized images and data sets based on multi-source remote sensing image data. However, the function of this design is relatively single, and it cannot meet the supervision of various illegal and irregular behaviors of shipping ships. In addition, there is a delay in the acquisition speed of ship images, and there are certain problems in the timely handling of ship violations.

[0003] Therefore, it is urgent to propose a water surface detection and identification method for water supervision and law enforcement security guards to solve the above technical problems. Summary of the invention

[0004] The present invention aims to solve the problems that the existing detection and identification methods have single functions and cannot meet the supervision needs of various illegal and irregular behaviors of shipping ships, and the delay in the acquisition speed of ship images leads to the inability to deal with the illegal and irregular behaviors of ships in a timely manner. Therefore, a water surface detection and identification method for water supervision and law enforcement security guards is proposed.

[0005] The technical solution adopted by the present invention to solve the above-mentioned problem is: the steps of the present invention include:

[0006] Step 1: Collect relevant information from multiple channels and build a comprehensive data set using tools such as YoloLabel;

[0007] Step 2: Establish an efficient image and video stitching algorithm;

[0008] Step 3: Based on the Yolov5 network model, the comprehensive data set of step 1 is trained in batches, and different grouping methods are used to perform multi-dimensional training on the recognition network to improve the accuracy and reliability of the recognition network;

[0009] Step 4: Install the software system constructed in steps 1 to 3 on the hardware of the water supervision and law enforcement security guard to achieve rapid image recognition of the spliced ​​video stream.

[0010] Furthermore, the comprehensive data set in step 1 includes a target ship data set and a river and sea waterway condition data set; the ship data set includes ship numbers and ship types.

[0011] Furthermore, step 2 specifically includes:

[0012] Step 201: pre-process the video frames using Gaussian filtering to reduce noise interference, and then apply the SURF (Speeded Up Robust Features) algorithm to extract features from the video stream using convolution kernels of different scales to accurately locate the splicing points in the channel scenes captured by the two cameras;

[0013] Step 202: By evaluating the similarity of the splicing points, the points with high matching degree in the two video streams are connected one by one, and the two video streams are seamlessly spliced ​​into a real-time panoramic waterway video stream by using image transformation and fusion technology to achieve monitoring without blind spots.

[0014] Furthermore, step 3 specifically includes:

[0015] Step 301, ship overspeed identification;

[0016] Step 302: Identify whether a ship has crossed the boundary or entered a prohibited navigation zone;

[0017] Step 303: Identification of excess draft of the ship;

[0018] Step 304: Ship fire identification.

[0019] Furthermore, step 301 specifically includes: identifying the characteristics of the ship when the ship is sailing, and when the image is recognized next time, comparing it with the previously recognized ship characteristics, and by comparing the ship positions at different times, identifying and calculating the speed of the ship in the channel; when it is recognized that the ship speed exceeds the maximum allowable speed of this channel, an alarm is automatically issued, and the current ship characteristics are recorded.

[0020] Furthermore, step 302 specifically includes: by dividing the ship restricted navigation zone in advance, analyzing and marking the characteristics of the ships in the channel, and importing the data set images and labels annotated by the YoloLabel software into Yolov5 for training, so that the software system can identify the ship. When a ship in the restricted navigation zone is identified, an alarm is automatically issued, and the characteristics of the currently identified ship are recorded.

[0021] Furthermore, step 303 specifically includes: identifying the distance between the ship's waterline and the water surface through a camera, automatically issuing an alarm when the identified ship's waterline is found to be abnormal, and recording the current ship characteristics.

[0022] Furthermore, step 304 specifically includes: using the temperature recognition technology carried by the gun-type camera to detect the temperature changes of navigable ships based on edge detection; when it is found that the edge temperature changes of the ship are abnormal, an alarm is automatically issued, and the current ship information is recorded and output to the cloud platform to remind the ship and waterway management department to handle it.

[0023] Furthermore, step 4 specifically includes: analyzing and marking the characteristics of ship violations in the waterway, importing the data set images and labels annotated by YoloLabel software into Yolov5 for training, so that the software system can identify ship violations. According to different image recognition features, the ship information in the waterway splicing panoramic video is determined and the identified ship violations are processed.

[0024] The beneficial effects of the present invention are:

[0025] 1. The present invention adopts a panoramic video stitching algorithm design, which can stitch the real-time video streams of two cameras into a panoramic video based on the stitching algorithm. On the one hand, it can complete the full coverage of the waterway scene, and on the other hand, it can also provide a panoramic video of the waterway for the illegal and irregular behavior recognition algorithm;

[0026] 2. The present invention relies on image recognition algorithms and can use camera terminals equipped with recognition algorithms to instantly process various target detection and recognition events and transmit data back, thereby realizing rapid and accurate recognition of illegal and irregular behaviors in the waterway. In addition, multiple recognition algorithms can realize the recognition of different illegal and irregular behaviors, thereby improving the versatility of the device and enhancing the waterway supervision effect;

[0027] 3. The present invention utilizes multiple camera visual perception terminals + exclusive visual recognition algorithms to achieve unmanned waterway safety supervision, greatly reducing the use of manpower and having advantages in economic benefits. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] Figure 1 It is a flow chart of the present invention. DETAILED DESCRIPTION

[0029] Specific implementation method 1: Figure 1 As shown, a method for detecting and identifying water surface security guards for water supervision and law enforcement is provided, and the specific steps include:

[0030] Step 1, construct a data set; the data set includes a target ship data set and a river and sea channel condition data set; the ship data set includes ship numbers and ship types. Through cooperation with waterway bureaus, public security departments and other institutions, obtain various types of original videos about ship appearance features, ship speeding, crossing boundaries, entering restricted navigation areas, overdraft, fire and other illegal behaviors, then clean the original videos, filter out pictures containing target feature information, and then import the pictures into YoloLabel software to select and mark the target features, and obtain a comprehensive data set containing pictures and corresponding target feature labels in the pictures;

[0031] Step 2: Establish an image and video stitching algorithm; specifically including:

[0032] Step 201: Based on Gaussian filtering and SURF algorithm, different convolution kernels are used to convolve the channel video stream from the camera, and the splicing points of the channel scenes in the two video streams are identified;

[0033] Step 202: Compare the splicing points, connect the splicing points with high similarity in the two video streams one by one, and realize splicing the two channel video streams into a real-time panoramic channel video stream;

[0034] Step 3: Train the data set in step 1 in batches, use different grouping methods to train the recognition network, and improve the accuracy and reliability of the recognition network; specifically, include:

[0035] Step 301, ship speeding identification: identify the ship's characteristics when the ship is sailing, compare the ship's characteristics with those previously identified when identifying the image next time, and identify the ship's speed in the channel by comparing the ship's positions at different times; when it is identified that the ship's speed exceeds the maximum allowable speed of the channel, an alarm is automatically issued and the current ship characteristics are recorded;

[0036] Step 302, ship crossing the boundary and entering the restricted navigation zone identification: use the comprehensive data set to train the training module of the YoloV5 network model, obtain the optimal weight file after iterative calculation, input it into the derivation module of the YoloV5 network model, and realize the rapid and accurate identification of the characteristics of the ship in the waterway. By dividing the restricted navigation zone for ships in advance, when a ship in the restricted navigation zone is identified, an alarm is automatically issued, and the characteristics of the currently identified ship are recorded;

[0037] Step 303, ship overdraft identification: using a camera to identify the distance between the ship's waterline and the water surface, and automatically sounding an alarm when the identified ship's waterline is abnormal, and recording the current ship characteristics;

[0038] Step 304, ship fire identification: using the temperature recognition technology carried by the gun-type camera, based on edge detection, the temperature change of the navigable ship is detected; when it is found that the edge temperature change of the ship is abnormal, an alarm is automatically issued, and the current ship information is recorded and output to the cloud platform to remind the ship and waterway management department to handle it;

[0039] Step 4: Perform image recognition on the spliced ​​video streams; determine the ship information in the spliced ​​panoramic video of the waterway based on different image recognition features and handle the identified illegal and irregular behaviors of the ships.

[0040] Example

[0041] An embodiment of the video stitching algorithm is: by reading the real-time video streams of two cameras, the feature points of the images of the two video streams at the same time are recognized, specifically including: based on the Gaussian convolution method, the stitching feature points of the two images are recognized; one-to-one matching of the recognized feature points; based on the recognized feature points, the two images are stitched together; after the images are transformed into a specified output form; the video stream image of the next frame is convoluted, recognized, and stitched;

[0042] The ship identification algorithm includes but is not limited to ship information identification, ship overspeed identification, ship overdraft identification, ship crossing the boundary, entering the restricted navigation zone identification, and ship fire identification;

[0043] An embodiment of the ship recognition algorithm is as follows: images containing ship objects and ship numbers are intercepted from the original video data and labeled with YoloLabel software to form a ship entity and ship number data set, and the data set consisting of images containing ship information is grouped; the grouped data sets are respectively input into the training module of the Yolov5 network model for training, and recognition training is performed on the characteristic information of the ship number and ship type; ship image information different from the training set is collected to construct a test set to test the performance of the trained neural network, and the network model is adjusted according to the test results until the requirements are met. After the performance meets the standard, the weight file obtained by training is input into the reasoning module of the Yolov5 network model, the video stream input by the camera is efficiently and accurately recognized, and corresponding information is returned for corresponding behaviors in the system;

[0044] An embodiment of the ship speeding identification algorithm is as follows: based on the ship identification algorithm, real-time identification of navigable ships in the waterway is performed; in the video stream, the ship position images in each time interval are compared; by identifying and one-to-one matching the feature points of the ship image in each time interval, the ship speed in the current time interval is obtained; and the speed is compared with the speed regulations set by the navigation department; after detecting that the ship is speeding, an autonomous alarm is issued and an autonomous decision is made through hardware;

[0045] An embodiment of the ship overdraft recognition algorithm is as follows: based on the ship recognition algorithm, real-time recognition of navigable ships in the waterway is performed; the draft of the navigable ship is recognized in the image of the panoramic video stream; when abnormal draft of the ship is detected, an autonomous alarm is issued and an autonomous decision is made through hardware;

[0046] An embodiment of the ship crossing the boundary and entering the prohibited navigation zone identification algorithm is as follows: intercepting pictures containing ship violations and illegal behaviors from the original video data and labeling them with YoloLabel software to form a ship violation and illegal behavior data set, and grouping the data set composed of pictures containing ship violations and illegal behaviors; inputting the grouped data sets into the training module of the Yolov5 network model for training, and performing recognition training on the characteristic information of ship violations and illegal behaviors; collecting ship violation and illegal behavior information different from the training set to construct a test set to test the performance of the trained neural network, and adjusting the network model according to the test results until the requirements are met. After the performance meets the standard, the weight file obtained by training is input into the reasoning module of the Yolov5 network model, and the video stream input by the camera is efficiently and accurately identified, and the corresponding information is set in the system for the corresponding behavior to be returned; the navigation department predetermines the prohibited parking and prohibited navigation sections and other areas in the waterway; based on the ship recognition algorithm, the navigable ships in the real-time panoramic video stream are identified; when the characteristics of navigable ships are detected in the target prohibited navigation and prohibited parking area, an autonomous alarm is given and an autonomous decision is made through hardware;

[0047] An embodiment of the ship fire identification algorithm is as follows: based on the temperature identification sensor built into the gun-type camera, the temperature inside the navigable ship is detected; based on the edge detection method, when a target with abnormally high temperature is detected, edge identification is performed; the ship detected in the video stream is compared to determine the ship where the fire occurred; an autonomous alarm is issued and an autonomous decision is made through hardware, and the report is notified to the waterway supervision department for processing.

[0048] How it works

[0049] When the present invention is working, based on the video stitching algorithm, the video streams input by two cameras are stitched in real time; the ship recognition algorithm is used to autonomously identify the navigable ships in the waterway in real time; and based on different illegal and irregular behavior algorithms, the illegal and irregular behaviors of navigable ships are autonomously identified in real time, and the detected illegal and irregular ship information is autonomously decided through hardware and output to the cloud platform.

[0050] The above is only a preferred embodiment of the present invention and does not limit the present invention in any form. Although the present invention has been disclosed as a preferred embodiment as above, it is not used to limit the present invention. Any technician familiar with this profession can make some changes or modify the technical contents disclosed above into equivalent embodiments without departing from the scope of the technical solution of the present invention. However, any simple modification, equivalent replacement and improvement made to the above embodiments without departing from the content of the technical solution of the present invention, based on the technical essence of the present invention, within the spirit and principles of the present invention, still fall within the protection scope of the technical solution of the present invention.

Claims

1. A method for detecting and identifying water surface security guards for water supervision and law enforcement, characterized in that: The specific steps include: Step 1: Build a data set; Step 2: Establish an image and video stitching algorithm; Step 3: Train the data set in step 1 in batches, use different grouping methods to train the Yolov5 recognition network model, and improve the accuracy and reliability of the recognition network; Step 4: Perform image recognition on the spliced ​​video stream.

2. A method for detecting and identifying water surface for water supervision and law enforcement security guards according to claim 1, characterized in that: The data set in step 1 includes a target ship data set and a river and sea channel condition data set; the ship data set includes a ship number and a ship type.

3. The method for detecting and identifying water surface security guards for water supervision and law enforcement according to claim 1 is characterized in that: Step 2 specifically includes: Step 201: Based on Gaussian filtering and SURF algorithm, different convolution kernels are used to convolve the channel video stream from the camera, and the splicing points of the channel scenes in the two video streams are identified; Step 202: Compare the splicing points, connect the splicing points with high similarity in the two video streams one by one, and realize splicing the two channel video streams into a real-time panoramic channel video stream.

4. The method for detecting and identifying water surface security guards for water supervision and law enforcement according to claim 1 is characterized in that: Step 3 specifically includes: Step 301, ship overspeed identification; Step 302: Identify whether a ship has crossed the boundary or entered a prohibited navigation zone; Step 303: Identification of excess draft of the ship; Step 304: Ship fire identification.

5. A method for detecting and identifying water surface for water supervision and law enforcement security guards according to claim 4, characterized in that: Step 301 specifically includes: identifying the characteristics of the ship when the ship is sailing, and comparing the characteristics of the ship with those previously identified when the image is identified next time, and identifying the speed of the ship in the channel by comparing the positions of the ship at different times; when it is identified that the speed of the ship exceeds the maximum allowable speed of the channel, an alarm is automatically issued, and the current ship characteristics are recorded.

6. A method for detecting and identifying water surface for water supervision and law enforcement security guards according to claim 4, characterized in that: Step 302 specifically includes: by dividing the ship restricted navigation zone in advance, using the Yolov5 network model trained based on the comprehensive data set to identify the characteristics of the ship in the channel, when a ship in the restricted navigation zone is identified, an alarm is automatically issued, and the currently identified ship characteristics are recorded.

7. A method for detecting and identifying water surface for water supervision and law enforcement security guards according to claim 4, characterized in that: Step 303 specifically includes: identifying the distance between the ship's waterline and the water surface through a camera, automatically issuing an alarm when the identified ship's waterline is found to be abnormal, and recording the current ship characteristics.

8. The method for detecting and identifying water surface security guards for water supervision and law enforcement according to claim 4 is characterized in that: Step 304 specifically includes: using the temperature recognition technology carried by the gun-type camera to detect the temperature changes of navigable ships based on edge detection; when it is found that the edge temperature changes of the ship are abnormal, an alarm is automatically issued, and the current ship information is recorded and output to the cloud platform to remind the ship and waterway management department to handle it.

9. The method for detecting and identifying water surface security guards for water supervision and law enforcement according to claim 1 is characterized in that: Step 4 specifically includes: determining the ship information in the channel stitching panoramic video based on different image recognition features and using the Yolov5 network model trained based on the comprehensive data set to process the identified ship violations.

Images