Ship anti-collision safety early warning method and system based on single-frame look-around image

Abstract

The invention provides a ship anti-collision safety early warning method and system based on a single-frame look-around image, and the method comprises the steps: obtaining a single-frame look-around image shot by a ship through a look-around camera at fixed time intervals; inputting the obtained single-frame look-around image into the trained ship instance segmentation model to obtain instance segmentation results of all ships in the frame image; the distance between each ship and the ship is calculated based on the instance segmentation result of the ship in the frame image, and ship images with the distances smaller than a set threshold value are cut out; inputting the cut ship images into a ship bow orientation regression model to obtain bow orientations of all ships; and carrying out ship anti-collision safety early warning based on the prow orientation recognition result. The method does not depend on other ship data sending conditions, can monitor the collision risk around the ship in real time only by depending on ship shooting information, and is a simple, effective and stable ship anti-collision safety early warning method.

Description

Technical Field

[0001] The present invention relates to the technical field of ship anti-collision, and in particular to a ship anti-collision safety warning method and system based on a single-frame surround view image. Background Art

[0002] When a ship collides with other ships during its travel, it will consume a lot of manpower, material resources and financial resources. Therefore, it is very necessary to provide timely warning of collision risks during the travel of the ship. Currently, collision risk warnings are mostly processed through manual lookout or AIS data. Manual lookout is the most primitive and effective anti-collision safety warning method. This method requires the crew to concentrate on strictly performing lookout at the specified time to determine the collision situation around the ship. This will consume a lot of manpower and is heavily dependent on the concentration and experience of the lookout crew. Despite this, there are still safety hazards of collision in the lookout blind spot.

[0003] Anti-collision safety warning through AIS data is based on the received AIS data of the target ship and the AIS data of the own ship, and the future navigation trajectory of the target ship and the own ship is predicted. When the predicted trajectory has a collision risk, a collision warning is issued. This method is widely used in current ship anti-collision warning. Although this method is relatively effective, when the own ship cannot receive the AIS data of the target ship, the navigation trajectory of the target ship cannot be predicted, and the anti-collision safety warning method loses its effectiveness, and there is a collision risk. Summary of the invention

[0004] In view of the defects in the prior art, the purpose of the present invention is to provide a ship anti-collision safety warning method and system based on a single-frame surround view image. The method of the present invention does not rely on the data sending of other ships, but only relies on the shooting information of the ship to monitor the collision risk around the ship in real time. It is a simple, effective and stable ship anti-collision safety warning method.

[0005] To solve the above problems, the technical solution of the present invention is:

[0006] A ship anti-collision safety warning method based on a single-frame surround view image comprises the following steps:

[0007] At fixed time intervals, a single frame of surround view images captured by the ship is obtained through the surround view camera;

[0008] The acquired single-frame surround view image is input into the trained ship instance segmentation model to obtain the instance segmentation results of all ships in the frame image;

[0009] Based on the instance segmentation result of the ship in the frame image, the distance between each ship and the own ship is calculated, and the ship images whose distance is less than the set threshold are cut out;

[0010] The cut-out ship images are input into the ship bow orientation regression model to obtain the bow orientations of all ships;

[0011] Ship anti-collision safety warning is carried out based on the bow direction recognition results.

[0012] Preferably, in the step of inputting the acquired single-frame surround view image into a trained ship instance segmentation model to obtain instance segmentation results of all ships in the frame image, the instance segmentation model adopts the yolov8m-seg instance segmentation model.

[0013] Preferably, the construction and training of the single-frame surround view image ship instance segmentation model requires the collection of 20,000 surround view images of ships in different weather and environments during training, the instance segmentation and annotation of the ships in the collected images, the instance segmentation training using yolov8m-seg, and the acquisition of a trained ship instance segmentation model.

[0014] Preferably, in the step of calculating the distance of each ship from the own ship based on the instance segmentation result of the ship in the frame image, and cutting out the ship image with a distance less than a set threshold, the pixel distance of each ship closest to the edge of the own ship is calculated according to the instance segmentation result of each ship in the frame image, as the distance from each ship to the own ship. If the pixel distances from all ships to the own ship are greater than the set distance threshold, it is considered that there is no risk of collision with the own ship; otherwise, a ship with a pixel distance less than the set distance threshold is selected, and the ship image is cut out from the original image according to the maximum circumscribed rectangular box based on the instance segmentation result.

[0015] Preferably, in the step of inputting the cut-out ship image into a ship bow orientation regression model to obtain the bow orientations of all ships, the ship bow orientation regression model adopts a convolutional regression network.

[0016] Preferably, the ship bow orientation regression model is constructed and trained, during training, based on the instance segmentation and annotation results of 20,000 ship surround images, a ship image is cut out from the original image according to the maximum circumscribed rectangular box of the instance segmentation, the bow orientation of the ship in the ship image is annotated, with 0 degrees being the same as the orientation of the bow of the ship, and the directions are marked from 0 to 360 degrees in a clockwise direction, and a convolutional regression network is used to perform regression training on the annotated images to obtain a trained ship bow orientation regression model.

[0017] Preferably, in the step of issuing a ship anti-collision safety warning based on the bow direction recognition result, at the pixel point of the ship closest to the own ship, a direction arrow is drawn according to the predicted bow direction, and if the extension line of any one of the two vertical components of the direction arrow intersects with the own ship, it is considered that the bow of the ship is facing the own ship; if among the ships with predicted bow directions, the bows of one or more ships are facing the own ship, an anti-collision safety warning is issued.

[0018] Furthermore, the present invention also provides a ship collision avoidance safety warning system based on a single-frame surround view image, comprising a camera, a processor, and a memory for storing executable instructions of the processor, wherein the processor is configured to execute the ship collision avoidance safety warning method based on a single-frame surround view image as described above by executing the executable instructions.

[0019] Compared with the prior art, the ship anti-collision safety warning method based on single-frame surround view images of the present invention performs ship instance segmentation, bow direction recognition, and ship distance judgment on the single-frame surround view images taken by the ship at specified intervals, thereby realizing anti-collision safety warning. The method of the present invention does not rely on the data sending of other ships, but only relies on the shooting information of the ship to monitor the collision risk around the ship in real time. It is a simple, effective and stable ship anti-collision safety warning method. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Other features, objects and advantages of the present invention will become more apparent from the detailed description of non-limiting embodiments made with reference to the following drawings:

[0021] Figure 1 The present invention is a flowchart of a ship anti-collision safety warning method based on a single-frame surround view image. DETAILED DESCRIPTION

[0022] The present invention is described in detail below in conjunction with specific embodiments. The following embodiments will help those skilled in the art to further understand the present invention, but are not intended to limit the present invention in any form. It should be noted that, for those of ordinary skill in the art, several changes and improvements can also be made without departing from the concept of the present invention. These all belong to the protection scope of the present invention.

[0023] The present invention provides a ship anti-collision safety warning method based on a single-frame surround view image. The method of the present invention is applicable to any ship equipped with a surround view camera. The present invention does not rely on the data transmission of the target ship, but only relies on the shooting information of the ship. It is a simple, effective and stable ship anti-collision safety warning method. Specifically, Figure 1 As shown, the method comprises the following steps:

[0024] S1: At fixed time intervals, a single frame of surround view images captured by the ship is obtained through the surround view camera;

[0025] Specifically, in this embodiment, a frame of surround view image is acquired by the surround view camera at every fixed time interval, for example, 1 minute.

[0026] S2: Input the acquired single frame surround view image into the trained ship instance segmentation model to obtain instance segmentation results of all ships in the frame image;

[0027] Specifically, the single-frame surround view image ship instance segmentation model adopts the yolov8m-seg instance segmentation model. During the construction and training of the single-frame surround view image ship instance segmentation model, it is necessary to collect about 20,000 ship surround view images in different weather and environments, perform instance segmentation and annotation on the ships in the collected images, use yolov8m-seg for instance segmentation training, and obtain a trained ship instance segmentation model.

[0028] S3: Calculate the distance between each ship and the own ship based on the instance segmentation result of the ship in the frame image, and cut out the ship images whose distance is less than the set threshold;

[0029] Specifically, according to the instance segmentation result of each ship in the frame image, the pixel distance of each ship closest to the edge of the ship is calculated as the distance from each ship to the ship. If the pixel distance from all ships to the ship is greater than the set distance threshold, it is considered that there is no risk of collision with the ship; otherwise, the ship with a pixel distance less than the set distance threshold is selected, and the ship image is cut out from the original image according to the maximum circumscribed rectangular box based on the instance segmentation result.

[0030] S4: inputting the cut-out ship images into a ship bow orientation regression model to obtain the bow orientations of all ships;

[0031] Specifically, the ship bow direction regression model adopts a convolutional regression network. The ship bow direction regression model is constructed and trained. During training, based on the instance segmentation and annotation results of 20,000 ship surround images, a ship image is cut out from the original image according to the maximum circumscribed rectangular box of the instance segmentation, and the bow direction of the ship in the ship image is annotated, with the bow and the ship facing the same direction as 0 degrees, and clockwise, the directions are marked from 0 to 360 degrees. The annotated images are subjected to regression training using a convolutional regression network to obtain a trained ship bow direction regression model.

[0032] S5: Provide ship anti-collision safety warning based on the bow direction recognition result.

[0033] Specifically, in the step of issuing a ship anti-collision safety warning based on the bow direction recognition result, a ship whose pixel distance is less than a set distance threshold is selected, and the ship image is cut out from the original image according to the maximum circumscribed rectangular box based on the instance segmentation result. At the pixel point where the ship is closest to the ship, a direction arrow is drawn according to the predicted bow direction. If the extension line of any one of the two vertical components of the direction arrow intersects with the ship, it is considered that the bow of the ship is facing the ship. If the bows of one or more ships among the ships with predicted bow directions are facing the ship, an anti-collision safety warning is issued.

[0034] According to another aspect of an embodiment of the present application, a ship collision avoidance safety warning system based on a single-frame surround view image is provided, which includes a camera, a processor, and a memory for storing executable instructions of the processor, and the processor is configured to execute the ship collision avoidance safety warning method based on a single-frame surround view image described in the embodiment of the present application by executing the executable instructions.

[0035] The above describes the specific embodiments of the present invention. It should be understood that the present invention is not limited to the above specific embodiments, and those skilled in the art can make various changes or modifications within the scope of the claims, which does not affect the essence of the present invention. In the absence of conflict, the embodiments of the present application and the features in the embodiments can be combined with each other arbitrarily.

Claims

1. A ship anti-collision safety warning method based on a single-frame surround view image, characterized in that: The method comprises the following steps: At fixed time intervals, a single frame of surround view images captured by the ship is obtained through the surround view camera; The acquired single-frame surround view image is input into the trained ship instance segmentation model to obtain the instance segmentation results of all ships in the frame image; Based on the instance segmentation result of the ship in the frame image, the distance between each ship and the own ship is calculated, and the ship images whose distance is less than the set threshold are cut out; The cut-out ship images are input into the ship bow orientation regression model to obtain the bow orientations of all ships; Ship anti-collision safety warning is carried out based on the bow direction recognition results.

2. The ship anti-collision safety warning method based on a single-frame surround view image according to claim 1 is characterized in that: In the step of inputting the acquired single-frame surround view image into a trained ship instance segmentation model to obtain instance segmentation results of all ships in the frame image, the instance segmentation model adopts the yolov8m-seg instance segmentation model.

3. The ship anti-collision safety warning method based on single-frame surround view image according to claim 2 is characterized in that: The construction and training of the single-frame surround view image ship instance segmentation model requires the collection of 20,000 surround view images of ships in different weather and environments, the instance segmentation and annotation of the ships in the collected images, the instance segmentation training using yolov8m-seg, and the acquisition of a trained ship instance segmentation model.

4. The ship anti-collision safety warning method based on single-frame surround view image according to claim 1 is characterized in that: In the step of calculating the distance of each ship from the ship based on the instance segmentation result of the ship in the frame image, and cutting out the ship image whose distance is less than a set threshold, the pixel distance of each ship closest to the edge of the ship is calculated according to the instance segmentation result of each ship in the frame image, as the distance from each ship to the ship. If the pixel distances from all ships to the ship are greater than the set distance threshold, it is considered that there is no risk of collision between the ship and the ship; otherwise, a ship whose pixel distance is less than the set distance threshold is selected, and the ship image is cut out from the original image according to the maximum circumscribed rectangular frame based on the instance segmentation result.

5. The ship anti-collision safety warning method based on single-frame surround view image according to claim 4 is characterized in that: In the step of inputting the cut-out ship image into a ship bow orientation regression model to obtain the bow orientations of all ships, the ship bow orientation regression model adopts a convolutional regression network.

6. The ship anti-collision safety warning method based on single-frame surround view image according to claim 4 is characterized in that: The ship bow direction regression model is constructed and trained. During training, based on the instance segmentation and annotation results of 20,000 ship surround images, the ship image is cut out from the original image according to the maximum circumscribed rectangular box of the instance segmentation, and the bow direction of the ship in the ship image is annotated. The direction of the bow is the same as that of the ship as 0 degrees, and the directions are marked from 0 to 360 degrees in a clockwise direction. The annotated images are subjected to regression training using a convolutional regression network to obtain a trained ship bow direction regression model.

7. The ship anti-collision safety warning method based on single-frame surround view image according to claim 4 is characterized in that: In the step of issuing a ship anti-collision safety warning based on the bow direction recognition result, at the pixel point closest to the ship, a direction arrow is drawn according to the predicted bow direction. If the extension line of any one of the two vertical components of the direction arrow intersects with the ship, it is considered that the bow of the ship is facing the ship. If one or more ships among the ships with predicted bow directions have their bows facing the ship, an anti-collision safety warning is issued.

8. A ship anti-collision safety warning system based on a single-frame surround view image, characterized in that: The system includes a camera, a processor, and a memory for storing executable instructions of the processor, and the processor is configured to execute the ship collision avoidance safety warning method based on a single-frame surround view image as described in any one of claims 1 to 7 by executing the executable instructions.

Images