Image type ship active anti-collision system and method

Abstract

The invention relates to an image type ship active anti-collision system and method, and the system comprises a camera module which is used for obtaining a video stream in a river channel; the ship identification module is used for carrying out real-time target detection on each frame of image in the video stream so as to identify a ship and determine the position and the category of the ship; the perspective transformation module is used for converting the video image into a top view by adopting perspective transformation and calculating the distance between the ship and a preset dangerous area on the top view; the user interaction module is used for drawing a preset dangerous area of the river channel and displaying a ship detection result and distance information between the ship and the preset dangerous area in real time; and the alarm module is used for automatically triggering alarm when the distance between the ship and the preset dangerous area is smaller than or equal to a preset safety distance threshold value. The system can monitor the dynamic state of the ship in the river channel in real time, judges whether the ship enters the preset dangerous area or not through target recognition, can automatically give an alarm once the ship enters the preset dangerous area, and is low in cost.

Description

Technical Field

[0001] The present invention relates to the technical field of active collision avoidance, and in particular to an image-based active collision avoidance system and method for ships. Background Art

[0002] With the continuous increase in global waterway traffic, the issue of ship safety management has become increasingly prominent, especially in dangerous areas such as bridges and narrow waterways, where ship navigation safety faces huge challenges. Traditional ship monitoring methods mainly rely on manual real-time observation of video images in the monitoring room to determine whether the ship has entered a dangerous area. However, this method has many shortcomings:

[0003] First of all, manual monitoring requires long periods of concentration, which is a great test of the monitoring personnel's vision and energy. Staring at the screen for a long time can easily lead to visual fatigue and lack of concentration, thus affecting the monitoring effect and increasing the risk of safety accidents on ships.

[0004] Secondly, traditional monitoring methods often lack intelligent means to automatically identify and warn of potential dangers. When there are a large number of ships and their navigation trajectories are complex, manual monitoring is difficult to be comprehensive, timely and accurate. In addition, in conditions of poor visibility such as bad weather or at night, the effectiveness of manual monitoring is greatly reduced.

[0005] To solve these problems, another traditional method is to use the AIS system (Automatic Identification System) for ship monitoring. The AIS system can obtain the ship's position, heading, speed and other information in real time through communication between the ship and the shore, providing strong support for ship safety management. However, the AIS system also has some limitations:

[0006] On the one hand, the construction and maintenance costs of the AIS system are high. On the other hand, not all ships are equipped with AIS equipment, especially for some small ships, fishing boats, etc., due to cost, technology and other reasons, the popularity of AIS equipment is not high. This leads to the monitoring effect of the AIS system being unsatisfactory in some areas or time periods. Therefore, it is necessary to develop a new image-based active ship collision avoidance system and method.

[0007] Therefore, it is necessary to develop a new image-based active ship collision avoidance system and method. Summary of the invention

[0008] The purpose of the present invention is to provide an image-based active ship collision avoidance system and method, which can monitor the dynamics of ships in the river in real time, and determine whether the ship enters a preset danger zone through target recognition. Once the ship enters the preset danger zone, the system can automatically alarm, and the cost is relatively low.

[0009] In a first aspect, an image-based active ship collision avoidance system according to the present invention comprises:

[0010] A camera module for acquiring video streams in the river;

[0011] A ship recognition module, used for performing real-time target detection on each frame of the video stream to identify the ship and determine its position and category;

[0012] A perspective transformation module converts the video image into a bird's-eye view using perspective transformation, and calculates the distance between the ship and the preset danger zone on the bird's-eye view;

[0013] The user interaction module is used to draw the preset dangerous areas of the river and display the ship detection results and the distance information between the ship and the preset dangerous areas in real time;

[0014] The alarm module is used to automatically trigger an alarm when it is detected that the distance between the ship and the preset dangerous area is less than or equal to a preset safety distance threshold.

[0015] Optionally, the ship recognition module uses an improved YOLOv7x model to perform real-time target detection on each frame of the video stream to identify the ship and determine its position and category, wherein the improved YOLOv7x model is a YOLOv7 network structure with a smaller target feature fusion output module added, that is, the improved YOLOv7x model has a large target feature fusion output module, a medium target feature fusion output module, a small target feature fusion output module and a smaller target feature fusion output module. By adding a significantly smaller target feature fusion output module, the recognition performance of a small target ship at a long distance is improved.

[0016] Optionally, the large target feature fusion output module is a target detection head with a 20×20×1280 resolution feature map, which is used to detect larger-sized ship targets;

[0017] The medium-sized target feature fusion output module is a target detection head with a 40×40×640 resolution feature map, which is used to detect medium-sized ship targets;

[0018] The small target feature fusion output module is a target detection head with a 80×80×320 resolution feature map, which is used to detect smaller ship targets;

[0019] The smaller target feature fusion output module is a target detection head with a 160×160×160 resolution feature map, which is used to detect extremely small ship targets or distant ships. By designing target detection heads with different resolution feature maps (large target, medium target, small target, and smaller target feature fusion output modules), the system can accurately detect ship targets of various sizes. This multi-scale detection capability helps to effectively identify and track large ships at close range or small ships at long distances in complex river environments.

[0020] Optionally, the ship identification module includes:

[0021] Build a model unit to build a YOLOv7 network structure, and add a smaller target feature fusion output module on the basis of the YOLOv7 network structure to obtain an improved YOLOv7x model;

[0022] The training data set acquisition unit acquires a video stream using a camera module, splits each frame of the video stream, and screens out images containing ships, draws a bounding box for each ship in each image, and marks the category of the ship to obtain a first image set; and acquires a public data set, the public data set containing images of ships of various types and in various scenes, adds the public data set to the first image set to obtain a second image set; performs uniform size processing on the images in the second image set, and selects some images for random rotation, cropping, flipping and color change to form a training data set;

[0023] The model training unit trains the improved YOLOv7x model using the training data set to obtain a trained improved YOLOv7x model.

[0024] Optionally, the alarm module is arranged within the area monitored by the camera module, and is used to issue a warning through broadcasting when the system automatically triggers an alarm. The alarm will not stop until the distance between the ship and the preset danger zone is greater than a preset safety distance threshold.

[0025] In a second aspect, an image-based active ship collision avoidance method according to the present invention comprises the following steps:

[0026] Get the video stream in the river;

[0027] Performing real-time target detection on each frame of the video stream to identify the ship and determine its position and category;

[0028] Draw preset dangerous areas of the river on the video and display ship detection results in real time;

[0029] The video image is converted into a bird's-eye view using perspective transformation, and the distance between the ship and the preset danger zone is calculated on the bird's-eye view;

[0030] When the distance between the ship and the preset danger zone is less than or equal to the preset safety distance threshold, an alarm is automatically triggered.

[0031] Optionally, an improved YOLOv7x model is used to perform real-time target detection on each frame image in the video stream;

[0032] The improved YOLOv7x model adds a smaller target feature fusion output module on the basis of the YOLOv7 network structure, that is, the improved YOLOv7x model has a large target feature fusion output module, a medium target feature fusion output module, a small target feature fusion output module and a smaller target feature fusion output module.

[0033] Optionally, the large target feature fusion output module is a target detection head with a 20×20×1280 resolution feature map, which is used to detect larger-sized ship targets;

[0034] The medium-sized target feature fusion output module is a target detection head with a 40×40×640 resolution feature map, which is used to detect medium-sized ship targets;

[0035] The small target feature fusion output module is a target detection head with a 80×80×320 resolution feature map, which is used to detect smaller ship targets;

[0036] The smaller target feature fusion output module is a target detection head with a 160×160×160 resolution feature map, which is used to detect extremely small ship targets or long-distance ships.

[0037] Optionally, before performing target detection on each frame image in the video stream using the improved YOLOv7x model, the following steps are further performed:

[0038] Get the training data set, specifically:

[0039] A video stream is acquired by using a camera module, and each frame of the video stream is split, and images containing ships are screened out, a bounding box is drawn for each ship in each image, and the category of the ship is marked to obtain a first image set;

[0040] Obtain a public data set, wherein the public data set includes ship images of various types and in various scenes;

[0041] Adding the public dataset to the first image set to obtain a second image set;

[0042] The images in the second image set are processed to a uniform size, and some of the images are randomly rotated, cropped, flipped, and color-changed to form a training data set;

[0043] The improved YOLOv7x model is trained using the training data set to obtain a trained improved YOLOv7x model. The first image set is obtained by using the camera module to obtain the video stream and process it, which can provide accurate basic data for the training of subsequent models. By integrating the existing public data set, the public data set contains ship images of various types and scenes, thereby increasing the diversity of model training, helping the model to learn more ship features in different environments during the training process, thereby improving the adaptability of the model. The images in the second image set are uniformly resized to ensure the consistency of the image size input to the improved YOLOv7x model, avoiding the problem of training instability caused by inconsistent image sizes, and improving the efficiency and stability of model training.

[0044] Optionally, when the distance between the ship and the preset dangerous area is less than or equal to the preset safety distance threshold, an alarm is broadcast in the area monitored by the camera module, and the alarm will not stop until the distance between the ship and the preset dangerous area is greater than the preset safety distance threshold. The system can monitor the distance between the ship and the preset dangerous area in real time, and immediately trigger an alarm when the distance reaches or is less than the preset safety distance threshold. This immediate warning mechanism can quickly attract the attention of operators or relevant personnel, enabling them to take countermeasures in time to avoid potential dangers or collision accidents. By broadcasting an alarm in the area monitored by the camera module, it can be ensured that when the ship approaches the dangerous area, the surrounding personnel and ships can obtain warning information in time. This wide alarm coverage helps to improve the safety and reliability of the entire monitoring area. The automated alarm system can reduce human judgment errors and reaction time, especially in emergency situations. Operators can focus more on taking countermeasures without being distracted by judging whether an alarm is needed or how to alarm. The alarm system will not only be triggered when the ship approaches the preset dangerous area, but also stop after the ship leaves the preset dangerous area. This continuous monitoring and feedback mechanism helps operators understand the dynamic situation of the ship and take corresponding measures to ensure navigation safety.

[0045] The present invention has the following advantages:

[0046] 1. Real-time monitoring and efficient identification: The present invention uses the camera module to obtain the video stream in the river, and the ship identification module can perform target detection on each frame of the image in real time, quickly and accurately identifying the ships in the river. This provides continuous and real-time monitoring of ships, which helps to detect potential dangerous situations in a timely manner.

[0047] 2. Multi-dimensional information acquisition: The ship identification module can not only identify the ship, but also determine its location and category. This information is crucial for assessing the ship's navigation status, predicting its possible behavior, and judging potential collision risks.

[0048] 3. Intuitive understanding of the ship's position: The perspective transformation module converts the video image into a bird's-eye view, which helps to more intuitively understand the relative position of the ship in the river. The distance between the ship and the preset danger zone is calculated on the bird's-eye view, providing accurate data support for judging the risk of collision.

[0049] 4. Flexible user interaction: The user interaction module allows the drawing of preset dangerous areas of the river and displays the ship detection results and the distance information between the ship and the preset dangerous areas in real time. This interactivity makes the system more flexible and can meet the needs of different river channels and navigation conditions.

[0050] 5. Automated alarm function: The alarm module automatically triggers an alarm when it detects that the distance between the ship and the preset danger zone is less than or equal to the preset safety distance threshold. This automated function can greatly reduce human errors and reaction time, and improve the efficiency and reliability of the anti-collision system.

[0051] 6. Significantly improve navigation safety: Through real-time monitoring, accurate identification, distance calculation and automatic alarm functions, the present invention significantly improves the navigation safety of ships in rivers. It helps prevent collision accidents and protect the safety of people and property.

[0052] 7. Wide adaptability: The present invention can adapt to different river environments and navigation conditions. By adjusting parameters such as the preset danger zone and the preset safety distance threshold, the system can flexibly adapt to different application scenarios and needs.

[0053] 8. Low cost and high efficiency: Compared with the traditional ship monitoring system, the present invention adopts intelligent technology based on camera module and target recognition algorithm, which does not require expensive AIS equipment and reduces the construction and maintenance costs of the system. At the same time, through automation and intelligent means, it improves monitoring efficiency, reduces labor costs, and achieves high-efficiency safety management.

[0054] In summary, the present invention can monitor the dynamics of ships in the river in real time, and determine whether the ship has entered the preset danger zone through target recognition. Once the ship enters the preset danger zone, the system can automatically alarm and issue a warning through broadcasting. This intelligent monitoring system has the advantages of wide coverage, fast real-time response, low false alarm rate, and low cost. It does not require human intervention and can work around the clock, reducing labor costs and improving management efficiency. Through the application of this system, the early warning capability of ships entering the preset danger zone has been significantly improved, effectively ensuring the safety of waterway transportation. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] Figure 1 This is a principle block diagram of the image-based active ship collision avoidance system described in the embodiment of the present application;

[0056] Figure 2 This is a usage status diagram of the image-based active ship collision avoidance system described in the embodiment of the present application;

[0057] Figure 3 This is a schematic diagram of the network structure of the improved YOLOv7x model described in the embodiments of the present application;

[0058] Figure 4 This is a principle block diagram of the ship identification module described in the embodiment of the present application;

[0059] Figure 5 This is a flow chart of the image-based active ship collision avoidance method described in an embodiment of the present application;

[0060] Figure 6 This is a schematic diagram of shooting of the camera module described in the embodiment of the present application;

[0061] Figure 7 A detailed flowchart of the image-based active ship collision avoidance method described in an embodiment of the present application;

[0062] Figure 8 A schematic diagram of the software interface in the embodiment of the present application;

[0063] Fig. 9 A top view of a river channel in an embodiment of the present application;

[0064] Fig.10 This is an oblique view of a river channel in an embodiment of the present application;

[0065] Description of reference numerals:

[0066] 1. Camera module, 2. Ship identification module, 3. Perspective transformation module, 4. User interaction module, 5. Alarm module, 6. Preset danger zone, 7. Target box. DETAILED DESCRIPTION

[0067] The following will describe the embodiments of the present invention with reference to the accompanying drawings and preferred embodiments. Those skilled in the art can understand other advantages and effects of the present invention from the contents disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present invention. It should be understood that the preferred embodiments are only for illustrating the present invention, not for limiting the scope of protection of the present invention.

[0068] It should be noted that the illustrations provided in the following embodiments are only schematic illustrations of the basic concept of the present invention. The illustrations only show components related to the present invention rather than being drawn according to the number, shape and size of components in actual implementation. In actual implementation, the type, quantity and proportion of each component may be changed arbitrarily, and the component layout may also be more complicated.

[0069] like Figure 1 and Figure 2 As shown, in an embodiment of the present application, an image-based active ship collision avoidance system includes a camera module 1, a ship identification module 2, a perspective transformation module 3, a user interaction module 4 and an alarm module 5. The camera module 1 is used to obtain a video stream in a river channel. The ship identification module 2 is used to perform real-time target detection on each frame of the video stream to identify the ship and determine its position and category. The perspective transformation module 3 uses perspective transformation to convert the video image into a bird's-eye view, and calculates the distance between the ship and the preset danger zone 6 on the bird's-eye view. The user interaction module 4 is used to draw the preset danger zone 6 of the river channel, and display the ship detection results and the distance information between the ship and the preset danger zone 6 in real time. The alarm module 5 is used to automatically trigger an alarm when it is detected that the distance between the ship and the preset danger zone 6 is less than or equal to the preset safety distance threshold.

[0070] The system obtains the video stream in the river channel through the camera module 1, so that the ship identification module 2 can perform target detection on each frame of the image in real time, and quickly and accurately identify the ship in the river channel. This provides continuous and real-time monitoring of ships, which helps to detect potential dangerous situations in a timely manner. The ship identification module 2 can not only identify the ship, but also determine its location and category. This information is crucial for evaluating the navigation status of the ship, predicting its possible behavior, and judging the potential collision risk. The perspective transformation module 3 converts the video image into a bird's-eye view, which helps to more intuitively understand the relative position of the ship in the river channel. The distance between the ship and the preset danger area 6 is calculated on the bird's-eye view, providing accurate data support for judging the collision risk. The user interaction module 4 allows the preset danger area 6 of the river channel to be drawn, and the ship detection results and the distance information between the ship and the preset danger area 6 are displayed in real time. This interactivity makes the system more flexible and can meet the needs of different river channels and navigation conditions. When the alarm module 5 detects that the distance between the ship and the preset danger area 6 is less than or equal to the preset safety distance threshold, it automatically triggers an alarm. This automated function can greatly reduce human errors and reaction time, and improve the efficiency and reliability of the collision avoidance system. This system significantly improves the navigation safety of ships in rivers through functions such as real-time monitoring, precise identification, distance calculation and automatic alarm. It helps prevent collision accidents and protect the safety of people and property. This system can adapt to different river environments and navigation conditions. By adjusting parameters such as the preset danger zone 6 and the preset safety distance threshold, the system can flexibly adapt to different application scenarios and needs.

[0071] like Figure 3 As shown, in a possible embodiment, the ship identification module 2 uses an improved YOLOv7x model to perform real-time target detection on each frame of the video stream to identify the ship and determine its position and category, wherein the improved YOLOv7x model is to add a smaller target feature fusion output module on the basis of the YOLOv7 network structure. This system uses the YOLOv7x network structure as a benchmark algorithm for ship identification. Compared with the basic algorithm, the improved YOLOv7x model is optimized and expanded in network structure, and has a deeper and more complex network architecture. These improvements enable the improved YOLOv7x model to detect targets more accurately under complex backgrounds and diverse environmental conditions, especially for ships. The improved YOLOv7x model uses a convolutional neural network (CNN) to extract features from images and perform real-time target detection. It can quickly and efficiently process each frame of the video stream, identify the ship therein and determine its position and category. Since the ship is a small target at a long distance, the YOLOv7 network structure is improved to improve the recognition accuracy of the model for ships at a long distance.

[0072] like Figure 3As shown in the figure, the improved YOLOv7x model has a large target feature fusion output module, a medium target feature fusion output module, a small target feature fusion output module and a smaller target feature fusion output module. The large target feature fusion output module is a target detection head with a 20×20×1280 resolution feature map, which is used to detect larger ship targets. The medium target feature fusion output module is a target detection head with a 40×40×640 resolution feature map, which is used to detect medium-sized ship targets. The small target feature fusion output module is a target detection head with an 80×80×320 resolution feature map, which is used to detect smaller ship targets. The smaller target feature fusion output module is a target detection head with a 160×160×160 resolution feature map, which is used to detect extremely small ship targets or ships at a long distance. Although the network depth of the improved YOLOv7x model has increased, the recognition performance of small target ships at a long distance has been significantly improved.

[0073] like Figure 4 As shown, in a possible embodiment, the ship identification module 2 includes a model building unit 21, a training data set acquisition unit 22 and a model training unit 23. Among them, the model building unit 21 is used to build a YOLOv7 network structure, and add a smaller target feature fusion output module on the basis of the YOLOv7 network structure to obtain an improved YOLOv7x model. The training data set acquisition unit 22 uses the camera module 1 to obtain the video stream, splits each frame of the video stream, and filters out images containing ships, draws a bounding box for each ship in each image, and marks the category of the ship to obtain a first image set; and obtains a public data set, which contains ship images of various types and scenes, and adds the public data set to the first image set to obtain a second image set; the images in the second image set are uniformly sized, and some images are selected for random rotation, cropping, flipping and color change to form a training data set. The model training unit 23 uses the training data set to train the improved YOLOv7x model to obtain a trained improved YOLOv7x model.

[0074] In a possible embodiment, the alarm module 5 is arranged within the area monitored by the camera module, and is used to issue a warning by broadcasting when the system automatically triggers an alarm, and the alarm will not stop until the distance between the ship and the preset dangerous area 6 is greater than the preset safety distance threshold. The system can monitor the distance between the ship and the preset dangerous area 6 in real time, and immediately trigger the alarm when the distance reaches or is less than the preset safety distance threshold. This immediate warning mechanism can quickly attract the attention of operators or relevant personnel, so that they can take countermeasures in time to avoid potential dangers or collision accidents. By broadcasting the alarm in the area monitored by the camera module, it can be ensured that when the ship approaches the dangerous area, the surrounding personnel and ships can obtain warning information in time. This wide alarm coverage helps to improve the safety and reliability of the entire monitoring area. The automated alarm system can reduce human judgment errors and reaction time, especially in emergency situations. Operators can focus more on taking countermeasures without having to be distracted to judge whether an alarm is needed or how to alarm. The alarm system will not only be triggered when the ship approaches the preset dangerous area 6, but also stop after the ship leaves the preset dangerous area 6. This continuous monitoring and feedback mechanism helps operators understand the dynamic situation of the ship and take corresponding measures to ensure navigation safety.

[0075] like Figure 5 As shown, in an embodiment of the present application, an image-based active ship collision avoidance method includes the following steps:

[0076] Get the video stream in the river;

[0077] Perform real-time object detection on each frame of the video stream to identify the vessel and determine its location and category;

[0078] Draw the preset danger zone of the river on the video6 and display the ship detection results in real time;

[0079] The video image is converted into a bird's-eye view by using perspective transformation, and the distance between the ship and the preset danger zone 6 is calculated on the bird's-eye view;

[0080] When the distance between the ship and the preset danger zone 6 is less than or equal to the preset safety distance threshold, an alarm is automatically triggered.

[0081] In a possible embodiment, an improved YOLOv7x model is used to perform real-time target detection on each frame image in a video stream.

[0082] In a possible embodiment, before using the improved YOLOv7x model to perform target detection on each frame image in the video stream, the following steps are further performed:

[0083] Obtain a training data set, and use the training data set to train the improved YOLOv7x model to obtain a trained improved YOLOv7x model.

[0084] In the embodiment of the present application, the method for obtaining the training data set is as follows:

[0085] First, in order to train an efficient and accurate improved YOLOv7x model, the camera module is used to obtain the video stream, and each frame of the video stream is split, and the images containing ships are filtered out. A bounding box is drawn for each ship in each image, and the category of the ship is marked to obtain the first image set. In this way, a high-quality first image set is generated.

[0086] Next, in order to improve the recognition diversity and robustness of the model, existing public datasets were also incorporated, specifically:

[0087] Obtain a public data set, which contains images of ships of various types and in various scenes, which can increase the diversity of model training and make it more adaptable under different environmental conditions. At the same time, select images similar to the monitored river scenes, add the public data set to the first image set, and obtain the second image set, that is, expand and mix the data set. In this way, the model can not only learn the characteristics of different types of ships during training, but also adapt to ship recognition under complex conditions such as different lighting, weather, and viewing angles.

[0088] Then, the images in the second image set are resized to ensure that the images input to the improved YOLOv7x model can maintain consistency and avoid training instability due to inconsistent image sizes.

[0089] Finally, some images are selected for random rotation, cropping, flipping and color change to form a training data set, which can further improve the generalization ability of the improved YOLOv7x model.

[0090] Through such a rich and diverse training data set, we can ensure that the trained improved YOLOv7x model not only performs well in ideal environments, but also performs well in actual complex scenarios, with strong robustness and accuracy.

[0091] In a possible embodiment, the camera module can only be installed downward and at a certain angle to the vertical direction to capture a wider and broader river channel. Since the principle of the camera module to shoot video is optical imaging. When the object is closer to the lens, the viewing angle is larger, and the object will occupy more pixels in the picture, so the visual effect is greater; conversely, the visual effect is smaller, such as Figure 6 This results in inconsistent visual effects for the same object or distance at different locations, making it more difficult to judge the distance relationship of objects.

[0092] In order to accurately determine the distance between the ship and the preset danger zone 6, the video image is first converted into a top view using perspective transformation, and the position of the ship is determined on the image after the perspective transformation. Perspective transformation uses four reference points in the video (for example, any four of the 1# to 7# points, see Fig. 9 and Fig.10 The main function of the reference point is to obtain the conversion relationship between the image position coordinate system and the world position coordinate system. Generally, a position with obvious features and not easily changed is selected, such as a certain intersection point between the river and the shore, a protruding stone point, etc. Assume that four reference points are selected on the oblique view, and the four reference points on the oblique view are converted to the top view, that is, the corresponding relationship between them is found) to calculate the transformation parameters, such as formula (1) and formula (2), and then use formula (3) to convert the captured video into a top view. Among them, X, Y, Z represent the three-dimensional coordinates of the world position coordinate system, but because the world coordinate system is a three-dimensional coordinate, it is necessary to use the Z value to convert it into a two-dimensional coordinate, X', Y' represent the two-dimensional coordinates of the world position coordinate system, x, y represent the two-dimensional coordinates of the image position coordinate system, and a1, a2, a3, b1, b2, b3, c1, c2, c3 represent the transformation parameters respectively.

[0093]

[0094] In a possible embodiment, when the distance between the ship and the preset danger zone 6 is less than or equal to the preset safety distance threshold, an alarm is broadcast in the area monitored by the camera module 1 until the distance between the ship and the preset danger zone 6 is greater than the preset safety distance threshold, at which time the alarm stops.

[0095] like Figure 7 As shown, the image-based active ship collision avoidance system uses a camera module 1 to obtain a video stream in the river, sends it to a ship identification module 2 to detect ships, and displays the video stream on the interface of a user interaction module 4. If there is a ship in the video, a target frame 7 is drawn for the ship through the ship identification module 2, and the video stream continues to be monitored; if there is no ship, the video stream continues to be monitored. The user may draw a preset danger zone 6 on the video through the user interaction module 4, and use the perspective transformation module 3 to convert the video image into a top view, and calculate the distance between the ship and the preset danger zone 6 in real time (that is, the distance between the target frame 7 corresponding to the ship and the preset danger zone 6). Once the ship is found to be close to the preset danger zone 6 (that is, the distance between the ship and the preset danger zone 6 is less than or equal to the preset safety distance threshold), an alarm is issued through the alarm module 5.

[0096] The software interface and effect display involved in this system can be found in Figure 8. The user can read the camera video stream and display it by clicking on the camera selection, and can draw the preset danger zone 6 in the video by clicking on the draw danger zone. The user can manually adjust the confidence and other information, change the recognition accuracy, click on the Model box to drop down and select different models, and then click on play to start detecting ships. When ships are detected in the video, the number of ships is output in the result statistics, and once a ship is detected approaching the preset danger zone 6, the software will continue to issue an alarm until all ships are away from the preset danger zone, and the alarm will stop.

[0097] The above-mentioned embodiments are preferred implementation modes of the present invention, but the implementation modes of the present invention are not limited to the above-mentioned embodiments. Any other changes, modifications, substitutions, combinations and simplifications made without departing from the spirit and principle of the present invention shall be equivalent replacement modes and shall be included in the protection scope of the present invention.

Claims

1. An image-based active ship collision avoidance system, comprising the following steps: A camera module (1) is used to obtain a video stream in a river channel; A ship identification module (2) is used to perform real-time target detection on each frame of the video stream to identify the ship and determine its position and category; A perspective transformation module (3) converts the video image into a bird's-eye view by using perspective transformation, and calculates the distance between the ship and the preset danger zone (6) on the bird's-eye view; A user interaction module (4) is used to draw a preset danger zone (6) of the river and display the ship detection result and the distance information between the ship and the preset danger zone (6) in real time; The alarm module (5) is used to automatically trigger an alarm when it is detected that the distance between the ship and the preset dangerous area (6) is less than or equal to a preset safety distance threshold.

2. The image-based active ship collision avoidance system according to claim 1, characterized in that: The ship recognition module (2) uses an improved YOLOv7x model to perform real-time target detection on each frame of the video stream to identify the ship and determine its position and category, wherein the improved YOLOv7x model is a YOLOv7 network structure with a smaller target feature fusion output module added, that is, the improved YOLOv7x model has a large target feature fusion output module, a medium target feature fusion output module, a small target feature fusion output module and an even smaller target feature fusion output module.

3. The image-based active ship collision avoidance system according to claim 2, characterized in that: The large target feature fusion output module is a target detection head with a 20×20×1280 resolution feature map, which is used to detect larger ship targets; The medium-sized target feature fusion output module is a target detection head with a 40×40×640 resolution feature map, which is used to detect medium-sized ship targets; The small target feature fusion output module is a target detection head with a 80×80×320 resolution feature map, which is used to detect smaller ship targets; The smaller target feature fusion output module is a target detection head with a 160×160×160 resolution feature map, which is used to detect extremely small ship targets or long-distance ships.

4. The image-based active ship collision avoidance system according to claim 2, characterized in that: The ship identification module (2) comprises: Build a model unit to build a YOLOv7 network structure, and add a smaller target feature fusion output module on the basis of the YOLOv7 network structure to obtain an improved YOLOv7x model; The training data set acquisition unit acquires a video stream using a camera module, splits each frame of the video stream, and screens out images containing ships, draws a bounding box for each ship in each image, and marks the category of the ship to obtain a first image set; and acquires a public data set, the public data set containing images of ships of various types and in various scenes, adds the public data set to the first image set to obtain a second image set; performs uniform size processing on the images in the second image set, and selects some images for random rotation, cropping, flipping and color change to form a training data set; The model training unit trains the improved YOLOv7x model using the training data set to obtain a trained improved YOLOv7x model.

5. The image-based active ship collision avoidance system according to claim 1, characterized in that: The alarm module (5) is arranged within the area monitored by the camera module (1) and is used to issue a warning via broadcasting when the system automatically triggers an alarm. The alarm will not stop until the distance between the ship and the preset dangerous area (6) is greater than a preset safety distance threshold.

6. An image-based active ship collision avoidance method, characterized in that: The following steps are involved: Get the video stream in the river; Performing real-time target detection on each frame of the video stream to identify the ship and determine its position and category; Draw the preset danger zone of the river on the video (6) and display the ship detection results in real time; The video image is converted into a bird's-eye view by using perspective transformation, and the distance between the ship and the preset danger zone (6) is calculated on the bird's-eye view; When the distance between the ship and the preset danger zone (6) is less than or equal to a preset safety distance threshold, an alarm is automatically triggered.

7. The image-based active ship collision avoidance method according to claim 6, characterized in that: Using an improved YOLOv7x model to perform real-time target detection on each frame of the video stream; The improved YOLOv7x model adds a smaller target feature fusion output module on the basis of the YOLOv7 network structure, that is, the improved YOLOv7x model has a large target feature fusion output module, a medium target feature fusion output module, a small target feature fusion output module and a smaller target feature fusion output module.

8. The image-based active ship collision avoidance method according to claim 7, characterized in that: The large target feature fusion output module is a target detection head with a 20×20×1280 resolution feature map, which is used to detect larger ship targets; The medium-sized target feature fusion output module is a target detection head with a 40×40×640 resolution feature map, which is used to detect medium-sized ship targets; The small target feature fusion output module is a target detection head with a 80×80×320 resolution feature map, which is used to detect smaller ship targets; The smaller target feature fusion output module is a target detection head with a 160×160×160 resolution feature map, which is used to detect extremely small ship targets or long-distance ships.

9. The image-based active ship collision avoidance method according to claim 7, characterized in that: Before using the improved YOLOv7x model to perform target detection on each frame image in the video stream, the following steps are also performed: Get the training data set, specifically: A video stream is acquired by using a camera module, and each frame of the video stream is split, and images containing ships are screened out, a bounding box is drawn for each ship in each image, and the category of the ship is marked to obtain a first image set; Obtain a public data set, wherein the public data set includes ship images of various types and in various scenes; Adding the public dataset to the first image set to obtain a second image set; The images in the second image set are processed to a uniform size, and some of the images are randomly rotated, cropped, flipped, and color-changed to form a training data set; The improved YOLOv7x model is trained using the training data set to obtain a trained improved YOLOv7x model.

10. The image-based active ship collision avoidance method according to claim 6, characterized in that: When the distance between the ship and the preset dangerous area (6) is less than or equal to the preset safety distance threshold, an alarm is broadcast in the area monitored by the camera module (1), and the alarm will not stop until the distance between the ship and the preset dangerous area (6) is greater than the preset safety distance threshold.

Images