Ship marine target sensing method and system based on machine vision

Abstract

The invention relates to the technical field of ship maritime orientation perception, and discloses a ship maritime target perception method and system based on machine vision, and the method comprises the steps: carrying out the processing of transmitted image information through employing an AD-YOLOv9 algorithm, and obtaining the position information and category information of a target image; establishing a coordinate system according to the position information of the target image, marking a target on the target image by using a target prediction frame, determining the center point coordinate of the target prediction frame based on a deep learning network so as to determine the center point coordinate of the target, associating the ship head direction of the target, and determining the absolute orientation and the relative orientation of the target; determining the position of the target relative to the reference point of the ship, and determining the direction angle of the target; determining a straight line where a target center point and a reference point of the ship are located according to the direction angle of the target, and recording as a target direction line; calculating and formatting an angle label of the target according to the target direction line, and selecting a position on the target direction line to draw angle label information; and displaying the processed target image.

Description

Technical Field

[0001] The present invention relates to the technical field of ship maritime position perception, and in particular to a ship maritime target perception method and system based on machine vision. Background Art

[0002] In traditional navigation practice, crew members mainly rely on traditional navigation tools such as compass and radar to determine the heading of the ship, and use visual observation or analysis of radar images to identify the relative positions of surrounding ships, buoys and other targets. Although this method has been proven to be effective in actual operations, in port waters, due to the high density of ship traffic, narrow spacing and strong maneuverability, radar has encountered significant challenges in stably tracking targets, thus affecting the accuracy of target positioning data.

[0003] According to the radar equipment performance standard MSC.192 (79) issued by the International Maritime Organization (IMO), in calm seas and with a range not exceeding 1.5 nautical miles, the radar should have a range resolution of at least 40 meters between two targets and a azimuth resolution of 2.5 degrees within the range of 50% to 100%. However, when the distance between targets is less than 40 meters or the azimuth difference is less than 2.5 degrees, the radar may not be able to effectively distinguish these targets and it is difficult to maintain stable tracking. This situation is particularly prominent in port waters with dense ships. At the same time, the performance of the radar is also limited by adverse weather environmental factors such as weather and visibility, which also places high demands on the experience and skills of the crew.

[0004] Given the limitations of radar in close-range target detection, target orientation perception technology based on computer vision has gradually demonstrated its unique advantages.

[0005] In the task of sensing and identifying targets at sea, computer vision-based target orientation perception technology uses cameras installed on ships to monitor the environmental image information around the ship in real time. By using advanced image processing algorithms and machine learning techniques, the system can automatically identify and track other key targets around it, such as ships and buoys.

[0006] The method for judging heading, direction and side angle based on computer vision is as follows: Figure 1 As shown in the figure, A and B respectively indicate two different target objects. T Represents the true north direction, that is, the geographical North Pole direction. The target direction is measured based on the true north line, which is called the true direction angle. For example, TB A TB B, which is independent of the actual sailing direction of the ship, i.e. the true heading TC. On the other hand, the target side angle Q is measured based on the course line (CL). The positive or negative value of the side angle Q indicates whether the marine target (such as buoys, reefs, other ships, etc.) is located on the starboard or port side of the ship.

[0007] The mathematical relationship between heading, bearing and side angle can be accurately described by the following formula: ; Where TB is the true bearing of the target, TC is the true heading, and Q is the relative bearing of the target relative to the heading of the ship.

[0008] After obtaining the ship's real-time heading data (i.e., true heading TC), the system can accurately associate the target orientation captured in the image with the ship's heading, thereby automatically calculating the true orientation (TB) and relative orientation (Q) of these targets and presenting them to the ship operator in an intuitive and clear manner. This method improves the intelligence level of target orientation judgment and has high practical value and application potential.

[0009] However, the current ship maritime target perception and recognition technology using the YOLOv9 benchmark model still has the defects of many missed ship detections, low recognition accuracy, large amount of calculation, and poor real-time performance. Its detection performance in complex marine environments needs to be improved.

[0010] Based on this, the applicant proposed the present invention. Summary of the invention

[0011] In order to solve the problems of many missed ship detections and low recognition accuracy in the prior art, or the problems of large computational complexity and poor real-time performance in the prior art, the present invention provides a method for ship maritime target perception based on machine vision, which adopts the AD-YOLOv9 maritime target recognition algorithm (referred to as AD-YOLOv9 algorithm for short), can simultaneously complete target positioning and classification in one forward propagation, and has a lighter model and higher target classification and recognition accuracy.

[0012] The technical solution of the present invention is as follows: A method for sensing a target at sea based on machine vision comprises the following steps: Step S1: Use the AD-YOLOv9 algorithm to process the incoming image information to obtain the location information and category information of the target image; Step S2: establishing a coordinate system according to the position information of the target image, marking the target on the target image with a target prediction frame, determining the center point coordinates of the target prediction frame based on a deep learning network, thereby determining the center point coordinates of the target, and associating the center point coordinates of the target with the bow direction to determine the absolute orientation and relative orientation of the target; Step S3: determining the position of the target relative to a reference point of the ship according to the absolute position and relative position of the target to determine the direction angle of the target; Step S4: determining a straight line where the center point of the target and the reference point of the ship are located according to the direction angle of the target, and recording it as a target direction line; Step S5: Calculate and format the angle label of the target according to the target direction line, and select a position on the target direction line to draw the angle label information; Step S6: Display the processed target image.

[0013] Furthermore, step S1 includes: using the AD-YOLOv9 algorithm to identify ships, buoy colors, bridges, and island obstacles.

[0014] Optionally, the method further includes step S0: monitoring the environmental information around the ship in real time through a camera on the ship, and the obtained video or image information is imported into the AD-YOLOv9 algorithm model.

[0015] Furthermore, in step S2, the center point coordinates of the target prediction box are Determined according to the following formula: ; in, Indicates the horizontal coordinate value of the upper left corner of the target prediction box. Indicates the horizontal coordinate value of the lower right corner of the target prediction box; ; in, Indicates the upper left corner ordinate value of the target prediction box. Indicates the vertical coordinate value of the lower right corner of the target prediction box.

[0016] Furthermore, based on the bow direction, the orientation of other targets at sea is determined according to the coordinates of the center point of the target prediction frame: ; ; in, Represented as the horizontal coordinate value of the center point of the target prediction box, Represented as the horizontal coordinate value of the center point of the target prediction box, h is the length of the screen, ω is the width of the screen, Q Indicates the direction of other targets.

[0017] Furthermore, in step S6, the target identified in the target image and its corresponding position information are dynamically displayed in a graphical form on the terminal display screen, so that the operator can directly observe the position of the target and its corresponding position information.

[0018] The present invention also provides a machine vision-based ship marine target perception system, comprising: Information extraction module: used to process the incoming image information using the AD-YOLOv9 algorithm to obtain the location information and category information of the target image; Image processing module: used to establish a coordinate system according to the position information of the target image, mark the target on the target image with a target prediction frame, determine the center point coordinates of the target prediction frame based on a deep learning network, thereby determining the center point coordinates of the target, and associate the center point coordinates of the target with the bow direction to determine the absolute orientation and relative orientation of the target; Target direction angle determination module: used to determine the position of the target relative to the reference point of the ship according to the absolute orientation and relative orientation of the target, so as to determine the direction angle of the target; Direction line marking module: used to determine the straight line where the center point of the target and the reference point of the ship are located according to the direction angle of the target, and record it as the target direction line; Angle drawing module: used for calculating and formatting the angle label of the target according to the target direction line, and drawing the angle label information at a selected position on the target direction line; Image display module: used to display the processed target image.

[0019] Beneficial effects of the above technical solution: The method and / or system proposed in the present invention realizes the automatic identification of other targets at sea and the visual judgment of their approximate relative position. By integrating the real-time image recognition capability of the AD-YOLOv9 deep learning algorithm with the bow direction data, the system can dynamically display the identified targets and their corresponding position information in a graphical form on the terminal display screen. The method and / or system of the present invention enables operators to more intuitively observe the positions and relative positions of surrounding targets, thereby supporting them to make faster and more accurate decisions. During the visualization process, the system also uses a variety of colors and icons to distinguish different target types and their relative positions, so as to further enhance the visualization of information. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0021] Figure 1 A schematic diagram of a method for determining heading, orientation and side angle based on computer vision; Figure 2 A schematic diagram of a process of a method for sensing a target at sea by a ship based on machine vision according to the present invention; Figure 3 This is a schematic diagram of the structure of a machine vision-based ship marine target perception system of the present invention; Figure 4 This is the AD-YOLOv9 network framework diagram used in the present invention; Figure 5 This is a schematic diagram of the prediction of the marine target position adopted by the present invention; Figure 6 This is a schematic diagram of the effect of using the YOLOv9 benchmark model to identify ship targets in an actual navigation scenario; Figure 7 The figure is a schematic diagram of the ship target perception effect using the AD-YOLOv9 model of the present invention in an actual navigation scenario. DETAILED DESCRIPTION

[0022] The technical solutions in the embodiments of the present invention will be described clearly and completely below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments.

[0023] Examples of the embodiments are shown in the accompanying drawings, wherein the same or similar symbols throughout represent the same or similar elements or elements with the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present invention, and should not be construed as limiting the present invention.

[0024] like Figure 2 As shown, it is a flow chart of a method for perceiving a target at sea by a ship based on machine vision according to the present invention. The method for perceiving a target at sea by a ship based on machine vision comprises the following steps: Step S1: Use the AD-YOLOv9 algorithm to process the incoming image information to obtain the location information and category information of the target image; Step S2: establishing a coordinate system according to the position information of the target image, marking the target on the target image with a target prediction frame, determining the center point coordinates of the target prediction frame based on a deep learning network, thereby determining the center point coordinates of the target, and associating the center point coordinates of the target with the bow direction to determine the absolute orientation and relative orientation of the target; Step S3: determining the position of the target relative to a reference point of the ship according to the absolute position and relative position of the target to determine the direction angle of the target; Step S4: determining a straight line where the center point of the target and the reference point of the ship are located according to the direction angle of the target, and recording it as a target direction line; Step S5: Calculate and format the angle label of the target according to the target direction line, and select a position on the target direction line to draw the angle label information; Step S6: Display the processed target image.

[0025] Specifically, in step S1, the AD-YOLOv9 algorithm is used to identify ships, buoy colors, bridges, and island obstacles.

[0026] In order to obtain image information, the present invention also includes step S0: monitoring the environmental information around the ship in real time through the camera on the ship, and the obtained video or image information is imported into the AD-YOLOv9 algorithm model.

[0027] like Figure 3 As shown, the present invention provides a ship's heading-based intelligent perception system for the orientation of a target at sea, comprising: Information extraction module 10: used to process the target image using the AD-YOLOv9 algorithm to obtain the location information and category information of the target image; Image processing module 20: used to establish a coordinate system according to the position information of the target image, mark the target on the target image with a target prediction frame, determine the center point coordinates of the target prediction frame based on a deep learning network, thereby determining the center point coordinates of the target, and associate the center point coordinates of the target with the bow direction to determine the absolute orientation and relative orientation of the target; The target direction angle determination module 30 is used to determine the position of the target relative to the reference point of the ship according to the absolute position and relative position of the target, so as to determine the direction angle of the target; A direction line marking module 40 is used to determine the straight line where the center point of the target and the reference point of the ship are located according to the direction angle of the target, and record it as the target direction line; Angle drawing module 50: used for calculating and formatting the angle label of the target according to the target direction line, and drawing the angle label information at a selected position on the target direction line; Image display module 60: used to display the processed target image.

[0028] The present invention improves the YOLOv9 network by introducing a triple attention mechanism and replacing Conv with a lightweight DualConv convolution structure, and proposes an AD-YOLOv9 algorithm that integrates the attention mechanism and lightweight design. Figure 4 As shown in the figure, first, the triple attention mechanism is introduced in the head network of YOLOv9. This mechanism includes channel attention, spatial attention and cross attention, which respectively focus on the importance of features in different dimensions, effectively enhancing the model's ability to capture key information; especially in marine ship detection, due to the complex and changeable background and different target sizes, the triple attention mechanism can help the model more accurately focus on the learning of details such as color, texture, and shape, thereby enhancing the accuracy of target recognition and reducing false detections and missed detections.

[0029] Secondly, a strategy of replacing the standard convolution layer in YOLOv9 with a lightweight DualConv convolution is proposed. DualConv is an efficient convolution structure that significantly reduces the computational complexity and number of parameters while maintaining similar feature extraction capabilities by decomposing the standard convolution into two smaller convolution operations (usually a combination of 1×1 convolution and 3×3 convolution). This optimization strategy not only significantly reduces the computational burden of the model, but also improves the real-time performance, making it more practical in marine application scenarios with limited computing resources.

[0030] The AD-YOLOv9 algorithm constructed by the present invention can simultaneously complete the positioning and classification of targets in one forward propagation, and has a lighter model and higher target classification and recognition accuracy. Therefore, the algorithm is more suitable for the real-time recognition scenario of ship navigation targets at sea than the ordinary YOLOv9 algorithm. In the specific task of marine target recognition, AD-YOLOv9 can accurately identify key targets such as ships, buoy colors, bridges, and island obstacles, laying a solid foundation for subsequent target position judgment.

[0031] When implementing the present invention, it is proposed to first make the following settings for the method of determining the target orientation: (1) The ship navigation monitoring equipment is assumed to be sufficiently stable and free of screen jitter; (2) The coordinates of the bottom midpoint of the image taken by the motion camera (w / 2,h) are set as the starting point of the ship's heading line, and the error is negligible; (3) Since it is difficult to obtain the bow data of the ship at every moment of the voyage for the collected image data, it is assumed that the bow data remains unchanged within a short period of time.

[0032] In a specific embodiment of the present invention, the input image is processed in the AD-YOLOv9 algorithm network to obtain the location information and category label of the target.

[0033] like Figure 5 As shown in the figure, a coordinate system with the upper left corner of the input image as the origin is established, in which each target is surrounded by a prediction box. This box provides the coordinate information of the target in the image. The deep learning network identifies the coordinate information of the upper left and lower right corners of the target prediction box. The origin of the coordinates is the coordinate axis in the deep learning network. Among them, (x1, y1) is the coordinate of the upper left corner of the bounding box, and (x2, y2) is the coordinate of the lower right corner of the bounding box. These coordinates are used to locate the target on the image.

[0034] Through these coordinate information, the center point coordinates of the target can be obtained by calculating the diagonal midpoint of the bounding box or simply averaging the bounding box coordinates.

[0035] The inverse tangent function and angle conversion formula are used to calculate the position of the target relative to the ship's navigation monitoring recorder (i.e., the reference point).

[0036] According to the calculated direction angle, a straight line is drawn from the reference point to the center point of the target on the image. This line represents the direction of other targets encountered in the marine navigation scenario.

[0037] Calculates and formats angle labels to display the target's direction angle and category information.

[0038] Then, select a suitable location on the direction line to draw the angle label text so that the crew can clearly read the ship's direction angle information.

[0039] Finally, the processed image can be optionally displayed or saved. This setting allows the user to view the detection results of the deep learning network as well as the orientation information of the target.

[0040] In the present invention, the center point position coordinates of the prediction frame of other targets at sea The calculation method is as follows: ; in, Indicates the horizontal coordinate value of the upper left corner of the target prediction box. Indicates the horizontal coordinate value of the lower right corner of the target prediction box; ; in, Indicates the upper left corner ordinate value of the target prediction box. Indicates the vertical coordinate value of the lower right corner of the target prediction box.

[0041] Based on the bow direction of the ship, calculate the direction of other targets at sea. The calculation method is as follows: ; ; in, Represented as the horizontal coordinate value of the center point of the target prediction box, Represented as the horizontal coordinate value of the center point of the target prediction box, ω is the width of the screen, h is the length of the screen.

[0042] Will Convert the radian system into the angle system and calculate the side angle of the target. The calculation method is as follows: ; In order to demonstrate the superiority of the machine vision-based ship maritime target perception method and / or system of the present invention, the applicant conducted a comparative experiment during the actual maritime navigation of the ship, recording the target recognition effect using the YOLOv9 benchmark model and the target perception effect using the AD-YOLOv9 model of the present invention.

[0043] In the actual sea voyage of ships, the data collected from an experimental ship sailing to the sea area near Guangzhou Port were selected as the experimental scene. The waterway in this scene is complex, with many types of ships, and common marine targets such as bridges, islands, reefs, and buoys. It can effectively verify the generalization of the proposed algorithm.

[0044] The recognition results in the actual navigation scene of the ship with bad weather conditions and complex targets are as follows: Figure 6 and Figure 7 shown.

[0045] Figure 6 The figure below is a diagram of the target recognition effect using the YOLOv9 benchmark model. The model identified five targets, including two navigation marks (buoy red 0.66 and buoy green 0.48) and three ships (other ship 0.71, other ship 0.75, and other ship 0.62). There are many ships that were missed. The recognition accuracy of the red buoy is 66%, and the recognition accuracy of the green buoy is 48%. The recognition accuracy of other ships is 71%, 75%, and 62%, respectively.

[0046] Figure 7The figure shows the target recognition effect of the AD-YOLOv9 model of the present invention. The model recognizes 6 targets, including 2 navigation marks (buoy red 0.68 and buoy green 0.50) and 4 ships (other ship 0.43, other ship 0.73, other ship 0.70 and other ship 0.66). Figure 7 It can be clearly seen that the accuracy of small targets in the current scene has been improved. The recognition accuracy of red and green buoys has increased by 2%, reaching 68% and 50% respectively, and one more target (other ship 0.43, Q:11, TB:331) has been recognized, thus reducing the missed detection rate. In addition, the scene also realizes the orientation perception of the target. According to the coordinate information obtained by deep learning, combined with the angle information of the bow, where Q is the side angle, the port side is negative, the starboard side is positive, and TB is the absolute orientation, the relative orientation of the target to the ship can be judged. For example, for the red buoy, a pink line is used, and the Q value and TB value are marked at a certain position on the line; for the green buoy, a yellow line is used, and the Q value and TB value are marked at a certain position on the line; for the other three target ships, green lines are used, and the Q value and TB value are marked at a certain position on the line.

[0047] The test results of the AD-YOLOv9 algorithm proposed in the present invention in actual navigation scenarios show that this method uses the deep learning YOLOv9 framework to improve the ability to distinguish different buoy colors and determine the target direction, effectively enhances the recognition accuracy of different targets at sea, and can improve the intelligent perception effect of targets during ship navigation at sea.

[0048] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not drive the essence of the corresponding technical solutions away from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for sensing marine targets based on machine vision, characterized in that: The steps include: Step S1: Use the AD-YOLOv9 algorithm to process the incoming image information to obtain the location information and category information of the target image; Step S2: establishing a coordinate system according to the position information of the target image, marking the target on the target image with a target prediction frame, determining the center point coordinates of the target prediction frame based on a deep learning network, thereby determining the center point coordinates of the target, and associating the center point coordinates of the target with the bow direction to determine the absolute orientation and relative orientation of the target; Step S3: determining the position of the target relative to a reference point of the ship according to the absolute position and relative position of the target to determine the direction angle of the target; Step S4: determining a straight line where the center point of the target and the reference point of the ship are located according to the direction angle of the target, and recording it as a target direction line; Step S5: Calculate and format the angle label of the target according to the target direction line, and select a position on the target direction line to draw the angle label information; Step S6: Display the processed target image.

2. The method for sensing marine targets based on machine vision according to claim 1 is characterized in that: The AD-YOLOv9 algorithm is an improvement of the YOLOv9 algorithm. A triple attention mechanism is introduced into the head network of YOLOv9. In the standard convolution layer of YOLOv9, a strategy of replacing it with a lightweight DualConv convolution is proposed.

3. The method for sensing marine targets based on machine vision according to claim 2 is characterized in that: The triple attention is channel attention, spatial attention and cross attention.

4. The method for sensing marine targets based on machine vision according to claim 1 is characterized in that: Step S1 includes: using the AD-YOLOv9 algorithm to identify ships, buoy colors, bridges, and island obstacles.

5. The method for sensing marine targets based on machine vision according to claim 1 is characterized in that: Also includes: Step S0: The environmental information around the ship is monitored in real time by the camera on the ship, and the obtained video or image information is imported into the AD-YOLOv9 model.

6. The method for sensing marine targets based on machine vision according to claim 1 is characterized in that: In step S2, the center point coordinates of the target prediction box Determined according to the following formula: ； in, Indicates the horizontal coordinate value of the upper left corner of the target prediction box, Indicates the horizontal coordinate value of the lower right corner of the target prediction box; ； in, Indicates the upper left corner ordinate value of the target prediction box. Indicates the vertical coordinate value of the lower right corner of the target prediction box.

7. The method for sensing marine targets based on machine vision according to claim 6 is characterized in that: Based on the bow direction, determine the position of other targets at sea according to the coordinates of the center point of the target: ； ； in, Represented as the horizontal coordinate value of the center point of the target prediction box, Represented as the horizontal coordinate value of the center point of the target prediction box, h is the length of the screen, ω is the width of the screen, Q Indicates the direction of other targets.

8. The method for sensing marine targets based on machine vision according to claim 1, characterized in that: Step S6 includes: dynamically displaying the target identified in the target image and its corresponding position information on the terminal display screen in a graphical form, so that the operator can directly observe the position of the target and its corresponding position information.

9. A machine vision-based ship marine target perception system, characterized in that: include: Information extraction module: used to process the incoming image information using the AD-YOLOv9 algorithm to obtain the location information and category information of the target image; Image processing module: used to establish a coordinate system according to the position information of the target image, mark the target on the target image with a target prediction frame, determine the center point coordinates of the target prediction frame based on a deep learning network, thereby determining the center point coordinates of the target, and associate the center point coordinates of the target with the bow direction to determine the absolute orientation and relative orientation of the target; Target direction angle determination module: used to determine the position of the target relative to the reference point of the ship according to the absolute position and relative position of the target, so as to determine the direction angle of the target; Direction line marking module: used to determine the straight line where the center point of the target and the reference point of the ship are located according to the direction angle of the target, and record it as the target direction line; Angle drawing module: used for calculating and formatting the angle label of the target according to the target direction line, and drawing the angle label information at a selected position on the target direction line; Image display module: used to display the processed target image.

Images