Attitude estimation method for ship in infrared image

Abstract

The invention discloses a method for estimating the attitude of a ship in an infrared image. The method comprises the following steps: S1, establishing an infrared ship image data set; s2, establishing a YOLOv8 neural network model, and inputting the infrared ship image into the YOLOv8 neural network model to obtain a ship region image; s3, carrying out image binarization segmentation processing on the ship area image based on a Chan-Vese model to obtain a ship contour map; s4, calculating the pixel-level length, height and width of the ship based on the ship position information in the ship contour map; s5, the ship orientation is obtained through judgment based on the ship contour map; s6, establishing a support phasor regression model, and performing training; and estimating the ship deflection angle based on the trained support phasor regression model. According to the method, the support vector regression model is used for predicting the two deflection angles of the pitch angle and the azimuth angle of the ship, the deflection angle of the ship can be accurately estimated under the condition that a small amount of prior information is used, and therefore the method can be applied to real-time investigation of sea surface targets, automatic monitoring is achieved, and a decision basis is provided for analysts. The method is of great significance in improving the working efficiency of ships on the sea and reducing the probability of accidents.

Description

Technical Field

[0001] The present invention relates to the technical field of ship attitude estimation, and in particular to a method for estimating the attitude of a ship in an infrared image. Background Art

[0002] In a complex ocean environment, the navigation of ships is easily affected by environmental factors such as sea breeze and waves, which makes the movement of ships full of uncertainty. This uncertainty brings safety risks to the ships' operations at sea.

[0003] Therefore, it is necessary to accurately estimate the attitude of the ship to ensure the safety of navigation and the stable operation of the shipboard equipment. In the prior art, there are five methods for estimating the deflection angle of the target in the image, namely: a method based on Fourier transform, a method based on linear fitting, a method based on Hough transform, a method based on the nearest neighbor connected domain, and a method based on projection; the Fourier transform method performs Fourier transform on the target image and analyzes the feature information of the image in the frequency domain, thereby realizing the detection of the target deflection angle. However, since this method detects the deflection angle of the image in the frequency domain, it is easily affected by noise and has the problems of high computational complexity and low efficiency; the linear fitting method solves the problem of high computational complexity , the deflection angle of the image is determined by fitting the boundary pixels with a straight line, but it is easily affected by image noise during the fitting process; the Hough transform method has strong anti-interference ability, and uses the transformation relationship between the two coordinate spaces to map the lines with the same shape in the space to a point in the parameter space, but the calculation is time-consuming and requires a large storage space; the nearest neighbor connected domain method establishes a connected domain based on the spatial relationship between the target edge points, and calculates the straight line angles between all edge points and the starting point in the connected domain, and takes the average value to solve the target deflection angle, but the algorithm has a large amount of calculation; the projection method detects the deflection angle according to the maximum value of the sum of the four maximum projection values, but it is overly dependent on the straight line characteristics of the target edge. Summary of the invention

[0004] The present invention provides a method for estimating the attitude of a ship in an infrared image, so as to overcome the technical problems existing in the prior art of estimating the attitude of a ship, such as being susceptible to noise, having a large amount of calculation, and being overly dependent on the straight line features of the target edge.

[0005] In order to achieve the above object, the technical solution of the present invention is:

[0006] A method for estimating the posture of a ship in an infrared image, characterized in that the specific steps include:

[0007] S1: Use an infrared camera to shoot the ship model to obtain several initial infrared ship images, and mark the real deflection angle of the ship in the infrared ship images to establish an infrared ship image dataset;

[0008] S2: Establish a YOLOv8 neural network model, input the infrared ship image into the YOLOv8 neural network model, perform a preliminary separation of the ship area and the sea and sky background in the infrared ship image, and obtain a ship area image;

[0009] S3: Based on the Chan-Vese model, the ship area image is subjected to image binary segmentation processing to obtain the ship contour map;

[0010] S4: Calculate the pixel-level length, height and width of the ship based on the ship position information in the ship outline image;

[0011] S5: Determine the ship's direction based on the ship's outline image;

[0012] S6: Establish a support phasor regression model, and train the support phasor regression model based on the actual ship deflection angle and the pixel-level length, width, height and ship orientation of the ship; estimate the ship deflection angle, that is, the pitch angle and azimuth angle of the ship based on the trained support phasor regression model.

[0013] 2. The method for estimating the posture of a ship in an infrared image according to claim 1 is characterized in that, in S3, the image of the ship region is subjected to image binarization segmentation processing based on the Chan-Vese model to obtain a ship contour map by:

[0014] The energy functional of the initial Chan-Vese model is set to:

[0015]

[0016] Where, μ and ν are the weight coefficients of the smoothing term, which represent the length weight of the segmentation boundary C and the area weight of the included area respectively; λ1 and λ2 are the weight coefficients of the area term, which represent the energy coefficients of the ship itself and the part outside the ship respectively; C represents the segmentation boundary; L(C) is the length of the segmentation boundary C, A(C) is the area of ​​the segmentation boundary C; I(x, y) is the ship area image; Ω0 and Ω b They represent the ship part and the background part outside the ship area image I(x, y) divided by the segmentation boundary C; c1 and c2 represent Ω0, Ω b The grayscale mean of ;

[0017] The signed distance function φ(x,y) is selected as the level set function to represent the ship part and the part outside the ship in the ship area image, which can be expressed as:

[0018]

[0019] Where d is the distance from the high-dimensional space point to the level set;

[0020] According to the level set function, and by introducing the ideal step and Dirac function, the energy functional of the initial Chan-Vese model in equation (1) is improved and expressed as follows:

[0021]

[0022] H(φ) and H′(φ) represent the ideal step and Dirac function, respectively, which are:

[0023]

[0024] In the formula, ε is a constant, which can keep φ(x,y) unchanged;

[0025] According to the Euler equation minimization formula (3), we get:

[0026]

[0027] Keep c1 and c2 unchanged, add the time variable t, and use the gradient descent method to obtain the partial differential equation, which is expressed as:

[0028]

[0029] Based on the iterative solution of partial differential equations When , φ is the minimum value;

[0030] Substituting the minimum value of φ into equation (3), we can obtain the minimum value of the energy functional F(c1, c2, C), and at this point, we can get the ship outline.

[0031] 3. The method for estimating the posture of a ship in an infrared image according to claim 2, characterized in that, in S4, the process of calculating the pixel-level length, height and width of the ship based on the ship position information in the ship outline image is:

[0032] 1) Determine the position information of the ship outline, including the coordinates of the bow, stern and highest point of the island;

[0033] The process of calculating the pixel-level length of a ship is:

[0034] According to the bow coordinates and stern coordinates, that is, the coordinates of the two leftmost and rightmost pixels in the ship outline, expressed as p1(x1,y1) and p2(x2,y2), the pixel-level length of the ship, that is, the deck line length, can be obtained based on the straight-line distance between p1 and p2;

[0035] 2) The process of calculating the pixel-level height of the ship is:

[0036] Assume the coordinates of the highest point of the island to be p3(x3,y3), and fit the equation of the straight line based on p1(x1,y1) and p2(x2,y2);

[0037] The intersection of p3 in the vertical direction with the straight line equation is the coordinate of the center point of the ship p4 (x4, y4), where, since x3 = x4, the pixel-level height of the ship can be obtained based on the difference between p3 and p4 on the Y axis;

[0038] 3) The height of the ship area image is used as the pixel-level width of the ship;

[0039] 4. The method for estimating the attitude of a ship in an infrared image according to claim 3, characterized in that, in S5, the process of determining the orientation of the ship based on the ship outline is as follows:

[0040] Extract the ship deck line in the ship outline image, and determine whether the ship deck line is parallel to the bottom of the ship outline image. If the ship deck line is parallel to the bottom of the ship outline image, it means that the ship outline image is a head-on infrared ship image, otherwise the ship outline image is a head-down infrared ship image;

[0041] If the ship outline image is a head-up infrared ship image, the process of determining the ship's direction is as follows:

[0042] Get the head-up template image;

[0043] The bow position and stern position in the image are determined according to the pixel coordinates of the bow and stern in the head-up template image and each ship outline image, and then the area near the bow position to 1 / 5 of the stern position is taken as the bow area, and the area near the stern position to 1 / 5 of the bow position is taken as the stern area;

[0044] Calculate the pixel difference σ between the stern area and the bow area in the head-up template image m , expressed as:

[0045] σ m =β m -α m (8)

[0046] In the formula, α m is the sum of the pixel values ​​of the bow area in the head-up template image; β m is the sum of the pixel values ​​of the stern area in the head-up template image;

[0047] Then calculate the sum of the pixel values ​​α of the bow area in each ship outline image i , the sum of the pixel values ​​in the stern area β i and σ m The pixel difference between i , expressed as:

[0048] σi =β i -α i -σ m ,i∈N (9)

[0049] Where N represents the total number of ship outlines, and i is any ship outline;

[0050] Finally, based on σ i The sign function is used to determine the direction of the ship. The sign function expression is:

[0051]

[0052] In the formula, Λ represents the ship's heading value, where Λ is 1, which means the ship is heading away from the camera, that is, outward, Λ is 0, which means the ship's azimuth is 90°, and Λ is -1, which means the ship is heading close to the camera, that is, inward;

[0053] If the ship outline image is a top-down infrared ship image, the ship orientation is obtained using the imaging principle of the bird's-eye view. The process is as follows:

[0054] Get a top-view template image;

[0055] The bow position and stern position in the image are determined according to the pixel coordinates of the bow and stern in the top-view template image and each ship outline image, and then the coordinates p4 (x4, y4) of the center point of the ship are determined according to the bow position and stern position. The top-view template image and each ship outline image are cut into four areas horizontally and vertically with p4 (x4, y4) as the origin;

[0056] Calculate the pixel difference σ between the upper and lower areas on the right side of the top-down template image mm , expressed as:

[0057] σ mm =α mo -β mo (11)

[0058] In the formula, α mo , β mo are the sum of the pixel values ​​corresponding to the upper and lower areas on the right;

[0059] Then calculate the sum of the pixel values ​​α of the upper and lower areas on the right side of each ship outline mi , β mi and σ mm The pixel difference between mo , expressed as:

[0060] σ mo =α mi -β mi -σ mm,i∈N (12)

[0061] Finally, based on σ mo The sign function is used to determine the direction of the ship. The sign function expression is:

[0062]

[0063] In the formula, Λ' represents the ship's heading value, where Λ' is 1, which means the ship is heading away from the camera, that is, outward; Λ' is 0, which means the ship's azimuth is 90°; and Λ' is -1, which means the ship is heading close to the camera, that is, inward.

[0064] 5. The method for estimating the attitude of a ship in an infrared image according to claim 4 is characterized in that, in S6, a support phasor regression model is established, and the process of training the support phasor regression model based on the actual ship deflection angle and the pixel-level length, width, height and ship orientation of the ship is as follows:

[0065] The support phasor SVM standard model is established and expressed as:

[0066]

[0067] In the formula, ω is the weight coefficient vector, is the mapping function, b is the bias variable, and f(x) is the prediction function to be fitted;

[0068] Through the principle of risk minimization, the inequality constraint optimization problem in the standard support vector machine SVM model is transformed into an equality constraint optimization problem. The transformed equality constraint formula is:

[0069]

[0070] In the formula, e is the difference between the predicted value and the true value, n is the number of samples in the training set, C' is the regularization constant, and l ε is the ε-insensitive loss function;

[0071] Introducing the slack variable ξ i and Convert equation (15) into:

[0072]

[0073] Introducing the Lagrange multiplier α i , and α i ≥0, Thus, equation (16) can be transformed into:

[0074]

[0075] Mapping formula (17) to a higher-dimensional space, we get formula (18), which supports the phasor regression model and is expressed as:

[0076]

[0077] Among them, x i The input variables for the training set are the pixel-level length, width, height, and orientation of the ship.

[0078] Beneficial effects: The present invention proposes a method for estimating the posture of a ship in an infrared image. By automatically extracting the length, width, height, and orientation of the ship in the infrared image sequence, a support vector regression model is used to predict the two deflection angles of the ship, namely the pitch angle and the azimuth angle. This method can accurately estimate the deflection angle of the ship using a small amount of prior information, and is applied to real-time detection of sea surface targets to achieve automated monitoring, thereby providing a decision-making basis for analysts. The method predicts the posture of the ship during navigation to ensure the stable use of shipboard equipment and navigation safety, which is of great significance to improving the work efficiency of ships at sea and reducing the probability of accidents. BRIEF DESCRIPTION OF THE DRAWINGS

[0079] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0080] Figure 1 It is a flow chart of a method for estimating the posture of a ship in an infrared image in the present invention;

[0081] Figure 2 Schematic diagram of the deflection of a ship in an embodiment of the present invention. DETAILED DESCRIPTION

[0082] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0083] This embodiment provides a method for estimating the attitude of a ship in an infrared image. Figure 1 As shown, the specific steps include:

[0084] S1: Use an infrared camera to shoot the ship model at multiple azimuths, multiple pitch angles, and multiple distances to obtain several initial infrared ship images, and annotate the real deflection angles of the ship in the infrared ship images to establish an infrared ship image dataset;

[0085] Specifically, infrared detection technology has good concealment, passive detection capability, strong anti-interference and all-weather applicability, especially at night or in low light conditions, and can obtain target information that is difficult to capture with images, and can identify heat sources, so it has become a key equipment for detecting and tracking ships at sea. Therefore, this embodiment uses an infrared camera to shoot the ship model at multiple azimuths, multiple pitch angles, and multiple distances, and marks the real angles of the ship in the infrared image, and establishes an infrared ship model data set containing 1020 infrared ship images. Figure 2 As shown in the figure, in each infrared ship image, point A is the bow position of the ship, point B is the stern position of the ship, point C is the position of the camera, and point D is the center position of the ship; θ1 is the azimuth angle of the ship to be estimated, and θ1 is defined as the angle between the projection of the camera optical center direction on the horizontal plane and the bow direction, which ranges from 0° to 180°; θ2 is the pitch angle of the ship to be estimated, and θ2 is defined as the angle between the camera optical center direction and the horizontal plane, which ranges from -90° to 90°.

[0086] S2: Establish a YOLOv8 neural network model, input the infrared ship image into the YOLOv8 neural network model, perform a preliminary separation of the ship area and the sea and sky background in the infrared ship image, and obtain a ship area image;

[0087] Specifically, ship attitude estimation is crucial for maritime monitoring, ship identification and security defense. However, real infrared ship images contain targets and backgrounds such as waves and clouds. These complex backgrounds will cause serious interference to the ship attitude estimation and may cause large errors in the ship attitude estimation. In order to overcome the interference of the background, the position information of the target ship in the image is obtained. Specifically, since YOLOv8 is a real-time target detection algorithm, it can quickly detect different types of objects in images or videos, including people, vehicles, animals, etc., and can accurately locate their positions. Therefore, in this embodiment, the infrared image ship is separated from the complex sea and sky background based on the YOLOv8 model.

[0088] S3: Based on the Chan-Vese model, the ship area image is subjected to image binary segmentation processing to obtain the ship contour map;

[0089] Specifically, S2 is to achieve the preliminary separation of the ship and the sea and sky background. However, due to the presence of a large number of interfering backgrounds such as clouds in the infrared ship image, the YOLOv8 algorithm can achieve the preliminary separation of most of the background and the ship, and extract the "ship area" in the detection frame. However, there are still a small number of background areas and ship parts in the "ship area". In order to further extract the ship, the Chan-Vese model is used to perform image binarization segmentation processing on the "ship area", that is, the ship area image, to obtain the ship contour map.

[0090] Specifically, the Chan-Vese model is an image segmentation model based on regional information, which segments the image by minimizing the grayscale difference of pixels in the region. The basic idea is to find a closed curve, namely the segmentation boundary C, to divide the image into an internal region and an external region. The regional term of the energy function is formed by the integral of the square of the grayscale difference between the inner and outer parts of the curve and the constant. If only the regional term of the model is considered, at the end of the curve evolution, the integral of the square of the grayscale difference between the inner and outer parts of the curve and the constant is minimized, that is, the intra-class variance is minimized.

[0091] S4: Calculate the pixel-level length, height and width of the ship based on the ship position information in the ship outline image;

[0092] S5: Determine the ship's direction based on the ship's outline image;

[0093] S6: Establish a support phasor regression model, and train the support phasor regression model based on the actual ship deflection angle and the pixel-level length, width, height and ship orientation of the ship; estimate the ship deflection angle, that is, the pitch angle and azimuth angle of the ship based on the trained support phasor regression model.

[0094] In a specific embodiment, in S3, the process of performing image binarization segmentation processing on the ship area image based on the Chan-Vese model to obtain the ship contour map is as follows:

[0095] The energy functional of the initial Chan-Vese model is set to:

[0096]

[0097] Where, μ and ν are the weight coefficients of the smoothing term, which represent the length weight of the segmentation boundary C and the area weight of the included area respectively; λ1 and λ2 are the weight coefficients of the area term, which represent the energy coefficients of the ship itself and the part outside the ship respectively; C represents the segmentation boundary; L(C) is the length of the segmentation boundary C, A(C) is the area of ​​the segmentation boundary C; I(x, y) is the ship area image; Ω0 and Ω bThey represent the ship part and the background part outside the ship area image I(x, y) divided by the segmentation boundary C; c1 and c2 represent Ω0, Ω b The grayscale mean of ;

[0098] Specifically, the energy functional obtains the minimum value when and only when the segmentation boundary C moves to the ship region boundary of the ship region image. Therefore, the infrared image ship contour segmentation problem can be transformed into the problem of finding the minimum value of the energy functional. In this embodiment, the signed distance function φ(x, y) is selected as the level set function to represent the ship itself and the part outside the ship of the ship region image, which is expressed as:

[0099]

[0100] Where d is the distance from the high-dimensional space point to the level set;

[0101] According to the level set function, and by introducing the ideal step and Dirac function, the energy functional of the initial Chan-Vese model in equation (1) is improved and expressed as follows:

[0102]

[0103] H(φ) and H′(φ) represent the ideal step and Dirac function, respectively, which are:

[0104]

[0105] In the formula, ε is a constant, which can keep φ(x,y) unchanged;

[0106] According to the Euler equation minimization formula (3), we get:

[0107]

[0108] Keep c1 and c2 unchanged, add the time variable t, and use the gradient descent method to obtain the partial differential equation, which is expressed as:

[0109]

[0110] Based on the iterative solution of partial differential equations When , φ is the minimum value;

[0111] Substituting the minimum value of φ into equation (3), we can obtain the minimum value of the energy functional F(c1, c2, C), and at this point, we can get the ship outline.

[0112] In a specific embodiment, in S4, the process of calculating the pixel-level length, height and width of the ship based on the ship position information in the ship outline image is:

[0113] 1) Determine the position information of the ship outline, including the coordinates of the bow, stern and highest point of the island;

[0114] The process of calculating the pixel-level length of a ship is:

[0115] According to the bow coordinates and stern coordinates, that is, the coordinates of the two leftmost and rightmost pixels in the ship outline, expressed as p1(x1,y1) and p2(x2,y2), the pixel-level length of the ship, that is, the deck line length, can be obtained based on the straight-line distance between p1 and p2;

[0116] 2) The process of calculating the pixel-level height of the ship is:

[0117] Assume the coordinates of the highest point of the island to be p3(x3,y3), and fit the equation of the straight line based on p1(x1,y1) and p2(x2,y2);

[0118] The intersection of p3 in the vertical direction with the straight line equation is the coordinate of the center point of the ship p4 (x4, y4), where, since x3 = x4, the pixel-level height of the ship can be obtained based on the difference between p3 and p4 on the Y axis;

[0119] 3) The height of the ship area image is used as the pixel-level width of the ship;

[0120] In a specific embodiment, in S5, the process of determining the direction of the ship based on the ship outline image is as follows:

[0121] Extract the ship deck line in the ship outline image, and determine whether the ship deck line is parallel to the bottom of the ship outline image. If the ship deck line is parallel to the bottom of the ship outline image, it means that the ship outline image is a head-on infrared ship image, otherwise the ship outline image is a head-down infrared ship image;

[0122] If the ship outline is a head-up infrared ship image, the process of determining the ship's direction based on line perspective is as follows:

[0123] The ship outline image of the ship with an azimuth angle of 90° at the same distance is used as the head-up template image;

[0124] The bow position and stern position in the image are determined according to the pixel coordinates of the bow and stern in the head-up template image and each ship outline image. Specifically, in the ship outline image, the coordinates of the first non-zero pixel point from the leftmost end are the "stern position", and the coordinates of the first non-zero pixel point from the rightmost end are the "bow position", and the area where the bow position is close to 1 / 5 of the stern position is taken as the bow area, and the area where the stern position is close to 1 / 5 of the bow position is taken as the stern area;

[0125] Calculate the pixel difference σ between the stern area and the bow area in the head-up template imagem , expressed as:

[0126] σ m =β m -α m (8)

[0127] In the formula, α m is the sum of the pixel values ​​of the bow area in the head-up template image; β m is the sum of the pixel values ​​of the stern area in the head-up template image;

[0128] Then calculate the sum of the pixel values ​​α of the bow area in each ship outline image i , the sum of the pixel values ​​in the stern area β i and σ m The pixel difference between i , expressed as:

[0129] σ i =β i -α i -σ m ,i∈N (9)

[0130] Where N represents the total number of ship outlines, and i is any ship outline;

[0131] Finally, based on σ i The sign function is used to determine the direction of the ship. The sign function expression is:

[0132]

[0133] In the formula, Λ represents the ship's heading value, where Λ is 1, which means the ship is heading away from the camera, that is, outward, Λ is 0, which means the ship's azimuth is 90°, and Λ is -1, which means the ship is heading close to the camera, that is, inward;

[0134] If the ship outline image is a top-down infrared ship image, the ship orientation is obtained using the imaging principle of the bird's-eye view. The process is as follows:

[0135] The ship outline image of the ship at the same pitch angle and distance and with an azimuth angle of 90° is used as the top-view template image;

[0136] The bow position and stern position in the image are determined according to the pixel coordinates of the bow and stern in the overhead template image and each ship outline image, and then the coordinates p4 (x4, y4) of the center point of the ship are determined according to the bow position and stern position. The overhead template image and each ship outline image are cut into four regions from left to right and from top to bottom in the horizontal and vertical directions with p4 (x4, y4) as the origin;

[0137] Calculate the pixel difference σ between the upper and lower areas on the right side of the top-down template imagemm , expressed as:

[0138] σ mm =α mo -β mo (11)

[0139] In the formula, α mo , β mo are the sum of the pixel values ​​corresponding to the upper and lower areas on the right;

[0140] Then calculate the sum of the pixel values ​​α of the upper and lower areas on the right side of each ship outline mi , β mi and σ mm The pixel difference between mo , expressed as:

[0141] σ mo =α mi -β mi -σ mm ,i∈N (12)

[0142] Specifically, in this embodiment, the experimental range of the azimuth angle is set to 0° to 180°. Therefore, in the infrared ship image data set, the bow is always on the right side, so it is only necessary to calculate the difference in the pixel values ​​on the right side to make the judgment;

[0143] Finally, based on σ mo The sign function is used to determine the direction of the ship. The sign function expression is:

[0144]

[0145] In the formula, Λ' represents the ship's heading value, where Λ' is 1, which means the ship is heading away from the camera, that is, outward; Λ' is 0, which means the ship's azimuth is 90°; and Λ' is -1, which means the ship is heading close to the camera, that is, inward.

[0146] Specifically, the ship orientation is obtained according to the imaging principle of the infrared camera's bird's-eye view, that is, in the bird's-eye view, the infrared camera viewpoint is relatively high, so in the camera imaging, the part close to the infrared camera viewpoint appears larger and is located at the bottom of the image, while the part far from the infrared camera viewpoint appears smaller and is located at the top of the image. In infrared imaging, for an inward-facing ship, the stern is close to the infrared camera viewpoint, the bow is far from the infrared camera viewpoint, and the bow appears above the image; while for an outward-facing ship, the bow is close to the infrared camera viewpoint, the stern is far from the infrared camera viewpoint, and the bow appears below the image. When the ship's deflection angle is 90° or 180°, the ship and the infrared camera are in the same plane. At this time, after the infrared camera forms an image, the ship's deck line is relatively in the middle of the image; when the ship's deflection angle is less than 90° or greater than 270° and less than 360°, the distance between the ship's bow and the infrared camera is relatively far. After the camera forms an image, the bow is above the image; when the ship's deflection angle is greater than 90° and less than 180°, the distance between the ship's bow and the camera is relatively close. After the camera forms an image, the bow is below the image. Therefore, according to the perspective principle, the orientation of the ship in the overhead infrared ship image can be determined.

[0147] Specifically, this embodiment determines the direction of the ship through the imaging mechanism of the infrared camera, and adds it as a feature to the support vector regression model, which can significantly improve the prediction accuracy of the ship's deflection angle, thereby realizing the ship's posture estimation and completing the tracking and monitoring of maritime targets.

[0148] In a specific embodiment, in S6, a supporting phasor regression model is established, and the process of training the supporting phasor regression model based on the actual ship deflection angle and the pixel-level length, width, height and ship orientation of the ship is as follows:

[0149] The support phasor SVM standard model is established and expressed as:

[0150]

[0151] In the formula, ω is the weight coefficient vector, is the mapping function, b is the bias variable, and f(x) is the prediction function to be fitted;

[0152] Through the principle of risk minimization, the inequality constraint optimization problem in the standard support vector machine SVM model is transformed into an equality constraint optimization problem. The transformed equality constraint formula is:

[0153]

[0154] In the formula, e is the difference between the predicted value and the true value, n is the number of samples in the training set, C' is the regularization constant, and l ε is the ε-insensitive loss function;

[0155] In order to make the model tolerate larger errors, a slack variable ξ is introduced i and Convert equation (15) into:

[0156]

[0157] In order to balance the complexity and error tolerance of the model, the Lagrange multiplier α is introduced i , and α i ≥0, Thus, equation (16) can be transformed into:

[0158]

[0159] In this embodiment, referring to the support vector classification algorithm, and with the help of the kernel function, formula (17) is mapped to a higher-dimensional space so that the training samples are linearly separable in this space, and formula (18) is obtained, that is, the support phasor regression model, which is expressed as:

[0160]

[0161] Among them, x i The input variables for the training set are the pixel-level length, width, height, and orientation of the ship.

[0162] To verify the effectiveness of the method proposed in this embodiment, experiments were conducted based on an infrared ship model data set. The pitch attitude estimation and azimuth attitude estimation results of the ship in some infrared images are shown in Tables 1 and 2, respectively. The average error of the estimated ship pitch angle is about 2.78°, and the average error of the ship azimuth is about 7.07°. Therefore, the ship attitude estimation using the method proposed in this embodiment can achieve a higher prediction level.

[0163] Table 1:

[0164] Image number True pitch angle (°) Experimental results (°) Error(°) 01 -3 -4.472 1.472 02 -16 -17.652 1.652 03 -24 -24.500 0.500 04 -34 -33.034 0.966 05 -44 -44.869 0.869 06 -46 -48.127 2.127 07 -50 -49.164 0.836

[0165] Table 2:

[0166] Image number True azimuth (°) Experimental results (°) Error(°) 1 100 95.267 4.733 02 110 112.126 2.126 03 30 32.166 2.166 04 70 73.637 3.637 05 160 157.668 2.332 06 60 62.360 2.360 07 90 88.005 1.995 08 140 143.978 3.978

[0167] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for estimating the attitude of a ship in an infrared image, characterized in that: The specific steps include: S1: Use an infrared camera to shoot the ship model to obtain several initial infrared ship images, and mark the real deflection angle of the ship in the infrared ship images to establish an infrared ship image dataset; S2: Establish a YOLOv8 neural network model, input the infrared ship image into the YOLOv8 neural network model, perform a preliminary separation of the ship area and the sea and sky background in the infrared ship image, and obtain a ship area image; S3: Based on the Chan-Vese model, the ship area image is subjected to image binary segmentation processing to obtain the ship contour map; S4: Calculate the pixel-level length, height and width of the ship based on the ship position information in the ship outline image; S5: Determine the ship's direction based on the ship's outline image; S6: Establish a support phasor regression model, and train the support phasor regression model based on the actual ship deflection angle and the pixel-level length, width, height and ship orientation of the ship; estimate the ship deflection angle, that is, the pitch angle and azimuth angle of the ship based on the trained support phasor regression model.

2. The method for estimating the attitude of a ship in an infrared image according to claim 1, characterized in that: In S3, the image of the ship area is subjected to image binary segmentation based on the Chan-Vese model, and the process of obtaining the ship contour map is as follows: The energy functional of the initial Chan-Vese model is set to: Where, μ and ν are the weight coefficients of the smoothing term, which represent the length weight of the segmentation boundary C and the area weight of the included area respectively; λ1 and λ2 are the weight coefficients of the area term, which represent the energy coefficients of the ship itself and the part outside the ship respectively; C represents the segmentation boundary; L(C) is the length of the segmentation boundary C, A(C) is the area of ​​the segmentation boundary C; I(x, y) is the ship area image; Ω0 and Ω b They represent the ship part and the background part outside the ship area image I(x, y) divided by the segmentation boundary C; c1 and c2 represent Ω0, Ω b The grayscale mean of ; The signed distance function φ(x,y) is selected as the level set function to represent the ship part and the part outside the ship in the ship area image, which can be expressed as: Where d is the distance from the high-dimensional space point to the level set; According to the level set function, and by introducing the ideal step and Dirac function, the energy functional of the initial Chan-Vese model in equation (1) is improved and expressed as follows: H(φ) and H′(φ) represent the ideal step and Dirac function, respectively, which are: In the formula, ε is a constant, which can keep φ(x,y) unchanged; According to the Euler equation minimization formula (3), we get: Keep c1 and c2 unchanged, add the time variable t, and use the gradient descent method to obtain the partial differential equation, which is expressed as: Based on the iterative solution of partial differential equations When , φ is the minimum value; Substituting the minimum value of φ into equation (3), we can obtain the minimum value of the energy functional F(c1, c2, C), and at this point, we can get the ship outline.

3. The method for estimating the attitude of a ship in an infrared image according to claim 2, characterized in that: In S4, the process of calculating the pixel-level length, height and width of the ship based on the ship position information in the ship outline map is as follows: 1) Determine the position information of the ship outline, including the coordinates of the bow, stern and highest point of the island; The process of calculating the pixel-level length of a ship is: According to the bow coordinates and stern coordinates, that is, the coordinates of the two leftmost and rightmost pixels in the ship outline, expressed as p1(x1,y1) and p2(x2,y2), the pixel-level length of the ship, that is, the deck line length, can be obtained based on the straight-line distance between p1 and p2; 2) The process of calculating the pixel-level height of the ship is: Assume the coordinates of the highest point of the island to be p3(x3,y3), and fit the equation of the straight line based on p1(x1,y1) and p2(x2,y2); The intersection of p3 in the vertical direction with the straight line equation is the coordinate of the center point of the ship p4 (x4, y4), where, since x3 = x4, the pixel-level height of the ship can be obtained based on the difference between p3 and p4 on the Y axis; 3) The height of the ship area image is used as the pixel-level width of the ship.

4. The method for estimating the attitude of a ship in an infrared image according to claim 3, characterized in that: In S5, the process of determining the ship's direction based on the ship's outline is as follows: Extract the ship deck line in the ship outline image, and determine whether the ship deck line is parallel to the bottom of the ship outline image. If the ship deck line is parallel to the bottom of the ship outline image, it means that the ship outline image is a head-on infrared ship image, otherwise the ship outline image is a head-down infrared ship image; If the ship outline image is a head-up infrared ship image, the process of determining the ship's direction is as follows: Get the head-up template image; The bow position and stern position in the image are determined according to the pixel coordinates of the bow and stern in the head-up template image and each ship outline image, and then the area near the bow position to 1 / 5 of the stern position is taken as the bow area, and the area near the stern position to 1 / 5 of the bow position is taken as the stern area; Calculate the pixel difference σ between the stern area and the bow area in the head-up template image m , expressed as: s m =b m -a m (8) In the formula, α m is the sum of the pixel values ​​of the bow area in the head-up template image; β m is the sum of the pixel values ​​of the stern area in the head-up template image; Then calculate the sum of the pixel values ​​α of the bow area in each ship outline image i , the sum of the pixel values ​​in the stern area β i and σ m The pixel difference between i , expressed as: s i =b i -a i -s m ,i∈N (9) Where N represents the total number of ship outlines, and i is any ship outline; Finally, based on σ i The sign function is used to determine the direction of the ship. The sign function expression is: In the formula, Λ represents the ship's heading value, where Λ is 1, which means the ship is heading away from the camera, that is, outward, Λ is 0, which means the ship's azimuth is 90°, and Λ is -1, which means the ship is heading close to the camera, that is, inward; If the ship outline image is a top-down infrared ship image, the ship orientation is obtained using the imaging principle of the bird's-eye view. The process is as follows: Get a top-view template image; The bow position and stern position in the image are determined according to the pixel coordinates of the bow and stern in the top-view template image and each ship outline image, and then the coordinates p4 (x4, y4) of the center point of the ship are determined according to the bow position and stern position. The top-view template image and each ship outline image are cut into four areas horizontally and vertically with p4 (x4, y4) as the origin; Calculate the pixel difference σ between the upper and lower areas on the right side of the top-down template image mm , expressed as: s mm =a mo -b mo (11) In the formula, α mo , β mo are the sum of the pixel values ​​corresponding to the upper and lower areas on the right; Then calculate the sum of the pixel values ​​α of the upper and lower areas on the right side of each ship outline mi , β mi and σ mm The pixel difference between mo , expressed as: s mo =a mi -b mi -s mm ,i∈N (12) Finally, based on σ mo The sign function is used to determine the direction of the ship. The sign function expression is: In the formula, Λ' represents the ship's heading value, where Λ' is 1, which means the ship is heading away from the camera, that is, outward; Λ' is 0, which means the ship's azimuth is 90°; and Λ' is -1, which means the ship is heading close to the camera, that is, inward.

5. The method for estimating the attitude of a ship in an infrared image according to claim 4, characterized in that: In S6, a support phasor regression model is established. The process of training the support phasor regression model based on the actual ship deflection angle and the pixel-level length, width, height and ship orientation of the ship is as follows: The support phasor SVM standard model is established and expressed as: In the formula, ω is the weight coefficient vector, is the mapping function, b is the bias variable, and f(x) is the prediction function to be fitted; Through the risk minimization principle, the inequality constraint optimization problem in the support vector machine (SVM) standard model is transformed into an equality constraint optimization problem. The transformed equality constraint formula is: In the formula, e is the difference between the predicted value and the true value, n is the number of samples in the training set, C' is the regularization constant, and l ε is the ε-insensitive loss function; Introducing the slack variable ξ i and Convert equation (15) into: Introducing the Lagrange multiplier α i , and α i ≥0, Thus, equation (16) can be transformed into: Mapping formula (17) to a higher-dimensional space, we get formula (18), which supports the phasor regression model and is expressed as: Among them, x i The input variables for the training set are the pixel-level length, width, height, and orientation of the ship.

Images