A method for estimating the pose of a ship in an infrared image

Abstract

The application discloses a kind of for the posture estimation method of ship in infrared image, S1: establish infrared ship image dataset;S2: establish YOLOv8 neural network model, infrared ship image is input into YOLOv8 neural network model, obtain ship region image;S3: based on Chan-Vese model, ship region image is carried out image binaryzation segmentation processing, obtain ship contour chart;S4: based on the ship position information in ship contour chart, the pixel level length, height and width of ship are calculated;S5: based on ship contour chart discrimination, obtain ship orientation;S6: establish support phasor regression model, and training;based on the support phasor regression model of training ship deflection angle estimation.The present application uses support vector regression model, the two deflection angles of ship pitch angle and azimuth angle are predicted, in the case where a small amount of prior information is used, the deflection angle of ship can be accurately estimated, so it can be applied to real-time investigation of sea target, realize automatic monitoring, provide decision basis for analyst.It has important significance to improve the efficiency of ship at sea and reduce the probability of accident.

Description

Technical Field

[0001] This invention relates to the field of ship attitude estimation technology, and in particular to a method for estimating the attitude of ships in infrared images. Background Technology

[0002] In complex marine environments, ship navigation is easily affected by environmental factors such as sea winds and waves, resulting in uncertainties in ship movement. This uncertainty poses safety hazards to ship operations at sea.

[0003] Therefore, accurate estimation of ship attitude is necessary to ensure safe navigation and stable operation of shipboard equipment. Existing technologies include five methods for estimating target deflection angles in images: Fourier transform-based methods, linear fitting-based methods, Hough transform-based methods, nearest neighbor connected component-based methods, and projection-based methods. The Fourier transform method analyzes the image's feature information in the frequency domain by performing a Fourier transform on the target image, thereby detecting the target deflection angle. However, this method is susceptible to noise due to its frequency domain-based detection of deflection angles and suffers from high computational complexity and low efficiency. The linear fitting method solves the problem of high computational complexity. One method determines the deflection angle of an image by fitting straight lines to boundary pixels, but it is easily affected by image noise during the fitting process. The Hough transform method has strong anti-interference ability and uses the transformation relationship between two coordinate spaces to map lines with the same shape in space to a point in the parameter space, but it is computationally time-consuming and requires a large amount of storage space. The nearest neighbor connected component method establishes a connected component based on the spatial relationship between target edge points and calculates the straight line angle between all edge points and the starting point in the connected component, and takes the average value to solve the target deflection angle, but this algorithm has a large computational load. The projection method detects the deflection angle based on the maximum value of the sum of four maximum projection values, but it relies too much on the straight line features of the target edge. Summary of the Invention

[0004] This invention provides a method for estimating the attitude of ships in infrared images, overcoming the technical problems of existing technologies in estimating ship attitude, such as susceptibility to noise, high computational load, and over-reliance on the linear features of target edges.

[0005] To achieve the above objectives, the technical solution of the present invention is as follows:

[0006] A method for attitude estimation of ships in infrared images, comprising the following steps:

[0007] S1: Use an infrared camera to take pictures of the ship model to obtain several initial infrared ship images, and annotate the actual deflection angle of the ship in the infrared ship images to establish an infrared ship image dataset.

[0008] S2: Establish a YOLOv8 neural network model, input the infrared ship image into the YOLOv8 neural network model, perform preliminary separation of the ship region and the sea and sky background in the infrared ship image, and obtain the ship region image.

[0009] S3: Based on the Chan-Vese model, the ship region image is binarized and segmented to obtain the ship outline image;

[0010] S4: Calculate the ship's pixel-level length, height, and width based on the ship's position information in the ship outline diagram;

[0011] S5: Determine the ship's orientation based on the ship's outline diagram;

[0012] S6: Establish a support phasor regression model and train the support phasor regression model based on the actual ship yaw angle and the ship's pixel-level length, width, height and orientation; estimate the ship yaw angle, i.e. the ship's pitch angle and azimuth angle, based on the trained support phasor regression model.

[0013] Furthermore, in S3, the process of performing image binarization segmentation on the ship region image based on the Chan-Vese model to obtain the ship outline image is as follows:

[0014] The initial energy functional of the Chan-Vese model is set as follows:

[0015]

[0016] In the formula, μ and ν are both smoothness term weight coefficients, representing the length weight of the segmentation boundary C and the area weight of the region it contains, respectively; λ1 and λ2 are both region term weight coefficients, representing the energy coefficients of the ship itself and the region outside the ship, respectively; C represents the segmentation boundary; L(C) is the length of the segmentation boundary C, and A(C) is the area of ​​the region contained in the segmentation boundary C; I(x,y) is the ship region image; Ω0 and Ω b c1 and c2 represent the ship itself and the background portion (the area outside the ship) of the global image I(x,y) divided by the segmentation boundary C; c1 and c2 represent Ω0 and Ω... b The average gray level;

[0017] The symbolic distance function φ(x,y) is chosen as the level set function to represent the ship itself and the area outside the ship in the ship region image, as follows:

[0018]

[0019] In the formula, d is the distance from a point in the high-dimensional space to the level set;

[0020] Based on the level set function, and by introducing the ideal step and Dirac function, the energy functional of the initial Chan-Vese model in equation (1) is improved, and the improved form is expressed as:

[0021]

[0022] H(φ) and H′(φ) represent the ideal step and Dirac function, respectively, and are as follows:

[0023]

[0024] In the formula, ε is a constant, which keeps φ(x,y) unchanged;

[0025] According to the minimization equation (3) of the Euler equation, we get:

[0026]

[0027] Keeping c1 and c2 constant, adding the time variable t, and using the gradient descent method, we obtain the partial differential equation, which is expressed as:

[0028]

[0029] Based on iterative solution of partial differential equations when When φ reaches its minimum value;

[0030] Substituting the minimum value of φ into equation (3), we obtain the minimum value of the energy functional F(c1,c2,C), at which point we can obtain the ship outline diagram.

[0031] Furthermore, in S4, the process of calculating the ship's pixel-level length, height, and width based on the ship's position information in the ship outline diagram is as follows:

[0032] 1) Determine the location information of the ship's outline, including the coordinates of the bow, stern, and the highest point of the island superstructure;

[0033] The process of calculating the pixel-level length of a ship is as follows:

[0034] Based on the bow and stern coordinates, which are the coordinates of the leftmost and rightmost pixels in the ship's outline diagram, denoted as p1(x1,y1) and p2(x2,y2), the pixel-level length of the ship, i.e., the deck line length, can be obtained based on the straight-line distance between p1 and p2.

[0035] 2) The process of calculating the pixel-level height of the ship is as follows:

[0036] Let the coordinates of the highest point of the island be p3(x3,y3), and the equation of the straight line be obtained by fitting p1(x1,y1) and p2(x2,y2);

[0037] The intersection of p3 with the straight line equation in the vertical direction is the coordinate p4(x4,y4) of the ship's center point. Since x3 = x4, the pixel-level height of the ship can be obtained based on the difference between p3 and p4 on the Y-axis.

[0038] 3) Use the height of the ship area image as the pixel-level width of the ship;

[0039] In S5, the process of determining the ship's orientation based on the ship's outline diagram is as follows:

[0040] Extract the ship deck line from the ship outline and determine whether the ship deck line is parallel to the bottom of the ship outline. If the ship deck line is parallel to the bottom of the ship outline, it means that the ship outline is a head-up infrared ship image; otherwise, the ship outline is a top-down infrared ship image.

[0041] If the ship outline diagram is a head-up infrared image of the ship, the process of determining the ship's orientation is as follows:

[0042] Obtain the head-up template image;

[0043] Based on the head view template image and the pixel coordinates of the bow and stern in each ship outline image, determine the bow and stern positions in the image. Then, the area where the bow position is close to 1 / 5 of the stern position is taken as the bow area, and the area where the stern position is close to 1 / 5 of the bow position is taken as the stern area.

[0044] Calculate the pixel difference σ between the stern and bow regions in the head-up template image. m , represented as:

[0045] σ m =β m -α m (8)

[0046] In the formula, α m β is the sum of pixel values ​​in the bow region of the head view template image; m This is the sum of pixel values ​​in the stern region of the head-up template image;

[0047] Then, the sum of pixel values ​​α in the bow region of each ship's outline map is calculated. i The sum of pixel values ​​β in the stern region i and σ m Pixel difference σ i , represented as:

[0048] σ i =β i -α i -σ m ,i∈N (9)

[0049] Where N represents the total number of ship outline drawings, and i is any ship outline drawing;

[0050] Finally based on σ i The ship's orientation is determined using a predefined symbolic function, the expression of which is:

[0051]

[0052] In the formula, Λ represents the ship's orientation value, where Λ = 1 means the ship is facing away from the camera, i.e., outwards; Λ = 0 means the ship's azimuth angle is 90°; and Λ = -1 means the ship is facing closer to the camera, i.e., inwards.

[0053] If the ship outline is a top-down infrared image of the ship, the ship's orientation is determined using the imaging principle of a bird's-eye view. The process is as follows:

[0054] Obtain the top-view template image;

[0055] Based on the pixel coordinates of the bow and stern in the top-view template image and each ship outline, determine the bow and stern positions in the image. Then, based on the bow and stern positions, determine the ship center point coordinates p4(x4,y4). Using p4(x4,y4) as the origin, cut the top-view template image and each ship outline into four regions in the horizontal and vertical directions.

[0056] Calculate the pixel difference σ between the upper and lower regions on the right side of the top-view template image. mm , represented as:

[0057] σ mm =α mo -β mo (11)

[0058] In the formula, α mo β mo These are the sums of the pixel values ​​corresponding to the upper and lower regions on the right, respectively;

[0059] Then calculate the sum of pixel values ​​α in the upper and lower right regions of each ship's outline map. mi β mi and σ mm Pixel difference σ mo , represented as:

[0060] σ mo =α mi -β mi -σ mm ,i∈N (12)

[0061] Finally based on σ mo The ship's orientation is determined using a predefined symbolic function, the expression of which is:

[0062]

[0063] In the formula, Λ' represents the ship's orientation value, where Λ' = 1 means the ship is facing away from the camera, i.e., outwards, Λ' = 0 means the ship's azimuth angle is 90°, and Λ' = -1 means the ship is facing closer to the camera, i.e., inwards.

[0064] Furthermore, in S6, the process of establishing a support phasor regression model and training it based on the actual ship yaw angle, as well as the ship's pixel-level length, width, height, and orientation, is as follows:

[0065] Establish a standard model that supports phasor SVM, represented as follows:

[0066]

[0067] In the formula, ω is the weight coefficient vector. Let f(x) be the mapping function, b be the bias variable, and f(x) be the prediction function to be fitted.

[0068] By applying the principle of risk minimization, the inequality-constrained optimization problem in the standard Support Vector Machine (SVM) model is transformed into an equality-constrained optimization problem. The transformed equality-constrained formula is as follows:

[0069]

[0070] In the formula, e is the difference between the predicted value and the true value, n is the number of samples in the training set, C' is the regularization constant, and l ε It is an ε-insensitive loss function;

[0071] Introducing slack variable ξ i and Equation (15) can be converted to:

[0072]

[0073] Introducing the Lagrange multiplier α i , and α i ≥0, Thus, equation (16) is transformed into:

[0074]

[0075] Mapping formula (17) to a higher-dimensional space, we get formula (18), which supports the phasor regression model, expressed as:

[0076]

[0077] Where, x iInput variables for the training set, namely the ship's pixel-level length, width, height, and orientation.

[0078] Beneficial Effects: This invention proposes a method for estimating the attitude of ships in infrared images. By automatically extracting the ship's length, width, height, and orientation features from infrared image sequences, and using a support vector regression model, it predicts the ship's pitch and azimuth angles. This method can accurately estimate the ship's yaw angles using only a small amount of prior information. It can be applied to real-time reconnaissance of sea targets, achieving automated monitoring and providing decision-making support for analysts. Predicting the ship's attitude during navigation ensures the stable operation of shipboard equipment and navigation safety, which is of great significance for improving the working efficiency of ships at sea and reducing the probability of accidents. Attached Figure Description

[0079] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0080] Figure 1 This is a flowchart of a method for estimating the attitude of a ship in an infrared image according to the present invention;

[0081] Figure 2 This is a schematic diagram of the ship's deflection in an embodiment of the present invention. Detailed Implementation

[0082] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0083] This embodiment provides a method for attitude estimation of ships in infrared images, such as... Figure 1 As shown, the specific steps include:

[0084] S1: Use an infrared camera to take pictures of the ship model from multiple azimuth angles, multiple pitch angles and multiple distances to obtain several initial infrared ship images, and mark the actual deflection angle of the ship in the infrared ship images to establish an infrared ship image dataset.

[0085] Specifically, infrared detection technology, due to its excellent concealment, passive detection capability, strong anti-interference ability, and all-weather applicability, especially at night or in low-light conditions, can acquire target information that is difficult to capture in images and can identify heat sources, thus becoming a key piece of equipment for the detection and tracking of ships at sea. Therefore, this embodiment uses an infrared camera to take pictures of a ship model from multiple azimuths, elevation angles, and distances, and annotates the actual angles of the ship in the infrared images to establish an infrared ship model dataset containing 1020 infrared ship images. Figure 2 As shown, in each infrared ship image, point A is the bow position of the ship, point B is the stern position of the ship, point C is the position of the camera, and point D is the center position of the ship; θ1 is the azimuth angle of the ship to be estimated, defined as the angle between the projection of the camera's optical center direction onto the horizontal plane and the bow direction, with a range of 0° to 180°; θ2 is the pitch angle of the ship to be estimated, defined as the angle between the camera's optical center direction and the horizontal plane, with a range of -90° to 90°.

[0086] S2: Establish a YOLOv8 neural network model, input the infrared ship image into the YOLOv8 neural network model, perform preliminary separation of the ship region and the sea and sky background in the infrared ship image, and obtain the ship region image.

[0087] Specifically, ship attitude estimation is crucial for maritime surveillance, ship identification, and security defense. However, real infrared ship images contain targets as well as backgrounds such as waves and clouds. These complex backgrounds can severely interfere with ship attitude estimation, potentially leading to significant errors. To overcome background interference and obtain the target ship's position information within the image, this embodiment uses the YOLOv8 model to separate the infrared image of the ship from the complex sea and sky background. YOLOv8 is a real-time target detection algorithm capable of quickly detecting different types of objects in images or videos, including people, vehicles, and animals, and accurately locating their positions.

[0088] S3: Based on the Chan-Vese model, the ship region image is binarized and segmented to obtain the ship outline image;

[0089] Specifically, S2 achieves the initial separation of the ship from the sea and sky background. However, due to the presence of a large number of interfering backgrounds such as clouds in the infrared ship image, the YOLOv8 algorithm can achieve the initial separation of most of the background from the ship and extract the "ship region" in the detection box. However, the "ship region" still contains a small amount of background area and ship part. In order to further extract the ship, the Chan-Vese model is used to perform image binarization segmentation processing on the "ship region" image, that is, the ship region image, to obtain the ship outline map.

[0090] Specifically, the Chan-Vese model is an image segmentation model based on region information, which segments the image by minimizing the difference in gray levels of pixels within a region. Its basic idea is to find a closed curve, i.e., the segmentation boundary C, to divide the image into an internal region and an external region. The region term of the energy function is formed by the integral of the square of the difference between the gray levels of the images inside and outside the curve and the constant. If only the region term of the model is considered, at the end of the curve evolution, the integral of the square of the difference between the gray levels of the images inside and outside the curve and the constant is minimized, that is, the intra-class variance is minimized.

[0091] S4: Calculate the ship's pixel-level length, height, and width based on the ship's position information in the ship outline diagram;

[0092] S5: Determine the ship's orientation based on the ship's outline diagram;

[0093] S6: Establish a support phasor regression model and train the support phasor regression model based on the actual ship yaw angle and the ship's pixel-level length, width, height and orientation; estimate the ship yaw angle, i.e. the ship's pitch angle and azimuth angle, based on the trained support phasor regression model.

[0094] In a specific embodiment, S3, the process of performing image binarization segmentation on the ship region image based on the Chan-Vese model to obtain the ship outline image is as follows:

[0095] The initial energy functional of the Chan-Vese model is set as follows:

[0096]

[0097] In the formula, μ and ν are both smoothness term weight coefficients, representing the length weight of the segmentation boundary C and the area weight of the region it contains, respectively; λ1 and λ2 are both region term weight coefficients, representing the energy coefficients of the ship itself and the region outside the ship, respectively; C represents the segmentation boundary; L(C) is the length of the segmentation boundary C, and A(C) is the area of ​​the region contained in the segmentation boundary C; I(x,y) is the ship region image; Ω0 and Ω b c1 and c2 represent the ship itself and the background portion (the area outside the ship) of the global image I(x,y) divided by the segmentation boundary C; c1 and c2 represent Ω0 and Ω... b The average gray level;

[0098] Specifically, the energy functional reaches its minimum value if and only if the segmentation boundary C moves to the ship region boundary of the ship region image. Therefore, the problem of ship contour segmentation in infrared images can be transformed into the problem of finding the minimum value of the energy functional. In this embodiment, the signed distance function φ(x,y) is selected as the level set function to represent the ship itself and the part outside the ship in the ship region image, expressed as:

[0099]

[0100] In the formula, d is the distance from a point in the high-dimensional space to the level set;

[0101] Based on the level set function, and by introducing the ideal step and Dirac function, the energy functional of the initial Chan-Vese model in equation (1) is improved, and the improved form is expressed as:

[0102]

[0103] H(φ) and H′(φ) represent the ideal step and Dirac function, respectively, and are as follows:

[0104]

[0105] In the formula, ε is a constant, which keeps φ(x,y) unchanged;

[0106] According to the minimization equation (3) of the Euler equation, we get:

[0107]

[0108] Keeping c1 and c2 constant, adding the time variable t, and using the gradient descent method, we obtain the partial differential equation, which is expressed as:

[0109]

[0110] Based on iterative solution of partial differential equations when When φ reaches its minimum value;

[0111] Substituting the minimum value of φ into equation (3), we obtain the minimum value of the energy functional F(c1,c2,C), at which point we can obtain the ship outline diagram.

[0112] In a specific embodiment, S4, the process of calculating the ship's pixel-level length, height, and width based on the ship's position information in the ship outline diagram is as follows:

[0113] 1) Determine the location information of the ship's outline, including the coordinates of the bow, stern, and the highest point of the island superstructure;

[0114] The process of calculating the pixel-level length of a ship is as follows:

[0115] Based on the bow and stern coordinates, which are the coordinates of the leftmost and rightmost pixels in the ship's outline diagram, denoted as p1(x1,y1) and p2(x2,y2), the pixel-level length of the ship, i.e., the deck line length, can be obtained based on the straight-line distance between p1 and p2.

[0116] 2) The process of calculating the pixel-level height of the ship is as follows:

[0117] Let the coordinates of the highest point of the island be p3(x3,y3), and the equation of the straight line be obtained by fitting p1(x1,y1) and p2(x2,y2);

[0118] The intersection of p3 with the straight line equation in the vertical direction is the coordinate p4(x4,y4) of the ship's center point. Since x3 = x4, the pixel-level height of the ship can be obtained based on the difference between p3 and p4 on the Y-axis.

[0119] 3) Use the height of the ship area image as the pixel-level width of the ship;

[0120] In a specific embodiment, S5, the process of determining the ship's orientation based on the ship's outline diagram is as follows:

[0121] Extract the ship deck line from the ship outline and determine whether the ship deck line is parallel to the bottom of the ship outline. If the ship deck line is parallel to the bottom of the ship outline, it means that the ship outline is a head-up infrared ship image; otherwise, the ship outline is a top-down infrared ship image.

[0122] If the ship outline is a head-up infrared image, the process of determining the ship's orientation based on perspective distance is as follows:

[0123] Use the ship outline of a ship at the same distance and with a 90° azimuth angle as the head-up template image;

[0124] The bow and stern positions in the image are determined based on the head-up template image and the pixel coordinates of the bow and stern in each ship outline image. Specifically, in the ship outline image, the coordinates of the first non-zero pixel from the leftmost end are the "stern position", and the coordinates of the first non-zero pixel from the rightmost end are the "bow position". The area 1 / 5 of the bow position close to the stern position is designated as the bow area, and the area 1 / 5 of the stern position close to the bow position is designated as the stern area.

[0125] Calculate the pixel difference σ between the stern and bow regions in the head-up template image. m , represented as:

[0126] σ m =β m -α m (8)

[0127] In the formula, α m β is the sum of pixel values ​​in the bow region of the head view template image; m This is the sum of pixel values ​​in the stern region of the head-up template image;

[0128] Then, the sum of pixel values ​​α in the bow region of each ship's outline map is calculated. iThe sum of pixel values ​​β in the stern region i and σ m Pixel difference σ i , represented as:

[0129] σ i =β i -α i -σ m , i∈N (9)

[0130] Where N represents the total number of ship outline drawings, and i is any ship outline drawing;

[0131] Finally based on σ i The ship's orientation is determined using a predefined symbolic function, the expression of which is:

[0132]

[0133] In the formula, Λ represents the ship's orientation value, where Λ = 1 means the ship is facing away from the camera, i.e., outwards; Λ = 0 means the ship's azimuth angle is 90°; and Λ = -1 means the ship is facing closer to the camera, i.e., inwards.

[0134] If the ship outline is a top-down infrared image of the ship, the ship's orientation is determined using the imaging principle of a bird's-eye view. The process is as follows:

[0135] Use the ship outline of a ship with an azimuth angle of 90° at the same pitch angle and distance as the top view template image;

[0136] Based on the pixel coordinates of the bow and stern in the top-view template image and each ship outline, the positions of the bow and stern in the image are determined. Then, based on the positions of the bow and stern, the coordinates of the ship's center point p4(x4,y4) are determined. Using p4(x4,y4) as the origin, the top-view template image and each ship outline are divided into four regions from left to right and from top to bottom, both horizontally and vertically.

[0137] Calculate the pixel difference σ between the upper and lower regions on the right side of the top-view template image. mm , represented as:

[0138] σ mm =α mo -β mo (11)

[0139] In the formula, α mo β mo These are the sums of the pixel values ​​corresponding to the upper and lower regions on the right, respectively;

[0140] Then calculate the sum of pixel values ​​α in the upper and lower right regions of each ship's outline map. mi β miand σ mm Pixel difference σ mo , represented as:

[0141] σ mo =α mi -β mi -σ mm , i∈N (12)

[0142] Specifically, in this embodiment, the experimental range of the azimuth angle is set to 0° to 180°. Therefore, in the infrared ship image dataset, the bow is always on the right side, so it is only necessary to calculate the difference in pixel values ​​on the right side to make a judgment.

[0143] Finally based on σ mo The ship's orientation is determined using a predefined symbolic function, the expression of which is:

[0144]

[0145] In the formula, Λ' represents the ship's orientation value, where Λ' = 1 means the ship is facing away from the camera, i.e., outwards, Λ' = 0 means the ship's azimuth angle is 90°, and Λ' = -1 means the ship is facing closer to the camera, i.e., inwards.

[0146] Specifically, the ship's orientation is determined based on the imaging principle of the infrared camera's bird's-eye view. In a top-down view, the infrared camera's viewpoint is relatively high. Therefore, in the camera's image, the portion closer to the infrared camera's viewpoint appears larger and is located at the bottom of the image, while the portion farther away appears smaller and is located at the top. In infrared imaging, for ships facing inwards, the stern is closer to the infrared camera's viewpoint, and the bow is farther away, with the bow appearing at the top of the image; while for ships facing outwards, the bow is closer to the infrared camera's viewpoint, and the stern is farther away, with the bow appearing at the bottom of the image. When the ship's deflection angle is 90° or 180°, the ship and the infrared camera are on the same plane. In this case, after the infrared camera images the ship, the ship's deck line is relatively in the center of the image. When the ship's deflection angle is less than 90° or greater than 270° but less than 360°, the ship's bow is relatively far from the infrared camera, and after the camera images the ship, the bow is at the top of the image. When the ship's deflection angle is greater than 90° but less than 180°, the ship's bow is relatively close to the camera, and after the camera images the ship, the bow is at the bottom of the image. Therefore, based on this perspective principle, the ship's orientation in the overhead infrared ship image can be determined.

[0147] Specifically, this embodiment uses the imaging mechanism of an infrared camera to determine the ship's orientation and incorporates it as a feature into a support vector regression model, which can significantly improve the prediction accuracy of the ship's yaw angle, thereby achieving ship attitude estimation and completing the tracking and monitoring of maritime targets.

[0148] In a specific embodiment, S6, the process of establishing a support phasor regression model and training the support phasor regression model based on the actual ship yaw angle and the ship's pixel-level length, width, height, and orientation is as follows:

[0149] Establish a standard model that supports phasor SVM, represented as follows:

[0150]

[0151] In the formula, ω is the weight coefficient vector. Let f(x) be the mapping function, b be the bias variable, and f(x) be the prediction function to be fitted.

[0152] By applying the principle of risk minimization, the inequality-constrained optimization problem in the standard Support Vector Machine (SVM) model is transformed into an equality-constrained optimization problem. The transformed equality-constrained formula is as follows:

[0153]

[0154] In the formula, e is the difference between the predicted value and the true value, n is the number of samples in the training set, C' is the regularization constant, and l ε It is an ε-insensitive loss function;

[0155] To make the model more tolerant of errors, a slack variable ξ is introduced. i and Equation (15) can be converted to:

[0156]

[0157] To balance model complexity and error tolerance, Lagrange multipliers α are introduced. i , and α i ≥0, Thus, equation (16) is transformed into:

[0158]

[0159] In this embodiment, referring to the support vector classification algorithm and using a kernel function, formula (17) is mapped to a higher-dimensional space, making the training samples linearly separable in this space, resulting in formula (18), which is the support phasor regression model, expressed as:

[0160]

[0161] Where, x i Input variables for the training set, namely the ship's pixel-level length, width, height, and orientation.

[0162] To verify the effectiveness of the method proposed in this embodiment, experiments were conducted based on an infrared ship model dataset. The pitch angle and azimuth angle attitude estimation results of the ship in some infrared images are shown in Table 1 and Table 2, respectively. The average error of the estimated ship pitch angle is about 2.78° and the average error of the ship azimuth angle is about 7.07°. Therefore, the ship attitude estimation using the method proposed in this embodiment can achieve a high prediction level.

[0163] Table 1:

[0164] Image number True pitch angle (°) Experimental results (°) Error (°) 01 -3 -4.472 1.472 02 -16 -17.652 1.652 03 -24 -24.500 0.500 04 -34 -33.034 0.966 05 -44 -44.869 0.869 06 -46 -48.127 2.127 07 -50 -49.164 0.836

[0165] Table 2:

[0166] Image number True azimuth (°) Experimental results (°) Error (°) 1 100 95.267 4.733 02 110 112.126 2.126 03 30 32.166 2.166 04 70 73.637 3.637 05 160 157.668 2.332 06 60 62.360 2.360 07 90 88.005 1.995 08 140 143.978 3.978

[0167] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for attitude estimation of ships in infrared images, characterized in that, The specific steps include: S1: Use an infrared camera to take pictures of the ship model to obtain several initial infrared ship images, and annotate the actual deflection angle of the ship in the infrared ship images to establish an infrared ship image dataset. S2: Establish a YOLOv8 neural network model, input the infrared ship image into the YOLOv8 neural network model, perform preliminary separation of the ship region and the sea and sky background in the infrared ship image, and obtain the ship region image. S3: Based on the Chan-Vese model, the ship region image is binarized and segmented to obtain the ship outline image; S4: Calculate the ship's pixel-level length, height, and width based on the ship's position information in the ship outline diagram; In S4, the process of calculating the ship's pixel-level length, height, and width based on the ship's position information in the ship outline diagram is as follows: 1) Determine the location information of the ship's outline, including the coordinates of the bow, stern, and the highest point of the island superstructure; The process of calculating the pixel-level length of a ship is as follows: Based on the bow and stern coordinates, which are the coordinates of the leftmost and rightmost pixels in the ship's outline diagram, denoted as p1(x1,y1) and p2(x2,y2), the pixel-level length of the ship, i.e., the deck line length, can be obtained based on the straight-line distance between p1 and p2. 2) The process of calculating the pixel-level height of the ship is as follows: Let the coordinates of the highest point of the island be p3(x3,y3), and the equation of the straight line be obtained by fitting p1(x1,y1) and p2(x2,y2); The intersection of p3 with the straight line equation in the vertical direction is the coordinate p4(x4,y4) of the ship's center point. Since x3 = x4, the pixel-level height of the ship can be obtained based on the difference between p3 and p4 on the Y-axis. 3) Use the height of the ship area image as the pixel-level width of the ship; S5: Determine the ship's orientation based on the ship's outline diagram; In S5, the process of determining the ship's orientation based on the ship's outline diagram is as follows: Extract the ship deck line from the ship outline and determine whether the ship deck line is parallel to the bottom of the ship outline. If the ship deck line is parallel to the bottom of the ship outline, it means that the ship outline is a head-up infrared ship image; otherwise, the ship outline is a top-down infrared ship image. If the ship outline diagram is a head-up infrared image of the ship, the process of determining the ship's orientation is as follows: Obtain the head-up template image; Based on the head view template image and the pixel coordinates of the bow and stern in each ship outline image, determine the bow and stern positions in the image. Then, the area where the bow position is close to 1 / 5 of the stern position is taken as the bow area, and the area where the stern position is close to 1 / 5 of the bow position is taken as the stern area. Calculate the pixel difference σ between the stern and bow regions in the head-up template image. m , is represented as: s m =b m -a m (1) In the formula, α m β is the sum of pixel values ​​in the bow region of the head view template image; m This is the sum of pixel values ​​in the stern region of the head-up template image; Then, the sum of pixel values ​​α in the bow region of each ship's outline map is calculated. i The sum of pixel values ​​β in the stern region i and σ m Pixel difference σ i , represented as: s i =b i -a i -s m ,i∈N (2) Where N represents the total number of ship outline drawings, and i is any ship outline drawing; Finally based on σ i The ship's orientation is determined using a predefined symbolic function, the expression of which is: In the formula, Λ represents the ship's orientation value, where Λ = 1 means the ship is facing away from the camera, i.e., outwards; Λ = 0 means the ship's azimuth angle is 90°; and Λ = -1 means the ship is facing closer to the camera, i.e., inwards. If the ship outline is a top-down infrared image of the ship, then the ship's orientation is determined using the imaging principle of a bird's-eye view. The process is as follows: Obtain the top-view template image; Based on the pixel coordinates of the bow and stern in the top-view template image and each ship outline, determine the bow and stern positions in the image. Then, based on the bow and stern positions, determine the ship center point coordinates p4(x4,y4). Using p4(x4,y4) as the origin, cut the top-view template image and each ship outline into four regions in the horizontal and vertical directions. Calculate the pixel difference σ between the upper and lower regions on the right side of the top-view template image. mm , is represented as: s mm =a mo -b mo (4) In the formula, α mo β mo These are the sums of the pixel values ​​corresponding to the upper and lower regions on the right, respectively. Then calculate the sum of pixel values ​​α in the upper and lower right regions of each ship's outline map. mi β mi and σ mm Pixel difference σ mo , represented as: s mo =a mi -b mi -s mm ,i∈N (5) Finally based on σ mo The ship's orientation is determined using a predefined symbolic function, the expression of which is: In the formula, Λ' represents the ship's orientation value, where Λ' = 1 means the ship's orientation is away from the camera, i.e., outward; Λ' = 0 means the ship's azimuth angle is 90°; and Λ' = -1 means the ship's orientation is close to the camera, i.e., inward. S6: Establish a support phasor regression model and train the support phasor regression model based on the actual ship yaw angle and the ship's pixel-level length, width, height and orientation; estimate the ship yaw angle, i.e. the ship's pitch angle and azimuth angle, based on the trained support phasor regression model.

2. The method for attitude estimation of ships in infrared images according to claim 1, characterized in that, In S3, the process of performing image binarization segmentation on the ship region image based on the Chan-Vese model to obtain the ship outline image is as follows: The initial energy functional of the Chan-Vese model is set as follows: In the formula, μ and ν are both smoothness term weight coefficients, representing the length weight of the segmentation boundary C and the area weight of the region it contains, respectively; λ1 and λ2 are both region term weight coefficients, representing the energy coefficients of the ship itself and the region outside the ship, respectively; C represents the segmentation boundary; L(C) is the length of the segmentation boundary C, and A(C) is the area of ​​the region contained in the segmentation boundary C; I(x,y) is the ship region image; Ω0 and Ω b c1 and c2 represent the ship itself and the background portion, respectively, of the global image I(x,y) divided by the segmentation boundary C into the ship region portion and the background portion, respectively; c1 and c2 represent Ω0 and Ω... b The average gray level; The symbolic distance function φ(x,y) is chosen as the level set function to represent the ship itself and the area outside the ship in the ship region image, as follows: In the formula, d is the distance from a point in the high-dimensional space to the level set; Based on the level set function, and by introducing the ideal step and Dirac function, the energy functional of the initial Chan-Vese model in equation (7) is improved, and the improved form is expressed as: H(φ) and H′(φ) represent the ideal step and Dirac function, respectively, and are as follows: In the formula, ε is a constant, which keeps φ(x,y) unchanged; According to the minimization equation (9) of the Euler equation, we get: Keeping c1 and c2 constant, adding the time variable t, and using the gradient descent method, we obtain the partial differential equation, which is expressed as: Based on iterative solution of partial differential equations when When φ reaches its minimum value; Substituting the minimum value of φ into equation (9), we obtain the minimum value of the energy functional F(c1,c2,C), at which point we can obtain the ship outline diagram.

3. The method for attitude estimation of ships in infrared images according to claim 2, characterized in that, In S6, the process of establishing a support phasor regression model and training it based on the actual ship yaw angle, as well as the ship's pixel-level length, width, height, and orientation, is as follows: Establish a standard model that supports phasor SVM, represented as follows: In the formula, ω is the weight coefficient vector. Let f(x) be the mapping function, b be the bias variable, and f(x) be the prediction function to be fitted. By applying the principle of risk minimization, the inequality-constrained optimization problem in the standard Support Vector Machine (SVM) model is transformed into an equality-constrained optimization problem. The transformed equality-constrained formula is as follows: In the formula, e is the difference between the predicted value and the true value, n is the number of samples in the training set, C' is the regularization constant, and l ε It is an ε-insensitive loss function; Introducing slack variable ξ i and Equation (15) can be converted to: Introducing the Lagrange multiplier α i , and α i ≥0, Thus, equation (16) is transformed into: Mapping formula (17) to a higher-dimensional space, we get formula (18), which supports the phasor regression model, expressed as: Where, x i Input variables for the training set, namely the ship's pixel-level length, width, height, and orientation.

Images