An intelligent identification method for key parts of a ship in an infrared image

Abstract

The application discloses an intelligent identification method for key parts of a ship in an infrared image, comprising the following steps: constructing a ship prior information database, establishing a three-dimensional entity model of each ship and key parts thereof in a three-dimensional coordinate system based on the ship prior information database, constructing an infrared ship detection network model of an improved YOLOv7, acquiring a ship infrared image, separating the ship in the ship infrared image from a sea-sky background according to the model and acquiring ship feature information, establishing a ship two-dimensional coordinate system based on the ship feature information, acquiring the separated ship image, performing gray equalization processing on the ship image, estimating the ship posture in the ship image, acquiring key parts in the ship image, and performing geometric matching on the key parts of the ship in the two-dimensional coordinate system and the key parts in the ship image and acquiring a matching result. The application realizes intelligent and automatic extraction of key parts of a sea surface target, and realizes fast monitoring in real-time reconnaissance of the sea surface target.

Description

Technical Field

[0001] This invention relates to the field of intelligent identification of key parts of ships, and in particular to an intelligent identification method for key parts of ships in infrared images. Background Technology

[0002] With the rapid development of high technology, the infrared characteristics of ship targets have become an important research topic in ship target surveillance and tracking. Infrared imaging technology has been widely used in ship target detection and identification. It can acquire target information at night or in low light conditions and can identify heat sources that traditional visible light images cannot detect. Especially in complex weather and sea conditions, when visible light images are limited, infrared images have unique advantages in ship target identification.

[0003] In recent years, intelligent recognition algorithms based on infrared target images have become a key research focus as a crucial technology for infrared target detection. These methods mainly fall into two categories: one extracts features from infrared images to identify the target ship; the other applies templates for matching and identifies the target based on the matching results. However, because infrared images lack rich color information and contain significant noise, the performance of these two methods deteriorates significantly when applied to identifying critical parts of infrared ships, where higher accuracy, recognition rate, and robustness are required. Summary of the Invention

[0004] This invention provides an intelligent identification method for key parts of ships in infrared images to overcome the above-mentioned technical problems.

[0005] A method for intelligent identification of key parts of ships in infrared images, comprising:

[0006] Step 1: Construct a ship prior information database. This database stores ship name, length, type, beam, key component parameters, and key component features. Based on this database, establish 3D solid models of each ship and its key components in a 3D coordinate system.

[0007] Step 2: Construct an improved YOLOv7 infrared ship detection network model to acquire infrared images of ships against a sea and sky background. Using the improved YOLOv7 infrared ship detection network model, separate the ship from the sea and sky background in the infrared images and obtain ship feature information. This ship feature information includes the bow, stern, length, and centerline of the ship. Establish a two-dimensional coordinate system for the ship based on this feature information.

[0008] Step 3: Acquire the separated ship images and perform grayscale equalization on them. Determine the angle from which the ship was photographed based on the separated images. When the angle is a top-down view, the ship's attitude in the image is estimated based on the ship's two-dimensional coordinate system. When the angle is a side-view view, the ship's attitude is estimated based on the ship's three-dimensional solid model in a three-dimensional coordinate system.

[0009] Step 4: Obtain key parts from the ship image. Based on the estimated ship attitude, rotate each ship in the three-dimensional coordinate system and project it to the two-dimensional coordinate system. Perform geometric matching between the key parts of the ship projected in the two-dimensional coordinate system and the key parts in the ship image. The geometric matching includes center matching and width matching. When both the center and width meet the matching conditions, the key parts of the ship in the two-dimensional coordinate system are replaced with the key parts of the ship image. When neither the center nor the width meets the matching conditions, the ship image is marked as an unrecognized image.

[0010] Preferably, the construction of the improved YOLOv7 infrared ship detection network model includes adding an infrared image preprocessing module to the YOLOv7 detection network model, replacing the convolution kernel with a deformable convolution, adding a coordinate attention module, and using EIOU as the loss function.

[0011] Preferably, the infrared image preprocessing module includes three sub-modules: image decomposition, image denoising, and image reconstruction. The image decomposition sub-module is used to decompose the infrared image layer by layer according to the Mallat decomposition algorithm. The image denoising sub-module is used to denoise the decomposed infrared image according to the Haar wavelet and adaptive threshold selection algorithm. The image reconstruction sub-module is used to reconstruct the denoised infrared image according to the Mallat reconstruction algorithm.

[0012] Preferably, the geometric matching of the key parts of the ship projected in the two-dimensional coordinate system with the key parts in the ship image includes approximating the key parts of the ship in the two-dimensional coordinate system and the key parts in the ship image as rectangles, and determining whether the key parts of the ship in the two-dimensional coordinate system match the key parts in the ship image according to formula (1). If formula (1) is satisfied, the key parts of the ship in the two-dimensional coordinate system are replaced with the key parts of the ship image. If formula (1) is not satisfied, the ship image is marked as an unrecognized image.

[0013]

[0014] In this context, the geometric center coordinates of the key parts of the ship in the two-dimensional coordinate system are P(x1,y1), the geometric center coordinates of the key parts in the ship image are P1(x2,y2), the geometric center coordinates of the key parts in the image are P2(x1cosθ,y1sinθ), θ is the offset angle, the ship width is R = a1, the width of the key parts in the ship image is R1 = a2, the width of the key parts in the image is R2 = a1cosθ, ε1 is the geometric center deviation, and ε2 is the width deviation.

[0015] Preferably, the grayscale equalization processing of the ship image includes performing grayscale equalization processing according to formula (2).

[0016]

[0017] In the formula, T represents a ship image with continuous grayscale; j represents the pixel grayscale level; L j This represents the equalized grayscale distribution.

[0018] This invention provides an intelligent identification method for key parts of ships in infrared images. It constructs a priori information database of ships and uses deep learning to obtain ship target detection categories from infrared images. Based on the detection results, it performs geometric matching with the priori information database to intelligently and automatically extract key parts of specific ship targets at sea. This technology can not only quickly and accurately analyze and process large amounts of accumulated historical image data, but also be applied to real-time reconnaissance of sea targets for rapid monitoring, greatly reducing the workload of image analysts and improving work efficiency. Attached Figure Description

[0019] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0020] Figure 1 This is a flowchart of the method of the present invention;

[0021] Figure 2 This is a three-dimensional model of the ship of the present invention;

[0022] Figure 3 This is the infrared image preprocessing module of the present invention;

[0023] Figure 4 This is a diagram showing the ship's offset angle estimation under a top-down view, as presented in this invention.

[0024] Figure 5 This invention describes the extraction process for key ship components. Detailed Implementation

[0025] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0026] Figure 1 This is a flowchart of the method of the present invention, as shown below. Figure 1 As shown, the method in this embodiment may include:

[0027] Step 1: Construct a ship prior information database. This database stores ship name, length, type, beam, key component parameters, and key component features. Based on this database, establish 3D solid models of each ship and its key components in a 3D coordinate system.

[0028] Step 2: Construct an improved YOLOv7 infrared ship detection network model to acquire infrared images of ships against a sea and sky background. Using the improved YOLOv7 infrared ship detection network model, separate the ship from the sea and sky background in the infrared images and obtain ship feature information. This ship feature information includes the bow, stern, length, and centerline of the ship. Establish a two-dimensional coordinate system for the ship based on this feature information.

[0029] Step 3: Acquire the separated ship images and perform grayscale equalization on them. Determine the angle from which the ship was photographed based on the separated images. When the angle is a top-down view, the ship's attitude in the image is estimated based on the ship's two-dimensional coordinate system. When the angle is a side-view view, the ship's attitude is estimated based on the ship's three-dimensional solid model in a three-dimensional coordinate system.

[0030] Step 4: Obtain key parts from the ship image. Based on the estimated ship attitude, rotate each ship in the three-dimensional coordinate system and project it to the two-dimensional coordinate system. Perform geometric matching between the key parts of the ship projected in the two-dimensional coordinate system and the key parts in the ship image. The geometric matching includes center matching and width matching. When both the center and width meet the matching conditions, the key parts of the ship in the two-dimensional coordinate system are replaced with the key parts of the ship image. When neither the center nor the width meets the matching conditions, the ship image is marked as an unrecognized image.

[0031] Based on the above scheme, a prior information database of ships is constructed, and ship target detection categories are obtained from infrared images through deep learning. Geometric matching is then performed between the detection results and the prior information database to intelligently and automatically extract key parts of specific ship targets at sea. This technology can not only quickly and accurately analyze and process large amounts of accumulated historical image data, but also be applied to real-time reconnaissance of sea targets for rapid monitoring, greatly reducing the workload of image analysts and improving work efficiency.

[0032] Step 1: Construct a ship prior information database. This database stores ship name, length, type, beam, key component parameters, and key component features. Based on this database, establish 3D solid models of each ship and its key components in a 3D coordinate system.

[0033] Specifically, key components of the target ship are obtained based on prior information. After collecting ship information from multiple sources, it is organized and filtered to construct a database containing textual information such as ship type, length, beam, and key component parameters, as well as feature information of these key components. Required key ship parameters can be retrieved from this database. A three-dimensional coordinate system for the ship is constructed based on these key parameters. To comprehensively and accurately represent the target ship, this embodiment comprehensively analyzes and fuses ship information from multiple sources to obtain more complete and reliable ship data. In addition to extracting useful information from infrared images, basic ship information (such as length, beam, and draft) can be searched online, and visible light images of the ship can be retrieved from multiple channels. Finally, this multi-source data is fused to obtain more unified and richer ship information. Simultaneously, to better aggregate and present this different types of information, a knowledge graph-based storage management model can be adopted. After organizing the information, a knowledge graph including basic ship information and feature information of key ship components can be constructed, from which required key ship information can be retrieved.

[0034] To establish a relatively complete ship intelligence system with as few parameters as possible, this embodiment focuses on the key parts of the ship. Specifically, the parameters that need to be characterized for the key parts of the ship are: ① the three-dimensional shape of each key part; ② the center point of key parts with different shapes.

[0035] Based on the ship's length, width, and height, the external features and coordinate system are determined. The center point of each key component is given, thus determining the location of these key components on the ship. A unified coordinate system allows for the determination of the location of key components on the ship within the infrared image. Given the specificity and complexity of key components on different types of ships, their 3D shapes are simplified to cuboids, cylinders, etc., which also helps in subsequently determining the center points of key components. A 3D ship model established from some characterization parameters (excluding numerical values) is shown below. Figure 2 As shown.

[0036] Step 2: Construct an improved YOLOv7 infrared ship detection network model to acquire infrared images of ships against a sea and sky background. Using the improved YOLOv7 infrared ship detection network model, separate the ship from the sea and sky background in the infrared images and obtain ship feature information. This ship feature information includes the bow, stern, length, and centerline of the ship. Establish a two-dimensional coordinate system for the ship based on this feature information.

[0037] Estimating ship attitude effectively from infrared images against a complex sea-sky background remains a challenging problem. This is due to the complexity of the image background. The image background consists of two parts: the sea background and the sky background. The sky background has relatively gentle undulations, while the sea background has more dramatic undulations. The grayscale value of the sky background is generally higher than that of the sea background due to the presence of clouds. The grayscale value near the horizon line is between the sea and sky backgrounds, representing a grayscale transition between the two backgrounds, which makes ship attitude estimation difficult. Therefore, to overcome background interference and obtain the size information of the target ship in the image, this patent employs deep learning to separate the target ship from the background.

[0038] YOLOv7 is a real-time object detection algorithm that can quickly detect different types of objects in images or videos, including people, vehicles, and animals, and accurately locate their positions. However, since YOLOv7 is a target detection algorithm based on visible light images, it is difficult to accurately distinguish object edges and texture details in infrared images with lower resolution and weaker contrast. This may lead to a high false detection or false negative rate. To better match the features of infrared ship images and obtain better output results, improvements are made to the single-stage YOLOv7 algorithm to improve its performance in infrared images.

[0039] The construction of the improved YOLOv7 infrared ship detection network model includes adding an infrared image preprocessing module to the YOLOv7 detection network model, replacing the convolution kernels with deformable convolutions, adding a coordinate attention module, and using EIOU as the loss function. The YOLOv7 detection network model includes a backbone network, a head network, an attention mechanism module, Rep, and Conv. The infrared image processed by the infrared image preprocessing module is input to the backbone network. The 3*3 convolution kernels of the backbone network are replaced with deformable convolutions. The output of the backbone network is connected to the input of the head network. The output of the head network is connected to the coordinate attention module, and then the predicted result is output after passing through Rep and Conv.

[0040] The infrared image preprocessing module comprises three sub-modules: image decomposition, image denoising, and image reconstruction. Its purpose is to eliminate noise and clutter in the infrared image to be detected, remove redundant data, improve image quality and signal-to-noise ratio, and thus enhance the efficiency of ship target detection. Compared to traditional image processing methods that cannot balance image denoising and detail enhancement, wavelet transform, as a time-frequency analysis technique with multi-scale and multi-resolution characteristics, has been successfully applied to the image field. The preprocessing flow is as follows: Figure 3 As shown.

[0041] The image decomposition submodule is used to decompose the infrared image layer by layer according to the Mallat decomposition algorithm. The Mallat decomposition algorithm first decomposes the original image into N row vectors or N column vectors, and then extracts the low-frequency and high-frequency components of each vector through low-pass and high-pass filters to separate low-frequency components and high-frequency details. Downsampling is used to reduce the sampling rate, thus enabling the decomposition and representation of the image signal at different frequencies, achieving the purpose of multi-scale image analysis. The image denoising submodule is used to denoise the decomposed infrared image according to the Haar wavelet and adaptive threshold selection algorithm. The Haar wavelet is a discrete wavelet basis function, composed of a parent wavelet function and a child wavelet function. The parent wavelet function is a smooth step function used to represent low-frequency information, approximating the overall trend and average value of the image; the child wavelet function is an oscillating function used to represent high-frequency details, capturing the image's edges, textures, and other detailed features. By translating and scaling the parent and child wavelet functions, multiple Haar wavelet basis functions at different scales and positions can be constructed. After wavelet decomposition, the coefficients of each frequency sub-band are obtained. Next, an adaptive threshold selection algorithm is used to process the high-frequency detail signal, which determines a suitable threshold based on the noise characteristics and signal energy distribution in the image. Generally, the high-frequency detail signal corresponding to the noise can be set to zero to achieve denoising.

[0042] The image reconstruction submodule is used to reconstruct the denoised infrared image according to the Mallat reconstruction algorithm. The image after threshold selection is reconstructed using a discrete binary wavelet transform combined with the Mallat reconstruction algorithm. The inverse wavelet transform is the inverse operation of the discrete wavelet transform. After threshold selection, the reconstructed images of each frequency subband have the effect of noise removal and detail enhancement.

[0043] The YOLOv7 convolution kernel is improved. Traditional convolution kernels are usually of fixed size, while the actual length and width ratio of ships varies greatly. Therefore, traditional convolution kernels have poor generalization ability for ship images. To solve this problem, deformable convolution is introduced. For each position w0 of the input feature map, the traditional convolution operation can be represented as:

[0044]

[0045] Where w n p represents each position of the convolution kernel, and R represents the 9 positions of the 3x3 convolution sampling.

[0046] Deformable convolution introduces an offset Δw for each point, based on traditional convolution. n For each position w0 in the input feature map, the convolution kernel is weighted and summed with the feature points at its offset positions (up, down, left, right, and diagonal) around R to obtain the pixel value corresponding to the new feature map position w0.

[0047]

[0048] Since the introduced offset is usually a decimal, while the actual pixels in the image are integers, the two cannot be correlated. Therefore, interpolation is needed to obtain the offset pixel value, usually bilinear interpolation.

[0049] Taking advantage of the reduced feature density of infrared images compared to visible light images, a lightweight and efficient Coordinate Attention (CA) module is introduced to reduce computational overhead while maintaining detection accuracy. The CA module embeds positional information into channel attention, a simple and efficient approach that is plug-and-play, enabling mobile networks to acquire information over a wider area without incurring additional computational burden.

[0050] The implementation of CA attention can be considered as two parallel stages: First, global average pooling is performed on the input feature map in both the width and height directions to obtain feature maps in those directions. Then, the two parallel stages are merged, the width and height are transposed to the same dimension, and then stacked to combine the width and height features. Features are then obtained using convolution, normalization, and activation functions. Next, the process is split into two parallel stages again, separating the width and height and transposing them to obtain two feature layers. Then, 1x1 convolutions are used to adjust the number of channels, and a sigmoid function is applied to obtain the attention status in the width and height dimensions. This is then multiplied by the original features. This approach takes into account the relationships between the target's channels and the spatial orientation and positional sensitivity information, further improving the model's detection performance for infrared targets and reducing the false negative rate.

[0051] YOLOv7 uses GIOU_Loss as its loss function, but it cannot address the positional relationship when the ground truth bounding box contains the predicted bounding box. This necessitates convergence through iterative processing, making the computation cumbersome, and it doesn't consider the balance between easy and difficult samples. To address these issues, the EIOU method is introduced.

[0052] The EIOU penalty term is based on the CIOU penalty term, but the aspect ratio influence factor is split and calculated separately for the length and width of the target box and anchor box. This loss function consists of three parts: overlap loss (L... IOU ), center distance loss (L dis Width and height loss (L) asp The first two parts continue the method in CIOU, but the width and height loss directly minimizes the difference in width and height between the target box and the anchor box, resulting in faster convergence.

[0053] L IOU =1-IOU (3)

[0054] L IOU For overlap loss, IOU is the ratio of the intersection to the union of the ground truth bounding boxes and the predicted bounding boxes, which can be used to evaluate the degree of overlap between the predicted and ground truth bounding boxes.

[0055]

[0056] L dis The center distance loss measures the positional or shape deviation of the predicted bounding box. Where p... 2 (b,b gt The distance () represents the distance between the center point of the predicted bounding box and the center point of the ground truth bounding box, and c is a scaling factor used to adjust the proportion of the distance. The distance loss is calculated in the form of squared difference, encouraging the center points of the predicted bounding boxes to be as close as possible to the center points of the ground truth bounding boxes.

[0057]

[0058] L asp For width and height loss (L) asp This is used to constrain the difference between the aspect ratio of the predicted bounding box and the aspect ratio of the actual labeled bounding box. Where p... 2 (w,w gt ) represents the difference between the width of the predicted bounding box and the width of the actual labeled bounding box, p 2 (h,h gt ) represents the difference between the predicted bounding box height and the actual labeled bounding box height, c w and c h This is a scaling factor used to adjust the aspect ratio. The aspect ratio loss is calculated using the squared difference method, encouraging the predicted bounding box to have an aspect ratio as close as possible to the actual bounding box.

[0059] The penalty term formula can then be expressed as follows:

[0060]

[0061] Compared to conventional penalty terms, the EIOU penalty term comprehensively considers factors such as position, shape, aspect ratio, and area, which can more accurately evaluate the quality of target detection results in infrared images; moreover, it can better utilize the edge information of the target and provide more accurate target detection results.

[0062] By making the above improvements to YOLOv7, it is possible to better obtain the target ship's feature information, identify the bow and stern, calculate the ship's length, determine the ship's centerline, and establish the ship's coordinate system in the image, so as to achieve the attitude estimation of the target ship.

[0063] Step 3: Obtain the separated ship image and perform grayscale equalization processing on the ship image. In order to achieve key part extraction, after separating the ship from the background using the improved YOLOv7 network model, the ship part of the infrared image is processed using grayscale histogram, the number of grayscale levels after processing is counted, and the grayscale levels are arranged at equal intervals throughout the grayscale range.

[0064] The grayscale equalization process for the ship image includes performing grayscale equalization according to formula (7).

[0065]

[0066] In the formula, T represents a ship image with continuous grayscale; j represents the pixel grayscale level; L j This represents the equalized grayscale distribution.

[0067] The angle at which the ship was photographed is determined based on the separated ship images. When the angle is a top-down view, the ship's attitude in the image is estimated based on the ship's two-dimensional coordinate system. When the angle is a side-view view, the ship's attitude is estimated based on the ship's three-dimensional solid model in a three-dimensional coordinate system. To meet the need for target ship attitude estimation, the ship's offset angle is estimated from both top-down and side-view perspectives.

[0068] The estimation of the ship's offset angle in a top-down view is as follows: Figure 4 As shown. First, the bow and stern are determined based on the ship's external features, and then the centerline of the target ship is obtained. A two-dimensional Cartesian coordinate system is established with the bow of the target ship as the origin O, the upward direction of the infrared image as the positive Y-axis, and the rightward direction of the infrared image as the positive X-axis. The angle θ between the centerline of the target ship's bow and the Y-axis is the offset angle of the target ship in the top-down view.

[0069] In a side-view scenario, it is assumed that the obtained infrared image information is the projection of the target ship onto the background. Based on the above principle, and according to the detected ship target type, prediction bounding box information, infrared camera resolution, and focal length, feature points are arranged in a three-dimensional coordinate system based on the ship's prior information. Knowing the positions of these feature points in the ship's coordinate system, the target's attitude is determined based on the changes in the geometric shape formed by the feature points. The ship is then positioned in the three-dimensional coordinate system O... c -x c y c z c The physical model is projected onto the infrared image plane coordinate system to estimate the ship's deflection angle and calibrate key parts of the ship.

[0070] First, based on the camera's sensor information and the parameter information of the captured image, the camera's intrinsic parameters are obtained. Since ship images are generally taken from a distance, the influence of distortion coefficients can be ignored. Assume the camera's focal length parameter is f. x f y The principal point offset is c x c y Its intrinsic parameter matrix K can be expressed as:

[0071]

[0072] Suppose there is a pixel feature point P in the infrared image. t (x t ,y t Its corresponding three-dimensional point in the world coordinate system is P. w (x w ,y w ,z w Based on the principle of pinhole imaging, the following transformation matrix can be obtained:

[0073]

[0074] Where R is a 3x3 rotation matrix, determined by the rotation angle of the camera in the world coordinate system, and T is a 3x1 translation vector, representing the translation of the camera in the world coordinate system.

[0075] Assume feature point P in the camera coordinate system c (x c ,y c ,z c If the rotation angles around the x, y, and z axes of the world coordinate system are α, β, and γ respectively, then the rotation matrix R can be expressed as:

[0076]

[0077] Once a sufficient number of feature points are available, the inherent geometric relationships between these feature points are combined with the camera's intrinsic parameters and the projection relationship between the known object's spatial coordinates and the corresponding image coordinates to form multiple feature point equations. The solution with the minimum reprojection error is obtained through iteration as the optimal solution to the problem, thereby estimating the ship's rotation matrix and attitude.

[0078] By calculating the ship's attitude, the three-dimensional key parts of the ship can be projected onto the image plane to obtain the projection points of the key parts that match the features. For ease of comparison, both the key parts obtained based on prior information and the key parts of the ship projected onto the image plane are approximated as rectangles.

[0079] Step 4: Obtain key parts from the ship image. Based on the estimated ship attitude, rotate each ship in the three-dimensional coordinate system and project it to the two-dimensional coordinate system. Perform geometric matching between the key parts of the ship projected in the two-dimensional coordinate system and the key parts in the ship image. The geometric matching includes center matching and width matching. When both the center and width meet the matching conditions, the key parts of the ship in the two-dimensional coordinate system are replaced with the key parts of the ship image. When neither the center nor the width meets the matching conditions, the ship image is marked as an unrecognized image.

[0080] The geometric matching of the key parts of the ship projected in the two-dimensional coordinate system with the key parts in the ship image includes approximating the key parts of the ship in the two-dimensional coordinate system and the key parts in the ship image as rectangles, and judging whether the key parts of the ship in the two-dimensional coordinate system match the key parts in the ship image according to formula (11). If formula (11) is satisfied, the key parts of the ship in the two-dimensional coordinate system are replaced with the key parts of the ship image. Further, by combining multi-source information such as visible light images, the fine contour features of the key parts of the ship target are obtained, thereby achieving high-precision extraction of key parts of the ship. If formula (11) is not satisfied, the ship image is marked as an unrecognized image.

[0081]

[0082] In this context, the geometric center coordinates of the key parts of the ship in the two-dimensional coordinate system are P(x1,y1), the geometric center coordinates of the key parts in the ship image are P1(x2,y2), the geometric center coordinates of the key parts in the image are P2(x1cosθ,y1sinθ), θ is the offset angle, the ship width is R = a1, the width of the key parts in the ship image is R1 = a2, the width of the key parts in the image is R2 = a1cosθ, ε1 is the geometric center deviation, and ε2 is the width deviation.

[0083] The matching and calibration process is as follows: Figure 5 As shown, specifically, the matching comparison result is determined by first determining whether the geometric centers of the two rectangles are sufficiently close, and then judging whether the widths of the rectangles are consistent. Let the geometric center coordinates of the key part in the actual ship be P(x1,y1), and the geometric center coordinates of the image segmentation be P1(x2,y2). Then, based on the deflection angle θ, the geometric center coordinates of the key part in the image should be P2(x1cosθ,y1sinθ). Let the width in the actual ship be R = a1, and the image segmentation width be R1 = a2. Then, based on the deflection angle θ, the width of the key part in the image should be R2 = a1cosθ. Due to the low image resolution, there is an error; therefore, a geometric center deviation ε1 and a width deviation ε2 can be set. If the spatial difference between P1 and P2 is not greater than the geometric center deviation ε1, then the geometric center of the key part judged by the image segmentation is considered consistent with the information of the actual ship. If the geometric centers match, then the widths of the approximate rectangles are judged to match. If the spatial difference between R1 and R2 is not greater than the width deviation ε2, then the width of the key part judged by the image segmentation is considered consistent with the information of the actual ship. By further combining information from multiple sources, such as visible light images, fine contour features of key parts of the ship target are obtained, thereby achieving high-precision extraction of key parts of the ship.

[0084] Overall beneficial effects:

[0085] This invention provides an intelligent identification method for key parts of ships in infrared images. It constructs a priori information database of ships and uses deep learning to obtain ship target detection categories from infrared images. Based on the detection results, it performs geometric matching with the priori information database to intelligently and automatically extract key parts of specific ship targets at sea. This technology can not only quickly and accurately analyze and process large amounts of accumulated historical image data, but also be applied to real-time reconnaissance of sea targets for rapid monitoring, greatly reducing the workload of image analysts and improving work efficiency.

[0086] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for intelligent identification of key parts of ships in infrared images, characterized in that, include, Step 1: Construct a ship prior information database. This database stores ship name, length, type, beam, key component parameters, and key component features. Based on this database, establish 3D solid models of each ship and its key components in a 3D coordinate system. Step 2: Construct an improved YOLOv7 infrared ship detection network model to acquire infrared images of ships against a sea and sky background. Using the improved YOLOv7 infrared ship detection network model, separate the ship from the sea and sky background in the infrared images and obtain ship feature information. This ship feature information includes the bow, stern, length, and centerline of the ship. Establish a two-dimensional coordinate system for the ship based on this feature information. Step 3: Acquire the separated ship images and perform grayscale equalization on them. Determine the angle from which the ship was photographed based on the separated images. When the angle is a top-down view, the ship's attitude in the image is estimated based on the ship's two-dimensional coordinate system. When the angle is a side-view view, the ship's attitude is estimated based on the ship's three-dimensional solid model in a three-dimensional coordinate system. Step 4: Obtain key parts from the ship image. Based on the estimated ship attitude, rotate each ship in the three-dimensional coordinate system and project it to the two-dimensional coordinate system. Perform geometric matching between the key parts of the ship projected in the two-dimensional coordinate system and the key parts in the ship image. The geometric matching includes center matching and width matching. When both the center and width meet the matching conditions, the key parts of the ship in the two-dimensional coordinate system are replaced with the key parts of the ship image. When neither the center nor the width meets the matching conditions, the ship image is marked as an unrecognized image.

2. The intelligent identification method for key parts of ships in infrared images according to claim 1, characterized in that, The construction of the improved YOLOv7 infrared ship detection network model includes adding an infrared image preprocessing module to the YOLOv7 detection network model, replacing the convolution kernel with deformable convolution, adding a coordinate attention module, and using EIOU as the loss function.

3. The intelligent identification method for key parts of ships in infrared images according to claim 2, characterized in that, The infrared image preprocessing module includes three sub-modules: image decomposition, image denoising, and image reconstruction. The image decomposition sub-module is used to decompose the infrared image layer by layer according to the Mallat decomposition algorithm. The image denoising sub-module is used to denoise the decomposed infrared image according to the Haar wavelet and adaptive threshold selection algorithm. The image reconstruction sub-module is used to reconstruct the denoised infrared image according to the Mallat reconstruction algorithm.

4. The intelligent identification method for key parts of ships in infrared images according to claim 1, characterized in that, The geometric matching of key parts of the ship projected in the two-dimensional coordinate system with key parts in the ship image includes approximating the key parts of the ship in the two-dimensional coordinate system and the key parts in the ship image as rectangles, and determining whether the key parts of the ship in the two-dimensional coordinate system match the key parts in the ship image according to formula (1). If formula (1) is satisfied, the key parts of the ship in the two-dimensional coordinate system are replaced with the key parts of the ship image. If formula (1) is not satisfied, the ship image is marked as an unrecognized image. (1) Among them, the geometric center coordinates of the key parts of the ship in the two-dimensional coordinate system are: The geometric center coordinates of key parts in the ship image are The geometric center coordinates of key parts of the ship in the image in a two-dimensional coordinate system are: , The offset angle is the ship's width. The width of key parts in the ship image is The width of key parts of the ship in the image in a two-dimensional coordinate system is: , For geometric center deviation, This refers to the width deviation.

5. The intelligent identification method for key parts of ships in infrared images according to claim 1, characterized in that, The grayscale equalization process for the ship image includes performing grayscale equalization according to formula (2). (2) In the formula, The image is a ship image with continuous grayscale. For pixel grayscale levels; This represents the equalized grayscale distribution.

Images