Ship classification method, device, equipment and system
By combining the local random erase algorithm in which the ship target is blocked during the training process and the YOLO-v7 detection classification model, the problem of deep learning poor detection effect in ship target classification is solved, and the classification accuracy is significantly improved. High resolution radar imaging for marine ship surveillance.
Patent Information
- Application Number
- CN202411488052.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-10-24
- Publication Date
- 2024-11-22
- Estimated Expiration
- Not applicable · inactive patent
AI Technical Summary
The existing deep learning-based ship target classification method has poor detection effect on ISAR images and SAR images and has low classification accuracy, making it difficult to meet the high-resolution radar imaging requirements for marine ship monitoring.
The preset local random erase algorithm is used to adjust the training radar image to simulate the effect of the ship's target being blocked, thereby training the deep learning detection classification model, improving the model's understanding of the ship's target characteristics, and using YOLO-v7 detection The classification model and CARAFE operator sample and adjust the feature map to improve detection effect and classification accuracy.
Through training to simulate occlusion effects, the detection effect and classification accuracy of ship detection classification based on deep learning are significantly improved, and the ship target can be identified more accurately, and is suitable for high-resolution radar imaging in marine ship monitoring.
Smart Images

Figure CN119006941A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of image processing technology, and in particular to a ship classification method, device, equipment and system. Background Art
[0002] Ships are the key targets of marine observation, and marine ship monitoring and detection are the key to maintaining marine safety. The synthetic aperture radar onboard the drone uses active microwave remote sensing imaging technology to emit electromagnetic wave signals to the target, and the information carried in the echo is used to accurately invert the target parameters; compared with passive detection methods such as optical observation, it does not rely on the lighting environment, so it has the advantage of high-resolution radar imaging observations all day and all weather, and can give full play to the reconnaissance efficiency of drones.
[0003] For airborne sea detection radar, the ability to identify ship targets and judge their categories is an important research direction. At present, for two-dimensional airborne synthetic aperture radar (SAR) images and inverse synthetic aperture radar (ISAR) images, the ship target classification methods mostly use the traditional manually designed feature training classifier to locate candidate targets in the image. However, this method requires designing different features for different detection problems, selecting features for combination and selecting classifiers for training, and the process is extremely complicated; the robustness is poor, the degree of automation depends on prior knowledge, and often requires human experience and intervention.
[0004] In recent years, deep learning technology has been widely used in various fields with its powerful feature expression ability and autonomous learning ability. Although there are schemes that directly apply deep learning-based target classification methods to the classification of ship targets in ISAR images and SAR images, the detection effect of this scheme is poor and the classification accuracy is low. Therefore, how to improve the detection effect and classification accuracy of ship detection and classification based on deep learning is an urgent problem to be solved today. Summary of the invention
[0005] The purpose of the present invention is to provide a ship classification method, device, equipment and system to improve the detection effect and classification accuracy of ship detection and classification based on deep learning.
[0006] In order to solve the above technical problems, the present invention provides a ship classification method, comprising:
[0007] Acquire a training radar image of preset specifications; wherein each of the training radar images is annotated with a corresponding ship type and a marked rectangular frame of a ship area;
[0008] According to the marked rectangular frame, the training radar image is adjusted by using a preset local random erasing algorithm to obtain a target training image;
[0009] The preset deep learning detection and classification model is trained using the target training image to obtain the preset deep learning detection and classification model after the training is completed, and the preset deep learning detection and classification model is used to detect and classify the ship image collected by the airborne radar to obtain the position and ship type of the ship in the ship image.
[0010] On the other hand, the preset local random erasing algorithm is a local random erasing algorithm based on a small sample target frame.
[0011] On the other hand, adjusting the training radar image according to the marked rectangular frame using a preset local random erasing algorithm to obtain a target training image includes:
[0012] According to the coordinates of the annotated rectangular frame corresponding to the current training radar image, a rectangular occlusion area in the current training radar image is randomly generated; wherein the current training radar image is any of the training radar images; the coordinates of the annotated rectangular frame corresponding to the current training radar image are (T x ,T y ,T w ,T h ), the coordinates of the upper left vertex of the rectangular occlusion area are (x e ,y e ); , , , , , S is the area of the annotated rectangular box corresponding to the current training radar image, S e is the area of the rectangular occlusion area, a is a preset parameter greater than 0 and less than 1, W e is the width of the rectangular occlusion area, H e is the height of the rectangular occlusion area, (T x ,T y ) is the coordinate of the upper left vertex of the annotated rectangular box corresponding to the current training radar image, T w and T h are the width and height of the annotated rectangle corresponding to the current training radar image, and random(·) is a random function;
[0013] According to the preset pixel value as the pixel amplitude in the rectangular occlusion area, a target training image corresponding to the current training radar image is obtained.
[0014] On the other hand, before randomly generating a rectangular occlusion area in the current training radar image according to the coordinates of the annotated rectangular frame corresponding to the current training radar image, the method further includes:
[0015] Determine whether the area of the annotated rectangular box corresponding to the current training radar image is less than a preset area threshold;
[0016] If yes, then executing the step of randomly generating a rectangular occlusion area in the current training radar image according to the coordinates of the annotated rectangular frame corresponding to the current training radar image;
[0017] If not, the current training radar image is used as the target training image.
[0018] On the other hand, the step of obtaining a radar image for training of preset specifications includes:
[0019] Acquire a training image set; wherein the training image set includes simulation images and images collected by airborne radar, and the images in the training image set are all marked with respective corresponding ship types and marked rectangular frames of ship areas;
[0020] After performing a sampling process of a preset multiple on the images in the training image set in the azimuth direction, a sampling image corresponding to each image in the training image set is obtained;
[0021] Cutting off a region of a preset size in each of the sampled images to obtain a cut-off image corresponding to each of the sampled images;
[0022] The captured image is filtered to obtain the training radar image.
[0023] On the other hand, filtering the captured image to obtain the training radar image includes:
[0024] The intercepted image is filtered using a Lee filter to obtain the training radar image.
[0025] On the other hand, obtaining a training image set comprises:
[0026] Acquire a ship image data set; wherein the ship image data set includes simulated images and images collected by airborne radar;
[0027] The ship image dataset is divided into a training image set, a verification image set and a test image set according to a ratio of 8:1:1;
[0028] The minimum circumscribed rectangle of the ship area in the images of the training image set, the verification image set and the test image set is annotated by a rotating box method to obtain an annotated rectangular box of the ship area corresponding to each image in the training image set, the verification image set and the test image set.
[0029] On the other hand, after the preset deep learning detection classification model is trained using the target training image to obtain the trained preset deep learning detection classification model, the method further includes:
[0030] Using the images in the test image set to perform a performance test on the preset deep learning detection classification model after the training is completed, to obtain a test result;
[0031] Adjusting the parameters of the preset deep learning detection classification model after the training according to the test results to obtain the preset deep learning detection classification model after the training is adjusted;
[0032] The performance of the adjusted and trained preset deep learning detection and classification model is verified using the images in the verification image set to obtain a verified preset deep learning detection and classification model, and the verified preset deep learning detection and classification model is used to detect and classify ship images collected by airborne radar to obtain the position and ship type of the ship in the ship image.
[0033] In another aspect, the training radar image is an ISAR image.
[0034] On the other hand, the preset deep learning detection and classification model is a YOLO-v7 detection and classification model; the YOLO-v7 detection and classification model uses a content-aware feature reconstruction CARAFE operator to sample and adjust the feature map output by the backbone feature extraction network.
[0035] On the other hand, the method of training a preset deep learning detection classification model using the target training image to obtain a preset deep learning detection classification model after training includes:
[0036] Obtain a feature map output by a backbone feature extraction network of the preset deep learning detection classification model;
[0037] predicting a recombinant kernel based on the content of each target position in the feature map;
[0038] The feature map is mapped and adjusted using the reorganized kernel to obtain an adjusted feature map.
[0039] The present invention also provides a ship classification device, comprising:
[0040] An acquisition module is used to acquire training radar images of preset specifications; wherein each of the training radar images is annotated with a corresponding ship type and a marked rectangular frame of the ship area;
[0041] An adjustment module, configured to adjust the training radar image according to the marked rectangular frame by using a preset local random erasing algorithm to obtain a target training image;
[0042] A training module is used to use the target training image to train a preset deep learning detection and classification model to obtain the preset deep learning detection and classification model after the training is completed, so as to use the preset deep learning detection and classification model after the training to detect and classify the ship image collected by the airborne radar to obtain the position and ship type of the ship in the ship image.
[0043] The present invention also provides a ship classification device, comprising:
[0044] Memory for storing computer programs;
[0045] A processor is used to implement the following steps when executing the computer program: using a preset deep learning detection and classification model to detect and classify the acquired ship image to obtain the position and ship type of the ship in the ship image; wherein the preset deep learning detection and classification model is trained by a target training image; the target training image is a training radar image of preset specifications obtained by adjusting a preset local random erasing algorithm, and each of the training radar images is annotated with a corresponding ship type and a labeled rectangular box of the ship area.
[0046] In addition, the present invention also provides a ship classification system, comprising:
[0047] A data processor is used to process the echo data received by the airborne radar of the UAV to obtain a ship image;
[0048] A ship classification device, used for detecting and classifying the ship image using a preset deep learning detection and classification model to obtain the position and ship type of the ship in the ship image; wherein the preset deep learning detection and classification model is obtained by training a target training image; the target training image is a training radar image of preset specifications obtained by adjusting a preset local random erasing algorithm, and each of the training radar images is annotated with a corresponding ship type and a marked rectangular frame of a ship area;
[0049] The display is used to display the position and type of the ship in the ship image on the human-computer interaction interface.
[0050] A ship classification method provided by the present invention includes: obtaining a training radar image of preset specifications; wherein each training radar image is annotated with a corresponding ship type and a labeled rectangular frame of the ship area; according to the labeled rectangular frame, the training radar image is adjusted using a preset local random erasing algorithm to obtain a target training image; a preset deep learning detection and classification model is trained using the target training image to obtain the preset deep learning detection and classification model after the training is completed, and the preset deep learning detection and classification model is used after the training is completed to detect and classify the ship images collected by the airborne radar to obtain the position and ship type of the ship in the ship image.
[0051] It can be seen that the present invention adjusts the training radar image according to the marked rectangular frame using a preset local random erasing algorithm to obtain a target training image, and can simulate the effect of ship target being blocked on the training radar image during the training process of the detection and classification model based on deep learning, so as to strengthen the model's cognition of ship target characteristics, so that the model after training can more accurately detect and classify ship images collected by airborne radar, thereby improving the detection effect and classification accuracy. In addition, the present invention also provides a ship classification device, equipment and system, which also have the above-mentioned beneficial effects. BRIEF DESCRIPTION OF THE DRAWINGS
[0052] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying creative work.
[0053] Figure 1 A flow chart of a ship classification method provided by an embodiment of the present invention;
[0054] Figure 2 A flow chart of another ship classification method provided by an embodiment of the present invention;
[0055] Figure 3 A schematic diagram of a CARAFE operator framework provided by an embodiment of the present invention;
[0056] Figure 4 A structural block diagram of a ship classification device provided by an embodiment of the present invention;
[0057] Figure 5 A schematic diagram of the structure of a ship classification device provided in an embodiment of the present invention. DETAILED DESCRIPTION
[0058] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0059] Please refer to Figure 1 , Figure 1 A flow chart of a ship classification method provided by an embodiment of the present invention. The method may include:
[0060] Step 101: Acquire a training radar image of preset specifications; wherein each training radar image is annotated with a corresponding ship type and a marked rectangular frame of the ship area.
[0061] It can be understood that the preset specifications in this embodiment can be the specifications of the input image of the preset deep learning detection and classification model, such as 270 pixels × 90 pixels; the training radar image in this embodiment can be the radar image required for training the preset deep learning detection and classification model (such as SAR images or ISAR images); wherein each training radar image can be annotated with the corresponding ship type (such as cargo ship or warship, etc.) and a marked rectangular box of the ship area to facilitate subsequent model training.
[0062] The present embodiment does not limit the specific radar image type of the radar images used for training. For example, the radar images used for training may all be ISAR images, or may all be SAR images with ship targets.
[0063] Correspondingly, the specific acquisition method of the training radar image of the preset specification in this embodiment can be set by the designer according to the usage scenario and user needs, such as the processor can directly receive the training radar image of the preset specification. The processor can also generate a training image set by itself; wherein the images in the training image set are all marked with the corresponding ship type and the marked rectangular frame of the ship area; after the images in the training image set are sampled by a preset multiple (such as 2 times) in the azimuth direction, the sampled images corresponding to each image in the training image set are obtained; the area of the preset specification in each sampled image is intercepted to obtain the intercepted image corresponding to each sampled image; the intercepted image is filtered to obtain the training radar image; that is, the processor can sample, intercept and filter the images of the original specifications (such as 2048 pixels × 2048 pixels) after being marked in the training image set by itself to obtain the corresponding training radar image of the preset specification. This embodiment does not impose any restrictions on this.
[0064] Accordingly, the training image set may include simulated images and images collected by airborne radar, that is, the training radar images may include simulated images and images collected by airborne radar. In other words, the images in the training image set may include not only images collected by airborne radar, such as ISAR images collected by ISAR radar of UAV, but also simulated images obtained by simulation (such as ISAR simulated images), so as to improve the richness of images used for model training and improve the training effect of the model by adding simulated images.
[0065] Similarly, this embodiment does not limit the specific acquisition method of the training image set, such as the processor can directly receive the training image set. The processor can also generate the training image set by itself; for example, the processor can obtain a ship image data set; wherein the ship image data set includes a simulation image and an image collected by an airborne radar; the ship image data set is divided according to a ratio of 8:1:1 to obtain a training image set, a verification image set, and a test image set; the minimum circumscribed rectangle of the ship area in the images of the training image set, the verification image set, and the test image set is annotated by a rotating box method, and the annotated rectangular box of the ship area corresponding to each image in the training image set, the verification image set, and the test image set is obtained; that is, the processor can divide and annotate the original specification images in the ship image data set by itself, and obtain the annotated training image set, the verification image set, and the test image set. Correspondingly, the processor can also directly receive the annotated ship image data set, and divide the annotated ship image data set according to a ratio of 8:1:1 to obtain the annotated training image set, the verification image set, and the test image set.
[0066] Step 102: According to the marked rectangular frame, a preset local random erasing algorithm is used to adjust the training radar image to obtain a target training image.
[0067] It is understandable that experiments have found that the schemes in related technologies that directly apply deep learning-based target classification methods to the classification of ship targets in radar images (such as ISAR images and SAR images) have poor detection effects and low classification accuracy. This is because these target classification methods were initially designed to mainly target RGB (red, green, and blue) optical images, which truly and intuitively present the texture, color and other features of the detection target through the superposition and combination of red, green, and blue bands. However, radar images belong to microwave imaging. The information on the image is the reflection of the ground objects to the radar beam, which is mainly the image information formed by the backscattering of the ground objects. Generally, grayscale values are used to reflect the image information. The greater the reflection intensity of the target, the higher the grayscale value. For example, a calm lake appears dark in the radar grayscale image, while a city appears as a bright area. This makes it more difficult to mine semantic information in radar images. In addition, when ships perform radar imaging, it is often the case that part of the hull structure is submerged in the waves due to complex sea conditions, high waves and fast water, resulting in partial missing of the imaging results. In addition, due to insufficient estimation of the ship's motion parameters during the imaging process, part of the hull structure appears defocused. From an intuitive point of view, these images seem to be shielded as if the ship target is obscured, which will lead to missed detections and false detections in subsequent ship detections.
[0068] Therefore, in this embodiment, the preset local random erasing algorithm is used to adjust the training radar image, which can simulate the effect of the ship target being blocked, so that the preset deep learning detection and classification model trained using the adjusted target training image can better recognize the characteristics of the ship target, thereby improving the detection effect and classification accuracy of the preset deep learning detection and classification model.
[0069] Correspondingly, the specific type of the preset local random erasure algorithm in this embodiment can be set by the designer according to practical scenarios and user needs. For example, the preset local random erasure algorithm can adopt a local random erasure algorithm based on a small sample target frame, such as the Mosaic-REL data enhancement algorithm. The Mosaic-REL data enhancement algorithm is a local random erasure algorithm (random erase-local, REL) based on a small sample target frame; the preset local random erasure algorithm can also adopt other REL algorithms, and this implementation does not impose any restrictions on this.
[0070] In this embodiment, the specific method of adjusting the training radar image according to the marked rectangular frame by using the preset local random erasing algorithm to obtain the target training image can be set by the designer. For example, when the preset local random erasing algorithm adopts the local random erasing algorithm based on the small sample target frame, the processor can randomly generate a rectangular occlusion area in the current training radar image according to the coordinates of the marked rectangular frame corresponding to the current training radar image; wherein the current training radar image is any training radar image; the coordinates of the marked rectangular frame corresponding to the current training radar image are (T x ,T y ,T w ,T h ), the coordinates of the upper left vertex of the rectangular occlusion area are (x e ,y e ); , , , , , S is the area of the annotated rectangular box corresponding to the current training radar image, S e is the area of the rectangular occlusion area, a is a preset parameter greater than 0 and less than 1, such as a can be 58% (ie 0.58); W e is the width of the rectangular occlusion area, H e is the height of the rectangular occlusion area, (T x ,T y ) is the coordinate of the upper left vertex of the annotated rectangular box corresponding to the current training radar image, T w and T h are the width and height of the annotated rectangular box corresponding to the current training radar image, that is, S=T w *T h ; random(·) is a random function; according to the preset pixel value as the pixel amplitude in the rectangular occlusion area, the target training image corresponding to the current training radar image is obtained.
[0071] Step 103: Use the target training image to train the preset deep learning detection and classification model to obtain the trained preset deep learning detection and classification model, and use the trained preset deep learning detection and classification model to detect and classify the ship image collected by the airborne radar to obtain the position and ship type of the ship in the ship image.
[0072] It should be noted that the preset deep learning detection and classification model in this embodiment can be a pre-set detection and classification model based on deep learning. This embodiment does not limit the specific model type of the preset deep learning detection and classification model. For example, the preset deep learning detection and classification model can adopt the YOLO-v7 detection and classification model to ensure the mining effect of the semantic information in the radar image and improve the detection and recognition effect of the model. Furthermore, since radar images (such as ISAR images) have the characteristics of low resolution and small targets to be detected, the quality loss during the amplification of radar images will further deteriorate the model recognition effect. Therefore, in this embodiment, the CARAFE (Content-Aware ReAssembly of Features) operator can be used in the YOLO-v7 detection and classification model to sample and adjust the feature map output by the backbone feature extraction network. The sampling operator on the features used in the network architecture of the YOLO-v7 detection and classification model is the nearest neighbor difference, which has the advantages of small computational complexity, simple algorithm and fast speed. However, it only uses the grayscale value of the pixel closest to the sampling point to be tested as the grayscale value of the sampling point, which makes it difficult to capture the rich semantic information required for intensive prediction tasks, resulting in obvious discontinuity of the grayscale value after sampling, and produces obvious mosaic and sawtooth phenomena. In this embodiment, the CARAFE operator is used to improve the YOLO-v7 detection and classification model. The CARAFE operator uses the underlying content information to predict the reorganization kernel and reassemble the predefined nearby area features. The adaptive and optimized reorganization kernel can be used at different locations, so that the image quality of the radar image will not be excessively lost due to image magnification, thereby improving the model recognition effect. The preset deep learning detection and classification model can adopt other detection and classification models based on deep learning, and this embodiment does not impose any restrictions on this.
[0073] For example, in this step, the processor can input the target training image into the YOLO-v7 detection classification model for model training, and the pre-trained weights are used in the model training process; in order to better activate the performance of the backbone feature extraction network of the YOLO-v7 detection classification model, the training method can adopt frozen training, so that the overall model training process can include two stages: the freezing stage and the unfreezing stage. Among them, the YOLO-v7 detection classification model can use the content-aware feature reorganization CARAFE operator to sample and adjust the feature map output by the backbone feature extraction network.
[0074] Correspondingly, in this embodiment, after obtaining the preset deep learning detection and classification model after training, the processor can also use the preset deep learning detection and classification model to detect and classify the ship images collected by the drone's airborne radar, and obtain the position of the ship in the ship image (such as the position of the detection box mark) and the ship type; display the position and ship type of the ship in the ship image through the display.
[0075] For example, after obtaining the preset deep learning detection and classification model after training, the processor can use images in the test image set to perform a performance test on the preset deep learning detection and classification model after training to obtain a test result; adjust the parameters of the preset deep learning detection and classification model after training according to the test result to obtain the adjusted preset deep learning detection and classification model after training; use images in the verification image set to perform performance verification on the adjusted preset deep learning detection and classification model to obtain a verified preset deep learning detection and classification model, and use the verified preset deep learning detection and classification model to detect and classify ship images collected by airborne radar to obtain the position and ship type of the ship in the ship image.
[0076] In this embodiment, the embodiment of the present invention adjusts the training radar image according to the marked rectangular box and utilizes a preset local random erasing algorithm to obtain a target training image. During the training process of the detection and classification model based on deep learning, the effect of occlusion of the ship target can be simulated on the training radar image to enhance the model's recognition of the characteristics of the ship target, so that the model after training can more accurately detect and classify the ship images collected by the airborne radar, thereby improving the detection effect and classification accuracy.
[0077] Based on the above embodiment, the present invention also provides another ship classification method to further improve the detection effect and classification accuracy. Figure 2 , Figure 2 A flowchart of another ship classification method provided by an embodiment of the present invention. The method may include:
[0078] Step 201: Acquire a ship image dataset; wherein the ship image dataset includes simulated images and images collected by airborne radar.
[0079] It can be understood that in this embodiment, ISAR simulation images and ISAR ship images collected by an aircraft (such as a drone) during actual flight can be obtained and mixed to form an ISAR ship image dataset to improve the richness of images used in model training and improve the training effect of the model.
[0080] Step 202: Divide the ship image dataset into a training image set, a verification image set, and a test image set according to a ratio of 8:1:1.
[0081] In this step, the images of the ship image dataset 1 can be divided into a ratio of 8:1:1 to obtain a training image set in the training phase, a verification image set in the verification phase, and a test image set in the test phase.
[0082] Step 203: Using a rotating frame method to mark the minimum circumscribed rectangle of the ship area in the images of the training image set, the verification image set and the test image set, and obtaining a marked rectangular frame of the ship area corresponding to each image in the training image set, the verification image set and the test image set.
[0083] That is to say, in this step, the minimum circumscribed rectangle of the ship area in the training image set, the verification image set and the test image set is annotated by a rotating frame method to obtain the annotated rectangular frame of the ship area corresponding to each image. For example, the processor can use RolabelImg (a rotating frame annotation software) annotation tool to annotate the minimum circumscribed rectangle of the target hull (i.e., the ship area) in the image in a rotating frame method to obtain the annotated rectangular frame of the image.
[0084] Correspondingly, in this embodiment, the ship types corresponding to the images in the training image set, the verification image set, and the test image set can also be labeled according to the received labeling control instruction. This embodiment does not limit the logical sequence of step 202 and step 203. As shown in this embodiment, the ship image data set can be divided first and then labeled; or the images in the ship image data set can be labeled first and then divided.
[0085] Step 204: After performing a sampling process of a preset multiple on the images in the training image set in the azimuth direction, a sampling image corresponding to each image in the training image set is obtained.
[0086] Step 205: intercepting a region of a preset size in each sampled image to obtain a corresponding intercepted image of each sampled image.
[0087] Step 206: Filter the captured image to obtain a radar image for training.
[0088] For example, for images in the training image set with an original size of 2048 pixels × 2048 pixels (such as the original ISAR simulation image and ISAR ship image), in this embodiment, after performing a 2-fold (i.e., a preset multiple) sampling process in the azimuth direction, an image area of 270 pixels × 90 pixels (i.e., a preset size) containing the main structure of the ship (such as the ship area within the marked rectangular frame) is filtered to obtain a corresponding training radar image.
[0089] Correspondingly, since the coherent speckle noise in the radar image (such as the ISAR image) will reduce the correlation between adjacent elements in the image, increase the entropy value of the image, and bring certain difficulties to target detection and recognition, in this embodiment, a Lee filter (a filter based on a window function) can be used to filter and denoise the intercepted image. In other words, in step 206, the Lee filter can be used to filter the intercepted image to obtain a radar image for training.
[0090] Step 207: According to the marked rectangular frame, the training radar image is adjusted using a preset local random erasing algorithm to obtain a target training image.
[0091] For example, the preset local random erasing algorithm in this embodiment can adopt a local random erasing algorithm based on a small sample target frame (such as a Mosaic-REL data enhancement algorithm). In this step, the processor can randomly generate a rectangular occlusion area in the current training radar image according to the coordinates of the annotated rectangular frame corresponding to the current training radar image; according to the preset pixel value as the pixel amplitude in the rectangular occlusion area, the target training image corresponding to the current training radar image is obtained. Among them, the current training radar image is any training radar image; the coordinates of the annotated rectangular frame corresponding to the current training radar image are (T x ,T y ,T w ,T h ), the coordinates of the upper left vertex of the rectangular occlusion area are (x e ,y e ); , , , , , S is the area of the annotated rectangular box corresponding to the current training radar image, S e is the area of the rectangular occlusion area, a is a preset parameter greater than 0 and less than 1, W e is the width of the rectangular occlusion area, H e is the height of the rectangular occlusion area, (T x ,T y ) is the coordinate of the upper left vertex of the annotated rectangular box corresponding to the current training radar image, T w and T h are the width and height of the annotated rectangle corresponding to the current training radar image, and random(·) is a random function.
[0092] Correspondingly, before randomly generating a rectangular occlusion area in the current training radar image according to the coordinates of the labeled rectangular box corresponding to the current training radar image, it is also possible to determine whether the area of the labeled rectangular box corresponding to the current training radar image is smaller than a preset area threshold; if so, execute the step of randomly generating a rectangular occlusion area in the current training radar image according to the coordinates of the labeled rectangular box corresponding to the current training radar image; if not, use the current training radar image as the target training image, that is, since there is no obvious difference in the detection effect of large targets between local and global random erasing, in this embodiment, large targets with an area greater than the preset area threshold can be not erased to reduce the computational load.
[0093] For example, the processor can first read the true coordinates (T x , T y , T w , T h ) of the ship target (i.e., the labeled rectangular box) in the current training radar image, and compare the area S of the labeled rectangular box in the current training radar image with the preset area threshold S0 to determine the target size; if S < S0, initialize the relevant parameters of the erasing rectangular box I e (i.e., the rectangular occlusion area), such as randomly setting the aspect ratio of the erasing rectangular box to r e and the erasing area to S e ; randomly select a point P e = (x e , y e ) in the current training radar image, generate a rectangular area I e with P e as the upper left vertex, an area of S e , an aspect ratio of r e , and a size of (W e , H e ), and assign values to the pixels in the area to achieve the effect of occluding the ship target, and output the processed image (i.e., the target training image).
[0094] Among them, the limiting condition of P e can be , that is, a = 58%.
[0095] If S ≥ S0, it is determined as a large target, and the current training radar image can be directly output as a target training image to reduce the computational load.
[0096] Step 208: Use the target training image to train the YOLO-v7 detection and classification model to obtain a preset deep learning detection and classification model after training; the YOLO-v7 detection and classification model uses the CARAFE operator to sample and adjust the feature map output by the backbone feature extraction network.
[0097] Among them, the improved YOLO-v7 detection classification model in this embodiment introduces a lightweight and efficient CARAFE operator to upsample the feature map output by its backbone feature extraction network. That is to say, in the process of training the YOLO-v7 detection classification model using the target training image in this embodiment, the process of upsampling the feature map by the CARAFE operator can be included, such as obtaining the feature map output by the backbone feature extraction network of the YOLO-v7 detection classification model, predicting the reorganization kernel according to the content of each target position in the feature map; using the reorganization kernel, mapping and adjusting the feature map to obtain the adjusted feature map.
[0098] For example, if Figure 3 As shown, if the shape of the feature map currently output by the backbone feature extraction network of the YOLO-v7 detection classification model (i.e., the current feature map) is ; H represents the width of the feature map, W represents the height of the feature map, and C represents the number of channels of the feature map; the processor can first compress the number of channels of the current feature map to obtain a compressed feature map, that is, , to reduce the amount of calculation in the subsequent process; in the compressed feature map, use a The convolutional layer is used to predict the upsampling kernel; the number of channels of the compressed feature map is , the upsampling ratio is , then the number of output channels can be ; where k up is the size of the reorganization kernel; is the content encoder size, , so as to achieve a good balance between performance and efficiency. Expanding the channel dimension in the space dimension gives a shape of The upsampling kernel is normalized using softmax (normalization function) so that the sum of the convolution kernel weights is 1. For the output feature map ( ) and maps it back to the current feature map; Indicates The current feature map centered at position Sub-region, take out the sub-region centered on it The region and the predicted upsampling kernel of the point ( ) to obtain the output value by performing a dot product, thereby obtaining the adjusted current feature map. For any target position of the adjusted current feature map , there is a corresponding source position at the current feature map input before adjustment Therefore, the image quality will not be excessively lost due to image enlargement, making the detection network more fully aware of the semantic information.
[0099] Correspondingly, other processes in the training process of the YOLO-v7 detection classification model are similar to the training method of the YOLO-v7 detection classification model in the related art, and will not be repeated here.
[0100] Step 209: Use the images in the test image set to perform a performance test on the preset deep learning detection classification model after training to obtain a test result.
[0101] Step 210: Adjust the parameters of the trained YOLO-v7 detection classification model according to the test results to obtain the adjusted trained YOLO-v7 detection classification model.
[0102] Step 211: Use the images in the verification image set to verify the performance of the adjusted and trained YOLO-v7 detection and classification model to obtain a verified YOLO-v7 detection and classification model, and use the verified YOLO-v7 detection and classification model to detect and classify the ship images collected by the airborne radar to obtain the position and ship type of the ship in the ship image.
[0103] It can be understood that in this embodiment, the images in the test image set can be used to perform a performance test on the trained YOLO-v7 detection and classification model, and according to the test results, the YOLO-v7 detection and classification model is adjusted, such as adjusting the settings of the optimizer scheme and parameters (such as the maximum learning rate and the minimum learning rate of the model), to obtain the adjusted trained YOLO-v7 detection and classification model; then, the images in the verification image set are used to perform performance verification on the adjusted trained YOLO-v7 detection and classification model, such as obtaining a capture image of a preset specification corresponding to the image in the verification image set through a similar process from step 204 to step 206 or a target recognition and image capture process before the YOLO-v7 detection and classification model is used in actual applications, and then the captured image is input into the adjusted trained YOLO-v7 detection and classification model for performance verification, so as to obtain a verified YOLO-v7 detection and classification model, so that the verified YOLO-v7 detection and classification model can be used to detect and classify the ship image collected by the airborne radar of the aircraft (such as a drone) to obtain the position and ship type of the ship in the ship image.
[0104] Correspondingly, this step may also include using a verified preset deep learning detection and classification model to detect and classify the ship image collected by the drone's airborne radar to obtain the position of the ship in the ship image (such as the position of the detection box mark) and the ship type; and display the position and ship type of the ship in the ship image through a display.
[0105] In this embodiment, the embodiment of the present invention mixes the simulated images and the ship images collected by the aircraft (such as a drone) during actual flight to form a ship image dataset, which can improve the richness of the images used for model training and improve the training effect of the model.
[0106] Corresponding to the above method embodiment, an embodiment of the present invention further provides a ship classification device. The ship classification device described below and the ship classification method described above can correspond to each other.
[0107] Please refer to Figure 4 , Figure 4 This is a structural block diagram of a ship classification device provided by an embodiment of the present invention. The device may include:
[0108] The acquisition module 10 is used to acquire a training radar image of preset specifications; wherein each training radar image is annotated with a corresponding ship type and a marked rectangular frame of a ship area;
[0109] An adjustment module 20 is used to adjust the training radar image according to the marked rectangular frame using a preset local random erasing algorithm to obtain a target training image;
[0110] The training module 30 is used to train the preset deep learning detection and classification model using the target training image to obtain the preset deep learning detection and classification model after the training is completed, so as to use the preset deep learning detection and classification model after the training to detect and classify the ship images collected by the airborne radar to obtain the position and ship type of the ship in the ship image.
[0111] In some embodiments, the preset local random erasing algorithm is a local random erasing algorithm based on a small sample target frame.
[0112] In some embodiments, the adjustment module 20 may include:
[0113] The random submodule is used to randomly generate a rectangular occlusion area in the current training radar image according to the coordinates of the annotated rectangular box corresponding to the current training radar image; wherein the current training radar image is any training radar image; the coordinates of the annotated rectangular box corresponding to the current training radar image are (T x ,T y ,T w ,T h ), the coordinates of the upper left vertex of the rectangular occlusion area are (x e ,y e ); , , , , , S is the area of the annotated rectangular box corresponding to the current training radar image, Se is the area of the rectangular occlusion area, a is a preset parameter greater than 0 and less than 1, W e is the width of the rectangular occlusion area, H e is the height of the rectangular occlusion area, (T x ,T y ) is the coordinate of the upper left vertex of the annotated rectangular box corresponding to the current training radar image, T w and T h are the width and height of the annotated rectangle corresponding to the current training radar image, and random(·) is a random function;
[0114] The occlusion submodule is used to obtain a target training image corresponding to the current training radar image according to the preset pixel value as the pixel amplitude in the rectangular occlusion area.
[0115] In some embodiments, the adjustment module 20 may further include:
[0116] The area judgment submodule is used to judge whether the area of the annotated rectangular box corresponding to the current training radar image is less than a preset area threshold; if so, a start signal is sent to the random submodule; if not, the current training radar image is used as a target training image.
[0117] In some embodiments, the acquisition module 10 may include:
[0118] The training acquisition submodule is used to acquire a training image set; wherein the training image set includes simulation images and images collected by airborne radar, and the images in the training image set are all annotated with respective corresponding ship types and annotated rectangular frames of ship areas;
[0119] A sampling submodule, used to perform a sampling process on the images in the training image set in the azimuth direction at a preset multiple, so as to obtain a sampling image corresponding to each image in the training image set;
[0120] A capture submodule is used to capture a region of a preset size in each sampled image to obtain a captured image corresponding to each sampled image;
[0121] The filtering submodule is used to filter the captured image to obtain a radar image for training.
[0122] In some embodiments, the filtering submodule may be specifically used to filter the captured image using a Lee filter to obtain a radar image for training.
[0123] In some embodiments, the training acquisition submodule may include:
[0124] An image acquisition unit, used to acquire a ship image data set; wherein the ship image data set includes a simulated image and an image collected by an airborne radar;
[0125] An image division unit is used to divide the ship image data set into a training image set, a verification image set and a test image set according to a ratio of 8:1:1;
[0126] The image annotation unit is used to annotate the minimum circumscribed rectangle of the ship area in the images of the training image set, the verification image set and the test image set by using a rotating frame method, and obtain the annotated rectangular frame of the ship area corresponding to each image in the training image set, the verification image set and the test image set.
[0127] In some embodiments, the apparatus may further include:
[0128] A testing module is used to perform a performance test on a preset deep learning detection and classification model after training using images in a test image set to obtain a test result;
[0129] A test adjustment module is used to adjust the parameters of the preset deep learning detection classification model after training according to the test results to obtain the preset deep learning detection classification model after the training is adjusted;
[0130] The verification module is used to verify the performance of the adjusted and trained preset deep learning detection and classification model using the images in the verification image set, so as to obtain the verified preset deep learning detection and classification model, and to use the verified preset deep learning detection and classification model to detect and classify the ship images collected by the airborne radar, so as to obtain the position and ship type of the ship in the ship image.
[0131] In some embodiments, the training radar image is an ISAR image.
[0132] In some embodiments, the preset deep learning detection classification model is a YOLO-v7 detection classification model; the YOLO-v7 detection classification model uses a content-aware feature reconstruction CARAFE operator to sample and adjust the feature map output by the backbone feature extraction network.
[0133] In some embodiments, the training module 30 may include:
[0134] A feature map acquisition submodule is used to obtain a feature map output by a backbone feature extraction network of a preset deep learning detection classification model;
[0135] The prediction submodule is used to predict the reorganization kernel based on the content of each target position in the feature map;
[0136] The feature adjustment submodule is used to use the reorganization kernel to perform mapping adjustment on the feature map to obtain an adjusted feature map.
[0137] In this embodiment, the embodiment of the present invention adjusts the training radar image according to the marked rectangular frame by adjusting the module 20 using a preset local random erasing algorithm to obtain a target training image. During the training process of the detection and classification model based on deep learning, the effect of the ship target being blocked can be simulated on the training radar image to enhance the model's recognition of the ship target characteristics, so that the model after training can more accurately detect and classify the ship images collected by the airborne radar, thereby improving the detection effect and classification accuracy.
[0138] Corresponding to the above method embodiment, an embodiment of the present invention further provides a ship classification device. The ship classification device described below and the ship classification method described above can refer to each other.
[0139] Please refer to Figure 5 , Figure 5 This is a schematic diagram of the structure of a ship classification device provided by an embodiment of the present invention. The ship classification device may include:
[0140] A memory D1, for storing computer programs;
[0141] Processor D2 is used to execute the following steps of the computer program: using a preset deep learning detection and classification model to detect and classify the acquired ship image to obtain the position and ship type of the ship in the ship image; wherein the preset deep learning detection and classification model is trained by a target training image; the target training image is a training radar image of preset specifications adjusted by a preset local random erasing algorithm, and each training radar image is annotated with a corresponding ship type and a labeled rectangular frame of the ship area.
[0142] It can be understood that the preset deep learning detection and classification model used by the processor D2 of the ship classification device in this embodiment can be a model trained in the above method embodiment, such as a preset deep learning detection and classification model trained or verified in the above method embodiment. In other words, the ship classification device provided in this embodiment can directly use the preset deep learning detection and classification model trained in the above method embodiment to detect and classify the acquired ship images (such as ship images collected in real time by airborne radar), and it does not need to train the preset deep learning detection and classification model itself.
[0143] Specifically, the ship classification device may be a computer or a server.
[0144] Correspondingly, the memory D1 can also be used to store ship images (such as ship images collected in real time by airborne radar) to facilitate ship detection and classification of the ship images.
[0145] Corresponding to the above method embodiment, an embodiment of the present invention further provides a ship classification system, and a computer-readable storage medium described below and a ship classification method described above can be referenced to each other.
[0146] A ship classification system, comprising:
[0147] A data processor is used to process the echo data received by the airborne radar of the UAV to obtain a ship image;
[0148] The ship classification device is used to detect and classify the ship image using a preset deep learning detection and classification model to obtain the position and ship type of the ship in the ship image; wherein the preset deep learning detection and classification model is trained by a target training image; the target training image is a training radar image of preset specifications adjusted by a preset local random erasing algorithm, and each training radar image is annotated with a corresponding ship type and a marked rectangular frame of the ship area;
[0149] The display is used to display the position and type of the ship in the ship image on the human-computer interaction interface.
[0150] The data processor in this embodiment may be a processor disposed in an aircraft (such as a drone), such as an airborne multi-function radar of a drone. The ship classification device in this embodiment may be a small computer or server equipped with a graphics processing unit (GPU), on which a ship intelligent classification program is stored, which is used to input the ship image (such as an ISAR ship image) output by the data processor into a trained preset deep learning detection classification model (such as a YOLO-v7 detection classification model) after preprocessing, and output the ship category and detection frame (such as a rectangular frame marking the position of the ship), that is, the ship classification device in this embodiment may be the ship classification device in the above-mentioned device embodiment.
[0151] Corresponding to the above method embodiment, an embodiment of the present invention further provides a computer-readable storage medium. The computer-readable storage medium described below and the ship classification method described above can be referred to each other.
[0152] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps of the ship classification method of the above method embodiment.
[0153] The computer-readable storage medium may be a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk, etc., which can store program codes.
[0154] The various embodiments in the specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other. For the devices, equipment, systems and computer-readable storage media disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the description is relatively simple, and the relevant parts can be referred to the method part description.
[0155] The above is a detailed introduction to a ship classification method, device, equipment and system provided by the present invention. This article uses specific examples to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea. It should be pointed out that for ordinary technicians in this technical field, without departing from the principle of the present invention, the present invention can also be improved and modified, and these improvements and modifications also fall within the scope of protection of the present invention.
Claims
1. A ship classification method, characterized in that: include: Acquire a training radar image of preset specifications; wherein each of the training radar images is annotated with a corresponding ship type and a marked rectangular frame of a ship area; According to the marked rectangular frame, the training radar image is adjusted by using a preset local random erasing algorithm to obtain a target training image; The preset deep learning detection and classification model is trained using the target training image to obtain the preset deep learning detection and classification model after the training is completed, and the preset deep learning detection and classification model is used to detect and classify the ship image collected by the airborne radar to obtain the position and ship type of the ship in the ship image.
2. The ship classification method according to claim 1, characterized in that: The preset local random erasing algorithm is a local random erasing algorithm based on a small sample target frame.
3. The ship classification method according to claim 2, characterized in that: The step of adjusting the training radar image according to the marked rectangular frame by using a preset local random erasing algorithm to obtain a target training image includes: According to the coordinates of the annotated rectangular frame corresponding to the current training radar image, a rectangular occlusion area in the current training radar image is randomly generated; wherein the current training radar image is any of the training radar images; the coordinates of the annotated rectangular frame corresponding to the current training radar image are (T x ,T y ,T w ,T h ), the coordinates of the upper left vertex of the rectangular occlusion area are (x e ,y e ); , , , , , S is the area of the annotated rectangular box corresponding to the current training radar image, S e is the area of the rectangular occlusion area, a is a preset parameter greater than 0 and less than 1, W e is the width of the rectangular occlusion area, H e is the height of the rectangular occlusion area, (T x ,T y ) is the coordinate of the upper left vertex of the annotated rectangular box corresponding to the current training radar image, T w and T h are the width and height of the annotated rectangle corresponding to the current training radar image, and random(·) is a random function; According to the preset pixel value as the pixel amplitude in the rectangular occlusion area, a target training image corresponding to the current training radar image is obtained.
4. The ship classification method according to claim 3, characterized in that: Before randomly generating a rectangular occlusion area in the current training radar image according to the coordinates of the annotated rectangular frame corresponding to the current training radar image, the method further includes: Determine whether the area of the annotated rectangular box corresponding to the current training radar image is less than a preset area threshold; If yes, then executing the step of randomly generating a rectangular occlusion area in the current training radar image according to the coordinates of the annotated rectangular frame corresponding to the current training radar image; If not, the current training radar image is used as the target training image.
5. The ship classification method according to claim 1, characterized in that: The step of obtaining a radar image for training of preset specifications includes: Acquire a training image set; wherein the training image set includes simulation images and images collected by airborne radar, and the images in the training image set are all marked with respective corresponding ship types and marked rectangular frames of ship areas; After performing a sampling process of a preset multiple on the images in the training image set in the azimuth direction, a sampling image corresponding to each image in the training image set is obtained; Cutting off a region of a preset size in each of the sampled images to obtain a cut-off image corresponding to each of the sampled images; The captured image is filtered to obtain the training radar image.
6. The ship classification method according to claim 5, characterized in that: The filtering process is performed on the intercepted image to obtain the training radar image, including: The intercepted image is filtered using a Lee filter to obtain the training radar image.
7. The ship classification method according to claim 5, characterized in that: The step of obtaining a training image set comprises: Acquire a ship image data set; wherein the ship image data set includes simulated images and images collected by airborne radar; The ship image dataset is divided into a training image set, a verification image set and a test image set according to a ratio of 8:1:1; The minimum circumscribed rectangle of the ship area in the images of the training image set, the verification image set and the test image set is annotated by a rotating box method to obtain an annotated rectangular box of the ship area corresponding to each image in the training image set, the verification image set and the test image set.
8. The ship classification method according to claim 7, characterized in that: After the preset deep learning detection classification model is trained by using the target training image to obtain the trained preset deep learning detection classification model, the method further includes: Using the images in the test image set to perform a performance test on the preset deep learning detection classification model after the training is completed, to obtain a test result; Adjusting the parameters of the preset deep learning detection classification model after the training according to the test results to obtain the preset deep learning detection classification model after the training is adjusted; The performance of the adjusted and trained preset deep learning detection and classification model is verified using the images in the verification image set to obtain a verified preset deep learning detection and classification model, and the verified preset deep learning detection and classification model is used to detect and classify ship images collected by airborne radar to obtain the position and ship type of the ship in the ship image.
9. The ship classification method according to claim 1, characterized in that: The training radar image is an ISAR image.
10. The ship classification method according to any one of claims 1 to 9, characterized in that: The preset deep learning detection and classification model is a YOLO-v7 detection and classification model; the YOLO-v7 detection and classification model uses a content-aware feature reorganization CARAFE operator to sample and adjust the feature map output by the backbone feature extraction network.
11. The ship classification method according to claim 10, characterized in that: The method of training a preset deep learning detection classification model using the target training image to obtain a preset deep learning detection classification model after training, includes: Obtain a feature map output by a backbone feature extraction network of the preset deep learning detection classification model; predicting a recombinant kernel based on the content of each target position in the feature map; The feature map is mapped and adjusted using the reorganized kernel to obtain an adjusted feature map.
12. A ship classification device, characterized in that: include: An acquisition module is used to acquire training radar images of preset specifications; wherein each of the training radar images is annotated with a corresponding ship type and a marked rectangular frame of the ship area; An adjustment module, configured to adjust the training radar image according to the marked rectangular frame by using a preset local random erasing algorithm to obtain a target training image; A training module is used to use the target training image to train a preset deep learning detection and classification model to obtain the preset deep learning detection and classification model after the training is completed, so as to use the preset deep learning detection and classification model after the training to detect and classify the ship image collected by the airborne radar to obtain the position and ship type of the ship in the ship image.
13. A ship classification device, characterized in that: include: Memory for storing computer programs; A processor is used to implement the following steps when executing the computer program: using a preset deep learning detection and classification model to detect and classify the acquired ship image to obtain the position and ship type of the ship in the ship image; wherein the preset deep learning detection and classification model is trained by a target training image; the target training image is a training radar image of preset specifications obtained by adjusting a preset local random erasing algorithm, and each of the training radar images is annotated with a corresponding ship type and a labeled rectangular box of the ship area.
14. A ship classification system, characterized in that: include: A data processor is used to process the echo data received by the airborne radar of the UAV to obtain a ship image; A ship classification device, used for detecting and classifying the ship image using a preset deep learning detection and classification model to obtain the position and ship type of the ship in the ship image; wherein the preset deep learning detection and classification model is obtained by training a target training image; the target training image is a training radar image of preset specifications obtained by adjusting a preset local random erasing algorithm, and each of the training radar images is annotated with a corresponding ship type and a marked rectangular frame of a ship area; The display is used to display the position and type of the ship in the ship image on the human-computer interaction interface.
Citation Information
Patent Citations
Method for synthetic aperture radar shelter from shelter from object identification based on supporting vector machine
CN101135729A
Remote sensing SAR target detection method based on combination of network pruning and parameter quantification
CN114170512A
Ocean vortex detection method and system for synthetic aperture radar (SAR) image
CN116719031A
Contextual visual-based SAR target detection method and apparatus, and storage medium
US20230184927A1
Cited By
Ship component decoupling identification method and device based on high-resolution range profile
CN121878647A
