Ship target detection method in complex sea surface environment, electronic equipment and medium
By enhancing the remote sensing image data set of sea surface vessels and improving the RTMDet network model, the accuracy and speed problems of ship target detection in complex sea surface environments are solved, and efficient target detection is achieved.
Patent Information
- Application Number
- CN202410927778.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-11
- Publication Date
- 2024-11-12
- Estimated Expiration
- 2044-07-11
AI Technical Summary
In complex sea surface environments, existing target detection models are difficult to effectively deal with sea surface vessel targets with rotation angles, and the feature extraction capability of interfered targets is limited, resulting in low detection accuracy and speed.
Data enhancement by sea surface vessel remote sensing image dataset, including adding cloud occlusion, Gaussian noise and salt and pepper noise, use the RTMDet network model and adjust the convolution kernel of the CSPLayer module to set the 5*5 ordinary convolution kernel and 7*7 expansion volume The kernel is built to construct deep separable convolutions, expand the receptive field to improve detection performance.
It realizes high-precision and rapid detection of ship targets in complex sea surface environments, improves detection accuracy and speed, and enhances the model's processing capability of the interfered environment.
Smart Images

Figure CN118941942A_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the field of deep learning target detection, and in particular relates to a method, electronic equipment, and medium for detecting ship targets in a complex sea environment. Background Art
[0002] With the increasing application of remote sensing satellites in the fields of maritime warfare, geological survey, urban planning, emergency response, etc., the efficiency of manual information processing is becoming increasingly low. In the field of target detection, if human eyes are used for detection, it is very easy to miss detection or make false detection when facing large-scale, multi-target, and dense target detection.
[0003] With the rapid development of deep learning technology in recent years, more and more tasks are moving towards unmanned, information-based, and networked development. By training deep neural networks, they are transformed into models that can automatically detect targets. Target detection models represented by Yolo and FasterRCNN can ensure high-precision and high-speed target detection, but the following problems may arise when facing remote sensing targets:
[0004] 1. Remote sensing targets such as ships on the sea surface have rotation angles. If a horizontal detection frame is used for detection, a large number of non-target areas will be included. In the case of dense targets, large-scale overlap between detection frames will occur.
[0005] 2. The ability to extract features from disturbed targets is limited, which affects the performance of model training and reasoning. Summary of the invention
[0006] In view of the deficiencies in the prior art, the present invention provides a method, electronic equipment and medium for detecting ship targets in a complex sea environment.
[0007] In a first aspect, an embodiment of the present invention provides a method for detecting a ship target in a complex sea environment, the method specifically comprising:
[0008] Acquire a remote sensing image dataset of sea surface ships; add cloud and fog occlusion, Gaussian noise and salt and pepper noise interference to the remote sensing image dataset of sea surface ships to obtain a remote sensing image dataset of ships simulating a complex sea surface environment;
[0009] Data enhancement is performed on the ship remote sensing image dataset simulating complex sea surface environment; including: random rotation, random flipping, median filtering, mean filtering, dark channel prior image defogging;
[0010] A target detection model is constructed based on the RTMDet network model, wherein the convolution kernel of the CSPLayer module in the target detection model is set to a 5*5 common convolution kernel and a 7*7 dilated convolution kernel, which together constitute a depth-separable convolution with a receptive field of 23; a ship remote sensing image dataset simulating a complex sea environment is input into the target detection model for training;
[0011] The remote sensing image of the sea surface ship to be detected is input into the trained target detection model for detection, and the position and category confidence of the target in the interfered environment are obtained.
[0012] In a second aspect, an embodiment of the present invention provides an electronic device, comprising a memory and a processor, wherein the memory is coupled to the processor; wherein the memory is used to store program data, and the processor is used to execute the program data to implement the above-mentioned ship target detection method in a complex sea environment.
[0013] In a third aspect, an embodiment of the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the above-mentioned method for detecting ship targets in a complex sea environment.
[0014] In a fourth aspect, an embodiment of the present invention provides a computer program product, including a computer program / instruction, which, when executed by a processor, implements the above-mentioned method for detecting ship targets in a complex sea environment.
[0015] The beneficial effects of the present invention are as follows: the present invention performs data enhancement on a ship remote sensing image data set simulating a complex sea surface environment; including: random rotation, random flipping, median filtering, mean filtering, dark channel prior image defogging; and improves the RTMDet network model to construct a target detection model, so that the present invention can realize automatic detection of ship targets in a complex sea surface environment with high accuracy and detection speed. BRIEF DESCRIPTION OF THE DRAWINGS
[0016] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings in the following description are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative labor.
[0017] Figure 1 is a flow chart of a method for detecting a ship target in a complex sea environment according to an exemplary embodiment;
[0018] Figure 2 is a schematic diagram of an improved CSPLayer model according to an exemplary embodiment;
[0019] Figure 3 is a schematic diagram showing an example of a detection result according to an exemplary embodiment;
[0020] Figure 4 It is a schematic diagram of an electronic device. DETAILED DESCRIPTION
[0021] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0022] It should be noted that, in the absence of conflict, the features in the following embodiments and implementations may be combined with each other.
[0023] like Figure 1 As shown, an embodiment of the present invention provides a method for detecting ship targets in a complex sea environment, and the method specifically includes:
[0024] Step S1, obtaining a remote sensing image dataset of sea surface ships; adding cloud and fog occlusion, Gaussian noise and salt and pepper noise interference to the remote sensing image dataset of sea surface ships to obtain a remote sensing image dataset of ships simulating a complex sea surface environment.
[0025] Specifically, in this example, random cloud and fog occlusions are added to the high-definition sea surface ship remote sensing image dataset to simulate the atmospheric cloud and fog occlusions that the remote sensing satellite is subject to during the actual shooting process; random Gaussian noise and salt and pepper noise are added to simulate the interference of images caused by external factors such as electromagnetic waves and radiation during the operation of the remote sensing satellite.
[0026] Step S2, performing data enhancement on the ship remote sensing image dataset simulating a complex sea environment; including: random rotation, random flipping, median filtering, mean filtering, and dark channel prior image defogging.
[0027] Since various types of noise and interference act on the original image, in order to retain more target semantic information, image preprocessing operations are performed according to the scope of interference from small to large.
[0028] The median filtering algorithm is used to denoise the salt and pepper noise of some pixels. The expression is as follows:
[0029] output(i,j,k)=median(K(i-1:i+1,j-1:j+1)) (6)
[0030] Among them, output represents the output pixel value of the pixel, i and j represent the plane coordinates of the pixel, k represents a channel, median(·) represents the median operation, and K(i-1:i+1,j-1:j+1) represents the pixel value of the 3*3 neighborhood centered on (i,j) on the channel corresponding to k.
[0031] The mean filtering algorithm is used to denoise the Gaussian noise of all pixels. The expression is as follows:
[0032] output(i,j,k)=mean(K(i-1:i+1,j-1:j+1)) (7)
[0033] Where mean(·) is the operation for calculating the mean. If the mean is not an integer, it is rounded down.
[0034] The dark channel prior image defogging algorithm is used to denoise the fog in the whole image. By estimating the atmospheric light intensity A and transmittance t, the foggy image is restored to a fog-free or less foggy image. The model of the foggy image is:
[0035] I(x)=J(x)t(x)+A(1-t(x)) (8)
[0036] Among them, I(x) is the foggy image, J(x) is the original image, t(x) is the atmospheric transmittance, and A is the atmospheric light intensity.
[0037] The transmittance t is estimated using dark channel prior theory:
[0038]
[0039] in, To estimate the transmittance, ω is a constant parameter between 0 and 1, and is generally taken as 0.95 to retain a little fog to be close to the real image. x and y are both image pixels, Ω(x) is a 15*15 pixel block, and c∈{r,g,b}.
[0040] Estimate the atmospheric illumination intensity A: Select one of the three RGB channels and obtain the top 0.1% brightest pixels in its corresponding dark channel. The pixel values of these pixels corresponding to the pixels in the foggy image are taken as the atmospheric illumination intensity of the channel.
[0041] According to the transmittance t and the atmospheric illumination intensity A, the foggy image I is restored to the defogged image J.
[0042] Step S3, constructing a target detection model based on the RTMDet network model, wherein the convolution kernel of the CSPLayer module in the target detection model is set to a 5*5 ordinary convolution kernel and a 7*7 dilated convolution kernel, which together constitute a depth-separable convolution with a receptive field of 23; inputting a ship remote sensing image dataset simulating a complex sea environment into the target detection model for training.
[0043] Specifically, Figure 2 As shown in the figure, this example replaces the 1*1 and 3*3 ordinary convolution kernels in the original CSPLayer module with 5*5 ordinary convolution kernels and 7*7 dilated convolution kernels, which together form a depth-wise separable convolution with a receptive field of 23. Compared with the original receptive field of 5, it can extract more background information around the target to improve the accuracy of target detection. The calculation method of the receptive field is as follows:
[0044]
[0045] Among them, RF represents the receptive field size, i represents the serial number of the convolution kernel, and n represents the total number of convolution kernels. In this module, a depth-separable convolution contains two convolution kernels. i represents the size of the i-th convolution kernel, d i Represents the dilation degree of the i-th convolution kernel.
[0046] In order to ensure the consistency of data before and after convolution, that is, to ensure that the feature map of size (m, n, c) still maintains the original scale (m, n, c) after depthwise separable convolution to ensure the normal operation of the entire network, other parameters of the convolution kernel need to be set. The specific parameters for 5*5 convolution are: dilation degree 1, step size 1, padding number 2, number c; the specific parameters for 7*7 convolution kernel are: dilation degree 3, step size 1, padding number 9, number c.
[0047] At the same time, in order to build a more lightweight network, the number of network parameters is reduced by reducing the number of channels of the feature map and the number of repeated stacking of certain modules. The original RTMDet backbone network will output feature maps of three scales. For an image with an input size of (1024, 1024, 3), the three feature map scales obtained are (80, 80, 256), (40, 40, 512), and (20, 20, 1024). The number of channels of each feature map is reduced to 0.375 of the original, and the three scale feature maps obtained are (80, 80, 96), (40, 40, 192), and (20, 20, 384), which greatly reduces the number of parameters for model operations. At the same time, some modules such as CSPNeXtBlock will be repeatedly stacked 3 or 6 times. In order to reduce the amount of calculation, the stacking times of these repeatedly stacked modules are set to 1.
[0048] It should be noted that the present invention adjusts the convolution kernel of the CSPLayer module, introduces depth-separable convolution and dilated convolution and increases the size of the convolution kernel, thereby expanding the receptive field of the network model, improving the ability to extract background information, and improving the detection performance of targets in complex environments. At the same time, in order to ensure the efficiency of model training and reasoning, the number of parameters is reduced by reducing the number of channels of the feature map and the number of repeated stacking of certain modules, so as to obtain faster network training and reasoning speed.
[0049] Step S4: Input the remote sensing image of the sea surface ship to be detected into the trained target detection model for detection, and obtain the location and category confidence of the target in the interference environment, as well as the precision and recall rate of the same category and the mAP of all categories. The result diagram is shown in the figure below. Figure 3 shown.
[0050] Furthermore, the present invention compares the improved network model with the original network model, and the detection accuracy is improved, as shown in Table 1 below, which proves the effectiveness of the method.
[0051] Table 1: Ship detection results in complex sea conditions
[0052]
[0053]
[0054] Accordingly, the present application also provides an electronic device, comprising: one or more processors; a memory for storing one or more programs; when the one or more programs are executed by the one or more processors, the one or more processors implement the above-mentioned ship target detection method in a complex sea environment. Figure 4 As shown, it is a hardware structure diagram of any device with data processing capability in the method for detecting ship targets in a complex sea surface environment provided by an embodiment of the present invention, except Figure 4 In addition to the processor, memory and network interface shown, any device with data processing capability in which the apparatus in the embodiment is located may also include other hardware, generally based on the actual functions of the device with data processing capability, which will not be described in detail.
[0055] Accordingly, the present application also provides a computer-readable storage medium on which computer instructions are stored, and when the instructions are executed by a processor, the ship target detection method in a complex sea environment as described above is implemented. The computer-readable storage medium can be an internal storage unit of any device with data processing capabilities described in any of the aforementioned embodiments, such as a hard disk or a memory. The computer-readable storage medium can also be an external storage device, such as a plug-in hard disk, a smart memory card (Smart Media Card, SMC), an SD card, a flash card (Flash Card), etc. equipped on the device. Furthermore, the computer-readable storage medium can also include both an internal storage unit of any device with data processing capabilities and an external storage device. The computer-readable storage medium is used to store the computer program and other programs and data required by any device with data processing capabilities, and can also be used to temporarily store data that has been output or is to be output.
[0056] The above embodiments are only used to illustrate the design ideas and features of the present invention, and their purpose is to enable those skilled in the art to understand the content of the present invention and implement it accordingly. The protection scope of the present invention is not limited to the above embodiments. Therefore, any equivalent changes or modifications made based on the principles and design ideas disclosed by the present invention are within the protection scope of the present invention.
Claims
1. A method for detecting ship targets in a complex sea environment, characterized in that: The method specifically comprises: Acquire a remote sensing image dataset of sea surface ships; add cloud and fog occlusion, Gaussian noise and salt and pepper noise interference to the remote sensing image dataset of sea surface ships to obtain a remote sensing image dataset of ships simulating a complex sea surface environment; Data enhancement is performed on the ship remote sensing image dataset simulating complex sea surface environment; including: random rotation, random flipping, median filtering, mean filtering, dark channel prior image defogging; A target detection model is constructed based on the RTMDet network model, wherein the convolution kernel of the CSPLayer module in the target detection model is set to a 5*5 common convolution kernel and a 7*7 dilated convolution kernel, which together constitute a depth-separable convolution with a receptive field of 23; a ship remote sensing image dataset simulating a complex sea environment is input into the target detection model for training; The remote sensing image of the sea surface ship to be detected is input into the trained target detection model for detection, and the position and category confidence of the target in the interfered environment are obtained.
2. The method for detecting ship targets in a complex sea environment according to claim 1, characterized in that: The process of data enhancement for the ship remote sensing image dataset simulating a complex sea environment includes: The median filtering algorithm is used to denoise the salt and pepper noise of some pixels. The expression is as follows: output(i,j,k)=median(K(i-1:i+1,j-1:j+1)) (1) Where output represents the output pixel value of the pixel, i and j represent the plane coordinates of the pixel, k represents a channel, median(·) represents the median operation, and K(i-1:i+1,j-1:j+1) represents the pixel value of the 3*3 neighborhood centered on (i,j) on the channel corresponding to k; The mean filtering algorithm is used to denoise the Gaussian noise of all pixels. The expression is as follows: output(i,j,k)=mean(K(i-1:i+1,j-1:j+1)) (2) Where mean(·) is the mean calculation operation, and when the mean is not an integer, it is rounded down; The dark channel prior image defogging algorithm is used to denoise the fog in the whole image. By estimating the atmospheric light intensity A and transmittance t, the foggy image is restored to a fog-free or less foggy image. The model of the foggy image is: I(x)=J(x)t(x)+A(1-t(x)) (3) Among them, I(x) is the foggy image, J(x) is the original image, t(x) is the atmospheric transmittance, and A is the atmospheric light intensity. The transmittance t is estimated using dark channel prior theory: in, To estimate the transmittance, ω is a constant parameter between 0 and 1, and is generally set to 0.95 to retain a little fog to be close to the real image. x and y are both image pixels, Ω(x) is a 15*15 pixel block, and c∈{r,g,b}; Estimate the atmospheric illumination intensity A: Select one of the three RGB channels and obtain the first K brightest pixels of its corresponding dark channel. The pixel values of these pixels corresponding to the pixels of the foggy image are used as the atmospheric illumination intensity of the channel. According to the transmittance t and the atmospheric illumination intensity A, the foggy image I is restored to the defogged image J.
3. The method for detecting ship targets in a complex sea environment according to claim 1, characterized in that: The receptive field is calculated as follows: Among them, RF represents the receptive field size, i represents the serial number of the convolution kernel, n represents the total number of convolution kernels, and k represents the total number of convolution kernels. i represents the size of the i-th convolution kernel, d i Represents the dilation degree of the i-th convolution kernel.
4. The method for detecting ship targets in a complex sea environment according to claim 1, characterized in that: The parameters of the 5*5 ordinary convolution kernel are: dilation degree 1, step size 1, padding number 2, and number c; the parameters of the 7*7 dilated convolution kernel are: dilation degree 3, step size 1, padding number 9, and number c.
5. The method for detecting ship targets in a complex sea environment according to claim 1, characterized in that: Reduce the number of channels of the feature map output by the target detection model.
6. The method for detecting ship targets in a complex sea environment according to claim 1, characterized in that: Set the number of stacking of the CSPNeXtBlock module in the target detection model to 1.
7. An electronic device comprising a memory and a processor, characterized in that: The memory is coupled to the processor; wherein the memory is used to store program data, and the processor is used to execute the program data to implement the ship target detection method in a complex sea environment as described in any one of claims 1-6 above.
8. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method for detecting a ship target in a complex sea environment as described in any one of claims 1 to 6 is implemented.
9. A computer program product comprising a computer program / instructions, characterized in that When the computer program / instruction is executed by a processor, the method for detecting a ship target in a complex sea environment as described in any one of claims 1 to 6 is implemented.
Citation Information
Patent Citations
Marine vessel target detection method and system based on improved YOLOX
CN116740572A
Cutter damage target detection method based on improved RTMDet
CN117197427A
Ship remote sensing image detection and classification method based on improved RTMdet
CN117671343A
Lightweight remote sensing image target detection method fused with large kernel attention convolution
CN117788996A
Cited By
Video and remote sensing fused spartina alterniflora intrusion detection method, equipment and medium
CN119763011A
