A method and device for ship target detection based on a light small twin network

Abstract

The application discloses a method for realizing ship target detection based on a light small twin network, and comprises the following steps: using a dark channel prior algorithm to perform defogging processing on a to-be-detected ship image; inputting the to-be-detected ship image after defogging processing and a target feature template into a twin full convolution network model, so that the twin full convolution network model outputs a target detection result of the to-be-detected ship image according to the target feature template; wherein the twin full convolution network model is obtained by training a ship image training set, and the target feature template is obtained by testing a ship image test set by using the twin full convolution network model after the twin full convolution network model is trained, and then performing credible feature fusion based on a test result. The application can accurately and timely detect complex and changeable ship targets on an embedded hardware platform.

Description

Technical Field

[0001] This invention belongs to the field of artificial intelligence, specifically relating to a method and apparatus for ship target detection based on a lightweight twin network. Background Technology

[0002] Maritime electro-optical reconnaissance systems are gradually shifting from manual to autonomous detection. Autonomous detection methods primarily rely on intelligent target detection based on convolutional neural networks, whose high accuracy meets the daily operational requirements of maritime military equipment. However, with the rapid development of military equipment technology, ship targets are becoming increasingly diverse in shape and size. Existing intelligent target detection methods, with their pre-trained convolutional models, cannot directly perform accurate detection and require retraining using newly collected ship target datasets. Furthermore, maritime electro-optical reconnaissance systems are often deployed on embedded hardware platforms, demanding extremely high detection speeds. The massive number of model parameters and diverse model architectures of intelligent target detection methods make it difficult to deploy them on embedded hardware platforms for energy-efficient computing. Therefore, how to accurately and in real-time detect complex and ever-changing ship targets on embedded hardware platforms has become a key research focus and challenge for maritime electro-optical reconnaissance systems. Summary of the Invention

[0003] To address the aforementioned problems in the existing technology, this invention provides a method and apparatus for ship target detection based on a lightweight twin network.

[0004] The technical problem to be solved by this invention is achieved through the following technical solution:

[0005] A method for ship target detection based on a lightweight twin network includes:

[0006] The dark channel prior algorithm is used to dehaze the images of the ships under test.

[0007] The dehazed image of the ship under test and the target feature template are input into the Siamese fully convolutional network model, so that the Siamese fully convolutional network model outputs the target detection result of the image of the ship under test according to the target feature template;

[0008] The twin fully convolutional network model is trained using a ship image training set. The target feature template is obtained by testing the ship image test set using the twin fully convolutional network model after it has been trained, and by performing reliable feature fusion based on the test results.

[0009] Optionally, the step of performing dehazing on the ship image under test using a dark channel prior algorithm is implemented by the following formula:

[0010]

[0011] Where I(x) represents the image of the ship under test before processing, x is the number of pixels, J(x) represents the image of the ship under test after dehazing, A represents the atmospheric light value, and t0 is the transmittance threshold. This represents the estimated transmittance for pixel x, ω∈[0,1] is the correction factor, Ω(x) represents an arbitrary window centered at pixel x, c∈{r,g,b}, where r,g,b represent the R, G, and B channels respectively, and A c I represents the atmospheric light value converted to channel c. c (y) represents the pixel value of the image of the ship under test in the c channel.

[0012] Optionally, the target detection result includes: the candidate region box corresponding to each pixel in the image of the ship to be tested, and the similarity between the image in each candidate region box and the target feature template;

[0013] The methods for obtaining the target feature template by performing reliable feature fusion based on test results include:

[0014] For each test sample in the ship image test set, a candidate region box corresponding to the pixel with the highest similarity is selected from its target detection results. The corresponding feature is then obtained from this candidate region box according to the target feature template. This feature is added to the feature pool, and the target feature template is updated using this feature. The update method is as follows:

[0015]

[0016]

[0017] Among them, F new_result η represents the acquired features, η represents the learning rate, and ω represents the learning rate. k F represents the weight of the k-th feature in the feature pool, n represents the original number of features in the feature pool, and F k Let C represent the k-th feature in the original feature pool, and F represent the update count. final This represents the updated target feature template, and N represents the upper limit of the number of features.

[0018] Optionally, the Siamese fully convolutional network model includes: a first AlexNet backbone network, a second AlexNet backbone network, a first conv3×3 classification network, a second conv3×3 classification network, a first conv3×3 regression network, a second conv3×3 regression network, a first DW convolutional network, a second DW convolutional network, a classification prediction network, and a candidate region box prediction network; wherein,

[0019] The output of the first AlexNet backbone network is connected to the inputs of the first conv3×3 classification network and the first conv3×3 regression network, respectively;

[0020] The output of the second AlexNet backbone network is connected to the inputs of the second conv3×3 classification network and the second conv3×3 regression network, respectively;

[0021] The outputs of the first conv3×3 classification network and the first conv3×3 regression network are both connected to the inputs of the first DW convolutional network and the second DW convolutional network;

[0022] The outputs of the second conv3×3 classification network and the second conv3×3 regression network are both connected to the inputs of the first DW convolutional network and the second DW convolutional network;

[0023] The output of the first DW convolutional network is connected to the input of the classification prediction network;

[0024] The output of the second DW convolutional network is connected to the input of the candidate region box prediction network;

[0025] The candidate region box prediction network outputs the candidate region box corresponding to each pixel in the image of the ship to be tested, and the classification prediction network outputs the similarity between the image in each candidate region box and the target feature template.

[0026] Optionally, the ship image training set is a visible light ship image training set; the ship image to be tested includes visible light ship images or infrared ship images.

[0027] The present invention also provides a device for ship target detection based on a lightweight twin network, comprising:

[0028] The preprocessing module is used to perform dehazing on the images of the ships under test using the dark channel prior algorithm;

[0029] The target detection module is used to input the dehazed image of the ship under test and the target feature template into the Siamese fully convolutional network model, so that the Siamese fully convolutional network model outputs the target detection result of the image of the ship under test according to the target feature template;

[0030] The twin fully convolutional network model is trained using a ship image training set. The target feature template is obtained by testing the ship image test set using the twin fully convolutional network model after it has been trained, and by performing reliable feature fusion based on the test results.

[0031] Optionally, the preprocessing module's operation of dehazing the ship image using the dark channel prior algorithm is implemented through the following formula:

[0032]

[0033] Where I(x) represents the image of the ship under test before processing, x is the number of pixels, J(x) represents the image of the ship under test after dehazing, A represents the atmospheric light value, and t0 is the transmittance threshold. This represents the estimated transmittance for pixel x, ω∈[0,1] is the correction factor, Ω(x) represents an arbitrary window centered at pixel x, c∈{r,g,b}, where r,g,b represent the R, G, and B channels respectively, and A c I represents the atmospheric light value converted to channel c. c (y) represents the pixel value of the image of the ship under test in the c channel.

[0034] Optionally, the target detection result includes: candidate region boxes corresponding to each point in the image of the ship to be tested, and the similarity between the image in each candidate region box and the target feature template;

[0035] The methods for obtaining the target feature template by performing reliable feature fusion based on test results include:

[0036] For each test sample in the ship image test set, a candidate region box corresponding to the point with the highest similarity is selected from its target detection results. The corresponding features are then obtained from this candidate region box according to the features contained in the target feature template. These features are added to the feature pool, and the target feature template is updated using these features. The update method is as follows:

[0037]

[0038]

[0039] Among them, F new_result η represents the acquired features, η represents the learning rate, and ω represents the learning rate. k F represents the weight of the k-th feature in the feature pool, n represents the original number of features in the feature pool, and F k Let C represent the k-th feature in the original feature pool, and F represent the update count. final This represents the updated target feature template, and N represents the upper limit of the number of features.

[0040] Optionally, the Siamese fully convolutional network model includes: a first AlexNet backbone network, a second AlexNet backbone network, a first conv3×3 classification network, a second conv3×3 classification network, a first conv3×3 regression network, a second conv3×3 regression network, a first DW convolutional network, a second DW convolutional network, a classification prediction network, and a candidate region box prediction network; wherein,

[0041] The output of the first AlexNet backbone network is connected to the inputs of the first conv3×3 classification network and the first conv3×3 regression network, respectively;

[0042] The output of the second AlexNet backbone network is connected to the inputs of the second conv3×3 classification network and the second conv3×3 regression network, respectively;

[0043] The outputs of the first conv3×3 classification network and the first conv3×3 regression network are both connected to the inputs of the first DW convolutional network and the second W convolutional network;

[0044] The outputs of the second conv3×3 classification network and the second conv3×3 regression network are both connected to the inputs of the first DW convolutional network and the second DW convolutional network;

[0045] The output of the first DW convolutional network is connected to the input of the classification prediction network;

[0046] The output of the second DW convolutional network is connected to the input of the candidate region box prediction network;

[0047] The candidate region box prediction network outputs the candidate region box corresponding to each pixel in the image of the ship to be tested, and the classification prediction network outputs the similarity between the image in each candidate region box and the target feature template.

[0048] Optionally, the ship image training set is a visible light ship image training set; the ship image to be tested includes visible light ship images or infrared ship images.

[0049] The method for ship target detection based on a lightweight twin network provided by this invention utilizes a dark channel prior method to dehaze the ship image under test. This reduces the impact of absorption and scattering by water vapor and particles in the atmosphere at sea, which leads to low contrast and dull colors in remote sensing images, thus improving the quality of remote sensing images and helping to improve the prediction accuracy of the model. This invention performs feature fusion based on the credible features of the target detection results, avoiding the reduced detection robustness caused by using fixed feature templates when the shape and size of the ship target change drastically, and improving the false negative rate. This invention uses a lightweight twin fully convolutional network to solve the problem of ship target detection methods being difficult to deploy and running slowly on embedded hardware platforms.

[0050] The present invention will now be described in further detail with reference to the accompanying drawings. Attached Figure Description

[0051] Figure 1 This is a flowchart of a method for ship target detection based on a lightweight twin network provided by an embodiment of the present invention;

[0052] Figure 2 yes Figure 1 A schematic diagram of the structure of the twin fully convolutional network model used in the method shown;

[0053] Figure 3 yes Figure 2 A schematic diagram of the classification prediction network and the candidate region box prediction network in the model shown;

[0054] Figure 4 This is a schematic diagram of obtaining the target feature template through trusted feature fusion in this invention. Detailed Implementation

[0055] The present invention will be further described in detail below with reference to specific embodiments, but the implementation of the present invention is not limited thereto.

[0056] To enable accurate and real-time detection of complex and ever-changing ship targets on embedded hardware platforms, this invention provides a method for ship target detection based on a lightweight twin network, such as... Figure 1 As shown, the method includes the following steps:

[0057] S10. Use the dark channel prior algorithm to dehaze the image of the ship under test.

[0058] Typically, in ocean scenes, at least one channel has pixels close to 0; these channels are called dark channels. Therefore, the dark channel prior method can be expressed as:

[0059]

[0060] Among them, J dark (x) represents the dark channel pixel value of the image, Ω(x) represents an arbitrary window centered at pixel x, c∈{r,g,b}, where r,g,b represent the R, G, and B channels respectively, I c (y) represents the pixel value of the image in the c channel.

[0061] Clouds and water vapor are suspended particles in the atmosphere. When these suspended particles are uniformly distributed in the atmosphere, the atmospheric scattering model for fog images is as follows:

[0062] I(x)=J(x)t(x)+A(1-t(x)) (Formula 2)

[0063] Where I(x) represents the foggy image, J(x) represents the fog-free image to be recovered, A represents the atmospheric light value, and t(x) represents the transmittance, i.e., the portion of light that is not scattered and reaches the image sensor. Equation 2 can be modified as follows:

[0064]

[0065] Among them, I c (x) represents the pixel value of the ship image under test in the c channel, A c This represents the atmospheric light value converted to channel c.

[0066] Assuming the global atmospheric light value A is known, and the transmittance t(x) of each window is constant, let it be... Then, by performing the minimum value operation twice on both sides of equation 2, we obtain equation 4:

[0067]

[0068] Equation 5 can be derived from Equation 1:

[0069]

[0070] Substituting Equation 5 into Equation 4, we obtain the estimated transmittance for pixel x.

[0071]

[0072] Considering the depth of field of the image, a correction factor ω∈[0,1] is introduced. After correction, the corrected transmittance as follows:

[0073]

[0074] ω can be 0.95, but it is not limited to this value.

[0075] In this embodiment of the invention, the dark channel plot J darkThe brightest pixels in the top 0.1% are mapped to the hazy image I(x), and the value of the brightest point is extracted as the global atmospheric light value A. Furthermore, due to transmittance... A smaller value will directly result in a larger value of J(x) in the haze-free image to be recovered. Therefore, a threshold t0 is set, for example, t0 = 0.1. A, Substituting these values ​​into Equation 1 yields the final image after dehazing. Therefore, the step of using the dark channel prior algorithm to dehaze the image of the ship under test in this embodiment of the invention can be achieved by the following formula:

[0076]

[0077] At this point, I(x) represents the image of the ship under test before processing, where x is a pixel; J(x) represents the image of the ship under test after dehazing; A represents the atmospheric light value; and t0 is the transmittance threshold. Let Ω(x) represent the estimated transmittance for pixel x, ω∈[0,1] be a correction factor, Ω(x) represent an arbitrary window centered at pixel x, c∈{r,g,b}, where r,g,b represent the R, G, and B channels respectively, and A c I represents the atmospheric light value converted to channel c. c (y) represents the pixel value of the image of the ship under test in the c channel.

[0078] S20. Input the dehazed ship image and target feature template into the Siamese fully convolutional network model so that the Siamese fully convolutional network model can output the target detection result of the ship image based on the target feature template.

[0079] The target detection results include: candidate region boxes corresponding to each pixel in the image of the ship under test, and the similarity between the image within each candidate region box and the target feature template. Understandably, the candidate region box corresponding to the pixel with the highest similarity can accurately identify the target.

[0080] In this embodiment of the invention, the Siamese fully convolutional network model is trained using a training set of ship images. See also... Figure 2 As shown, the Siamese fully convolutional network model includes: a first AlexNet backbone network, a second AlexNet backbone network, a first conv3×3 classification network, a second conv3×3 classification network, a first conv3×3 regression network, a second conv3×3 regression network, a first DW convolutional network, a second DW convolutional network, a classification prediction network, and a candidate region box prediction network.

[0081] Specifically, the output of the first AlexNet backbone network is connected to the inputs of the first conv3×3 classification network and the first conv3×3 regression network, respectively; the output of the second AlexNet backbone network is connected to the inputs of the second conv3×3 classification network and the second conv3×3 regression network, respectively; the outputs of the first conv3×3 classification network and the first conv3×3 regression network are both connected to the inputs of the first DW convolutional network and the second DW convolutional network; the outputs of the second conv3×3 classification network and the second conv3×3 regression network are both connected to the inputs of the first DW convolutional network and the second DW convolutional network; the output of the first DW convolutional network is connected to the input of the classification prediction network; the output of the second DW convolutional network is connected to the input of the candidate region box prediction network; the candidate region box prediction network outputs the candidate region box corresponding to each pixel in the ship image to be tested, and the classification prediction network outputs the similarity between the image in each candidate region box and the target feature template.

[0082] The structures of the candidate region bounding box prediction network and the classification prediction network can be found in [reference needed]. Figure 3 . Figure 3 In this context, Conv3×3 represents a 3×3 convolutional layer, Conv1×1 represents a 1×1 convolutional layer, and Relu represents the Relu activation function.

[0083] based on Figure 2 and Figure 3 The network structure shown demonstrates the training of a Siamese fully convolutional network model using a ship image training set. The ship image training set comprises multiple training samples, each a single ship image, with each sample labeled with the original bounding box indicating the location of a target. Points within the bounding box correspond to the target category, while points outside the box do not. During training, the model parameters are initialized to 1. For each training sample input to the Siamese fully convolutional network model, the model outputs a set of predicted target detection results. The model loss is calculated based on the difference between the predicted results and the labeled information. The model parameters are adjusted based on the loss to gradually converge the model towards a convergent state. The trained Siamese fully convolutional network model is obtained when the model loss gradually converges below a threshold, or when a sufficient number of training iterations are performed.

[0084] The model loss is calculated as follows:

[0085]

[0086] Where, p x,y This represents the predicted probability of a candidate region bounding box centered at point (x, y) in the classification feature map output by the classification prediction network. This indicates that the point (x, y) maps to the ground truth point on the input image of the model, tx,y This represents the candidate region box centered at point (x, y) in the regression feature map output by the candidate region box prediction network. L represents the original bounding box centered at point (x, y) in the ship images included in the training samples. cls This indicates the calculation of Focal loss, which is used to measure the classification loss of the model; L reg This represents the IOU loss, which measures the regression loss of the model. N pos This represents the number of positive samples. In practice, if the Intersection over Union (IOU) between the candidate bounding box predicted by the model for the training sample and the original labeled bounding box of the training sample is greater than 0.5, then the sample is a positive sample; otherwise, it is a negative sample. λ = 1 represents the balancing weight factor of the IOU loss. Indicates an indicator function, if If the function value is 1, then the function value is 0; otherwise, the function value is 0.

[0087] After training the twin fully convolutional network model, the twin fully convolutional network model is tested using a ship image test set, and reliable feature fusion is performed based on the test results to obtain the target feature template used in step S20.

[0088] For details, see Figure 4 Methods for obtaining target feature templates through reliable feature fusion based on target detection results include:

[0089] For each test sample in the ship image test set, a candidate region box corresponding to the point with the highest similarity is selected from the target detection results. The corresponding features are obtained from the candidate region box according to the features contained in the target feature template. The feature is added to the feature pool and the target feature template is updated using the feature.

[0090] The selection of the candidate region box with the highest score in the object detection results is represented as follows:

[0091]

[0092] B curr =B[x * ];

[0093] In the above formula, x * It is the point with the highest similarity, B[x] * ] represents x * The corresponding candidate region bounding box, Represents the similarity of point x. This represents the coordinate space of the similarity score graph, which is composed of the similarity scores of each point x.

[0094] In this embodiment of the invention, the similarity between the image in the candidate region box and the target feature template can be determined by the Euclidean distance between them.

[0095] Then, the target feature template is updated using the acquired features as follows:

[0096]

[0097]

[0098] Among them, F new_result η represents the acquired features, η represents the learning rate, and ω represents the learning rate. k F represents the weight of the k-th feature in the feature pool, where n represents the original number of features in the feature pool. k Let C represent the k-th feature in the feature pool, and F represent the number of updates. final This represents the updated target feature template, and N represents the upper limit of the number of features.

[0099] Therefore, by using the trained Siamese fully convolutional network model and the target feature template obtained through fusion and updating, target detection can be performed on new ship images.

[0100] In this embodiment of the invention, the ship image training set can be a visible light ship image training set. After training the twin fully convolutional network model with the visible light ship image training set, the model can be used not only for the detection of visible light ship images but also for the detection of infrared ship images. This is due to the dehazing process performed on the dark channel prior algorithm in step S10 and the target feature template used in step S20.

[0101] The embodiments of the present invention will be further illustrated below through simulation experiments.

[0102] In the experiment, a ship image training set was built, consisting of 506 ship images and 859 instances. The training set contained 1042 ship images, the validation set contained 232 ship images, and the test set contained 232 ship images. All images were scaled to 4800×2747, and the target feature template was a template image whose scale was determined relative to the actual size of the target in the ship image being tested. The model was loaded onto an NVIDIA GTX 1660Ti GPU for testing. The test results are shown in Table 1. It can be seen that the model's detection accuracy was 92.8%, and the computation speed was 50 FPS. Furthermore, the model also achieved an accuracy of over 90% for infrared ship image detection, demonstrating the effectiveness of this invention.

[0103] Table 1

[0104]

[0105] In summary, the embodiments of the present invention utilize the dark channel prior method to dehaze the images of the ships under test, reducing the impact of absorption and scattering by water vapor and particles in the atmosphere in the sea scene, which leads to low contrast and dull colors in the remote sensing images, thereby improving the quality of the remote sensing images and helping to improve the prediction accuracy of the model.

[0106] The embodiments of the present invention perform feature fusion based on the reliable features of the target detection results, which avoids the reduction in detection robustness caused by using a fixed feature template when the shape and size of the ship target changes drastically, and improves the detection false detection rate.

[0107] This invention, based on the network model in the SiamFC++ algorithm, improves the network architecture according to the uneven distribution of features in multi-target images. The improved Siamese fully convolutional network model can fully utilize the computing resources of DSP (Digital Signal Processing) chips to achieve efficient computation. Compared to existing technologies that mostly use multi-type custom operator layers for ship target detection, the Siamese fully convolutional network used in this invention has a simpler structure and therefore faster computation speed, solving the problem of existing ship target detection methods being difficult to deploy and running slowly on embedded hardware platforms.

[0108] The method provided in this invention can be applied to electronic devices. Specifically, the electronic device can be a desktop computer, a portable computer, a smart mobile terminal, a server, etc. No limitation is made herein; any electronic device that can implement this invention falls within the protection scope of this invention.

[0109] Based on the same inventive concept, embodiments of the present invention also provide a device for ship target detection based on a lightweight twin network, comprising:

[0110] The preprocessing module is used to perform dehazing on the images of the ships under test using the dark channel prior algorithm;

[0111] The target detection module is used to input the dehazed image of the ship under test and the target feature template into the Siamese fully convolutional network model, so that the Siamese fully convolutional network model can output the target detection result of the image of the ship under test according to the target feature template;

[0112] The twin fully convolutional network model is trained using the ship image training set. The target feature template is obtained by testing the ship image test set using the trained twin fully convolutional network model and performing reliable feature fusion based on the test results.

[0113] Optionally, the preprocessing module performs dehazing on the ship image using a dark channel prior algorithm, which is achieved through the following formula:

[0114]

[0115] Where I(x) represents the image of the ship under test before processing, x is the number of pixels, J(x) represents the image of the ship under test after dehazing, A represents the atmospheric light value, and t0 is the transmittance threshold. This represents the estimated transmittance for pixel x, ω∈[0,1] is the correction factor, Ω(x) represents an arbitrary window centered at pixel x, c∈{r,g,b}, where r,g,b represent the R, G, and B channels respectively, and A c I represents the atmospheric light value converted to channel c. c (y) represents the pixel value of the image of the ship under test in the c channel.

[0116] Optionally, the target detection results include: the candidate region bounding box corresponding to each point in the image of the ship to be tested, and the similarity between the image in each candidate region bounding box and the target feature template;

[0117] Methods for obtaining target feature templates through reliable feature fusion based on test results include:

[0118] For each test sample in the ship image test set, a candidate region box corresponding to the point with the highest similarity is selected from the target detection results. The corresponding features are then obtained from this candidate region box according to the features contained in the target feature template. These features are added to the feature pool, and the target feature template is updated using these features. The update method is as follows:

[0119]

[0120]

[0121] Among them, F new_result η represents the acquired features, η represents the learning rate, and ω represents the learning rate. k F represents the weight of the k-th feature in the feature pool, where n represents the original number of features in the feature pool. k Let C represent the k-th feature in the feature pool, and F represent the number of updates. final This represents the updated target feature template, and N represents the upper limit of the number of features.

[0122] Optionally, the Siamese fully convolutional network model includes: a first AlexNet backbone network, a second AlexNet backbone network, a first conv3×3 classification network, a second conv3×3 classification network, a first conv3×3 regression network, a second conv3×3 regression network, a first DW convolutional network, a second DW convolutional network, a classification prediction network, and a candidate region bounding box prediction network; wherein,

[0123] The output of the first AlexNet backbone network is connected to the inputs of the first conv3×3 classification network and the first conv3×3 regression network, respectively.

[0124] The output of the second AlexNet backbone network is connected to the inputs of the second conv3×3 classification network and the second conv3×3 regression network, respectively.

[0125] The outputs of the first conv3×3 classification network and the first conv3×3 regression network are both connected to the inputs of the first DW convolutional network and the second W convolutional network;

[0126] The outputs of the second conv3×3 classification network and the second conv3×3 regression network are both connected to the inputs of the first DW convolutional network and the second DW convolutional network;

[0127] The output of the first DW convolutional network is connected to the input of the classification prediction network;

[0128] The output of the second DW convolutional network is connected to the input of the candidate region box prediction network;

[0129] The candidate region bounding box prediction network outputs the candidate region bounding box corresponding to each pixel in the image of the ship under test, and the classification prediction network outputs the similarity between the image in each candidate region bounding box and the target feature template.

[0130] Optionally, the ship image training set is a visible light ship image training set; the ship images to be tested include visible light ship images to be tested or infrared ship images.

[0131] As the device embodiment is basically similar to the method embodiment, the description is relatively simple, and relevant parts can be found in the description of the method embodiment.

[0132] It should be noted that the terms "first," "second," etc., are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this disclosure described herein can be implemented in orders other than those illustrated or described herein. The implementations described in the following exemplary embodiments do not represent all implementations consistent with this disclosure. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this disclosure.

[0133] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the present invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features or characteristics described may be combined in any suitable manner in one or more embodiments or examples. In addition, those skilled in the art can combine and integrate the different embodiments or examples described in this specification.

[0134] Although this application has been described herein in conjunction with various embodiments, those skilled in the art, by reviewing the accompanying drawings and the disclosure, will understand and implement other variations of the disclosed embodiments in carrying out the claimed application. In the description of this invention, the word "comprising" does not exclude other components or steps, "a" or "an" does not exclude a plurality, and "a plurality" means two or more, unless otherwise explicitly specified. Furthermore, while different embodiments may describe certain measures, this does not mean that these measures cannot be combined to produce good results.

[0135] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (devices), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0136] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0137] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0138] The above description, in conjunction with specific preferred embodiments, provides a further detailed explanation of the present invention. It should not be construed that the specific implementation of the present invention is limited to these descriptions. For those skilled in the art, various simple deductions or substitutions can be made without departing from the concept of the present invention, and all such modifications and substitutions should be considered within the scope of protection of the present invention.

Claims

1. A method for ship target detection based on a lightweight twin network, characterized in that, include: The dark channel prior algorithm is used to dehaze the images of the ships under test. The dehazed image of the ship under test and the target feature template are input into the Siamese fully convolutional network model, so that the Siamese fully convolutional network model outputs the target detection result of the image of the ship under test according to the target feature template; The twin fully convolutional network model is trained using a ship image training set. The target feature template is obtained by testing the ship image test set with the twin fully convolutional network model after it has been trained, and by performing reliable feature fusion based on the test results. The twin fully convolutional network model includes: a first AlexNet backbone network, a second AlexNet backbone network, a first conv3×3 classification network, a second conv3×3 classification network, a first conv3×3 regression network, a second conv3×3 regression network, a first DW convolutional network, a second DW convolutional network, a classification prediction network, and a candidate region bounding box prediction network; wherein, The output of the first AlexNet backbone network is connected to the inputs of the first conv3×3 classification network and the first conv3×3 regression network, respectively; The output of the second AlexNet backbone network is connected to the inputs of the second conv3×3 classification network and the second conv3×3 regression network, respectively; The outputs of the first conv3×3 classification network and the first conv3×3 regression network are both connected to the inputs of the first DW convolutional network and the second DW convolutional network; The outputs of the second conv3×3 classification network and the second conv3×3 regression network are both connected to the inputs of the first DW convolutional network and the second DW convolutional network; The output of the first DW convolutional network is connected to the input of the classification prediction network; The output of the second DW convolutional network is connected to the input of the candidate region box prediction network; The candidate region box prediction network outputs the candidate region box corresponding to each pixel in the image of the ship to be tested, and the classification prediction network outputs the similarity between the image in each candidate region box and the target feature template.

2. The method according to claim 1, characterized in that, The step of using the dark channel prior algorithm to dehaze the image of the ship under test is implemented by the following formula: ； in, This represents the image of the ship under test before processing. For pixels, This represents the image of the ship under test after defogging. Indicates atmospheric light value, The transmittance threshold, Indicates a pixel Estimated transmittance As a correction factor, Represented by pixels Any window centered on the center , These represent the R, G, and B channels, respectively. Indicates conversion to Atmospheric light value of the channel, Indicates the image of the ship under test in The pixel value of the channel.

3. The method according to claim 1, characterized in that, The target detection results include: the candidate region bounding box corresponding to each pixel in the image of the ship to be tested, and the similarity between the image in each candidate region bounding box and the target feature template; The methods for obtaining the target feature template by performing reliable feature fusion based on test results include: For each test sample in the ship image test set, a candidate region box corresponding to the pixel with the highest similarity is selected from its target detection results. The corresponding feature is then obtained from this candidate region box according to the target feature template. This feature is added to the feature pool, and the target feature template is updated using this feature. The update method is as follows: ， ； in, Indicates the acquired features, Indicates the learning rate. Indicates the first feature in the feature pool k The weights of each feature n This indicates the original number of features in the feature pool. This indicates the original number of features in the feature pool. k One characteristic, C Indicates the number of updates. This represents the updated target feature template. Indicates the upper limit of the number of features.

4. The method according to claim 1, characterized in that, The ship image training set is a visible light ship image training set; the ship images to be tested include visible light ship images or infrared ship images.

5. A device for ship target detection based on a lightweight twin network, characterized in that, include: The preprocessing module is used to perform dehazing on the images of the ships under test using the dark channel prior algorithm; The target detection module is used to input the dehazed image of the ship under test and the target feature template into the Siamese fully convolutional network model, so that the Siamese fully convolutional network model outputs the target detection result of the image of the ship under test according to the target feature template; The twin fully convolutional network model is trained using a ship image training set. The target feature template is obtained by testing the ship image test set with the twin fully convolutional network model after it has been trained, and by performing reliable feature fusion based on the test results. The twin fully convolutional network model includes: a first AlexNet backbone network, a second AlexNet backbone network, a first conv3×3 classification network, a second conv3×3 classification network, a first conv3×3 regression network, a second conv3×3 regression network, a first DW convolutional network, a second DW convolutional network, a classification prediction network, and a candidate region bounding box prediction network; wherein, The output of the first AlexNet backbone network is connected to the inputs of the first conv3×3 classification network and the first conv3×3 regression network, respectively; The output of the second AlexNet backbone network is connected to the inputs of the second conv3×3 classification network and the second conv3×3 regression network, respectively; The outputs of the first conv3×3 classification network and the first conv3×3 regression network are both connected to the inputs of the first DW convolutional network and the second DW convolutional network; The outputs of the second conv3×3 classification network and the second conv3×3 regression network are both connected to the inputs of the first DW convolutional network and the second DW convolutional network; The output of the first DW convolutional network is connected to the input of the classification prediction network; The output of the second DW convolutional network is connected to the input of the candidate region box prediction network; The candidate region box prediction network outputs the candidate region box corresponding to each pixel in the image of the ship to be tested, and the classification prediction network outputs the similarity between the image in each candidate region box and the target feature template.

6. The apparatus according to claim 5, characterized in that, The preprocessing module uses a dark channel prior algorithm to dehaze the image of the ship under test. This is achieved through the following formula: ； in, This represents the image of the ship under test before processing. For pixels, This represents the image of the ship under test after defogging. Indicates atmospheric light value, The transmittance threshold, Indicates a pixel Estimated transmittance As a correction factor, Represented by pixels Any window centered on the center , These represent the R, G, and B channels, respectively. Indicates conversion to Atmospheric light value of the channel, Indicates the image of the ship under test in The pixel value of the channel.

7. The apparatus according to claim 5, characterized in that, The target detection results include: candidate region boxes corresponding to each point in the image of the ship to be tested, and the similarity between the image in each candidate region box and the target feature template; The methods for obtaining the target feature template by performing reliable feature fusion based on test results include: For each test sample in the ship image test set, a candidate region box corresponding to the point with the highest similarity is selected from its target detection results. The corresponding features are then obtained from this candidate region box according to the features contained in the target feature template. These features are added to the feature pool, and the target feature template is updated using these features. The update method is as follows: ， ； in, Indicates the acquired features, Indicates the learning rate. Indicates the first feature in the feature pool k The weights of each feature n This indicates the original number of features in the feature pool. This indicates the original number of features in the feature pool. k One characteristic, C Indicates the number of updates. This represents the updated target feature template. Indicates the upper limit of the number of features.

8. The apparatus according to claim 5, characterized in that, The ship image training set is a visible light ship image training set; the ship images to be tested include visible light ship images or infrared ship images.

Images