Lightweight harbor area ship target detection method

Abstract

The invention relates to a lightweight harbor district ship target detection method, which belongs to the technical field of harbor district management, and comprises the following steps: step 1, constructing a data set for detecting a single ship target in a remote sensing image, and dividing the data set into a training set and a verification set; 2, establishing a target detection framework; 3, a gating attention coding module is adopted in a preprocessing layer of the target detection framework to improve the coding capacity, and then coded information is input into a pulse encoder; step 4, the pulse encoder is staged, and the time step length of each stage is reduced by using a time converter; step 5, using the training set to train a gating attention coding module, a pulse encoder and a detection decoder in the network, and using the verification set to verify; and 6, performing ship target detection by using the trained target detection framework. According to the method, the calculation complexity is reduced, the energy efficiency of the model is improved, and the calculation power demand and the energy consumption of the model during front-end deployment are remarkably reduced.

Description

Technical Field

[0001] The invention belongs to the technical field of port area management, and in particular relates to a lightweight port area ship target detection method. Background Art

[0002] Unlike the task of identifying ships on the open sea, ships moored in ports are a challenging task in remote sensing scenarios due to the complex environment in the port area and the interference of buildings and equipment. Most target detection models (such as YOLO series, DETR series, etc.) are based on artificial neural networks (ANN) to identify the category of objects in images and accurately locate their positions. As people pursue higher detection accuracy, the scale of the network is becoming increasingly complex, and the demand for computing resources and energy consumption continues to rise, making these models difficult to deploy on front-end devices with limited computing power and power resources. Although detection models represented by YOLO have launched lightweight versions, it is still difficult to meet the stringent requirements of lightweight front-ends. In particular, the current ANN-based port ship target detection technology still has the following defects:

[0003] (1) Most object detection methods define object candidate regions through empirical design, such as sliding windows, anchor boxes, and reference points. Such methods require adjusting multiple parameters to adapt to specific application scenarios and lack flexibility.

[0004] (2) Since ANN is based on continuous floating-point number calculations, each forward propagation involves a large number of matrix multiplications, additions, and nonlinear function calculations, which leads to high computational complexity and high energy consumption of the target detection model. Its computational cost and energy consumption limit its application on devices with limited resources.

[0005] Therefore, port ship target detection urgently needs a lightweight target detection model that can provide high precision and low energy consumption. Summary of the invention

[0006] The purpose of the present invention is to provide a lightweight port ship target detection method to address the problems existing in the prior art, thereby achieving high-precision ship target detection under low energy consumption conditions.

[0007] The technical solution is as follows:

[0008] A lightweight port ship target detection method comprises the following steps:

[0009] Step 1: Construct a dataset for detecting single ship targets in remote sensing images and divide it into a training set and a validation set in a ratio of 8:2;

[0010] Step 2: Establish an object detection framework with DiffusionDet as the basic framework and replace its image encoder with a spike encoder based on a spike neural network;

[0011] Step 3: Use the Gated Attention Coding (GAC) module in the preprocessing layer of the object detection framework to improve the encoding ability, and then input the encoded information into the pulse encoder;

[0012] Step 4: Phase the pulse encoder and use a time converter to reduce the time step of each stage, thereby reducing time redundancy and computational complexity;

[0013] Step 5: Use the training set to train the gated attention encoding module, pulse encoder, and detection decoder in the network, and verify it with the validation set;

[0014] Step 6: Use the trained object detection framework to perform ship object detection.

[0015] Furthermore, the dataset for single ship target detection in step 1 is constructed by selecting images containing ship targets from the three datasets of DIOR, DOTA, and LEVIR, and removing labels of other categories in the images.

[0016] Furthermore, in step 2, the pulse encoder is composed of a plurality of pulse modules, and the pulse module uses a leaky integrate-and-fire (LIF) model, such as the formula:

[0017] in is the membrane potential of the ith neuron in the n+1th layer at time step t, τ is the attenuation factor of the leakage current, and the synaptic input is the synaptic weights of the previous layer n With j pulses The sum of , H(·) represents the Heaviside step function, which satisfies H(x)=1 when x≥0, otherwise H(x)=0, and the discharge activity of the neuron is determined by the threshold V th Control, when the neuron fires a pulse at time step t+1, V t+1,n+1 will be reset to the resting potential V rest The pulse encoder extracts image features by gradually downsampling the input image. The information is transmitted in the form of pulses of 0 or 1, which simplifies the matrix multiplication operation between the activation value and the weight into an addition operation, thereby reducing the computational complexity and the computational energy consumption in the feature extraction process, significantly improving energy efficiency.

[0018] Furthermore, the gated attention encoding in step 3 combines temporal attention and spatial channel attention to encode the static image into a representation with temporal dynamic characteristics, that is, assuming that the input of the static dataset , C represents the channel size, H and W represent the length and width of the image respectively. First, the convolutional layer is used to extract image features. Then this process is repeated at each time step, and the results are respectively sent to the LIF model and the gated attention unit (GAU) module. Finally, the outputs of the above two modules are gated.

[0019] Furthermore, the LIF model is calculated as follows: O = F G (f k×k (X))e SN(f k×k (X)), where O represents output, f k×k (·) represents a shared two-dimensional convolution operation with a kernel size of k×k, SN(·) represents the LIF model, e represents the Hadamard product, and F G (·) represents the GAU function; the specific calculation formula of GAU is as follows: F G (X) = σ(Me N) = σ(F T (X)eF SC (X)), where M and N represent the time vector and spatial channel matrix respectively, σ(·) represents the Sigmoid function, and F T (·) represents the temporal attention function, F SC (·) represents the spatial channel attention function. The GAU function can fuse the spatiotemporal channel information to better encode feature representation.

[0020] Furthermore, the temporal attention function is calculated by first performing a spatial feature map of the repeated input Perform a compression step, where T represents the simulation time step, C represents the channel size, H and W represent the length and width respectively, then use average pooling and maximum pooling to calculate the average and maximum values ​​of the input X, then pass through the RELU activation function respectively, and finally use the shared MLP network to convert the average pooling and maximum pooling features into a time vector M∈R T , the calculation formula is:

[0021] F T (X) = (W m (RELU(W n (AvgPool(X))))+W m (RELU(W n (MaxPool(X))))),where represents the weights of two shared dense layers, r represents the time dimension reduction factor; the spatial channel attention function is calculated by using a shared two-dimensional convolution operation at each time step to obtain the spatial channel matrix N = [N 1 ,N 2 ,L,N t ]∈? T×C×H×W , generate the spatial channel dynamics of the encoding result, the calculation formula is: Where W i,j is a learnable parameter and K represents the size of the convolution kernel.

[0022] Furthermore, in step 4, the phased pulse encoder is to divide the pulse encoder into 4 stages. As the number of network layers increases, the time step of each stage is shortened, that is, T1>T2>T3>T4. This method of gradually shortening the time step helps to effectively retain information transmission while reducing computational complexity; the specific method of using the time converter to reduce the time step of each stage is as follows: Assuming that the time steps of the two stages are T1 and T2, the data processed are and Where C, H and W represent the channel, height and width of the intermediate feature map respectively, and the output of the previous stage is recorded as The calculation of the time converter is shown in the formula: in is a global average descriptor based on the time step; then the temporal score for the T2 time step can be obtained through nonlinear transformation As shown in the formula: in is a learnable weight of the nonlinear transformation. The softmax function keeps the sum of the time score d as 1 to ensure the complete distribution of information. The time score d is used as the basis for redistributing O1 in the T2 time step. The sum of O1 in the T1 time step is calculated as the total information and distributed to the T2 time step according to d, resulting in As shown in the formula: in Indicates total information, Represents a multiplication operation with a broadcast mechanism. After the above transformation, I2 obtained has a time dimension that is compatible with the subsequent stage and can be used as the input of the subsequent stage to continue forward reasoning. In this way, the time step can be gradually reduced to an ultra-low value without causing significant information loss in the reduction process.

[0023] Furthermore, in step 5, the detection decoder uses a sparse, CNN-featured region (Sparse-regionwith CNN features, Sparse R-CNN) detection head and uses a proxy gradient to solve the problem that the spiking neural network cannot backpropagate. The approximate derivative is expressed as: Where a is a hyperparameter that controls the size of the gradient, which is used to adjust the amplitude of the approximate gradient to ensure that the integral of the gradient is 1 in the back propagation. The Sign function is used to determine the membrane potential. With threshold V th The relationship between The Sign function outputs 1 when , otherwise it outputs 0, ensuring that the approximate gradient will only work when the membrane potential is close to the threshold.

[0024] Furthermore, in step 6, the ship target detection is to first extract features from the model input image or video stream through a pulse encoder, and then input the extracted image features and the randomly generated noise boundary box into the detection decoder, and gradually perform denoising to finally obtain the position and category of the ship on the image.

[0025] Beneficial effects:

[0026] 1) The target detection framework of the pulse encoder based on the pulse neural network of the component of the present invention performs ship detection through the diffusion model, predicts the real target bounding box from the random noise bounding box, does not require object prior information, and does not need to adjust parameters such as anchor boxes and reference points, and is suitable for a variety of scenarios.

[0027] 2) The target detection framework of the present invention is combined with a pulse encoder based on a pulse neural network, which simplifies the multiplication operation of weights and activation values ​​in the ANN into an addition operation, realizes multiplication-free reasoning, reduces the computational complexity, and thus reduces computational energy consumption.

[0028] 3) The object detection framework of the present invention uses a gated attention encoding module to efficiently encode static images to reduce redundant information. The time step is shortened by a time converter, which reduces time redundancy and further reduces computational complexity. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Figure 1 It is a framework flow chart of the lightweight port area ship target detection method of the present invention;

[0030] Figure 2 This is the network structure diagram of the pulse encoder;

[0031] Figure 3 This is the network structure diagram of the pulse module in the pulse encoder;

[0032] Figure 4This is the network structure diagram of the gated attention encoding module;

[0033] Figure 5 This is the network structure diagram of the time converter in the pulse encoder. DETAILED DESCRIPTION

[0034] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments:

[0035] like Figure 1 A lightweight port ship target detection method is shown, comprising the following steps:

[0036] Step 1: Construct a dataset for detecting single ship targets in remote sensing images and divide it into a training set and a validation set in a ratio of 8:2;

[0037] Step 2: Establish an object detection framework with DiffusionDet as the basic framework and replace its image encoder with a spike encoder based on a spike neural network;

[0038] Step 3: Use the Gated Attention Coding (GAC) module in the preprocessing layer of the object detection framework to improve the encoding ability, and then input the encoded information into the pulse encoder;

[0039] Step 4: Phase the pulse encoder and use a time transformer to reduce the time step of each phase, thereby reducing time redundancy and computational complexity.

[0040] Step 5: Use the training set to train the gated attention encoding module, pulse encoder, and detection decoder in the network, and verify it with the validation set;

[0041] Step 6: Use the trained object detection framework to perform ship object detection.

[0042] The dataset for single ship target detection in step 1 is constructed by selecting images containing ship targets from the three datasets of DIOR, DOTA, and LEVIR, and removing labels of other categories in the images.

[0043] In step 2, the pulse encoder is composed of multiple pulse modules, and the pulse module uses a leaky integrate-and-fire (LIF) model, such as the formula: in is the membrane potential of the ith neuron in the n+1th layer at time step t, τ is the attenuation factor of the leakage current, and the synaptic input is the synaptic weights of the previous layer n With j pulses The sum of , H(·) represents the Heaviside step function, which satisfies H(x)=1 when x≥0, otherwise H(x)=0, and the discharge activity of the neuron is determined by the threshold V th Control, when the neuron fires a pulse at time step t+1, V t+1,n+1 will be reset to the resting potential V rest The pulse encoder extracts image features by gradually downsampling the input image. The information is transmitted in the form of pulses of 0 or 1, which simplifies the matrix multiplication operation between the activation value and the weight into an addition operation, thereby reducing the computational complexity and the computational energy consumption in the feature extraction process, significantly improving energy efficiency.

[0044] In step 3, the gated attention encoding combines temporal attention and spatial channel attention to encode the static image into a representation with temporal dynamic characteristics, that is, assuming that the input of the static dataset , C represents the channel size, H and W represent the length and width of the image respectively. First, the convolutional layer is used to extract image features, and then this process is repeated at each time step, and the results are respectively sent to the LIF model and the gated attention unit (GAU) module, and finally the outputs of the above two modules are gated; the LIF model is calculated as follows: O = F G (f k×k (X))e SN(f k×k (X)), where O represents output, f k×k (·) represents a shared two-dimensional convolution operation with a kernel size of k×k, SN(·) represents the LIF model, e represents the Hadamard product, and F G (·) represents the GAU function; the specific calculation formula of GAU is as follows:

[0045] F G (X) = σ(Me N) = σ(F T (X)eF SC (X)), where M and N represent the time vector and spatial channel matrix respectively, σ(·) represents the Sigmoid function, and F T (·) represents the temporal attention function, F SC (·) represents the spatial channel attention function. The GAU function can fuse the spatiotemporal channel information to better encode the feature representation. The calculation method of the temporal attention function is to firstly perform the spatial feature map of the repeated input Perform a compression step, where T represents the simulation time step, C represents the channel size, H and W represent the length and width respectively, then use average pooling and maximum pooling to calculate the average and maximum values ​​of the input X, then pass through the RELU activation function respectively, and finally use the shared MLP network to convert the average pooling and maximum pooling features into a time vector M∈R T , the calculation formula is:

[0046] F T (X)=(W m (RELU(W n (AvgPool(X))))+W m (RELU(W n (MaxPool(X))))),where represents the weights of two shared dense layers, r represents the time dimension reduction factor; the spatial channel attention function is calculated by using a shared two-dimensional convolution operation at each time step to obtain the spatial channel matrix N = [N 1 ,N 2 ,L,N t ]∈? T×C×H×W , generate the spatial channel dynamics of the encoding result, the calculation formula is: Where W i,j is a learnable parameter and K represents the size of the convolution kernel.

[0047] In step 4, the phased pulse encoder divides the pulse encoder into 4 stages. As the number of network layers increases, the time step of each stage is shortened, that is, T1>T2>T3>T4. This method of gradually shortening the time step helps to effectively retain information transmission while reducing computational complexity; the specific method of using the time converter to reduce the time step of each stage is as follows: Assuming that the time steps of the two stages are T1 and T2, the data processed are and Where C, H and W represent the channel, height and width of the intermediate feature map respectively, and the output of the previous stage is recorded as The calculation of the time converter is shown in the formula: in is a global average descriptor based on the time step; then the temporal score for the T2 time step can be obtained through nonlinear transformation As shown in the formula: in is a learnable weight of the nonlinear transformation. The softmax function keeps the sum of the time score d as 1 to ensure the complete distribution of information. The time score d is used as the basis for redistributing O1 in the T2 time step. The sum of O1 in the T1 time step is calculated as the total information and distributed to the T2 time step according to d, resulting in As shown in the formula: in Indicates total information. Represents a multiplication operation with a broadcast mechanism. After the above transformation, I2 obtained has a time dimension that is compatible with the subsequent stage and can be used as the input of the subsequent stage to continue forward reasoning. In this way, the time step can be gradually reduced to an ultra-low value without causing significant information loss in the reduction process.

[0048] In step 5, the detection decoder uses a sparse, CNN-featured region (Sparse-region with CNNfeatures, Sparse R-CNN) detection head and uses a proxy gradient to solve the problem that the spiking neural network cannot backpropagate. The approximate derivative is expressed as: Where a is a hyperparameter that controls the size of the gradient, which is used to adjust the amplitude of the approximate gradient to ensure that the integral of the gradient is 1 in the back propagation. The Sign function is used to determine the membrane potential. With threshold V th The relationship between The Sign function outputs 1 when , otherwise it outputs 0, ensuring that the approximate gradient will only work when the membrane potential is close to the threshold.

[0049] In step 6, the ship target detection is to first extract features from the model input image or video stream through a pulse encoder, and then input the extracted image features and the randomly generated noise boundary box into the detection decoder, and gradually perform denoising to finally obtain the position and category of the ship on the image.

[0050] Embodiment: The present invention constructs a new dataset specifically for single ship target detection in remote sensing images from publicly available datasets. The image sources cover a variety of sensors and platforms, including Google Earth, JL-1 satellite, China Resources Satellite Data and Application Center's GF-2 satellite, etc. The dataset contains images of various types of ships (such as yachts, liquefied petroleum tankers, ore carriers, bulk carriers, container ships, etc.) taken at different altitudes. Images containing ships are screened out from the three remote sensing datasets of DIOR, DOTA, and LEVIER, and other categories such as airplanes, storage tanks, baseball fields, tennis courts, etc. are removed, and all images are scaled to 800×800 pixels to speed up training and save training time. The final dataset contains a total of 2891 images, which are divided into training set and validation set in a ratio of 8:2.

[0051] DiffusionDet extracts high-level features from the input image through the ResNet image encoder, and inputs these features together with the noise bounding box into the detection decoder to decode the location and category of the ship. However, the ResNet image encoder belongs to ANN, which relies on continuous real numbers to transmit information. The network mainly performs multiplication and accumulation operations (MAC), resulting in high computational complexity and energy consumption. In order to solve this problem, the present invention uses a pulse encoder composed of pulse neurons to extract features, simplifying the multiplication operation to addition operation, thereby reducing computational complexity and energy consumption. The network structure of the pulse encoder is shown in the figure. Figure 2 shown.

[0052] Spiking neurons are the basic components of spiking neural networks. In the present invention, spiking neurons use a leaky integrate-and-fire (LIF) model, as shown in the formula:

[0053] in is the membrane potential of the ith neuron in the n+1th layer at time step t, τ is the attenuation factor of the leakage current, and the synaptic input is the synaptic weights of the previous layer n With j pulses The sum of , H(·) represents the Heaviside step function, which satisfies H(x)=1 when x≥0, otherwise H(x)=0, and the discharge activity of the neuron is determined by the threshold V th Control, when the neuron fires a pulse at time step t+1, V t+1,n+1 will be reset to the resting potential V rest .

[0054] The pulse encoder consists of multiple pulse modules and feature pyramid networks. The pulse module network structure is as follows: Figure 3 The detection decoder uses a sparse, CNN-featured region (Sparse-region with CNN features, Sparse R-CNN) detection head and uses a proxy gradient to solve the problem that the spike neural network cannot back propagate. The approximate derivative is expressed as:

[0055] Where a is a hyperparameter that controls the size of the gradient, which is used to adjust the amplitude of the approximate gradient to ensure that the integral of the gradient is 1 in the back propagation. The Sign function is used to determine the membrane potential. With threshold V th The relationship between When , the Sign function outputs 1, otherwise it outputs 0. This ensures that the approximate gradient only works when the membrane potential is close to the threshold.

[0056] Spiking neural networks are usually encoded directly through the first Conv-tdBN layer, but the pulse representations generated by this encoding method lack the temporal dynamic characteristics inherent in human vision. Repeating the Conv-tdBN operation cannot generate dynamic outputs, ignores redundant data, and thus does not fully utilize the spatiotemporal feature extraction capabilities of the subsequent pulse encoder. To solve this problem, the present invention adds a network structure such as Figure 4 The GAC module shown in the figure integrates the attention mechanism in neuroscience and the multi-dimensional attention mechanism inspired by human dynamic vision. By effectively establishing spatiotemporal relationships at each moment, static images are efficiently encoded into representations with temporal dynamic characteristics, minimizing redundancy, and thus establishing temporal associations between pulse neural network inputs. The specific calculation process is as follows:

[0057] Assuming a static dataset input , C represents the channel size, H and W represent the length and width of the image respectively. First, the convolutional layer is used to extract image features, and then this process is repeated at each time step, and the results are respectively sent to the LIF model and the gated attention unit (GAU) module. Finally, the outputs of the above two modules are gated. The specific calculation is as follows:

[0058] O=F G (f k×k (X))e SN(f k×k (X)), where O represents output, f k×k (·) represents a shared two-dimensional convolution operation with a kernel size of k×k, SN(·) represents the LIF model, e represents the Hadamard product, and F G (·) represents the GAU function. The specific calculation formula of GAU is as follows:

[0059] F G (X) = σ(Me N) = σ(F T (X)eF SC (X)), where M and N represent the time vector and spatial channel matrix respectively, σ(·) represents the Sigmoid function, and F T (·) represents the temporal attention function, F SC (·) represents the spatial channel attention function. The GAU function can fuse the spatiotemporal channel information to better encode feature representation.

[0060] F T The calculation process of (·) is as follows:

[0061] F T (X)=(W m (RELU(Wn (AvgPool(X))))+W m (RELU(W n (MaxPool(X))))),where represents the weights of two shared dense layers, r represents the time dimension reduction factor, and first the spatial feature map of the repeated input Perform a compression step, where T represents the simulation time step, C represents the channel size, H and W represent the length and width respectively, and then use average pooling and maximum pooling to calculate the average and maximum values ​​of the input X, and then pass through the RELU activation function respectively, and finally use the shared MLP network to convert the average pooling and maximum pooling features into a time vector M∈R T .

[0062] In order to generate the spatial channel dynamics of the encoded result, a shared 2D convolution operation is used at each time step to obtain the spatial channel matrix N = [N 1 ,N 2 ,L,N t ]∈? T×C×H×W , F SC The specific calculation formula of (·) is as follows: Where W i,j is a learnable parameter, and K represents the size of the convolution kernel.

[0063] Pulse neural networks usually require a longer time step to extract sufficiently valuable features from the input, but a longer time step will introduce more redundant data and increase computational complexity. In order to solve this problem, we divide the pulse encoder into 4 stages. As the number of network layers increases, the time step of each stage continues to shorten, that is, T1>T2>T3>T4. This gradual shortening of the time step helps to effectively retain information transmission while reducing computational complexity. For stages with different time steps, the time dimension of processing data is different. Therefore, the present invention uses a time converter to smoothly transmit information between two stages and retain effective information to the maximum extent. The network structure of the time converter is as follows: Figure 4 Assume that the time steps of the two stages are T1 and T2, and the processed data are and Where C, H and W represent the channel, height and width of the intermediate feature map respectively, and the output of the previous stage is recorded as The calculation process of the time converter is shown in the formula: in is the global average descriptor based on the time step. Then, the temporal score for the T2 time step can be obtained through nonlinear transformation As shown in the formula: in is a learnable weight for the nonlinear transformation. The Softmax function keeps the sum of the temporal scores d equal to 1 to ensure complete distribution of information. The temporal score d is used as the basis for redistributing O1 in the T2 time step. The sum of O1 in the T1 time step is calculated as the total information and distributed to the T2 time step according to d, resulting in As shown in the formula:

[0064] in Indicates total information; Represents a multiplication operation with a broadcast mechanism. After the above transformation, I2 obtained has a time dimension that is compatible with the subsequent stage and can be used as the input of the subsequent stage to continue forward reasoning. In this way, the time step can be gradually reduced to an ultra-low value without causing significant information loss in the reduction process.

[0065] The training is based on DiffusionDet, and the GAC module, pulse encoder and detection decoder in the network are trained from scratch. During the training process, the number of noise bounding boxes generated is set to 500. In order to reduce the deviation of data distribution and accelerate model convergence, the input image is standardized, the pixel mean is set to (123.675, 116.280, 103.530), the pixel standard deviation is set to (58.395, 57.120, 57.375), and the cropping function in data enhancement is turned on to crop the input image. The maximum number of iterations is 20000, the batch size is 8, the weight decay coefficient is 0.0001, the optimizer is AdamW, the warm-up factor is 0.01, the number of warm-up iterations is 1000, and the basic learning rate is 0.000025.

[0066] When using the trained object detection framework to detect ship targets, the resolution of the input image or video frame is first adjusted to 640 × 640. The image features are then extracted through a pulse encoder, and then the extracted features are sent to the detection decoder together with the randomly generated noise bounding box for further processing. Finally, the model outputs the bounding box coordinates and category information of the target in the input image or video frame.

[0067] Since the present invention uses a pulse encoder, a GAC ​​module and a time converter, the ship target detection calculation has the advantages of light weight and low energy consumption.

[0068] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the principles and spirit of the present invention should be included in the protection scope of the present invention.

Claims

1. A lightweight port ship target detection method, characterized by: The steps include: Step 1: Construct a dataset for detecting single ship targets in remote sensing images and divide it into a training set and a validation set in a ratio of 8:2; Step 2: Establish an object detection framework with DiffusionDet as the basic framework and replace its image encoder with a spike encoder based on a spike neural network; Step 3: Use a gated attention encoding module in the preprocessing layer of the object detection framework to improve the encoding ability, and then input the encoded information into the pulse encoder; Step 4: Phase the pulse encoder and use a time converter to reduce the time step of each stage, thereby reducing time redundancy and computational complexity; Step 5: Use the training set to train the gated attention encoding module, pulse encoder, and detection decoder in the network, and verify it with the validation set; Step 6: Use the trained object detection framework to perform ship object detection.

2. A lightweight port ship target detection method as claimed in claim 1, characterized in that: The single ship target detection dataset described in step 1 is constructed by selecting images containing ship targets from the three datasets of DIOR, DOTA, and LEVIR, and removing labels of other categories in the images.

3. A lightweight port ship target detection method as claimed in claim 1, characterized in that: The pulse encoder described in step 2 is composed of multiple pulse modules, and the pulse module uses a leaky integration-trigger model, such as the formula: in is the membrane potential of the ith neuron in the n+1th layer at time step t, τ is the attenuation factor of the leakage current, and the synaptic input is the synaptic weights of the previous layer n With j pulses The sum of , H(·) represents the Heaviside step function, which satisfies H(x)=1 when x≥0, otherwise H(x)=0, and the discharge activity of the neuron is determined by the threshold V th Control, when the neuron fires a pulse at time step t+1, V t+1,n+1 will be reset to the resting potential V rest The pulse encoder extracts image features by gradually downsampling the input image. The information is transmitted in the form of pulses of 0 or 1, which simplifies the matrix multiplication operation between the activation value and the weight into an addition operation, thereby reducing the computational complexity and reducing the computational energy consumption in the feature extraction process.

4. A lightweight port ship target detection method as claimed in claim 1, characterized in that: The gated attention encoding described in step 3 combines temporal attention and spatial channel attention to encode static images into representations with temporal dynamic characteristics, that is, assuming the input of a static dataset C represents the channel size, H and W represent the length and width of the image respectively. First, the convolutional layer is used to extract image features, and then this process is repeated at each time step. The results are respectively sent to the LIF model and the gated attention unit module, and finally the outputs of the above two modules are gated.

5. A lightweight port ship target detection method as claimed in claim 4, characterized in that: The LIF model is calculated as follows: O = F G (f k×k (X))e SN(f k×k (X)), where O represents output, f k×k (·) represents a shared 2D convolution operation with a kernel size of k×k, SN(·) represents the LIF model, e represents the Hadamard product, and F G (·) represents the GAU function; the specific calculation formula of GAU is as follows: F G (X) = σ(Me N) = σ(F T (X)eF SC (X)), where M and N represent the time vector and spatial channel matrix respectively, σ(·) represents the Sigmoid function, and F T (·) represents the temporal attention function, F SC (·) represents the spatial channel attention function.

6. A lightweight port ship target detection method as claimed in claim 5, characterized in that: The temporal attention function is calculated by first performing a spatial feature map of the repeated input Perform a compression step, where T represents the simulation time step, C represents the channel size, H and W represent the length and width respectively, then use average pooling and maximum pooling to calculate the average and maximum values ​​of the input X, then pass through the RELU activation function respectively, and finally use the shared MLP network to convert the average pooling and maximum pooling features into a time vector M∈R T , the calculation formula is: F T (X)=(W m (RELU(W n (AvgPool(X))))+W m (RELU(W n (MaxPool(X))))),where represents the weights of two shared dense layers, r represents the time dimension reduction factor; the calculation of the spatial channel attention function is to use a shared two-dimensional convolution operation at each time step to obtain the spatial channel matrix N = [N 1 ,N 2 ,L,N t ]∈? T×C×H×W , generate the spatial channel dynamics of the encoding result, the calculation formula is: Where W i,j is a learnable parameter and K represents the size of the convolution kernel.

7. A lightweight port ship target detection method as claimed in claim 1, characterized in that: The staged pulse encoder described in step 4 divides the pulse encoder into 4 stages. As the number of network layers increases, the time step of each stage is shortened, that is, T1>T2>T3>T4. This method of gradually shortening the time step helps to effectively retain information transmission while reducing computational complexity; the specific method of using the time converter to reduce the time step of each stage is as follows: Assume that the time steps of the two stages are T1 and T2, and the data processed are and Where C, H and W represent the channel, height and width of the intermediate feature map respectively, and the output of the previous stage is recorded as The calculation of the time converter is shown in the formula: in is a global average descriptor based on the time step; then the temporal score for the T2 time step can be obtained through nonlinear transformation As shown in the formula: in is a learnable weight of the nonlinear transformation. The softmax function keeps the sum of the time score d as 1 to ensure the complete distribution of information. The time score d is used as the basis for redistributing O1 in the T2 time step. The sum of O1 in the T1 time step is calculated as the total information and distributed to the T2 time step according to d, resulting in As shown in the formula: in Indicates total information, Represents a multiplication operation with a broadcast mechanism. After the above transformation, I2 obtained has a time dimension that is compatible with the subsequent stages and can be used as the input of the subsequent stages to continue forward reasoning.

8. A lightweight port ship target detection method as claimed in claim 1, characterized in that: The detection decoder described in step 5 uses a sparse, CNN-featured region detection head and uses a proxy gradient to solve the problem that the spiking neural network cannot backpropagate. The approximate derivative is expressed as: Where a is a hyperparameter that controls the size of the gradient, which is used to adjust the amplitude of the approximate gradient to ensure that the integral of the gradient is 1 in the back propagation. The Sign function is used to determine the membrane potential. With threshold V th The relationship between The Sign function outputs 1 when , otherwise it outputs 0, ensuring that the approximate gradient will only work when the membrane potential is close to the threshold.

9. A lightweight port ship target detection method as claimed in claim 1, characterized in that: The ship target detection described in step 6 is to first extract features from the model input image or video stream through a pulse encoder, and then input the extracted image features and the randomly generated noise boundary box into the detection decoder, gradually perform denoising, and finally obtain the position and category of the ship on the image.

Images