Sea surface target detection method based on biological nerve synapse simulation and step substitution function

Abstract

The invention discloses a sea surface target detection method based on biological nerve synapse simulation and a step substitution function, and belongs to the field of sea surface target detection. According to the invention, a spiking neuron LIF is adopted to replace an original activation function of YOLOV5; the loss derivation of the spiking neural network under the YOLOV5 + SNN architecture is completed by using a step substitution function, and abundant training strategies are provided for complex sea surface image input, so that error back propagation is carried out. According to the method, stimulation of neurons to data needs to be continuously accumulated in an actual sea surface ship target detection scene is considered, the method better fits a biological learning process, the ship target recognition rate can be effectively improved, ideal balance between delay and the recognition rate is achieved, and more effective information is reserved for a specific sea scene image.

Description

Technical Field

[0001] The invention belongs to the field of sea surface target detection, and in particular relates to a sea surface target detection method based on biological neural synapse simulation and step substitution function. Background Art

[0002] Object detection is a research direction in the field of computer vision, which aims to determine the location and type of the target object. Object detection technology plays an important role in autonomous driving, video surveillance, industrial defect detection and other fields. Using deep learning algorithms to learn and apply knowledge in object detection tasks is currently a popular research method. However, traditional artificial neural networks (ANNs) use continuous analog signals, not biological neural pulse signals, and therefore cannot imitate the spatiotemporal characteristics of biological brains well. At the same time, the activation method used by ANNs will cause a lot of computational overhead and energy consumption in the calculation process.

[0003] Spiking neural network (SNN) is a third generation neural network, which is connected by spiking neurons. It converts input and output into pulses for propagation in the computer. The computing unit is activated only when the neuron sends or receives a pulse, which will greatly save the computing cost of SNN in inference and prediction. At the same time, the pulse itself is composed of 0-1, which conforms to the underlying computing method of the computer, so it can well reduce computing energy consumption and facilitate the forward propagation of the neural network.

[0004] However, the current application of SNN in the field of target detection still has certain limitations, such as lower accuracy than ANN. This is usually due to the use of pulses instead of floating-point numbers for reasoning and training strategies. Researchers generally solve the latter by converting ANN to SNN, but the converted SNN has a larger time step and has no advantage over ANN in reasoning speed.

[0005] EMS-YOLO proposed a direct training SNN for target detection, which can achieve accuracy close to that of ANN of the same scale at a lower cost when the time step is small. However, in the directly trained SNN, because the function of generating pulses is not differentiable, a substitute function is needed to simulate it. However, the substitute function adopted by EMS-YOLO does not conform to the pulse generation process of the human perception process, which has a certain impact on the detection effect. Therefore, the existing technology still needs to be improved and developed. Summary of the invention

[0006] In view of the problems existing in the prior art, the present invention proposes a sea surface target detection method based on biological neural synapse simulation and step substitution function in combination with the practical problems of SNN in target detection, so as to overcome the limitations of EMS-YOLO in error back propagation and improve the accuracy of target detection.

[0007] The present invention provides a sea surface target detection method based on biological neural synapse simulation and step substitution function, comprising the following steps:

[0008] Step 1: Obtain an image dataset with sea surface targets, perform data enhancement on each image in the dataset, and construct a training set with the enhanced image dataset. Each sample in the training set includes an image and a corresponding sea surface target detection label;

[0009] Step 2: Build a pulse neural network under the YOLOV5+SNN framework and train it using the training set;

[0010] The spiking neural network under the YOLOV5+SNN framework adopts biological synapse simulation, and the biological synapse simulation adopts spiking neuron LIF to replace the original activation function of YOLOV5; the loss derivation of the spiking neural network under the YOLOV5+SNN architecture is completed by using a step replacement function, providing a rich training strategy for complex sea surface image input, so as to perform error back propagation;

[0011] Step 3: First preprocess the training set, perform data enhancement on the images to be detected in the preprocessed training set, and input the data-enhanced images to be detected into the pulse neural network under the trained YOLOV5+SNN architecture to obtain the sea surface target detection results.

[0012] Further, in step 2, the pulse neural network model under the YOLOV5+SNN architecture includes backbone, neck and head; the backbone includes multiple SNN_block, Conv, SPPF, which are used to extract features of video data; the neck includes Conv, Concat, SNN_block, which are used to complete the fusion and splicing of feature maps of different scales; the head includes Conv, Loss loss calculation module, and candidate box generation module, and uses an anchor-based method to complete the convolution of feature maps from three levels of the neck and the generation of predicted candidate boxes;

[0013] The SNN_block includes a pulse neuron LIF, Conv, and MaxPool3D. The pulse neuron LIF converts the output into a pulse form to be input to Conv.

[0014] Furthermore, step 2 specifically includes:

[0015] Step 2.1: The image is first sampled by convolution calculation through Conv in backbone, and then outputs feature maps of different scales to neck through several SNN_blocks:, and uses an SPPF to fuse multi-scale features;

[0016] Step 2.2: neck adopts the PAFPN path, samples the feature maps of different scales through Conv, samples them on the nearest interpolation, and fuses them together;

[0017] Step 2.3: The head convolves the three-layer feature map output by the neck, and then generates positive and negative samples for calculating the loss function based on the enhanced dataset and the ship target annotation box. Finally, the calculated loss is used for backward propagation.

[0018] Furthermore, in step 2.1, the pulse neuron LIF uses the input sea surface ship image as the neuron stimulus, and considering that the sea surface image has the characteristics of high exposure and motion blur, there is a certain misjudgment rate for target detection on the sea surface with changeable weather. Therefore, the input image is accumulated as the neuron stimulus in time. When the accumulated stimulus exceeds a certain value, a pulse is activated, and the characteristic value of the activated image is output, which is as follows:

[0019]

[0020] Where X(t) is the image of the sea surface ship input to the spike neuron LIF; leak is the leakage parameter, which is used to weaken the neuron's memory of the image at the previous time step; V(t) is the membrane potential, that is, the intermediate eigenvalue, that is, the accumulated stimulation of the neuron; V thr is the discharge threshold, and the characteristic information exceeding this threshold will generate a pulse; S(t) is the output pulse, the characteristic value finally generated by the spiking neuron LIF; h(x) is the step function used to generate the pulse;

[0021] When LIF receives the ship image data X(t), it starts to charge. When the membrane potential V(t) exceeds V thr When , the discharge generates a pulse S(t), the eigenvalue is output to the next layer, and V(t) is reset.

[0022] Furthermore, in step 2.3, the step substitution function is used to derive the loss function, which solves the problem that the step function cannot be derived; at the same time, considering the application scenario of sea surface target detection, the step substitution function can provide different sizes and discrete derivative values ​​compared with other substitution functions, while reducing the computational burden, providing a richer training strategy for sea surface target detection neurons, which can adapt to image input in a changeable and complex environment of the sea surface; the specific formula of the step substitution function is as follows:

[0023]

[0024] Where k is the slope of the step function; n is the number of segments; m is a natural number between 0 and n, used to indicate which segment of the function this is; the starting point is at s and the end point is at e; each a i are equal to each other, each b i They are also equal in pairs;

[0025] The derivative formula of the step substitution function is as follows:

[0026]

[0027]

[0028] Among them, grad output The gradient value corresponding to the feature output by the network; grad input is the gradient value corresponding to the input image or feature; sign() is the sign function; Input is the feature or image input to the neuron.

[0029] Furthermore, the preprocessing of the training set in step 3 is to perform K-means clustering statistics, and count the most frequently appearing Bboxes in the sea surface target detection data set as anchors, specifically including:

[0030] Step 3.1: Randomly select k Bboxes from all the Bboxes in the training set as the center of the category;

[0031] Step 3.2: Calculate the distance between each Bboxes and each category;

[0032] Step 3.3: Calculate the nearest category center for each Bboxes and assign it to the category closest to it;

[0033] Step 3.4: Recalculate the category center based on the Bboxes in each cluster;

[0034] Step 3.5: Repeat steps 3.3 to 3.4 until each category element no longer changes.

[0035] The present invention also provides a computer device / equipment / system, comprising a memory, a processor and a computer program stored in the memory, wherein when the processor executes the computer program, the steps of any of the above-mentioned methods for detecting sea surface targets based on biological neural synapse simulation and step substitution function are implemented.

[0036] The present invention also provides a computer-readable storage medium having a computer program / instruction stored thereon, which, when executed by a processor, implements the steps of any of the above-mentioned methods for detecting sea surface targets based on biological neural synapse simulation and step substitution function.

[0037] The present invention also provides a computer program product, including a computer program / instruction, characterized in that when the computer program / instruction is executed by a processor, the steps of any of the above-mentioned methods for sea surface target detection based on biological neural synapse simulation and step substitution function are implemented.

[0038] The beneficial effects of the present invention are:

[0039] The present invention uses the overall architecture of SNN to reduce the energy consumption of computer reasoning prediction and improve the detection effect. It can perform reasoning with lower energy consumption and more accuracy to guide decision-making and control during ship navigation. The selection of step substitution function enables network training to more reasonably simulate neuron pulses during error back propagation and retain more effective information for specific ocean scene images. This improvement takes into account the need to continuously accumulate information in the time dimension to stimulate neurons in the actual scenario of surface ship target detection. At the same time, different ocean scene images also need to use different training strategies. The improved model has a learning process that is more in line with biological organisms, has a higher ship target recognition rate, and can achieve an ideal balance between delay and recognition rate. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 It is the overall flow chart of the present invention;

[0041] Figure 2 This is a schematic diagram of the YOLOV5+SNN network structure proposed in this invention. DETAILED DESCRIPTION

[0042] The present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments.

[0043] The present invention discloses a sea surface target detection method based on biological neural synapse simulation and step substitution function, comprising the following steps:

[0044] (1) Construct and pre-train a spiking neural network (with YOLOV5+SNN as the overall architecture) on a public dataset for feature extraction of the ocean surface input video information stream; the dataset is the COCO dataset, and the main purpose of pre-training is to accelerate the convergence speed of network training.

[0045] (2) Video information streams are collected during the ship's navigation process to construct a dataset; the dataset mainly comes from the video of the ship's navigation process, covering port and open sea scenes, including various sea surface objects such as ships, shores, and buoys.

[0046] (3) Divide the video dataset in step (2) into a training set, a validation set, and a test set according to a ratio of 6:3:1.

[0047] (4) Perform K-means clustering statistics on the divided training set to count the most frequently occurring anchors; K-means clustering statistics is an unsupervised learning method that aims to automatically divide data into multiple categories without any prior knowledge, and make the objects in each category have a high degree of similarity, while objects in different categories show a high degree of difference. The network used in the present invention is based on the anchor-based method, and it is necessary to design anchors in order to learn a better surface ship detector. The K-means clustering method will obtain the classification result with the largest inter-class distance and the smallest intra-class distance, thereby calculating the anchor prior knowledge for the surface ship target.

[0048] (5) Based on the prior anchor knowledge counted in step (4), the data is first enhanced and then input into YOLOV5+SNN to fine-tune the pre-trained spiking neural network; pre-training data enhancement includes random perspective transformation, random HSV, random flipping, Mosaic, etc. Data enhancement can effectively avoid overfitting, improve model robustness, increase model generalization ability, and avoid sample imbalance. Random flipping arbitrarily reverses the image, and the annotation box also rotates accordingly; random HSV is to transform the hue of the image, including: brightness, contrast, saturation, and hue; random perspective transformation performs arbitrary perspective transformation on the image, changes the length, angle, parallelism, etc. of the image coordinates while retaining collinearity and correlation, and the annotation box is also transformed accordingly. The transformed background can be filled with any color. Mosaic splices 4 images by random scaling, random cropping, and random arrangement. This can greatly enrich the detection data set, add many small target instances to the training, and improve the model robustness and reduce the possibility of overfitting. Compared with the YOLOV3-Tiny+SNN architecture used by EMS-YOLO, the YOLOV5+SNN architecture contains larger model parameters and a deeper network. It can fuse feature maps with a larger perception field, enabling the network to detect farther ships and improve the multi-scale ship target detection effect.

[0049] The YOLOV5+SNN architecture consists of backbone, neck and head parts. The backbone consists of multiple SNN_blocks, Conv and SPPF to extract the features of sea surface video data. The neck part consists of Conv, Concat and SNN_block modules to complete the fusion and splicing of feature maps of different scales. The head part adopts the anchor-based method to complete the convolution of the feature maps of the three levels from the neck and the generation of predicted ship target candidate frames. The SNN_block in the backbone consists of pulse neurons LIF, Conv and MaxPool3D. Conv is responsible for performing a convolution calculation on different frame images of the sea surface video and then standardizing them. LIF replaces the role of the original activation function and converts the output into a pulse form to input to the Conv layer, which complies with the hardware transmission rules and greatly reduces the energy consumption of calculation. MaxPool3D is a maximum pooling layer that considers the time dimension of the sea surface video. It also takes into account the time dimension information while downsampling. The Conv in the backbone is the same as that in the SNN_block. The main function of SPPF is to fuse multi-scale ship features. It splices feature layers of different scales through three layers of maximum pooling, which can reduce the amount of calculation for deep features while greatly improving the fusion performance of multi-scale features. The Conv and SNN_block in the neck are the same as those in the backbone. Concat is used to splice feature maps of the same size in the channel dimension. The neck part of YOLOV5 refers to the PAFPN path, scales the feature layer from bottom to top through upsampling, fuses P3, P4 and P5, and then downsamples these feature layers from top to bottom and fuses them again. This bidirectional pyramid fusion method can effectively extract multi-scale ship target features and enhance the model's ability in multi-scale ship target detection tasks. The head convolves the three-layer feature map output by the neck, and generates predicted candidate boxes at three scales based on the prior anchor knowledge on the feature map results (expressed by the offset value of the candidate box relative to the anchor, that is, the offset t of the center coordinate x , t y and the offset of length and width t w , t h ). According to the positive and negative sample matching rules, a certain number of positive samples and negative samples are obtained to calculate the regression Bbox Loss, classification Cls.Loss and confidence loss Obj Loss respectively. The calculated loss is used for back propagation to adjust the weight of each layer, thereby training the entire ship target detection network.

[0050] (6) Use the step substitution function to complete the loss derivation of the spiking neural network under the YOLOV5+SNN architecture, so as to perform error back propagation; the step substitution function is a function used to replace the derivation process of the spiking neuron in the SNN. The step function used by the spiking neuron to generate pulses is not differentiable, so it is necessary to use a differentiable function with similar characteristics to the step function as a substitute function for error back propagation to provide the gradient value during the back propagation of the neural network. The present invention adopts a step substitution function so that the accumulation of spiking neurons for stimulation is staged, and the effective information generated in the intermediate process of accumulating from 0 to 1 can also serve the adjustment of the model weights.

[0051] (7) Use the trained model to predict on the test set, compare the predicted value with the groundtruth, and test and evaluate the performance of the model.

[0052] (8) The model is converted into TensorRT format and deployed to actual application scenarios.

[0053] Example 1

[0054] like Figure 1 The present invention proposes a method for detecting sea surface targets based on biological neural synapse simulation and step substitution function, which comprises the following steps:

[0055] Step S1: The COCO dataset used is a large dataset with rich objects. It is mainly captured from complex daily scenes, and the objects in the images have been accurately calibrated. The COCO dataset provides 80 categories, more than 330,000 images, of which 200,000 are labeled. The number of individuals in the entire dataset exceeds 1.5 million, making it one of the most suitable datasets for neural network pre-training.

[0056] Neural network pre-training is a pre-trained model or the process of pre-training a model. Because the actual data set used for training is usually not large enough, overfitting will occur when using a model with a large number of parameters. Therefore, the present invention uses a large data set (COCO data set) for pre-training, and then fine-tunes the training strategy on a self-built sea surface video data set.

[0057] Step S2: The video resolution collected during the ship's voyage is 1920*1080, the frame rate is 24, and 24 segments of one hour of video are collected. The video is sliced ​​and then manually annotated to obtain the ship target annotation box.

[0058] Step S4: Use the K-means method to complete the cluster analysis of the ship target annotation boxes in the data set to obtain the prior knowledge of the anchor (i.e. the length and width of the anchor on the three specific size feature maps). The steps are as follows:

[0059] 1. Randomly select k ship target annotation boxes as clustering category centers;

[0060] 2. Calculate the distance between each ship target annotation box and the center of each category;

[0061] 3. Assign each ship target annotation box to the category closest to it;

[0062] 4. Recalculate the category center based on the ship target annotation box in each cluster;

[0063] 5. Repeat 3 to 4 until each category element no longer changes.

[0064] In this way, the clustering results of the ship target annotation boxes can be obtained and input into the neural network as prior knowledge.

[0065] Step S5: The YOLOV5+SNN used in the present invention is as follows Figure 2 As shown in the figure. The images collected during the flight are input into the backbone for feature extraction after data enhancement. The image is first downsampled through a Conv layer for convolution calculation, and then through several SNN_blocks to output feature maps of different scales to the neck, and finally a SPPF is used to fuse multi-scale features. The neck part of YOLOV5+SNN adopts the PAFPN path, downsampling the feature maps of different scales through Conv, upsampling by nearest interpolation, and then fusing them together.

[0066] SNN_block consists of two branches. The main branch consists of two LIF-Conv-BN pathways, in which the LIF spike neuron acts as an activation function before convolution, mapping and activating the values ​​on the feature map, and the BN batch normalization limits the output scale and limits the propagated ship image data to a certain distribution. The shortcut branch consists of a MaxPool3D and a LIF-Conv-BN pathway. Its main purpose is to perform residual connection to prevent gradient explosion or gradient disappearance due to excessive network depth. MaxPool3D takes into account the time dimension of the navigation video data and performs maximum pooling in the three dimensions of time, length and width, so that the dimensions of the main branch and the shortcut branch are consistent to complete the summation operation.

[0067] LIF pulse neuron is used as the activation function before convolution, which is the key to SNN's simulation of human neuron perception. Different from the continuous activation value provided by the general activation function, LIF regards the input sea surface ship image as the stimulation of the neuron, and considering that the sea surface image has the characteristics of high exposure and motion blur, there is a certain misjudgment rate for target detection on the sea surface with changeable weather. Therefore, the input image is used as the stimulation of the neuron in time. When the cumulative stimulation of the neuron in the time dimension exceeds a certain value, a pulse is activated, that is, the characteristic value of the output image after activation. The calculation steps and formula are as follows:

[0068]

[0069] Where X(t) is the sea surface ship image input to LIF; leak is the leakage parameter, which is used to measure the speed at which neurons lose feature information over time and can be learned; V(t) is the membrane potential, that is, the intermediate feature value; V thr is the discharge threshold, that is, the characteristic information exceeding this threshold will generate a pulse; S(t) is the output pulse, that is, the characteristic value finally generated by LIF; h(x) is the step function used to generate the pulse. When LIF receives the ship image data X(t), it starts to charge. When the membrane potential V(t) exceeds V thr When , the discharge generates a pulse S(t), the eigenvalue is output to the next layer, and V(t) is reset.

[0070] Step S5: YOLOV5+SNN uses an anchor-based method to calculate the loss in the head for model weight training and outputs the final prediction results of multi-scale ship target features. The head predicts the position, size and category of the ship target bounding box at different scales through a series of convolutions. This design enables the model to perform ship target detection at different scales in parallel.

[0071] The three scale feature maps output by the neck are input into the head for convolution, and the output of pred∈(B,3,H,W,cls+1+4) dimension is obtained. B is the batch size, 3 refers to the three feature scales, (H,W) refers to the length and width of the feature map, cls refers to the number of types of objects in the ship dataset, 1 refers to the objectness used to determine whether there is an object in the candidate box, and 4 refers to the offset of the object from the anchor (t x ,t y ,t w ,t h ).

[0072] Then, based on the anchor prior knowledge and the ship target annotation box, positive and negative samples are generated for calculating the loss function. The positive and negative sample matching algorithm will select a grid at each ship target annotation box position, and use the center point position of the ship target annotation box to select the two nearest adjacent grids. These three grids will generate 9 anchors at three scales. Then these anchors are compared with the length and width of the ship target annotation box to meet Those that meet the conditions will be used as positive samples, and those that do not meet the conditions will be used as negative samples. gt , w gt ) is the length and width of the ship target in the image, (h a , w a ) is the length and width of the anchor.

[0073] For classification tasks, the model uses BCE (binary cross entropy) to calculate the classification loss Cls.Loss, where N is the number of samples, y is the true label of the data, and p is the probability of the model predicting the label:

[0074]

[0075] Before calculating the regression task, the predicted anchor offset representation (t x ,t y ,t w ,t h ) is converted into the (x, y, w, h) representation of the ship target annotation box. The conversion method is as follows, where (P w ,P h ) is the width and height of the anchor.

[0076]

[0077] For the regression task, the model uses CIoU to measure the overlap between the predicted value and the ship target annotation box as the prediction box regression loss Bbox Loss, which is calculated as follows, where (W G ,h G ), (W P , H P ) are the width and height of the ship target annotation box and the predicted box, d is the distance between the center points of the predicted box and the real box, c is the diagonal distance of the minimum circumscribed rectangle, v is the correction factor used to further adjust the loss function, α represents the scale of the box, and IoU represents the similarity of the two boxes:

[0078]

[0079] At the same time, the confidence loss Obj.Loss needs to be calculated. Objectness is used to measure the degree to which the predicted candidate box covers the object, that is, whether the candidate box contains the target ship. When calculating Obj Loss, the objectness (t obj ) is set to the value of IoU. The calculation formula of Obj.Loss is as follows. Where gr is the label smoothing coefficient, P obj is the predicted objectness, and IoU is the CIoU calculated in the previous step.

[0080] t obj =(1-gr)+gr*IoU

[0081]

[0082] Step S6: During forward propagation, the LIF neuron uses a step function to generate pulse eigenvalues. During error back propagation, a step substitution function is used to simulate the step function, and the derivative generated by the step substitution function is used for error back propagation. The step substitution function formula is as follows:

[0083]

[0084] The slope of the step function is k, the number of segments is n, and m is a natural number between 0 and n, which is used to indicate the first segment of the function. The starting point is s and the end point is e. i are equal to each other, each b i They are also equal.

[0085] The derivative formula of the step substitution function is as follows:

[0086]

[0087] Among them, grad output The gradient value corresponding to the feature output by the network; grad input is the gradient value corresponding to the input image or feature; sign() is the sign function; Input is the feature or image input to the neuron.

[0088] Step S7: Test and evaluate the model on the test set.

[0089] In particular, in some preferred embodiments of the present invention, a computer device is also provided, including a memory and a processor and a computer program stored in the memory, and when the processor executes the computer program, the steps of the sea surface target detection method based on biological neural synapse simulation and step substitution function described in any of the above embodiments are implemented.

[0090] In other preferred embodiments of the present invention, a computer-readable storage medium is provided, on which a computer program / instructions are stored. When the computer program is executed by a processor, the steps of the sea surface target detection method based on biological neural synapse simulation and step substitution function described in any of the above embodiments are implemented.

[0091] A person skilled in the art may understand that all or part of the processes in the above-mentioned embodiment method may be implemented by instructing related hardware through a computer program. The computer program may be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it may include the processes of the above-mentioned embodiment of the sea surface target detection method based on biological neural synapse simulation and step substitution function, which will not be repeated here.

[0092] It will be easily understood by those skilled in the art that the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A sea surface target detection method based on biological neural synapse simulation and step substitution function, characterized in that: The following steps are involved: Step 1: Obtain an image dataset with sea surface targets, perform data enhancement on each image in the dataset, and construct a training set with the enhanced image dataset. Each sample in the training set includes an image and a corresponding sea surface target detection label; Step 2: Build a pulse neural network under the YOLOV5+SNN framework and train it using the training set; The spiking neural network under the YOLOV5+SNN framework adopts biological synapse simulation, and the biological synapse simulation adopts spiking neuron LIF to replace the original activation function of YOLOV5; the loss derivation of the spiking neural network under the YOLOV5+SNN architecture is completed by using a step replacement function, providing a rich training strategy for complex sea surface image input, so as to perform error back propagation; Step 3: First preprocess the training set, perform data enhancement on the images to be detected in the preprocessed training set, and input the data-enhanced images to be detected into the pulse neural network under the trained YOLOV5+SNN architecture to obtain the sea surface target detection results.

2. The sea surface target detection method based on biological neural synapse simulation and step substitution function according to claim 1 is characterized in that: In step 2, the pulse neural network model under the YOLOV5+SNN architecture includes backbone, neck and head; the backbone includes multiple SNN_block, Conv, SPPF, which are used to extract the features of video data; the neck includes Conv, Concat, SNN_block, which are used to complete the fusion and splicing of feature maps of different scales; the head includes Conv, Loss loss calculation module, and candidate box generation module, and uses the anchor-based method to complete the convolution of feature maps from three levels of the neck and the generation of predicted candidate boxes; The SNN_block includes a pulse neuron LIF, Conv, and MaxPool3D. The pulse neuron LIF converts the output into a pulse form to be input to Conv.

3. The sea surface target detection method based on biological neural synapse simulation and step substitution function according to claim 2 is characterized in that: Step 2 specifically includes: Step 2.1: The image is first sampled by convolution calculation through Conv in backbone, and then outputs feature maps of different scales to neck through several SNN_blocks:, and uses an SPPF to fuse multi-scale features; Step 2.2: neck adopts the PAFPN path, samples the feature maps of different scales through Conv, samples them on the nearest interpolation, and fuses them together; Step 2.3: The head convolves the three-layer feature map output by the neck, and then generates positive and negative samples for calculating the loss function based on the enhanced dataset and the ship target annotation box. Finally, the calculated loss is used for backward propagation.

4. The method for detecting sea surface targets based on biological neural synapse simulation and step substitution function according to claim 3, characterized in that: In step 2.1, the pulse neuron LIF uses the input sea surface ship image as the neuron stimulus, and considering that the sea surface image has the characteristics of high exposure and motion blur, there is a certain misjudgment rate for target detection on the sea surface with changeable weather. Therefore, the input image is accumulated as the neuron stimulus in time. When the accumulated stimulus exceeds a certain value, a pulse is activated and the feature value of the activated image is output, which is as follows: Where X(t) is the image of the sea surface ship input to the spike neuron LIF; leak is the leakage parameter, which is used to weaken the neuron's memory of the image at the previous time step; V(t) is the membrane potential, that is, the intermediate eigenvalue, that is, the accumulated stimulation of the neuron; V thr is the discharge threshold, and the characteristic information exceeding this threshold will generate a pulse; S(t) is the output pulse, the characteristic value finally generated by the spiking neuron LIF; h(x) is the step function used to generate the pulse; When LIF receives the ship image data X(t), it starts to charge. When the membrane potential V(t) exceeds V thr When , the discharge generates a pulse S(t), the eigenvalue is output to the next layer, and V(t) is reset.

5. The method for detecting sea surface targets based on biological neural synapse simulation and step substitution function according to claim 3, characterized in that: In step 2.3, the step substitution function is used to derive the loss function, which solves the problem that the step function cannot be derived. Considering the application scenario of sea surface target detection, the step substitution function can provide different sizes of discrete derivative values ​​compared with other substitution functions. While reducing the computational burden, it provides a richer training strategy for sea surface target detection neurons, which can adapt to image input in a changeable and complex environment of the sea surface. The specific formula of the step substitution function is as follows: Where k is the slope of the step function; n is the number of segments; m is a natural number between 0 and n, used to indicate which segment of the function this is; the starting point is at s and the end point is at e; each a i are equal to each other, each b i They are also equal in pairs; The derivative formula of the step substitution function is as follows: Among them, grad output The gradient value corresponding to the feature output by the network; grad input is the gradient value corresponding to the input image or feature; sign() is the sign function; Input is the feature or image input to the neuron.

6. The method for detecting sea surface targets based on biological neural synapse simulation and step substitution function according to claim 1, characterized in that: The preprocessing of the training set in step 3 is to perform K-means clustering statistics, and count the most frequently appearing BBoxes in the sea surface target detection dataset as anchors, specifically including: Step 3.1: Randomly select k BBoxes from all the BBoxes in the training set as the center of the category; Step 3.2: Calculate the distance between each BBoxes and each category; Step 3.3: Calculate the nearest category center for each BBoxes and assign it to the category closest to it; Step 3.4: Recalculate the category center based on the BBoxes in each cluster; Step 3.5: Repeat steps 3.3 to 3.4 until each category element no longer changes.

7. A computer device / equipment / system comprising a memory, a processor and a computer program stored in the memory, characterized in that: The processor executes the computer program to implement the steps of the method according to any one of claims 1 to 6.

8. A computer-readable storage medium having a computer program / instruction stored thereon, characterized in that: When the computer program / instructions are executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

9. A computer program product comprising a computer program / instructions, characterized in that: When the computer program / instructions are executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images