Application method of pulse-driven target detection model

Abstract

The application provides an application method of a target detection model based on pulse driving, the target detection model is a pulse driving target detection model, and the method comprises the following steps: constructing a target detection model based on a YOLOv8 model, in the network architecture of the target detection model, two different SNN feature extraction modules are used to replace C2F feature extraction modules in the YOLOv8 model; obtaining a to-be-processed image, copying a plurality of times of static image data in the to-be-processed image and inputting the static image data into the target detection model, and inputting neuromorphic data in the to-be-processed image into the target detection model in frames; and applying the target detection model to process the static image data and the neuromorphic data respectively to obtain a target detection result of the to-be-processed image. Through the above method, the problem that a target detection model is difficult to be directly applied to an actual image detection scene due to low performance of the target detection model in the field of pulse neural networks is solved.

Description

Technical Field

[0001] This invention relates to the field of neural network technology, and in particular to an application method of a target detection model based on pulse-driven detection. Background Technology

[0002] Image object detection is a very challenging visual task with broad application prospects in fields such as autonomous driving and facial recognition. In the field of artificial neural networks (ANN), object detection models represented by YOLO have achieved remarkable results.

[0003] Due to the unique advantages of Spiking Neural Networks (SNNs), such as low energy consumption and biological rationality, some works have attempted to apply SNNs to image object detection tasks. For example, some have directly replaced the activation function in the traditional YOLO model with LIF neurons, or designed a meta-spikeformer-based classification network and fine-tuned it on object detection datasets. However, the object detection models described above suffer from low performance and are difficult to directly apply to real-world image detection scenarios.

[0004] How to achieve an accurate, efficient, and low-power target detection method is a technical problem that urgently needs to be solved in the field of computer vision. Summary of the Invention

[0005] This invention provides an application method for a pulse-driven target detection model to address the problem that target detection models in the field of spiking neural networks have low performance and are difficult to apply directly to practical image detection scenarios.

[0006] This invention provides an application method for a pulse-driven target detection model, comprising:

[0007] A target detection model is constructed based on the YOLOv8 model. In the network architecture of the target detection model, two different SNN feature extraction modules are used to replace the C2F feature extraction module in the YOLOv8 model.

[0008] The image to be processed is acquired, and the static image data in the image to be processed is copied multiple times and input into the target detection model. The neuromorphic data in the image to be processed is aggregated frame by frame and input into the target detection model.

[0009] The target detection model is applied to process the static image data and the neuromorphic data respectively to obtain the target detection result of the image to be processed.

[0010] According to the application method of the pulse-driven target detection model provided by the present invention, any SNN feature extraction module includes a SepConv module and a ChannelConv module; the replacement of the C2F feature extraction module in the YOLOv8 model with two different SNN feature extraction modules includes:

[0011] The C2F feature extraction module of the shallow network in the YOLOv8 model is replaced by a first SNN feature extraction module. The ChannelConv module in the first SNN feature extraction module is constructed based on standard convolution.

[0012] The C2F feature extraction module of the deep network in the YOLOv8 model is replaced by a second SNN feature extraction module. The ChannelConv module in the second SNN feature extraction module is constructed based on reparameterized convolution.

[0013] According to the application method of the target detection model based on pulse driving provided by the present invention, the ChannelConv module in the first SNN feature extraction module consists of two standard convolutions with a kernel size of 3×3.

[0014] According to the application method of the target detection model based on pulse driving provided by the present invention, the ChannelConv module in the second SNN feature extraction module is composed of two reparameterized convolutions with a kernel size of 3×3, and there are no nonlinear neurons inside either convolution kernel.

[0015] According to the application method of the target detection model based on pulse driving provided by the present invention, the SepConv module in any SNN feature extraction module includes a depthwise separable convolution and a tail depthwise convolution; wherein, the depthwise separable convolution is composed of two point convolutions and a depthwise convolution.

[0016] According to the application method of the target detection model based on pulse driving provided by the present invention, there are no nonlinear neurons between the depthwise separable convolution and the tail depthwise convolution.

[0017] According to the application method of the pulse-driven target detection model provided by the present invention, a target detection model is constructed based on the YOLOv8 model, including:

[0018] The connection method between the backbone, neck and detection head of the YOLOv8 model is maintained in the network architecture of the target detection model.

[0019] The present invention also provides an application device based on a pulse-driven target detection model, comprising:

[0020] The model building unit is used to build an object detection model based on the YOLOv8 model. In the network architecture of the object detection model, two different SNN feature extraction modules are used to replace the C2F feature extraction module in the YOLOv8 model.

[0021] The image processing unit is used to acquire the image to be processed, copy the static image data in the image to be processed multiple times and input them into the target detection model, and aggregate the neuromorphic data in the image to be processed frame by frame and input them into the target detection model.

[0022] The image detection unit is used to process the static image data and the neuromorphic data using the target detection model to obtain the target detection result of the image to be processed.

[0023] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the application method of the pulse-driven target detection model as described above.

[0024] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the application method of the pulse-driven target detection model as described above.

[0025] The present invention also provides a computer program product, including a computer program that, when executed by a processor, implements the application method of the pulse-driven target detection model as described above.

[0026] The application method of the pulse-driven target detection model provided by this invention replaces the C2F feature extraction module in the YOLOv8 model with two different SNN feature extraction modules, so that the target detection model adopts full addition pulse operation. This provides a target detection model with high accuracy and low energy consumption, which can achieve good detection performance on image data of both static datasets and neuromorphic datasets. It solves the problem that the target detection model in the field of spiking neural networks has low performance and is difficult to be directly applied to actual image detection scenarios. Attached Figure Description

[0027] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0028] Figure 1 This is a flowchart illustrating an application method of a pulse-driven target detection model provided by the present invention.

[0029] Figure 2 This is one of the schematic diagrams of the principle of a target detection model provided by the present invention.

[0030] Figure 3 This is the second schematic diagram of the principle of the target detection model provided by the present invention.

[0031] Figure 4 This is one of the schematic diagrams showing the test results of the target detection model provided by the present invention.

[0032] Figure 5 This is the second schematic diagram showing the test results of the target detection model provided by this invention.

[0033] Figure 6 This is the third schematic diagram showing the test results of the target detection model provided by this invention.

[0034] Figure 7 This is the fourth schematic diagram showing the test results of the target detection model provided by this invention.

[0035] Figure 8 This is a schematic diagram of the structure of an application device based on a pulse-driven target detection model provided by the present invention.

[0036] Figure 9 This is a schematic diagram of the structure of the electronic device provided by the present invention. Detailed Implementation

[0037] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0038] The terms "first," "second," etc., used in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such descriptions can be used interchangeably where appropriate to allow embodiments to be implemented in a sequence other than that illustrated or described in this application. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or device that includes a series of steps or modules is not necessarily limited to those explicitly listed, but may include other steps or modules not explicitly listed or inherent to such processes, methods, products, or devices. The naming or numbering of steps appearing in this application does not imply that the steps in the method flow must be performed in the chronological / logical order indicated by the naming or numbering. The execution order of named or numbered process steps can be changed according to the desired technical purpose, as long as the same or similar technical effect is achieved. The module division described in this application is a logical division. In practical applications, there may be other division methods. For example, multiple modules may be combined or integrated into another system, or some features may be ignored or not executed. In addition, the shown or discussed mutual coupling, direct coupling, or communication connection may be through some interface, and the indirect coupling or communication connection between units may be electrical or other similar forms, none of which are limited in this application. Furthermore, the modules or sub-modules described as separate components may or may not be physically separated, may or may not be physical units, or may be distributed in multiple circuit units. Some or all of the units can be selected to achieve the purpose of the solution in this application according to actual needs.

[0039] In an embodiment of the present invention, an application method based on a pulse-driven target detection model is provided. The target detection model in this embodiment of the present invention references the feature extraction architecture of YOLOv8 at the macro level and draws on the meta SNN module design of Meta SpikeFormer at the micro level.

[0040] The methods at the macro-design level include: spiking neural network blocks based on convolutional neural networks; and variations and fusion of resolution and channel count between network blocks. The methods at the micro-design level include: implementation methods for shallow spiking-driven feature extraction modules; implementation methods for deep spiking-driven feature extraction modules, such as large-kernel convolution to expand the model's receptive field, and the application of reparameterized convolution in spiking neural networks.

[0041] Based on the aforementioned architectural design, the model's performance is improved, achieving excellent object detection performance on the static dataset COCO. Furthermore, the model is versatile, capable of handling various morphological information, and also achieves good performance on the neuromorphic dataset Gen1. Simultaneously, because the model retains its spiking characteristic, its overall power consumption is low, fully utilizing the computational characteristics of spiking neural networks to significantly reduce computational costs and power consumption. This facilitates the edge deployment of high-performance, low-power spiking-driven object detection neural networks.

[0042] The following is combined with Figures 1-7 The specific contents of this invention are described below.

[0043] like Figure 1 As shown, this invention provides an application method for a target detection model based on pulse driving, including steps S101-S103.

[0044] Step S101: Construct an object detection model based on the YOLOv8 model. In the network architecture of the object detection model, two different SNN feature extraction modules are used to replace the C2F feature extraction module in the YOLOv8 model.

[0045] In one possible implementation, a target detection model is constructed based on the YOLOv8 model, specifically including maintaining the connection method between the backbone, neck, and detection head of the YOLOv8 model in the network architecture of the target detection model. In an embodiment of the present invention, the target detection model is a pulse-driven target detection model.

[0046] Specifically, such as Figure 2 The diagram shown illustrates the principle of an object detection model provided in an embodiment of the present invention. The network architecture of the object detection model maintains the connection method between the backbone, neck, and detection head of the YOLOv8 model; simultaneously, two different SNN feature extraction modules replace the C2F feature extraction module in the YOLOv8 model.

[0047] In one possible implementation, any SNN feature extraction module includes a SepConv module and a ChannelConv module; replacing the C2F feature extraction module in the YOLOv8 model with two different SNN feature extraction modules includes steps S101A and S101B.

[0048] In embodiments of the present invention, any SNN feature extraction module comprises a SepConv module and a ChannelConv module, such as... Figure 3As shown. SepConv represents the SepConv module, and ChannelConv represents the ChannelConv module. The difference between the two SNN feature extraction modules lies in the differences in the ChannelConv modules that make them up.

[0049] The direct relationship between the input and output of any SNN feature extraction module can be expressed by the formula:

[0050]

[0051]

[0052] in, This is the input data for the SepConv module in the SNN feature extraction module; This is the output data of the SepConv module, which is the input data of the ChannelConv module; This is the output of the SNN feature extraction module.

[0053] In the embodiments of the present invention, two different SNN feature extraction modules are used in the shallow and deep networks of the target detection model network architecture, as detailed below.

[0054] Step S101A: Replace the C2F feature extraction module of the shallow network in the YOLOv8 model with the first SNN feature extraction module. The ChannelConv module in the first SNN feature extraction module is constructed based on standard convolution.

[0055] In one possible implementation, the ChannelConv module in the first SNN feature extraction module consists of two standard convolutions with a kernel size of 3×3, and there are no nonlinear neurons inside either convolution kernel.

[0056] The first SNN feature extraction module provided by this invention is described below:

[0057] like Figure 3 As shown, in the shallow network, the ChannelConv module in the first SNN feature extraction module consists of two standard convolutions with a kernel size of 3×3, used to further enhance the extraction of local features. In the first SNN feature extraction module, the ChannelConv module can be described by the following formula:

[0058]

[0059] Step S101B: Replace the C2F feature extraction module of the deep network in the YOLOv8 model with the second SNN feature extraction module. The ChannelConv module in the second SNN feature extraction module is constructed based on reparameterized convolution.

[0060] In one possible implementation, the ChannelConv module in the second SNN feature extraction module consists of two reparameterized convolutions with a kernel size of 3×3, and there are no nonlinear neurons inside either convolution kernel.

[0061] The second SNN feature extraction module provided by this invention is described below:

[0062] like Figure 3 As shown, in the deep network, the ChannelConv module in the second SNN feature extraction module consists of two reparameterized convolutions with a kernel size of 3×3. The ChannelConv module does not contain nonlinear neurons, so it can be reparameterized into a standard convolution during inference. This design can effectively reduce the number of parameters in the network.

[0063] In one possible implementation, the SepConv module in any SNN feature extraction module includes a depthwise separable convolution and a tail depthwise convolution; wherein the depthwise separable convolution consists of two point convolutions and a depthwise convolution.

[0064] In one possible implementation, there are no nonlinear neurons between the depthwise separable convolution and the tail depthwise convolution.

[0065] Specifically, such as Figure 3 As shown, the SepConv module includes a depthwise separable convolution and a tail-level depthwise convolution. The depthwise separable convolution consists of two point convolutions and a depthwise convolution. The two point convolutions are used to expand and reduce the number of channels, while the tail-level depthwise convolution is used to perform spatial feature fusion on the expanded feature map. The depthwise separable convolution uses a large 7×7 kernel, which allows the network to have a large receptive field while increasing the number of parameters almost. There are no non-linear neurons between the tail-level depthwise convolution and the depthwise separable convolution, thus ensuring that the last point convolution of the depthwise separable convolution can be reparameterized into the tail-level depthwise convolution during inference.

[0066] The SepConv module can be described by the formula as follows:

[0067]

[0068] in, , , For the input of the SepConv module; for Figure 3 In the middle, the output of the SepConv module is the first point convolution of PWConv from top to bottom; This is the output of the first depthwise convolution DWConv in the SepConv model from top to bottom; subsequently, The output of the second point convolution of PWConv from the top of the SepConv module. This is the output of the second depthwise convolution DWConv from top to bottom in the SepConv module.

[0069] Step S102: Obtain the image to be processed. For static images, copy the image to be processed multiple times and input it into the target detection model. For neuromorphic data, aggregate it frame by frame and input it into the target detection model.

[0070] like Figure 2 As shown, the static image data from the image to be processed is copied multiple times and input into the object detection model; the neuromorphic data from the image to be processed is aggregated frame by frame and input into the object detection model. The object detection model obtained by the above network architecture design improves the performance of the object detection model, achieving excellent object detection performance on the static dataset COCO. In addition, this object detection model is also versatile, capable of handling various information morphologies, and also achieves excellent performance on the neuromorphic dataset Gen1. For details, please refer to the subsequent examples of performance testing of the object detection model.

[0071] Step S103: Apply the target detection model to process the static image data and the neuromorphic data respectively to obtain the target detection results of the image to be processed.

[0072] Specifically, object detection models are applied to process static image data and neuromorphic data, including: Figure 2 As shown, two SNN feature extraction modules are used in the backbone and neck of the target detection model to extract and fuse the data features of the input static image data and neuromorphic data, obtaining the output membrane potential of each target category and its location in the image data. The detection head of the target detection model averages the output results of each frame and then performs post-processing such as non-maximum suppression to obtain the target detection result of the image.

[0073] The following is an example of performance testing of the target detection model provided by the present invention.

[0074] For the first embodiment of performance testing, in the target detection model provided by this invention, performance tests for target detection were conducted on the COCO dataset for four parameter ranges: 13.2M, 23.1M, 48.1M, and 68.8M. Specific results are as follows: Figure 4 As shown in the table, T represents the time step, and D represents the quantization value of the neuron spike. This model achieved an accuracy of 66.2% mAP@50 and 48.9% mAP@50:95 at a time step of 68.8M and a neuron spike quantization value of 4, significantly exceeding the accuracy of other existing models.

[0075] Among them Figures 4 to 7 In this context, Architecture, Model, Param, and Power represent the model architecture, model name, number of parameters, and power consumption parameters, respectively. mAP@50 (%) represents the measurement accuracy at an IoU threshold of 0.50, focusing on the model's ability to correctly detect objects. mAP@50:95 (%) represents the average accuracy across a range of IoU thresholds, providing a comprehensive evaluation of detection performance. The IoU threshold measures the degree of overlap between the predicted bounding box and the ground reality bounding box; a higher value indicates higher localization accuracy.

[0076] In the neuromorphic dataset, this invention achieved 67.2% mAP@50 and 40.4% mAP@50:95% with 23.1M parameters, also surpassing other models, as shown in the results. Figure 5 As shown, this demonstrates that the present invention achieves good performance on both static and neuromorphic datasets.

[0077] For the second embodiment of performance testing, to demonstrate the effectiveness of the SNN feature extraction module provided by this invention, the architecture was converted to an ANN version, and the original YOLOv8 was directly converted to an SNN version. The results are as follows: Figure 6 As shown in the image. Tests revealed that directly converting YOLOv8 to the SNN version causes severe impulse degradation, leading to performance degradation.

[0078] For the third embodiment of performance testing, under a model with 23.1M parameters, this invention explored the specific framework of SpikeYOLO, and the results are as follows: Figure 7 As shown: The first row shows the baseline for accuracy and energy consumption of the 23.1M model.

[0079] First, regarding the reparameterized convolution used in this study, reparameterization is achieved by removing the nonlinearity within consecutive convolutions. Adding spiking neurons to consecutive convolutions results in a 1.7% decrease in accuracy. This demonstrates that reparameterized convolution plays a crucial role in the model's local feature extraction.

[0080] Secondly, regarding the two SNN feature extraction modules proposed in this invention, when we replace the shallow first SNN feature extraction module (SNN-Block-1) with the deep second SNN feature extraction module (SNN-Block-2), the accuracy decreased by 1.1%, indicating that the dense convolutions in the early stages of the object detection model can effectively achieve fast feature extraction. When we replace the deep second SNN feature extraction module (SNN-Block-2) with a Transformer Block, the model performance decreased by 1.2%, indicating that the Transformer Block's method of modeling global information is not suitable for the object detection model in this invention.

[0081] Finally, replacing the anchor-free head with an anchor-based head reduced the object detection model's performance by 2.8%. This is likely because spiking neurons can only output binary information. Unlike anchor-based heads that directly predict the target bounding box, anchor-free heads predict the probability distribution of the target bounding box, which can compensate for the information loss caused by binary spiking to some extent.

[0082] By employing the above method, the present invention can achieve at least one of the following beneficial effects:

[0083] 1. By replacing the C2F feature extraction module in the YOLOv8 model with two different SNN feature extraction modules, the target detection model adopts full addition pulse operation, which provides a high-precision and low-energy target detection model that can achieve good detection performance on image data of both static datasets and neuromorphic datasets.

[0084] 2. In the SepConv and Channel Conv modules of the SNN feature extraction module in this invention, the use of nonlinear neurons is minimized as much as possible, so that the convolution module can be reparameterized, effectively reducing the number of parameters in the target detection model and optimizing the performance of the target detection model.

[0085] The following describes the application device of the pulse-driven target detection model provided by the present invention. The application device of the pulse-driven target detection model described below and the application method of the pulse-driven target detection model described above can be referred to in correspondence.

[0086] like Figure 8 The diagram shown is a structural schematic of an application device for a pulse-driven target detection model provided by the present invention. The device includes:

[0087] The model building unit 810 is used to build an object detection model based on the YOLOv8 model. In the network architecture of the object detection model, two different SNN feature extraction modules are designed to replace the C2F feature extraction module in the YOLOv8 model.

[0088] The image processing unit 820 is used to acquire the image to be processed, copy the static image data in the image to be processed multiple times and input them into the target detection model, and aggregate the neuromorphic data in the image to be processed frame by frame and input them into the target detection model.

[0089] The image detection unit 830 is used to process static image data and neuromorphic data respectively using a target detection model to obtain the target detection result of the image to be processed.

[0090] In one possible implementation, any SNN feature extraction module includes a SepConv module and a ChannelConv module; the model building unit 810 includes:

[0091] The first model construction subunit is used to replace the C2F feature extraction module of the shallow network in the YOLOv8 model with the first SNN feature extraction module. The ChannelConv module in the first SNN feature extraction module is constructed based on standard convolution.

[0092] The second model building subunit is used to replace the C2F feature extraction module of the deep network in the YOLOv8 model with the second SNN feature extraction module. The ChannelConv module in the second SNN feature extraction module is built based on standard convolution.

[0093] In one possible implementation, the model building unit 810 further includes a second model building subunit for maintaining the connection between the backbone, neck, and detection head of the YOLOv8 model in the network architecture of the target detection model.

[0094] Figure 9 An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 9 As shown, the electronic device may include: a processor 910, a communication interface 920, a memory 930, and a communication bus 940, wherein the processor 910, the communication interface 920, and the memory 930 communicate with each other via the communication bus 940. The processor 910 can call logical instructions in the memory 930 to execute an application method based on a pulse-driven target detection model, the method including:

[0095] An object detection model is built based on the YOLOv8 model. In the network architecture of the object detection model, two different SNN feature extraction modules are used to replace the C2F feature extraction module in the YOLOv8 model.

[0096] The image to be processed is acquired, and the static image data in the image to be processed is copied multiple times and input into the target detection model. The neuromorphic data in the image to be processed is aggregated frame by frame and input into the target detection model.

[0097] The target detection model is applied to process static image data and neuromorphic data respectively to obtain the target detection results of the image to be processed.

[0098] Furthermore, the logical instructions in the aforementioned memory 930 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, essentially, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0099] On the other hand, the present invention also provides a computer program product, the computer program product comprising a computer program that can be stored on a non-transitory computer-readable storage medium, wherein when the computer program is executed by a processor, the computer is able to execute the application method of the pulse-driven target detection model provided by the above methods, the method comprising:

[0100] An object detection model is built based on the YOLOv8 model. In the network architecture of the object detection model, two different SNN feature extraction modules are used to replace the C2F feature extraction module in the YOLOv8 model.

[0101] Acquire the image to be processed, copy the static image data in the image to be processed multiple times and input them into the target detection model, or aggregate the neuromorphic data in the image to be processed frame by frame and input them into the target detection model;

[0102] The target detection model is applied to process static image data and neuromorphic data respectively to obtain the target detection results of the image to be processed.

[0103] In another aspect, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements an application method for the pulse-driven target detection model provided by the above methods, the method comprising:

[0104] An object detection model is built based on the YOLOv8 model. In the network architecture of the object detection model, two different SNN feature extraction modules are used to replace the C2F feature extraction module in the YOLOv8 model.

[0105] Acquire the image to be processed, copy the static image data in the image to be processed multiple times and input them into the target detection model, or aggregate the neuromorphic data in the image to be processed frame by frame and input them into the target detection model;

[0106] The target detection model is applied to process static image data and neuromorphic data respectively to obtain the target detection results of the image to be processed.

[0107] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0108] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0109] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. An application method for a target detection model based on pulse-driven detection, characterized in that, include: A target detection model is constructed based on the YOLOv8 model. In the network architecture of the target detection model, two different SNN feature extraction modules are used to replace the C2F feature extraction module in the YOLOv8 model. The image to be processed is acquired, and the static image data in the image to be processed is copied multiple times and input into the target detection model. The neuromorphic data in the image to be processed is aggregated frame by frame and input into the target detection model. The target detection model is applied to process the static image data and the neuromorphic data respectively to obtain the target detection result of the image to be processed; Any SNN feature extraction module includes the SepConv module and the ChannelConv module; The replacement of the C2F feature extraction module in the YOLOv8 model with two different SNN feature extraction modules includes: The C2F feature extraction module of the shallow network in the YOLOv8 model is replaced by a first SNN feature extraction module. The ChannelConv module in the first SNN feature extraction module is constructed based on standard convolution. The C2F feature extraction module of the deep network in the YOLOv8 model is replaced by a second SNN feature extraction module. The ChannelConv module in the second SNN feature extraction module is constructed based on reparameterized convolution.

2. The application method of the target detection model based on pulse driving according to claim 1, characterized in that, The ChannelConv module in the first SNN feature extraction module consists of two standard convolutions with a kernel size of 3×3.

3. The application method of the target detection model based on pulse driving according to claim 1, characterized in that, The ChannelConv module in the second SNN feature extraction module consists of two reparameterized convolutions with a kernel size of 3×3, and there are no nonlinear neurons inside either kernel.

4. The application method of the target detection model based on pulse driving according to claim 1, characterized in that, The SepConv module in any SNN feature extraction module includes a depthwise separable convolution and a tail depthwise convolution; wherein the depthwise separable convolution consists of two point convolutions and a depthwise convolution.

5. The application method of the target detection model based on pulse driving according to claim 4, characterized in that, There are no nonlinear neurons between the depthwise separable convolution and the tail depthwise convolution.

6. The application method of the target detection model based on pulse driving according to claim 1, characterized in that, A target detection model based on the YOLOv8 model is constructed, including: The connection method between the backbone, neck and detection head of the YOLOv8 model is maintained in the network architecture of the target detection model.

7. An application device based on a pulse-driven target detection model, characterized in that, include: The model building unit is used to build an object detection model based on the YOLOv8 model. In the network architecture of the object detection model, two different SNN feature extraction modules are used to replace the C2F feature extraction module in the YOLOv8 model. The image processing unit is used to acquire the image to be processed, copy the static image data in the image to be processed multiple times and input them into the target detection model, and aggregate the neuromorphic data in the image to be processed frame by frame and input them into the target detection model. An image detection unit is used to apply the target detection model to process the static image data and the neuromorphic data respectively, so as to obtain the target detection result of the image to be processed; Any SNN feature extraction module includes the SepConv module and the ChannelConv module; The replacement of the C2F feature extraction module in the YOLOv8 model with two different SNN feature extraction modules includes: The C2F feature extraction module of the shallow network in the YOLOv8 model is replaced by a first SNN feature extraction module. The ChannelConv module in the first SNN feature extraction module is constructed based on standard convolution. The C2F feature extraction module of the deep network in the YOLOv8 model is replaced by a second SNN feature extraction module. The ChannelConv module in the second SNN feature extraction module is constructed based on reparameterized convolution.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that, When the processor executes the computer program, it implements the application method of the pulse-driven target detection model as described in any one of claims 1 to 6.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the application method of the pulse-driven target detection model as described in any one of claims 1 to 6.

Images