Application method of target detection model based on pulse driving

Abstract

The invention provides an application method of a target detection model based on pulse driving, the target detection model is a pulse-driven target detection model, and the method comprises the following steps: constructing the target detection model based on a YOLOv8 model, and in a network architecture of the target detection model, replacing a C2F feature extraction module in the YOLOv8 model with two different SNN feature extraction modules; acquiring a to-be-processed image, copying multiple pieces of static image data in the to-be-processed image, inputting the copied static image data into the target detection model, and aggregating and inputting neuromorphic data in the to-be-processed image into the target detection model according to frames; and processing the static image data and the neuromorphic data by using the target detection model to obtain a target detection result of the to-be-processed image. Through the method, the problem that the target detection model is difficult to be directly applied to an actual image detection scene due to relatively low performance in the field of spiking neural networks is solved.

Description

Technical Field

[0001] The present invention relates to the technical field of neural networks, and in particular to an application method of a pulse-driven target detection model. Background Art

[0002] Object detection in images is a very challenging visual task, which has broad application prospects in fields such as autonomous driving and face recognition. In the field of artificial neural networks (ANN), object detection models represented by YOLO have achieved remarkable success.

[0003] Since the Spiking Neural Network (SNN) has unique advantages such as low energy consumption and biological plausibility, some works have tried to apply SNN to the task of object detection in images, such as directly replacing the activation function in the traditional YOLO model with LIF neurons, or designing a classification network based on meta-spikeformer and fine-tuning it on the object detection dataset. However, the object detection model in the above method is difficult to be directly applied to actual image detection scenarios due to the problem of low performance.

[0004] How to achieve accurate, efficient and low-power target detection methods is a technical problem that needs to be solved urgently in the field of computer vision. Summary of the invention

[0005] The present invention provides an application method of a pulse-driven target detection model to solve the problem that the target detection model in the field of pulse neural networks has low performance and is difficult to be directly applied to actual image detection scenarios.

[0006] The present invention provides an application method of a pulse-driven target detection model, comprising: Building a target detection model based on the YOLOv8 model, wherein in the network architecture of the target detection model, two different SNN feature extraction modules are used to replace the C2F feature extraction module in the YOLOv8 model; Acquire an image to be processed, copy multiple copies of static image data in the image to be processed and input them into the target detection model, and aggregate the neuromorphic data in the image to be processed by frames and input them into the target detection model; The target detection model is applied to process the static image data and the neuromorphic data respectively to obtain a target detection result of the image to be processed.

[0007] According to the application method of the pulse-driven target detection model provided by the present invention, any SNN feature extraction module includes a SepConv module and a ChannelConv module; the use of two different SNN feature extraction modules to replace the C2F feature extraction module in the YOLOv8 model includes: A first SNN feature extraction module is used to replace the C2F feature extraction module of the shallow network in the YOLOv8 model, wherein the ChannelConv module in the first SNN feature extraction module is constructed based on standard convolution; A second SNN feature extraction module is used to replace the C2F feature extraction module of the deep network in the YOLOv8 model, and the ChannelConv module in the second SNN feature extraction module is constructed based on re-parameterized convolution.

[0008] According to the application method of the pulse-driven target detection model provided by the present invention, the ChannelConv module in the first SNN feature extraction module is composed of two standard convolutions with a convolution kernel size of 3×3.

[0009] According to the application method of the pulse-driven target detection model provided by the present invention, the ChannelConv module in the second SNN feature extraction module is composed of two re-parameterized convolutions with a convolution kernel size of 3×3, and there are no nonlinear neurons inside any convolution kernel.

[0010] According to the application method of the pulse-driven target detection model provided by the present invention, the SepConv module in any SNN feature extraction module includes a depthwise separable convolution and a tail depthwise convolution; wherein the depthwise separable convolution is composed of two point convolutions and one depthwise convolution.

[0011] According to the application method of the pulse-driven target detection model provided by the present invention, there are no nonlinear neurons between the depthwise separable convolution and the tail depthwise convolution.

[0012] According to the application method of the pulse-driven target detection model provided by the present invention, a target detection model is constructed based on the YOLOv8 model, including: The connection between the trunk, neck and detection head of the YOLOv8 model is maintained in the network architecture of the target detection model.

[0013] The present invention also provides an application device based on the pulse-driven target detection model, comprising: A model building unit, used to build a target detection model based on the YOLOv8 model, wherein in the network architecture of the target detection model, two different SNN feature extraction modules are used to replace the C2F feature extraction module in the YOLOv8 model; An image processing unit, used to obtain an image to be processed, copy multiple copies of static image data in the image to be processed and input them into the target detection model, and aggregate the neuromorphic data in the image to be processed by frame and input them into the target detection model; An image detection unit is used to apply the target detection model to process the static image data and the neuromorphic data respectively to obtain a target detection result of the image to be processed.

[0014] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, an application method of a pulse-driven target detection model as described above is implemented.

[0015] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements an application method of a pulse-driven target detection model as described in any one of the above.

[0016] The present invention also provides a computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the computer program implements the application method of the pulse-driven target detection model as described in any one of the above.

[0017] The application method of the pulse-driven target detection model provided by the present invention replaces the C2F feature extraction module in the YOLOv8 model by designing two different SNN feature extraction modules, so that all full-addition pulse operations are adopted in the target detection model, and a target detection model with high precision and low energy consumption is provided. Good detection performance can be achieved on image data of static data sets and neural morphological data sets at the same time, which solves the problem that the target detection model in the field of pulse neural networks has low performance and is difficult to be directly applied to actual image detection scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0019] Figure 1It is a flow chart of an application method of a pulse-driven target detection model provided by the present invention.

[0020] Figure 2 It is one of the principle schematic diagrams of the principle schematic diagram of a target detection model provided by the present invention.

[0021] Figure 3 This is the second principle schematic diagram of the principle schematic diagram of a target detection model provided by the present invention.

[0022] Figure 4 This is one of the test result schematic diagrams of the performance test of the target detection model provided by the present invention.

[0023] Figure 5 This is the second schematic diagram of the test results of the performance test of the target detection model provided by the present invention.

[0024] Figure 6 This is the third schematic diagram of the test results of the performance test of the target detection model provided by the present invention.

[0025] Figure 7 This is the fourth schematic diagram of the test results of the performance test of the target detection model provided by the present invention.

[0026] Figure 8 It is a structural schematic diagram of an application device of a pulse-driven target detection model provided by the present invention.

[0027] Fig. 9 It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0028] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0029] The terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the descriptions used in this way can be interchanged where appropriate, so that the embodiments can be implemented in a sequence other than that illustrated or described in the present application. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or modules is not necessarily limited to those steps or modules that are clearly listed, but may include other steps or modules that are not clearly listed or inherent to these processes, methods, products or devices. The naming or numbering of the steps that appear in the present application does not mean that the steps in the method flow must be executed in the time / logical sequence indicated by the naming or numbering. The process steps that have been named or numbered can change the execution order according to the technical purpose to be achieved, as long as the same or similar technical effects can be achieved. The division of modules in this application is a logical division. There may be other division methods when it is implemented in actual applications. For example, multiple modules can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, and the indirect coupling or communication connection between units can be electrical or other similar forms, which are not limited in this application. In addition, the modules or submodules described as separate components may or may not be physically separated, may or may not be physical units, or may be distributed in multiple circuit units, and some or all of the units may be selected according to actual needs to achieve the purpose of the present application.

[0030] In an embodiment of the present invention, an application method of a pulse-driven target detection model is provided. The target detection model in the embodiment of the present invention refers to the feature extraction architecture of YOLOv8 at a macro level, and draws on the meta SNN module design of Meta SpikeFormer at a micro level.

[0031] The methods at the macro level include: pulse neural network blocks based on convolutional neural networks; changes and fusions in resolution and number of channels between network blocks. The methods at the micro level include: implementation methods of shallow pulse-driven feature extraction modules; implementation methods of deep pulse-driven feature extraction modules, such as large kernel convolution to expand the model receptive field, and the application of re-parameterized convolution in pulse neural networks.

[0032] Based on the above architecture design, the performance of the model has been improved, and good target detection performance has been achieved on the static dataset COCO. In addition, the model is also versatile and can process a variety of morphological information. It has also achieved good performance on the neuromorphic dataset Gen1. At the same time, because the model as a whole still maintains the characteristics of pulse drive, the overall power consumption of the model is low, and the computing characteristics of the pulse neural network can be fully utilized, which greatly reduces the computing cost and power consumption, and is conducive to the edge deployment of high-performance and low-power pulse-driven target detection neural networks.

[0033] Combine the following Figure 1-Figure 7 The specific contents of the present invention are described.

[0034] like Figure 1 As shown, an application method of a pulse-driven target detection model provided by the present invention includes steps S101-S103.

[0035] Step S101, building a target detection model based on the YOLOv8 model. In the network architecture of the target detection model, two different SNN feature extraction modules are used to replace the C2F feature extraction module in the YOLOv8 model.

[0036] In a possible implementation, building a target detection model based on a YOLOv8 model specifically includes: maintaining the connection between the trunk, the neck, and the detection head of the YOLOv8 model in the network architecture of the target detection model. In an embodiment of the present invention, the target detection model is a pulse-driven target detection model.

[0037] Specifically, if Figure 2 FIG. 1 is a schematic diagram of the principle of a target detection model provided by an embodiment of the present invention. The connection between the trunk, neck and detection head of the YOLOv8 model is maintained in the network architecture of the target detection model; at the same time, two different SNN feature extraction modules are used to replace the C2F feature extraction module in the YOLOv8 model.

[0038] In a possible implementation, any one of the SNN feature extraction modules includes a SepConv module and a ChannelConv module; using two different SNN feature extraction modules to replace the C2F feature extraction module in the YOLOv8 model includes step S101A and step S101B.

[0039] In an embodiment of the present invention, any SNN feature extraction module includes a SepConv module and a ChannelConv module, such as Figure 3As shown in the figure, SepConv represents the SepConv module, and ChannelConv represents the ChannelConv module. The difference between the two SNN feature extraction modules lies in the differences in the ChannelConv modules that constitute them.

[0040] The direct relationship between the input and output of any SNN feature extraction module can be expressed as:

[0041] in, It is the input data of SepConv module in SNN feature extraction module; is the output data of the SepConv module, i.e., the input data of the ChannelConv module; is the output of the SNN feature extraction module.

[0042] In an embodiment of the present invention, two different SNN feature extraction modules are used in the shallow network and the deep network in the network architecture of the target detection model, and the specific contents are as follows.

[0043] Step S101A, using the first SNN feature extraction module to replace the C2F feature extraction module of the shallow network in the YOLOv8 model, and the ChannelConv module in the first SNN feature extraction module is constructed based on standard convolution.

[0044] In a possible implementation, the ChannelConv module in the first SNN feature extraction module is composed of two standard convolutions with a convolution kernel size of 3×3, and there is no nonlinear neuron inside any convolution kernel.

[0045] The first SNN feature extraction module provided by the present invention is described below: like Figure 3 As shown in the figure, in the shallow network, the ChannelConv module in the first SNN feature extraction module consists of two standard convolutions with a convolution kernel size of 3×3, which is used to further enhance the extraction of local features. In the first SNN feature extraction module, the ChannelConv module can be described by the formula:

[0046] Step S101B, using the second SNN feature extraction module to replace the C2F feature extraction module of the deep network in the YOLOv8 model, and the ChannelConv module in the second SNN feature extraction module is constructed based on re-parameterized convolution.

[0047] In a possible implementation, the ChannelConv module in the second SNN feature extraction module is composed of two re-parameterized convolutions with a convolution kernel size of 3×3, and there is no nonlinear neuron inside any convolution kernel.

[0048] The second SNN feature extraction module provided by the present invention is described below: like Figure 3 As shown in the figure, in the deep network, the ChannelConv module in the second SNN feature extraction module consists of two re-parameterized convolutions with a convolution kernel size of 3×3. The ChannelConv module does not contain nonlinear neurons, so it can be re-parameterized into a standard convolution during inference. This design can effectively reduce the number of network parameters.

[0049] In a possible implementation, the SepConv module in any SNN feature extraction module includes a depthwise separable convolution and a tail depthwise convolution; wherein the depthwise separable convolution is composed of two pointwise convolutions and one depthwise convolution.

[0050] In one possible implementation, there is no nonlinear neuron between the depthwise separable convolution and the tail depthwise convolution.

[0051] Specifically, if Figure 3 As shown in the figure, the SepConv module includes a depthwise separable convolution and a tail depthwise convolution, where the depthwise separable convolution consists of two point convolutions and a depthwise convolution. The two point convolutions are used to expand and reduce the number of channels, while the tail depthwise convolution is used to perform spatial feature fusion on the feature map after the channels are expanded. The depthwise separable convolution uses a large 7×7 convolution kernel, which gives the network a larger receptive field while almost increasing the number of parameters. There are no nonlinear neurons between the tail depthwise convolution and the depthwise separable convolution, which ensures that the last point convolution of the depthwise separable convolution can be re-parameterized into the tail depthwise convolution during inference.

[0052] The SepConv module can be described by the formula:

[0053] in, , , It is the input of SepConv module; for Figure 3 In the figure, the first point from top to bottom of the SepConv module convolves the output of PWConv; is the output of the first depth convolution DWConv of the SepConv model from top to bottom; sequentially, The output of the second point convolution PWConv from top to bottom of the SepConv module, It is the output of the second depth convolution DWConv from top to bottom of the SepConv module.

[0054] Step S102, obtaining an image to be processed. For a static image, multiple copies of the image to be processed are input into a target detection model; for neuromorphic data, the image is aggregated frame by frame and input into the target detection model.

[0055] like Figure 2 As shown, multiple copies of the static image data in the image to be processed are input into the target detection model; the neural morphological data in the image to be processed is aggregated by frame and input into the target detection model. The target detection model obtained by the above network architecture design has improved performance and achieved good target detection performance on the static dataset COCO. In addition, the target detection model is also versatile and can process a variety of information forms, and has also achieved good performance on the neural morphological dataset Gen1. For details, please refer to the subsequent implementation example of the performance test of the target detection model.

[0056] Step S103, applying the target detection model to process the static image data and the neuromorphic data respectively to obtain the target detection result of the image to be processed.

[0057] Specifically, the object detection model is applied to process static image data and neuromorphic data respectively, including: Figure 2 As shown in the figure, two SNN feature extraction modules are used in the backbone (BackBone) and neck (Neck) of the target detection model to extract and fuse the data features of the input static image data and neuromorphic data, and obtain the output membrane potential of the category and location of each target in the image data. The detection head (Head) of the target detection model averages the output results of each frame, and obtains the target detection result of the image through post-processing such as non-maximum suppression.

[0058] The following is an example of a performance test of the target detection model provided by the present invention.

[0059] For the first embodiment of the performance test, in the target detection model provided by the present invention, the performance test of target detection is performed on the COCO dataset for four levels of parameter amounts, namely 13.2M, 23.1M, 48.1M, and 68.8M. The specific results are as follows: Figure 4 As shown in the table, T represents the time step and D represents the quantized value of the neuron pulse. This model achieved an accuracy of 66.2% mAP@50 and 48.9% mAP@50:95 at 68.8M and a neuron pulse quantization value of 4, far exceeding other existing models.

[0060] Among them Figures 4 to 7 In the figure, Architecture, Model, Param, and Power represent the model architecture, model name, parameter quantity, and energy consumption parameter, respectively; mAP@50 (%) is the accuracy measured when the IoU threshold is 0.50, focusing on the model's ability to correctly detect objects. mAP@50: 95 (%) is the average accuracy in a series of IoU thresholds, which comprehensively evaluates the detection performance. The IoU threshold is a parameter that measures the degree of overlap between the predicted bounding box and the ground truth bounding box. The higher the value, the higher the positioning accuracy.

[0061] In the neuromorphic dataset, the present invention achieved 67.2% mAP@50 (%) and 40.4% mAP@50: 95 (%) with 23.1M parameters, which also exceeded other models. The results are as follows Figure 5 It is shown that the present invention can achieve good performance in both static datasets and neuromorphic datasets.

[0062] For the second embodiment of the performance test, in order to prove the effectiveness of the SNN feature extraction module provided by the present invention, the present invention converts the present architecture into an ANN version and directly converts the original YOLOv8 into an SNN version. The results are as follows: Figure 6 The test found that directly converting YOLOv8 into the SNN version will cause serious pulse degradation problems, resulting in reduced performance.

[0063] For the third embodiment of the performance test, under the model with 23.1M parameters, the present invention conducted a specific framework exploration on SpikeYOLO, and the results are as follows: Figure 7 As shown: The first row shows the baseline of accuracy and energy consumption of the 23.1M model.

[0064] First, for the reparameterized convolution used in this study, reparameterization is achieved by removing the nonlinearity inside the continuous convolution. If spiking neurons are added to the continuous convolution, the accuracy will be reduced by 1.7%. This shows that the reparameterized convolution plays a crucial role in the local feature extraction of the model.

[0065] Secondly, for the two SNN feature extraction modules proposed in the present invention, when we replaced the shallow first SNN feature extraction module (SNN-Block-1) with the deep second SNN feature extraction module (SNN-Block-2), the accuracy dropped by 1.1%, indicating that the early dense convolution of the target detection model can effectively realize fast feature extraction. When we replaced the deep second SNN feature extraction module (SNN-Block-2) with Transformer Block, the model performance dropped by 1.2%, indicating that the way Transformer Block models global information is not suitable for the target detection model in the present invention.

[0066] Finally, after replacing the Anchor-free head with the Anchor-based head, the performance of the object detection model decreased by 2.8%. This may be because the spike neurons can only output binary information. Unlike the Anchor-based head that directly predicts the target bounding box, the Anchor-free head predicts the probability distribution of the target bounding box, which can compensate for the information loss caused by the binary spikes to a certain extent.

[0067] By adopting the above method, the present invention can achieve at least one of the following beneficial effects: 1. By using two different SNN feature extraction modules to replace the C2F feature extraction module in the YOLOv8 model, all-additive pulse operations are used in the target detection model, providing a high-precision and low-energy target detection model, which can achieve good detection performance on image data of both static datasets and neuromorphic datasets.

[0068] 2. In the SepConv module and Channel Conv module of the SNN feature extraction module of the present invention, the use of nonlinear neurons is reduced as much as possible, so that the convolution module can be re-parameterized, which effectively reduces the number of parameters of the target detection model and optimizes the performance of the target detection model.

[0069] The application device of the pulse-driven target detection model provided by the present invention is described below. The application device of the pulse-driven target detection model described below and the application method of the pulse-driven target detection model described above can be referred to each other.

[0070] like Figure 8 As shown, the present invention provides a schematic diagram of the structure of an application device based on a pulse-driven target detection model, the device comprising: A model building unit 810 is used to build a target detection model based on the YOLOv8 model. In the network architecture of the target detection model, two different SNN feature extraction modules are designed to replace the C2F feature extraction module in the YOLOv8 model. An image processing unit 820 is used to obtain an image to be processed, copy multiple copies of static image data in the image to be processed and input them into a target detection model, and aggregate the neuromorphic data in the image to be processed by frame and input them into the target detection model; The image detection unit 830 is used to apply the target detection model to process the static image data and the neuromorphic data respectively to obtain the target detection result of the image to be processed.

[0071] In a possible implementation, any SNN feature extraction module includes a SepConv module and a ChannelConv module; the model building unit 810 includes: The first model construction subunit is used to replace the C2F feature extraction module of the shallow network in the YOLOv8 model with the first SNN feature extraction module, and the ChannelConv module in the first SNN feature extraction module is constructed based on standard convolution.

[0072] The second model construction subunit is used to replace the C2F feature extraction module of the deep network in the YOLOv8 model with the second SNN feature extraction module, and the ChannelConv module in the second SNN feature extraction module is constructed based on standard convolution.

[0073] In a possible implementation, the model construction unit 810 further includes: a second model construction subunit, which is used to maintain the connection between the trunk, neck and detection head of the YOLOv8 model in the network architecture of the target detection model.

[0074] Fig. 9 An example of a physical structure diagram of an electronic device is shown in FIG. Fig. 9 As shown, the electronic device may include: a processor 910, a communication interface 920, a memory 930 and a communication bus 940, wherein the processor 910, the communication interface 920 and the memory 930 communicate with each other through the communication bus 940. The processor 910 may call the logic instructions in the memory 930 to execute the application method of the pulse-driven target detection model, which method includes: The target detection model is built based on the YOLOv8 model. In the network architecture of the target detection model, two different SNN feature extraction modules are used to replace the C2F feature extraction module in the YOLOv8 model. Obtain an image to be processed, copy multiple copies of static image data in the image to be processed and input them into a target detection model, and aggregate the neuromorphic data in the image to be processed by frames and input them into the target detection model; The object detection model is applied to process the static image data and the neuromorphic data respectively to obtain the object detection results of the image to be processed.

[0075] In addition, the logic instructions in the above-mentioned memory 930 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.

[0076] On the other hand, the present invention further provides a computer program product, the computer program product comprising a computer program, the computer program can be stored on a non-transitory computer-readable storage medium, and when the computer program is executed by a processor, the computer can execute the application method of the pulse-driven target detection model provided by the above methods, the method comprising: The target detection model is built based on the YOLOv8 model. In the network architecture of the target detection model, two different SNN feature extraction modules are used to replace the C2F feature extraction module in the YOLOv8 model. Obtain an image to be processed, copy multiple copies of static image data in the image to be processed and input them into the target detection model, or aggregate the neuromorphic data in the image to be processed by frames and input them into the target detection model; The object detection model is applied to process the static image data and the neuromorphic data respectively to obtain the object detection results of the image to be processed.

[0077] In another aspect, the present invention further provides a non-transitory computer-readable storage medium having a computer program stored thereon, which is implemented when the computer program is executed by a processor to execute the application method of the pulse-driven target detection model provided by the above methods, the method comprising: The target detection model is built based on the YOLOv8 model. In the network architecture of the target detection model, two different SNN feature extraction modules are used to replace the C2F feature extraction module in the YOLOv8 model. Obtain an image to be processed, copy multiple copies of static image data in the image to be processed and input them into the target detection model, or aggregate the neuromorphic data in the image to be processed by frames and input them into the target detection model; The object detection model is applied to process the static image data and the neuromorphic data respectively to obtain the object detection results of the image to be processed.

[0078] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0079] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0080] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. An application method of a pulse-driven target detection model, characterized in that: include: Building a target detection model based on the YOLOv8 model, wherein in the network architecture of the target detection model, two different SNN feature extraction modules are used to replace the C2F feature extraction module in the YOLOv8 model; Acquire an image to be processed, copy multiple copies of static image data in the image to be processed and input them into the target detection model, and aggregate the neuromorphic data in the image to be processed by frames and input them into the target detection model; The target detection model is applied to process the static image data and the neuromorphic data respectively to obtain a target detection result of the image to be processed.

2. The application method of the pulse-driven target detection model according to claim 1, characterized in that: Any SNN feature extraction module includes a SepConv module and a ChannelConv module; The two different SNN feature extraction modules are used to replace the C2F feature extraction module in the YOLOv8 model, including: A first SNN feature extraction module is used to replace the C2F feature extraction module of the shallow network in the YOLOv8 model, wherein the ChannelConv module in the first SNN feature extraction module is constructed based on standard convolution; A second SNN feature extraction module is used to replace the C2F feature extraction module of the deep network in the YOLOv8 model, and the ChannelConv module in the second SNN feature extraction module is constructed based on re-parameterized convolution.

3. The application method of the pulse-driven target detection model according to claim 2 is characterized in that: The ChannelConv module in the first SNN feature extraction module consists of two standard convolutions with a convolution kernel size of 3×3.

4. The application method of the pulse-driven target detection model according to claim 2 is characterized in that: The ChannelConv module in the second SNN feature extraction module is composed of two re-parameterized convolutions with a convolution kernel size of 3×3, and there is no nonlinear neuron inside any convolution kernel.

5. The application method of the pulse-driven target detection model according to claim 2, characterized in that: The SepConv module in any SNN feature extraction module includes a depthwise separable convolution and a tail depthwise convolution; wherein the depthwise separable convolution is composed of two pointwise convolutions and one depthwise convolution.

6. The application method of the pulse-driven target detection model according to claim 5, characterized in that: There is no nonlinear neuron between the depthwise separable convolution and the tail depthwise convolution.

7. The application method of the pulse-driven target detection model according to claim 1, characterized in that: Build a target detection model based on the YOLOv8 model, including: The connection between the trunk, neck and detection head of the YOLOv8 model is maintained in the network architecture of the target detection model.

8. An application device based on a pulse-driven target detection model, characterized in that: include: A model building unit, used to build a target detection model based on the YOLOv8 model, wherein in the network architecture of the target detection model, two different SNN feature extraction modules are used to replace the C2F feature extraction module in the YOLOv8 model; An image processing unit, used to obtain an image to be processed, copy multiple copies of static image data in the image to be processed and input them into the target detection model, and aggregate the neuromorphic data in the image to be processed by frame and input them into the target detection model; An image detection unit is used to apply the target detection model to process the static image data and the neuromorphic data respectively to obtain a target detection result of the image to be processed.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, it implements the public cultural resource recommendation device method based on graph neural network as described in any one of claims 1 to 7.

10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, it implements the public cultural resource recommendation device method based on graph neural network as described in any one of claims 1 to 7.

Images