Brain-like pulse neural network infrared target detection method and system
Through the infrared object detection method of brain-like pulse neural network, combined with pulse coding and transfer learning technology, the problem of low robustness of infrared object detection in the existing technology is solved, and efficient infrared object detection is achieved in the background of noise and complexity, and efficient integration is achieved on embedded devices.
Patent Information
- Application Number
- CN202311639999.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2023-12-04
- Publication Date
- 2025-06-06
AI Technical Summary
In the prior art, the infrared object detection method has low robustness, especially in the case of a large influence on the background environment and noise, it is difficult to effectively detect infrared objects.
The infrared object detection method of brain-like pulse neural network is adopted, and the infrared image data is converted into encoded feature vectors through the pulse encoding layer and input it to the first type of brain pulse object detection model for detection. At the same time, pulse data generated by visible light event cameras are introduced, and the model is trained using transfer learning to improve infrared object detection performance.
It improves the robustness and accuracy of infrared target detection, can effectively detect infrared targets in complex noise and background environments, and deploys the trained model on embedded devices to achieve efficient integration in resource-limited environments.
Smart Images
Figure CN120107638A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of computer vision and brain-like intelligence technology, and in particular to a brain-like pulse neural network infrared target detection method and system. Background Art
[0002] Infrared target detection has important application value in the field of unmanned systems. Using brain-like pulse neural networks to detect targets under infrared images can effectively improve the task energy consumption ratio. However, in the field of brain-like pulse research, infrared target detection faces more difficulties than visible light target detection. First, the sampling cost of infrared pulse data is high, and it is difficult to directly obtain infrared event cameras for data construction; although the use of temporal Poisson coding can simulate the random discharge behavior of neurons, this method has the defect of incomplete temporal information in the process of pulse time emission for a single image, and is sensitive to noise. Especially for infrared images, due to differences in background environment, atmospheric conditions, detector type and other factors, the target is more significantly affected by noise.
[0003] Pulse data is one of the core elements of brain-like infrared target detection systems. Unlike traditional neural networks, the input of brain-like pulse neural networks relies on discrete pulse data. Pulse neural networks represent the activation of neurons with discrete events, and simulate the pulse information transmission between biological neurons in an event-driven manner, that is, calculations are performed only when events occur. Pulse coding is still an important means of generating pulse data. How to effectively and accurately characterize the discrete pulse characteristics of the target is the key to improving the robustness of infrared target detection methods. Summary of the invention
[0004] The present invention provides a brain-like pulse neural network infrared target detection method and system, which can solve the technical problem of low robustness of infrared target detection methods in the prior art.
[0005] According to one aspect of the present invention, a brain-like pulse neural network infrared target detection method is provided, the method comprising:
[0006] S10, inputting the infrared image data into the pulse coding layer to obtain a coding feature vector corresponding to the infrared image data;
[0007] S20, inputting the encoded feature vector corresponding to the infrared image data into the first type of brain pulse target detection model to obtain an infrared domain target detection result;
[0008] S30, inputting the pulse data generated by the visible light event camera into the second type of brain pulse target detection model to obtain a visible light domain target detection result;
[0009] S40, repeating S10 to S30 to train the first type of brain pulse target detection model and the second type of brain pulse target detection model to obtain an optimal pulse coding layer and an optimal first type of brain pulse target detection model; wherein, during the training process, transfer learning processing is performed on the infrared domain target detection results and the visible light domain target detection results, so that the first type of brain pulse target detection model learns from the second type of brain pulse target detection model;
[0010] S50. Input the infrared target to be detected into the optimal pulse coding layer to obtain the coding feature vector corresponding to the infrared target to be detected, and input the coding feature vector corresponding to the infrared target to be detected into the optimal first-class brain pulse target detection model to realize the detection of the infrared target.
[0011] Preferably, the structure of the first type of brain pulse target detection model is the same as the structure of the second type of brain pulse target detection model, and both of the two brain pulse target detection models include a feature extraction part, a feature fusion part and a detection part which are connected in sequence.
[0012] Preferably, the feature extraction parts of the two brain-like pulse target detection models both include several pulse residual blocks, each pulse residual block includes two convolutional layers and two leakage integration and release neuron functions, and the input value of each pulse residual block is sequentially input into the first convolutional layer, the first leakage integration and release neuron function, and the second convolutional layer of the current pulse residual block to obtain the output value of the second convolutional layer of the current pulse residual block, and the input value of the current pulse residual block and the output value of the second convolutional layer are added and input into the second leakage integration and release neuron function to obtain the output value of the current pulse residual block.
[0013] Preferably, in S20 of the present invention, the encoded feature vector corresponding to the infrared image data is input into the first type of brain pulse target detection model, and the infrared domain target detection result is obtained, including:
[0014] S21, inputting the encoded feature vector corresponding to the infrared image data into the feature extraction part of the first type of brain pulse target detection model to obtain a plurality of first extracted feature vectors of different levels;
[0015] S22, inputting the first extracted feature vectors of several different levels into the feature fusion part of the first type of brain pulse target detection model to obtain a first fused feature vector;
[0016] S23, inputting the first fused feature vector into the detection part of the first type of brain pulse target detection model to obtain a number of infrared domain target detection results with different resolutions;
[0017] The number of levels of the first extracted feature vector is the same as the number of resolutions of the infrared domain target detection result.
[0018] Preferably, in S30 of the present invention, the pulse data generated by the visible light event camera is input into the second type of brain pulse target detection model, and the visible light domain target detection result is obtained, including:
[0019] S31, inputting the pulse data generated by the visible light event camera into the feature extraction part of the second type of brain pulse target detection model to obtain a plurality of second extracted feature vectors of different levels;
[0020] S32, inputting the second extracted feature vectors of several different levels into the feature fusion part of the second type brain pulse target detection model to obtain a second fused feature vector;
[0021] S33, inputting the second fused feature vector into the detection part of the second type of brain pulse target detection model to obtain a number of visible light domain target detection results with different resolutions;
[0022] The number of levels of the second extracted feature vector is the same as the number of resolutions of the visible light domain target detection result.
[0023] Preferably, in S40 of the present invention, S10 to S30 are repeated to train the first type of brain pulse target detection model and the second type of brain pulse target detection model, and obtaining the optimal pulse coding layer and the optimal first type of brain pulse target detection model includes:
[0024] S41, repeat S10 to S30, and obtain a number of first extracted feature vectors of different levels, a number of second extracted feature vectors of different levels, and infrared domain target detection results in each training process;
[0025] S42, based on the first extracted feature vectors of several different levels and the second extracted feature vectors of several different levels in each training process, obtaining the maximized mean difference between the first extracted feature vectors of several different levels and the second extracted feature vectors of several different levels in each training process;
[0026] S43, obtaining the classification loss and bounding box loss of the infrared domain target detection result based on the infrared domain target detection result in each training process;
[0027] S44, obtaining a total loss corresponding to each training based on the maximized mean difference between the first extracted feature vectors of several different levels and the second extracted feature vectors of several different levels in each training process, the classification loss and the bounding box loss of the infrared domain target detection result;
[0028] S45. The pulse coding layer and the first type of brain pulse target detection model corresponding to the minimum total loss are used as the optimal pulse coding layer and the optimal first type of brain pulse target detection model.
[0029] Preferably, the total loss corresponding to each training is obtained by the following formula:
[0030] Loss=L(D s ,D t )+L cls +L reg
[0031] In the formula, Loss represents the total loss, L(D s ,D t ) represents the maximum mean difference between the first extracted feature vectors of several different levels and the second extracted feature vectors of several different levels, L cls represents the classification loss of infrared domain target detection results, L reg Represents the bounding box loss of infrared domain object detection results.
[0032] Preferably, the maximum mean difference between the first extracted feature vectors of several different levels and the second extracted feature vectors of several different levels in each training process is obtained by the following formula:
[0033]
[0034] In the formula, x i Represents the second extracted feature vectors at several different levels, y i represents the first extracted feature vectors of several different levels, n represents the number of training samples in the visible light domain, m represents the number of training samples in the infrared domain, and f(x) represents the mapping of the extracted feature vector in the regenerated Hilbert space.
[0035] According to another aspect of the present invention, a brain-like pulse neural network infrared target detection system is provided, the system comprising:
[0036] An acquisition module, used for acquiring infrared targets to be measured and inputting the infrared targets to be measured into the embedded device;
[0037] An embedded device, in which the above-mentioned optimal pulse coding layer and the optimal first-class brain pulse target detection model are deployed; the optimal pulse coding layer is used to obtain a coding feature vector based on the infrared target to be detected, and the optimal first-class brain pulse target detection model is used to detect the coding feature vector.
[0038] The technical solution of the present invention is applied to solve the problem of difficulty in characterizing infrared pulse events through end-to-end learning. In addition, the pulse data generated by the visible light event camera is introduced into the pulse neural network training process, and the infrared target detection performance of the pulse neural network is improved by transfer learning. At the same time, in order to solve the problem of integrated implementation of the infrared target detection system under a platform with limited volume and power consumption, the trained model can be deployed on an embedded device, making the integrated implementation of infrared target detection in a resource-limited environment more efficient. BRIEF DESCRIPTION OF THE DRAWINGS
[0039] The included drawings are used to provide a further understanding of the embodiments of the present invention, which constitute a part of the specification, are used to illustrate the embodiments of the present invention, and together with the text description, explain the principles of the present invention. Obviously, the drawings in the following description are only some embodiments of the present invention, and for ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.
[0040] Figure 1 A flowchart of a brain-like pulse neural network infrared target detection method according to an embodiment of the present invention is shown;
[0041] Figure 2 An architecture diagram of a brain-like pulse neural network infrared target detection method provided according to an embodiment of the present invention is shown;
[0042] Figure 3 A schematic structural diagram of a brain-like pulse neural network infrared target detection system provided according to an embodiment of the present invention is shown. DETAILED DESCRIPTION
[0043] It should be noted that, in the absence of conflict, the embodiments in this application and the features in the embodiments can be combined with each other. The technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. The following description of at least one exemplary embodiment is actually only illustrative and is by no means intended to limit the present invention and its application or use. Based on the embodiments in the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0044] It should be noted that the terms used herein are only for describing specific embodiments and are not intended to limit the exemplary embodiments according to the present application. As used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. In addition, it should be understood that when the terms "comprise" and / or "include" are used in this specification, it indicates the presence of features, steps, operations, devices, components and / or combinations thereof.
[0045] Unless otherwise specifically stated, the relative arrangement of the parts and steps described in these embodiments, numerical expressions and numerical values do not limit the scope of the present invention. At the same time, it should be understood that, for ease of description, the sizes of the various parts shown in the accompanying drawings are not drawn according to the actual proportional relationship. The technology, method and equipment known to ordinary technicians in the relevant field may not be discussed in detail, but in appropriate cases, the technology, method and equipment should be regarded as a part of the authorization specification. In all examples shown and discussed here, any specific value should be interpreted as being merely exemplary, rather than as a limitation. Therefore, other examples of exemplary embodiments may have different values. It should be noted that similar reference numerals and letters represent similar items in the following drawings, so once a certain item is defined in an accompanying drawing, it does not need to be further discussed in subsequent drawings.
[0046] like Figure 1 As shown, the present invention provides a brain-like pulse neural network infrared target detection method, the method comprising:
[0047] S10, inputting the infrared image data into the pulse coding layer to obtain a coding feature vector corresponding to the infrared image data;
[0048] S20, inputting the encoded feature vector corresponding to the infrared image data into the first type of brain pulse target detection model to obtain an infrared domain target detection result;
[0049] S30, inputting the pulse data generated by the visible light event camera into the second type of brain pulse target detection model to obtain a visible light domain target detection result;
[0050] S40, repeating S10 to S30 to train the first type of brain pulse target detection model and the second type of brain pulse target detection model to obtain an optimal pulse coding layer and an optimal first type of brain pulse target detection model; wherein, during the training process, transfer learning processing is performed on the infrared domain target detection results and the visible light domain target detection results, so that the first type of brain pulse target detection model learns from the second type of brain pulse target detection model;
[0051] S50. Input the infrared target to be detected into the optimal pulse coding layer to obtain the coding feature vector corresponding to the infrared target to be detected, and input the coding feature vector corresponding to the infrared target to be detected into the optimal first-class brain pulse target detection model to realize the detection of the infrared target.
[0052] In the present invention, the target detection result includes target category attributes, confidence and target bounding box position information.
[0053] The present invention solves the problem of difficulty in characterizing infrared pulse events through end-to-end learning. In addition, the pulse data generated by the visible light event camera is introduced into the pulse neural network training process, and the transfer learning method is used to improve the infrared target detection performance of the pulse neural network.
[0054] According to one embodiment of the present invention, the structure of the first type of brain pulse target detection model is the same as the structure of the second type of brain pulse target detection model, and both of the two brain pulse target detection models include a feature extraction part, a feature fusion part and a detection part which are connected in sequence.
[0055] Specifically, Figure 2 As shown, the feature extraction parts of the two brain-like pulse target detection models include several pulse residual blocks, each of which includes two convolutional layers and two leakage integration and release neuron functions. The input value of each pulse residual block is sequentially input into the first convolutional layer, the first leakage integration and release neuron function, and the second convolutional layer of the current pulse residual block to obtain the output value of the second convolutional layer of the current pulse residual block, and the input value of the current pulse residual block and the output value of the second convolutional layer are added and input into the second leakage integration and release neuron function to obtain the output value of the current pulse residual block.
[0056] According to an embodiment of the present invention, in S20 of the present invention, the encoded feature vector corresponding to the infrared image data is input into the first type of brain pulse target detection model, and the infrared domain target detection result is obtained, including:
[0057] S21, inputting the encoded feature vector corresponding to the infrared image data into the feature extraction part of the first type of brain pulse target detection model to obtain a plurality of first extracted feature vectors of different levels;
[0058] S22, inputting the first extracted feature vectors of several different levels into the feature fusion part of the first type of brain pulse target detection model to obtain a first fused feature vector;
[0059] S23, inputting the first fused feature vector into the detection part of the first type of brain pulse target detection model to obtain a number of infrared domain target detection results with different resolutions;
[0060] The number of levels of the first extracted feature vector is the same as the number of resolutions of the infrared domain target detection result.
[0061] According to an embodiment of the present invention, in S30 of the present invention, the pulse data generated by the visible light event camera is input into the second type of brain pulse target detection model, and the visible light domain target detection result is obtained, including:
[0062] S31, inputting the pulse data generated by the visible light event camera into the feature extraction part of the second type of brain pulse target detection model to obtain a plurality of second extracted feature vectors of different levels;
[0063] S32, inputting the second extracted feature vectors of several different levels into the feature fusion part of the second type brain pulse target detection model to obtain a second fused feature vector;
[0064] S33, inputting the second fused feature vector into the detection part of the second type of brain pulse target detection model to obtain a number of visible light domain target detection results with different resolutions;
[0065] The number of levels of the second extracted feature vector is the same as the number of resolutions of the visible light domain target detection result.
[0066] According to an embodiment of the present invention, in S40 of the present invention, repeating S10 to S30 to train the first type of brain pulse target detection model and the second type of brain pulse target detection model to obtain the optimal pulse coding layer and the optimal first type of brain pulse target detection model includes:
[0067] S41, repeat S10 to S30, and obtain a number of first extracted feature vectors of different levels, a number of second extracted feature vectors of different levels, and infrared domain target detection results in each training process;
[0068] S42, based on the first extracted feature vectors of several different levels and the second extracted feature vectors of several different levels in each training process, obtaining the maximized mean difference between the first extracted feature vectors of several different levels and the second extracted feature vectors of several different levels in each training process;
[0069] The maximum mean difference between the first extracted feature vectors of several different levels and the second extracted feature vectors of several different levels in each training process is obtained by the following formula:
[0070]
[0071] In the formula, x i Represents the second extracted feature vectors at several different levels, y irepresents the first extracted feature vectors of several different levels, n represents the number of training samples in the visible light domain, m represents the number of training samples in the infrared domain, and f(x) represents the mapping of the extracted feature vector in the regenerated Hilbert space;
[0072] S43, obtaining the classification loss and bounding box loss of the infrared domain target detection result based on the infrared domain target detection result in each training process;
[0073] S44, obtaining a total loss corresponding to each training based on the maximized mean difference between the first extracted feature vectors of several different levels and the second extracted feature vectors of several different levels in each training process, the classification loss and the bounding box loss of the infrared domain target detection result;
[0074] Among them, the total loss corresponding to each training is obtained by the following formula:
[0075] Loss=L(D s ,D t )+L cls +L reg
[0076] In the formula, Loss represents the total loss, L(D s ,D t ) represents the maximum mean difference between the first extracted feature vectors of several different levels and the second extracted feature vectors of several different levels, L cls represents the classification loss of infrared domain target detection results, L reg Represents the bounding box loss of infrared domain object detection results;
[0077] S45. The pulse coding layer and the first type of brain pulse target detection model corresponding to the minimum total loss are used as the optimal pulse coding layer and the optimal first type of brain pulse target detection model.
[0078] In order to further understand the present invention, the following Figure 1 and Figure 2 The brain-like pulse neural network infrared target detection method of the present invention is described in detail, and specifically comprises the following steps:
[0079] Step 1: Input the infrared image data into the pulse coding layer to obtain the coding feature vector corresponding to the infrared image data; the pulse coding layer is responsible for completing the mapping function from the input image to the pulse event coding, and automatically learns to adjust the network parameters through the end-to-end training process to optimize the coding quality, thereby improving the overall performance of the target detection task. The collected infrared image is input into the pulse coding layer, which consists of three convolution blocks and one activation block to realize the mapping of the infrared image frame to the pulse sequence with a value of 0 / 1.
[0080] Step 2: Input the encoded feature vector corresponding to the infrared image data into the first type of brain pulse target detection model, where the brain pulse target detection model is constructed based on the feature pyramid network (FPN). By integrating features at different levels, a feature pyramid is constructed to solve multi-scale target detection. The FPN architecture includes three parts: bottom-up feature extraction, top-down feature propagation, and multi-scale feature fusion. Figure 2 As shown, the specific steps are as follows:
[0081] 2-1. Input the encoded feature vector corresponding to the infrared image data into the feature extraction part of the first type of brain pulse target detection model to obtain the first extracted feature vectors of 3 different levels. The feature extraction part is modified based on the classic backbone network residual neural network (ResNet-50, a total of 50 layers), and ResNet-50 consists of 5 residual blocks. In order to use ResNet-50 to extract features of the pulse sequence, it is necessary to reconstruct the pulse residual block so that each layer of input can receive the pulse encoding and timing information features contained in the output result of the previous layer. Then modify the activation function in each residual block, and replace the original linear rectification activation function (ReLU) with the leaky integrated release neuron (LIF) function. The LIF function is defined as:
[0082] U(t,pre)=τ decay U(t-1)+WX(t);
[0083] Where U(t,pre) represents the pre-discharge membrane potential at the t-th time step, U(t-1) represents the membrane potential at the t-1th time step, and τ decay represents the time decay constant describing the decay of the membrane potential, W represents the weight, and X(t) represents the binary map from the previous layer at the t-th time step. If U(t,pre) exceeds a threshold, the spiking neuron will generate a pulse and then the membrane voltage will reset to 0, as shown below:
[0084]
[0085] U(t)=U(t,pre)·(1-O(t));
[0086] Where V th Represents a given discharge threshold, which can be set to 0.5, O(t) represents the output of the leaky integrate-send neuron activation function, and U(t) represents the membrane potential at the tth time step.
[0087] 2-2. Top-down feature transfer and feature fusion. The first extracted feature vectors of three different levels output by the last three pulse residual blocks are upsampled and transferred downward to expand the size of the feature map. Figure 2As shown in the figure, taking the first extracted feature vector as an example with 3 levels and 3 detection heads, the first extracted feature vectors of 3 different levels output by the last 3 pulse residual blocks are respectively reduced in dimension by a 1*1 convolution to obtain the feature map after dimensionality reduction. Figure 1 One path is directly output to the detection head, the other path is upsampled and added to the feature map of the previous layer after dimensionality reduction. After addition, one path is output to the detection head after 3*3 convolution, and the other path is upsampled and added to the feature map of the previous layer after dimensionality reduction... and so on.
[0088] 2-3. Result output. The feature map is processed by three detection heads to obtain infrared domain target detection results at different resolutions, namely target category attributes, confidence, and target bounding box position information.
[0089] Step three: input the pulse data generated by the visible light event camera into the second type of brain pulse target detection model to obtain the visible light domain target detection result; the process is the same as step two and will not be repeated here.
[0090] Step 4: Repeat steps 1 to 3 to train the first type of brain pulse target detection model and the second type of brain pulse target detection model to obtain the optimal pulse coding layer and the optimal first type of brain pulse target detection model.
[0091] Step 5: Input the infrared target to be detected into the optimal pulse coding layer to obtain the coding feature vector corresponding to the infrared target to be detected, and input the coding feature vector corresponding to the infrared target to be detected into the optimal first-class brain pulse target detection model to realize the detection of the infrared target.
[0092] like Figure 3 As shown, the present invention provides a brain-like pulse neural network infrared target detection system, the system comprising:
[0093] An acquisition module, used for acquiring infrared targets to be measured and inputting the infrared targets to be measured into the embedded device;
[0094] An embedded device, in which the above-mentioned optimal pulse coding layer and the optimal first-class brain pulse target detection model are deployed; the optimal pulse coding layer is used to obtain a coding feature vector based on the infrared target to be detected, and the optimal first-class brain pulse target detection model is used to detect the coding feature vector.
[0095] In order to solve the problem of integrated implementation of infrared target detection system under a platform with limited volume and power consumption, the present invention can deploy the trained model on an embedded device, so that the integrated implementation of infrared target detection in an environment with limited resources is more efficient.
[0096] In summary, the present invention provides a brain-like pulse neural network infrared target detection method and system, which solves the problem of difficult characterization of infrared pulse events through end-to-end learning; in addition, the pulse data generated by the visible light event camera is introduced into the pulse neural network training process, and the transfer learning method is used to improve the performance of pulse neural network infrared target detection. At the same time, in order to solve the problem of integrated implementation of infrared target detection system under the platform with limited volume and power consumption, the trained model can be deployed on embedded devices, making the integrated implementation of infrared target detection in resource-limited environments more efficient.
[0097] Parts of the present invention that are not described in detail are well known to those skilled in the art.
[0098] In the description of the present invention, it is necessary to understand that the directions or positional relationships indicated by directional words such as "front, back, up, down, left, right", "lateral, vertical, perpendicular, horizontal" and "top, bottom" are usually based on the directions or positional relationships shown in the drawings. They are only for the convenience of describing the present invention and simplifying the description. Unless otherwise specified, these directional words do not indicate or imply that the devices or elements referred to must have a specific direction or be constructed and operated in a specific direction. Therefore, they cannot be understood as limiting the scope of protection of the present invention. The directional words "inside and outside" refer to the inside and outside relative to the contours of each component itself.
[0099] For ease of description, spatially relative terms such as "above", "above", "on the upper surface of", "above", etc. may be used here to describe the spatial positional relationship between a device or feature and other devices or features as shown in the figure. It should be understood that spatially relative terms are intended to include different orientations of the device in use or operation in addition to the orientation described in the figure. For example, if the device in the accompanying drawings is inverted, the device described as "above other devices or structures" or "above other devices or structures" will be positioned as "below other devices or structures" or "below other devices or structures". Thus, the exemplary term "above" can include both "above" and "below". The device can also be positioned in other different ways (rotated 90 degrees or in other orientations), and the spatially relative descriptions used here are interpreted accordingly.
[0100] In addition, it should be noted that the use of terms such as "first" and "second" to limit components is only for the convenience of distinguishing the corresponding components. If not otherwise stated, the above terms have no special meaning and therefore cannot be understood as limiting the scope of protection of the present invention.
[0101] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.
Claims
1. A brain-like pulse neural network infrared target detection method, It is characterized in that The method comprises: S10, inputting the infrared image data into the pulse coding layer to obtain a coding feature vector corresponding to the infrared image data; S20, inputting the encoded feature vector corresponding to the infrared image data into the first type of brain pulse target detection model to obtain an infrared domain target detection result; S30, inputting the pulse data generated by the visible light event camera into the second type of brain pulse target detection model to obtain a visible light domain target detection result; S40, repeating S10 to S30 to train the first type of brain pulse target detection model and the second type of brain pulse target detection model to obtain an optimal pulse coding layer and an optimal first type of brain pulse target detection model; wherein, during the training process, transfer learning processing is performed on the infrared domain target detection results and the visible light domain target detection results, so that the first type of brain pulse target detection model learns from the second type of brain pulse target detection model; S50. Input the infrared target to be detected into the optimal pulse coding layer to obtain the coding feature vector corresponding to the infrared target to be detected, and input the coding feature vector corresponding to the infrared target to be detected into the optimal first-class brain pulse target detection model to realize the detection of the infrared target.
2. The method according to claim 1, It is characterized in that The structure of the first type of brain pulse target detection model is the same as that of the second type of brain pulse target detection model. Both of the two brain pulse target detection models include a feature extraction part, a feature fusion part and a detection part that are connected in sequence.
3. The method according to claim 2, It is characterized in that The feature extraction parts of the two brain-like pulse target detection models both include several pulse residual blocks, each of which includes two convolutional layers and two leakage integration and release neuron functions. The input value of each pulse residual block is sequentially input into the first convolutional layer, the first leakage integration and release neuron function, and the second convolutional layer of the current pulse residual block to obtain the output value of the second convolutional layer of the current pulse residual block. The input value of the current pulse residual block and the output value of the second convolutional layer are added and input into the second leakage integration and release neuron function to obtain the output value of the current pulse residual block.
4. The method according to claim 2 or 3, It is characterized in that In S20 of the present invention, the encoded feature vector corresponding to the infrared image data is input into the first type of brain pulse target detection model, and the infrared domain target detection result is obtained, including: S21, inputting the encoded feature vector corresponding to the infrared image data into the feature extraction part of the first type of brain pulse target detection model to obtain a plurality of first extracted feature vectors of different levels; S22, inputting the first extracted feature vectors of several different levels into the feature fusion part of the first type of brain pulse target detection model to obtain a first fused feature vector; S23, inputting the first fused feature vector into the detection part of the first type of brain pulse target detection model to obtain a number of infrared domain target detection results with different resolutions; The number of levels of the first extracted feature vector is the same as the number of resolutions of the infrared domain target detection result.
5. The method according to claim 2 or 3, It is characterized in that In S30 of the present invention, the pulse data generated by the visible light event camera is input into the second type of brain pulse target detection model, and the visible light domain target detection result is obtained, including: S31, inputting the pulse data generated by the visible light event camera into the feature extraction part of the second type of brain pulse target detection model to obtain a plurality of second extracted feature vectors of different levels; S32, inputting the second extracted feature vectors of several different levels into the feature fusion part of the second type brain pulse target detection model to obtain a second fused feature vector; S33, inputting the second fused feature vector into the detection part of the second type of brain pulse target detection model to obtain a number of visible light domain target detection results with different resolutions; The number of levels of the second extracted feature vector is the same as the number of resolutions of the visible light domain target detection result.
6. The method according to any one of claims 1 to 5, It is characterized in that In S40 of the present invention, S10 to S30 are repeated to train the first type of brain pulse target detection model and the second type of brain pulse target detection model to obtain the optimal pulse coding layer and the optimal first type of brain pulse target detection model, including: S41, repeat S10 to S30, and obtain a number of first extracted feature vectors of different levels, a number of second extracted feature vectors of different levels, and infrared domain target detection results in each training process; S42, based on the first extracted feature vectors of several different levels and the second extracted feature vectors of several different levels in each training process, obtaining the maximized mean difference between the first extracted feature vectors of several different levels and the second extracted feature vectors of several different levels in each training process; S43, obtaining the classification loss and bounding box loss of the infrared domain target detection result based on the infrared domain target detection result in each training process; S44, obtaining a total loss corresponding to each training based on the maximized mean difference between the first extracted feature vectors of several different levels and the second extracted feature vectors of several different levels in each training process, the classification loss and the bounding box loss of the infrared domain target detection result; S45. The pulse coding layer and the first type of brain pulse target detection model corresponding to the minimum total loss are used as the optimal pulse coding layer and the optimal first type of brain pulse target detection model.
7. The method according to claim 1, It is characterized in that The total loss corresponding to each training is obtained by the following formula: Loss=L(D s ,D t )+L cls +L reg In the formula, Loss represents the total loss, L(D s ,D t ) represents the maximum mean difference between the first extracted feature vectors of several different levels and the second extracted feature vectors of several different levels, L cls represents the classification loss of infrared domain target detection results, L reg Represents the bounding box loss of infrared domain object detection results.
8. The method according to claim 7, It is characterized in that The maximum mean difference between the first extracted feature vectors of several different levels and the second extracted feature vectors of several different levels in each training process is obtained by the following formula: In the formula, x i Represents the second extracted feature vectors at several different levels, y i represents the first extracted feature vectors of several different levels, n represents the number of training samples in the visible light domain, m represents the number of training samples in the infrared domain, and f(x) represents the mapping of the extracted feature vector in the regenerated Hilbert space.
9. A brain-like pulse neural network infrared target detection system, It is characterized in that The system comprises: An acquisition module, used for acquiring infrared targets to be measured and inputting the infrared targets to be measured into the embedded device; An embedded device, in which the optimal pulse coding layer described in any one of claims 1-8 and the optimal first-class brain pulse target detection model described in any one of claims 1-8 are deployed; the optimal pulse coding layer is used to obtain a coding feature vector based on an infrared target to be detected, and the optimal first-class brain pulse target detection model is used to detect the coding feature vector.