Target detection method, device and equipment based on fully convolutional twin network, and medium

Abstract

The invention discloses a target detection method, device and equipment based on a fully convolutional twin network, and a medium. The method comprises the following steps: performing feature processing on a to-be-detected image and a preset template image through a convolutional layer in a preset full convolutional twin network to obtain a to-be-detected feature map and a template feature map; generating a corresponding pulse sequence from the to-be-detected feature map and the template feature map through a preset pulse frequency algorithm; performing similarity calculation on the to-be-detected feature map and the pulse sequence corresponding to the template feature map through the preset fully convolutional twin network to generate a response map; and performing offset calculation according to the response diagram to determine the target position of the to-be-detected target. By implementing the method provided by the invention, the problem that target detection cannot be efficiently carried out in the prior art can be solved.

Description

Technical Field

[0001] The present invention relates to the technical field of neural networks, and in particular to a target detection method, device, equipment and medium based on a fully convolutional twin network. Background Art

[0002] Target detection and tracking technology is a key technology in computer vision and is widely used in fields such as autonomous driving, intelligent monitoring, and drone navigation. With the development of deep learning, algorithms based on DNN (deep neural network) have made significant progress. These algorithms are mainly divided into two categories: generative and discriminative. Generative algorithms achieve detection and tracking by comparing target and background features, but they are prone to drift in complex environments. Discriminative algorithms such as Siamese-FC (fully convolutional twin network) detect by learning the difference between target and background, and the effect is good, but there are problems such as complex calculation, hardware dependence and high energy consumption. To solve these problems, researchers began to pay attention to SNN (spiking neural network) models. SNN simulates biological neuron activity and has advantages such as low power consumption, suitability for edge computing and bioinspiration. However, SNN has problems such as high training difficulty, limited feature extraction ability and network construction, which makes the existing technology unable to efficiently detect targets. Summary of the invention

[0003] The embodiments of the present invention provide a target detection method, device, equipment and medium based on a fully convolutional twin network, aiming to solve the problem that the prior art cannot efficiently perform target detection.

[0004] In the first aspect, an embodiment of the present invention provides a target detection method based on a fully convolutional twin network, comprising: performing feature processing on an image to be detected and a preset template image through a convolutional layer in a preset fully convolutional twin network to obtain a feature map to be detected and a template feature map; generating a corresponding pulse sequence from the feature map to be detected and the template feature map through a preset pulse frequency algorithm; performing similarity calculation on the pulse sequence corresponding to the feature map to be detected and the template feature map through the preset fully convolutional twin network to generate a response map; and performing offset calculation according to the response map to determine the target position of the target to be detected.

[0005] In the second aspect, an embodiment of the present invention also provides a target detection device based on a fully convolutional twin network, which includes: a feature processing unit, used to perform feature processing on the image to be detected and the preset template image through the convolution layer in the preset fully convolutional twin network to obtain a feature map to be detected and a template feature map; a sequence generation unit, used to generate a corresponding pulse sequence from the feature map to be detected and the template feature map through a preset pulse frequency algorithm; a calculation unit, used to perform similarity calculation on the pulse sequence corresponding to the feature map to be detected and the template feature map through the preset fully convolutional twin network to generate a response map; a determination unit, used to perform offset calculation according to the response map to determine the target position of the target to be detected.

[0006] In a third aspect, an embodiment of the present invention further provides a computer device, which includes a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the above method when executing the computer program.

[0007] In a fourth aspect, an embodiment of the present invention further provides a computer-readable storage medium, wherein the storage medium stores a computer program, wherein the computer program includes program instructions, and the program instructions can implement the above method when executed by a processor.

[0008] The embodiment of the present invention provides a method, device, equipment and medium for target detection based on a fully convolutional twin network. The method includes: performing feature processing on an image to be detected and a preset template image through a convolutional layer in a preset fully convolutional twin network to obtain a feature map to be detected and a template feature map; generating a corresponding pulse sequence from the feature map to be detected and the template feature map through a preset pulse frequency algorithm; performing similarity calculation on the pulse sequence corresponding to the feature map to be detected and the template feature map through the preset fully convolutional twin network to generate a response map; performing offset calculation based on the response map to determine the target position of the target to be detected. The embodiment of the present invention performs feature processing on the image to be detected and the preset template image through a preset full convolutional twin network to obtain a feature map, so as to facilitate subsequent comparison and matching based on the feature map, and efficiently converts the continuous value image data into a pulse sequence according to the pulse frequency algorithm. This conversion helps to reduce data redundancy and improve the efficiency of subsequent similarity calculations. By performing similarity calculations based on the preset full convolutional twin network to generate a response map, the target position is determined, thereby realizing a complete process from feature extraction to target position determination, and being able to perform target detection and tracking with low power consumption and high precision, thereby effectively improving the efficiency of target detection and tracking. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying any creative work.

[0010] Figure 1 A schematic diagram of a flow chart of a target detection method based on a fully convolutional twin network provided in an embodiment of the present invention;

[0011] Figure 2 A schematic diagram of a sub-process of a target detection method based on a fully convolutional twin network provided in an embodiment of the present invention;

[0012] Figure 3 A schematic diagram of a sub-process of a target detection method based on a fully convolutional twin network provided in an embodiment of the present invention;

[0013] Figure 4 A schematic diagram of a sub-process of a target detection method based on a fully convolutional twin network provided in an embodiment of the present invention;

[0014] Figure 5 A schematic diagram of a sub-process of a target detection method based on a fully convolutional twin network provided in an embodiment of the present invention;

[0015] Figure 6 A schematic diagram of a sub-process of a target detection method based on a fully convolutional twin network provided in an embodiment of the present invention;

[0016] Figure 7 A schematic block diagram of a target detection device based on a fully convolutional twin network provided by an embodiment of the present invention;

[0017] Figure 8 A schematic block diagram of a computer device provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0018] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0019] It should be understood that when used in this specification and the appended claims, the terms "include" and "comprises" indicate the presence of described features, integers, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or combinations thereof.

[0020] It should also be understood that the terms used in this specification of the present invention are only for the purpose of describing specific embodiments and are not intended to limit the present invention. As used in the specification of the present invention and the appended claims, unless the context clearly indicates otherwise, the singular forms "a", "an" and "the" are intended to include plural forms.

[0021] It should be further understood that the term "and / or" used in the present description and the appended claims refers to any and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0022] See also Figure 1 , Figure 1 A flow chart of a target detection method based on a fully convolutional twin network provided in an embodiment of the present invention. The target detection method based on a fully convolutional twin network in this embodiment can be applied to real-time tracking of targets (such as drone navigation, pedestrian detection and tracking in autonomous driving), edge computing devices (such as portable monitoring devices, camera modules in smart homes), and low-power smart terminals (such as Internet of Things devices, smart wearable devices). By using this method, the efficiency of target detection and tracking can be effectively improved.

[0023] Figure 1 1 is a flow chart of a target detection method based on a fully convolutional twin network provided by an embodiment of the present invention. As shown in the figure, the method includes the following steps S110-S140.

[0024] S110, performing feature processing on the image to be detected and the preset template image through the convolution layer in the preset fully convolutional twin network to obtain a feature map to be detected and a template feature map.

[0025] In this embodiment, the preset full convolution twin network is a network architecture after the full convolution twin network is trained and adjusted. Specifically, the core of the full convolution twin network (Siamese-FC, referred to as SiamFC) structure is a fully convolutional twin network. It contains two neural network branches with the same structure and shared weights, and these two branches process the template image and the image to be detected respectively. The image to be detected is an image that needs to be analyzed or matched. For example, in a target tracking task, this is the current frame in the video frame. The preset template image is a known image for comparison. For example, in target tracking, this is usually the first frame in the video sequence, where the target is clearly marked. Feature processing is performed by the convolution layer in the preset full convolution twin network to obtain the feature map to be detected and the template feature map. Specifically, the image to be detected and the preset template image are respectively sent to two branches (i.e., two identical sub-networks) in the preset full convolution twin network, which are composed of a series of convolution layers. The function of the convolution layer is to extract features in the image, such as edges, textures, shapes, etc. By stacking multiple convolution layers, the network can learn more and more complex feature representations. Since the two sub-networks share weights, they process their respective input images in the same way, thus ensuring the consistency of feature representation. After being processed by the convolutional layer, each input image generates a feature map, which contains a high-level feature representation of the input image. For the image to be detected and the template image, they generate a feature map to be detected and a template feature map, respectively. By obtaining the feature map to be detected and the template feature map according to the convolutional layer in the preset fully convolutional twin network, subsequent comparison and matching can be performed based on the feature map, thereby achieving the purpose of image matching and recognition.

[0026] In one embodiment, if Figure 2 As shown, the step S110 also includes steps S1101 - S1102 before the step S110.

[0027] S1101, adjusting the structure of the convolutional neural network in the initial fully convolutional twin network according to the pulse neural network structure;

[0028] S1102. Migrate the trained convolutional neural network parameters to the structurally adjusted convolutional neural network to determine the preset fully convolutional twin network.

[0029] In this embodiment, the initial full convolution twin network is a full convolution twin network that has not been trained and structurally adjusted, and the core of the full convolution twin network (Siamese-FC) is feature matching through a twin network structure. It is understandable that the architecture includes two shared convolutional neural networks, and the two convolutional neural networks have the same structure and weights. Convolutional Neural Network (CNN) is a deep learning model, and the spiking neural network (Spiking Neural Network, SNN) is a neural network model that can simulate the behavior of biological neurons in the brain. The two differ in structure, working principle and application scenario. The convolutional neural network in the initial full convolution twin network is structurally adjusted according to the spiking neural network structure. Specifically, the convolutional neural network (CNN) part in a traditional full convolution twin network is transformed to meet the structural requirements of the spiking neural network (SNN). For example, since the activation function of the spiking neuron is different from the functions such as ReLU and Sigmoid in CNN, the weights and offset values ​​in CNN are adjusted to adapt to the new activation mechanism to make it consistent with the structure of the spiking neural network (SNN). The trained convolutional neural network parameters are transferred to the structurally adjusted convolutional neural network to determine the preset full convolutional twin network. Specifically, a structurally adjusted convolutional neural network in the full convolutional twin network can be trained through a public training data set to obtain high-precision model parameters, and then the trained model parameters can be directly copied to another structurally adjusted convolutional neural network. It should be noted that because the convolutional neural network (CNN) is adjusted according to the structure of the spiking neural network (SNN), the adjusted CNN can also be regarded as an SNN. The specific training process is not limited, and high-precision model parameters can be obtained. By determining the preset full convolutional twin network according to the structurally adjusted convolutional neural network, the convolution module in the full convolutional twin network is converted into a spiking neural network, and a Spiking-SiamFC model suitable for edge device applications is constructed, so that the preset full convolutional twin network can be efficiently applied in resource-limited edge devices, and has the characteristics of low power consumption and high precision of the SNN model.

[0030] In one embodiment, if Figure 3 As shown, the step S1101 also includes steps S11011-S11013.

[0031] S11011, performing offset value removal processing on the convolutional neural network;

[0032] S11012, replacing the maximum pooling in the convolutional neural network with average pooling, and performing regular optimization on the weights in the convolutional neural network by a preset weight optimization method;

[0033] S11013. Perform network pruning on the convolutional neural network after regularization optimization.

[0034] In this embodiment, the offset value is a parameter in the neural network, which allows the output node to have an adjustable offset before the activation function. The convolutional neural network is subjected to offset removal processing, specifically, the offset value (bias) of the convolutional layer or the fully connected layer in the convolutional neural network (CNN) is removed or zeroed. It is understandable that, since the activation of neurons in the spiking neural network (SNN) is based on the accumulation process of membrane potential, the input fixed offset value is difficult to be accurately represented by the spiking neuron. Therefore, during the structural conversion process, all offset values ​​in the convolutional neural network are removed. The maximum pooling selects the maximum value in the pooling window as the output, while the average pooling calculates the average value of all values ​​in the pooling window as the output. The maximum pooling in the convolutional neural network is replaced by the average pooling. Specifically, the maximum pooling operation in the traditional CNN is nonlinear, and the features in the spiking neural network are represented in the form of discrete pulses, and nonlinear operations cannot be directly performed. Average pooling is used instead of maximum pooling when performing structural adjustment to ensure the stability of the feature extraction process. The weights in the convolutional neural network are regularly optimized by a preset weight optimization method. Specifically, weight regularization is a technique to prevent overfitting of the neural network. By using a preset weight optimization method (L2 regularization), the problem of too high or too low firing rate caused by uneven distribution of convolution weights of the structurally adjusted neural network can be avoided. The weights in the convolutional neural network are regularly optimized to ensure that the firing rate is distributed within a reasonable range, thereby optimizing the performance of the convolutional neural network (spiking neural network) after structural conversion. The convolutional neural network after regular optimization is network pruned. Specifically, network pruning is a method to reduce the complexity and computational complexity of the neural network. It usually involves removing redundant or less important connections, neurons or layers in the network. In this step, the neural network after regular optimization is pruned to simplify the model structure, reduce computing resource consumption or increase the reasoning speed of the model. The specific method of pruning may include pruning based on weight importance, pruning based on activation values, etc., or random pruning. Among them, the specific method of pruning is not limited. By adjusting the structure of the convolutional neural network and converting it into a spiking neural network (SNN), the Spiking-SiamFC model has the characteristics of low power consumption and high precision of the SNN model.

[0035] S120, generating a corresponding pulse sequence from the feature image to be detected and the template feature image through a preset pulse frequency algorithm.

[0036] In this embodiment, the preset pulse frequency algorithm is a method for converting a feature map into a pulse sequence, and a pulse sequence is a discrete sequence composed of 0 and 1 (or -1 and 1), which simulates the pulse emission behavior in biological neurons. The pulse frequency algorithm usually generates pulses based on the pixel value or feature intensity in the feature map, where a high intensity value may correspond to a higher pulse frequency (i.e., more pulses per unit time). The feature map to be detected and the template feature map are generated with a corresponding pulse sequence through a preset pulse frequency algorithm. Specifically, the feature map to be detected and the template feature map respectively convert the pixel values ​​in the feature maps according to the preset pulse frequency algorithm to generate a pulse sequence corresponding to the feature maps of the two. By converting both the feature map to be detected and the template feature map into pulse sequences, these sequences can be used for subsequent processing and analysis.

[0037] In one embodiment, if Figure 4 As shown, the step S120 includes steps S121-S122.

[0038] S121, inputting the pixel values ​​in the feature map to be detected and the template feature map into the preset pulse frequency algorithm to generate corresponding pulse times;

[0039] S122. Determine a pulse sequence within a preset unit time according to the number of pulses.

[0040] In this embodiment, the number of pulses indicates how many pulses the pixel should emit within a period of time (such as a unit of time). The pixel values ​​in the feature map to be detected and the template feature map are input into the preset pulse frequency algorithm to generate the corresponding number of pulses. Specifically, each pixel value in the feature map to be detected and the template feature map is input into the preset pulse frequency algorithm. The preset pulse frequency algorithm calculates the number of pulses that each pixel should generate based on these values. The preset pulse frequency algorithm is:

[0041]

[0042] Among them, the S i,j is the number of pulses, S is the total number of pixels, P i,j is the pixel value of the feature map, P max , P minThey are the maximum pixel value and the minimum pixel value of the feature map, respectively. It can be understood that the preset pulse frequency algorithm processes the pixel values ​​of the feature map to be detected and the pixel values ​​in the template feature map, respectively, so the above parameters can be used in two feature maps. The preset unit time is a fixed time period used to define the time scale of the pulse sequence. The selection of unit time usually depends on application requirements and algorithm design. The pulse sequence within the preset unit time is determined according to the number of pulses. Specifically, a pulse sequence is constructed for each pixel point within the preset unit time. The pulse sequence is a discrete sequence composed of 0 and 1 (or other symbols indicating the presence or absence of pulses). For example, within each unit time, if the number of pulses of a certain pixel point is greater than 0, the corresponding position in the pulse sequence at this time point is marked as, otherwise it is marked as 0 (or a symbol indicating no pulse). By determining the pulse sequence within the preset unit time according to the number of pulses, the pulse emission mode of the feature map to be detected and the template feature map within the preset unit time is represented, and these pulse sequences can be used for subsequent processing and analysis, so as to accurately track the target.

[0043] S130. Perform similarity calculation on the pulse sequence corresponding to the feature map to be detected and the template feature map through the preset fully convolutional twin network to generate a response map.

[0044] In this embodiment, the preset full convolution twin network has two neural networks after structural adjustment, so the pulse sequence corresponding to the feature map to be detected and the template feature map can be jointly generated. The pulse sequence corresponding to the feature map to be detected and the template feature map is calculated by the preset full convolution twin network for similarity to generate a response map. Specifically, the pulse sequence of the feature map to be detected and the pulse sequence of the template feature map are input into the preset full convolution twin network. The full convolution twin network consists of two branches with the same structure but shared parameters, which process the pulse sequences of the target to be detected and the template respectively, and compare the pulse sequences of the two to calculate the similarity between them. The result of the similarity calculation is a response map, which represents the degree of similarity between the target to be detected and the template at each position. Each value in the response map corresponds to a spatial position, and a higher value indicates that the position is more similar to the template. The preset full convolution twin network is used to perform similarity calculation to generate a response map, so as to understand the changes of the target to be detected in the two images, and the full convolution twin network and the pulse sequence can be used to achieve accurate positioning and tracking of the target to be detected.

[0045] In one embodiment, if Figure 5 As shown, the step S130 includes steps S131-S133.

[0046] S131, acquiring a corresponding pulse frequency according to a pulse sequence corresponding to the feature graph to be detected and the template feature graph;

[0047] S132, performing similarity calculation on the pulse frequencies corresponding to the feature graph to be detected and the template feature graph to obtain a similarity score;

[0048] S133: construct the response graph according to the similarity score.

[0049] In this embodiment, the pulse frequency refers to the number of pulses emitted per unit time, which reflects the intensity or activity of the pulse activity at the position. The corresponding pulse frequency is obtained according to the pulse sequence corresponding to the feature map to be detected and the template feature map. Specifically, a time window needs to be determined for calculating the pulse frequency. This time window should be long enough to contain enough pulse emission events so as to accurately reflect the activity of the pulse sequence. Within the determined time window, the number of pulses emitted by the pulse sequence at the position is counted. The pulse frequency at the position can be obtained by dividing the number of pulses obtained by the statistics by the length of the time window (usually seconds). Among them, the specific method for obtaining the pulse frequency is not limited. The similarity calculation is performed on the pulse frequencies corresponding to the feature map to be detected and the template feature map to obtain a similarity score. Specifically, the similarity calculation is performed after the pulse frequencies at the corresponding positions of the feature map to be detected and the template feature map are obtained. Among them, there are many methods for similarity calculation, such as element-by-element comparison, calculation of cosine similarity, Euclidean distance, etc. Which method to choose depends on the needs of the specific task and the characteristics of the data. The result of the similarity calculation is a similarity score matrix, which represents the similarity between each position in the feature map to be detected and the corresponding position of the template feature map. The higher the score, the more similar the two positions are. The response map is constructed according to the similarity score, wherein the response map is a two-dimensional matrix, and its size is usually the same as the feature map to be detected. Each value in the response map corresponds to a spatial position, and the value represents the similarity between the position and the template feature map. The response map can be directly constructed according to the similarity score matrix. Usually, each value in the similarity score matrix is ​​used as the response value of the corresponding position in the response map. In this way, the response map intuitively represents the similarity between each position in the feature map to be detected and the template feature map. The pulse frequency is obtained by the pulse sequence corresponding to the feature map to be detected and the template feature map, and the similarity calculation is performed, and finally the response map is constructed, so as to track the target to be detected and obtain the position of the target to be detected according to the response map.

[0050] S140: Perform offset calculation according to the response graph to determine the target position of the target to be detected.

[0051] In the present embodiment, the response map is a two-dimensional matrix, and its size is usually the same as the image to be detected or the feature map. Each value in the response map represents a spatial position, and the size of the value reflects the similarity between the position and the template or the reference feature. According to the response map, the target position of the target to be detected is determined by offset calculation. Specifically, the maximum value in the response map and the preset reference position calculation can be used to obtain the moving distance and direction of the target to be detected relative to the template, and finally the moving distance and direction relative to the template are mapped back to the actual image space to determine the exact position of the target to be detected in the image. By performing offset calculation according to the response map to determine the target position of the target to be detected, the accuracy of target positioning is improved, and the changes in the target morphology can be adapted to optimize the continuity of target tracking.

[0052] In one embodiment, if Figure 6 As shown, the step S140 includes steps S141-S142.

[0053] S141, performing deviation calculation on the maximum position parameter and the center position parameter in the response graph to obtain an offset value;

[0054] S142: Determine the target position of the target to be detected according to the offset value.

[0055] In this embodiment, the maximum position parameter refers to the coordinates of the position with the highest response value in the response map. This position usually represents the area most similar to the template, that is, the most likely target position. The center position parameter is a preset or calculated coordinate, which represents the center position of the response map or the expected target position. In practical applications, the center position parameter can be determined according to the size, shape or characteristics of the template to be detected. The maximum position parameter and the center position parameter in the response map are subjected to deviation calculation to obtain an offset value. Specifically, the deviation calculation refers to comparing the maximum position parameter with the center position parameter to calculate the difference between them. This difference is usually represented by a two-dimensional vector, namely the offset value. The offset value is the difference between the maximum position parameter and the center position parameter, which describes the offset direction and distance of the target to be detected relative to the expected position (or center position). Among them, the formula for deviation calculation can be: Δx=(x max -x center )·step. Among them, x max is the maximum position parameter, x centeris the center position parameter, and step is the network step size. The target position of the target to be detected is determined according to the offset value. Specifically, the coordinates of the target position are obtained by adding the offset value to the coordinates of the center position parameter. By determining the target position of the target to be detected according to the offset value, the target position of the target to be detected is accurately determined, thereby improving the accuracy of target positioning and optimizing the continuity of target tracking.

[0056] Figure 7 is a schematic block diagram of a target detection device 200 based on a fully convolutional twin network provided by an embodiment of the present invention. Figure 7 As shown, corresponding to the above target detection method based on the full convolutional twin network, the present invention also provides a target detection device based on the full convolutional twin network. The target detection device based on the full convolutional twin network includes a unit for executing the above target detection method based on the full convolutional twin network, and the device can be configured in a desktop computer, a tablet computer, a laptop, and other terminals. For details, please refer to Figure 7 The target detection device based on the fully convolutional twin network includes a feature processing unit 210, a sequence generation unit 220, a calculation unit 230 and a determination unit 240.

[0057] The feature processing unit 210 is used to perform feature processing on the image to be detected and the preset template image through the convolution layer in the preset fully convolutional twin network to obtain a feature map to be detected and a template feature map.

[0058] In one embodiment, the feature processing unit 210 includes an adjustment unit and a migration unit.

[0059] An adjustment unit, used for adjusting the structure of the convolutional neural network in the initial fully convolutional twin network according to the pulse neural network structure;

[0060] A migration unit is used to migrate the trained convolutional neural network parameters to the convolutional neural network after structural adjustment to determine the preset fully convolutional twin network.

[0061] In one embodiment, the feature processing unit 210 includes a removal unit, an optimization unit, and a cropping unit.

[0062] A removal unit, used for performing offset value removal processing on the convolutional neural network;

[0063] An optimization unit, used to replace the maximum pooling in the convolutional neural network with average pooling, and to perform regular optimization on the weights in the convolutional neural network by a preset weight optimization method;

[0064] The pruning unit is used to prune the convolutional neural network after regularization optimization.

[0065] The sequence generating unit 220 is used to generate a corresponding pulse sequence by using a preset pulse frequency algorithm to generate the feature image to be detected and the template feature image.

[0066] In one embodiment, the sequence generating unit 220 includes an input unit and a sequence determining unit.

[0067] An input unit, used to input the pixel values ​​in the feature map to be detected and the template feature map into the preset pulse frequency algorithm to generate a corresponding number of pulses;

[0068] The sequence determination unit is used to determine the pulse sequence within a preset unit time according to the pulse number.

[0069] The calculation unit 230 is used to perform similarity calculation on the pulse sequence corresponding to the feature map to be detected and the template feature map through the preset fully convolutional twin network to generate a response map.

[0070] In one embodiment, the calculation unit 230 includes a frequency acquisition unit, a score calculation unit, and a construction unit.

[0071] A frequency acquisition unit, used for acquiring a corresponding pulse frequency according to a pulse sequence corresponding to the feature graph to be detected and the template feature graph;

[0072] A score calculation unit, used for performing similarity calculation on the pulse frequencies corresponding to the feature graph to be detected and the template feature graph to obtain a similarity score;

[0073] A construction unit is used to construct the response graph according to the similarity score.

[0074] The determination unit 240 is configured to perform an offset calculation according to the response graph to determine a target position of the target to be detected.

[0075] In one embodiment, the determination unit 240 includes a deviation calculation unit and a position determination unit.

[0076] A deviation calculation unit, used for calculating the deviation between the maximum position parameter and the center position parameter in the response graph to obtain an offset value;

[0077] A position determination unit is used to determine the target position of the target to be detected according to the offset value.

[0078] It should be noted that technical personnel in the relevant field can clearly understand that the specific implementation process of the above-mentioned target detection device 200 based on the fully convolutional twin network and each unit can refer to the corresponding description in the aforementioned method embodiment, and for the convenience and conciseness of the description, it will not be repeated here.

[0079] The above-mentioned target detection device based on the fully convolutional twin network can be implemented in the form of a computer program. The computer program can be used in Figure 8 Runs on the computer device shown.

[0080] See also Figure 8 , Figure 8 5 is a schematic block diagram of a computer device provided in an embodiment of the present application. The computer device 500 may be a terminal or a server, wherein the terminal may be an electronic device with communication functions such as a smart phone, a tablet computer, a laptop computer, a desktop computer, a personal digital assistant, and a wearable device. The server may be an independent server or a server cluster composed of multiple servers.

[0081] See also Figure 8 The computer device 500 includes a processor 502 , a memory and a network interface 505 connected via a system bus 501 , wherein the memory may include a non-volatile storage medium 503 and an internal memory 504 .

[0082] The non-volatile storage medium 503 can store an operating system 5031 and a computer program 5032. The computer program 5032 includes program instructions, which, when executed, can enable the processor 502 to perform a target detection method based on a fully convolutional twin network.

[0083] The processor 502 is used to provide computing and control capabilities to support the operation of the entire computer device 500 .

[0084] The internal memory 504 provides an environment for the operation of the computer program 5032 in the non-volatile storage medium 503. When the computer program 5032 is executed by the processor 502, the processor 502 can execute a target detection method based on a fully convolutional twin network.

[0085] The network interface 505 is used to communicate with other devices over the network. Figure 8 The structure shown in the figure is merely a block diagram of a portion of the structure related to the solution of the present application, and does not constitute a limitation on the computer device 500 to which the solution of the present application is applied. The specific computer device 500 may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0086] The processor 502 is used to run a computer program 5032 stored in the memory to implement the steps of the above method.

[0087] It should be understood that in the embodiment of the present application, the processor 502 may be a central processing unit (CPU), and the processor 502 may also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Among them, the general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0088] It can be understood by those skilled in the art that all or part of the processes in the method for implementing the above embodiment can be completed by instructing the relevant hardware through a computer program. The computer program includes program instructions, and the computer program can be stored in a storage medium, which is a computer-readable storage medium. The program instructions are executed by at least one processor in the computer system to implement the process steps of the embodiment of the above method.

[0089] Therefore, the present invention also provides a storage medium. The storage medium may be a computer-readable storage medium. The storage medium stores a computer program, wherein the computer program includes program instructions. When the program instructions are executed by a processor, the processor executes the steps of the above method.

[0090] The storage medium may be a USB flash drive, a mobile hard disk, a read-only memory (ROM), a magnetic disk, or an optical disk, etc., which are computer-readable storage media that can store program codes.

[0091] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in terms of function in the above description. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0092] In the several embodiments provided by the present invention, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of each unit is only a logical function division, and there may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed.

[0093] The steps in the method of the embodiment of the present invention can be adjusted in order, combined and deleted according to actual needs. The units in the device of the embodiment of the present invention can be combined, divided and deleted according to actual needs. In addition, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0094] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a storage medium. Based on this understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, terminal, or network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention.

[0095] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed by the present invention, and these modifications or replacements should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention shall be based on the protection scope of the claims.

Claims

1. A target detection method based on a fully convolutional twin network, characterized in that: The method comprises: The image to be detected and the preset template image are processed by the convolution layer in the preset fully convolutional twin network to obtain the feature map to be detected and the template feature map; Generate a corresponding pulse sequence from the feature image to be detected and the template feature image through a preset pulse frequency algorithm; The pulse sequence corresponding to the feature map to be detected and the template feature map is subjected to similarity calculation through the preset fully convolutional twin network to generate a response map; The target position of the target to be detected is determined by performing an offset calculation according to the response map.

2. The method according to claim 1, characterized in that: Before the step of performing feature processing on the image to be detected and the preset template image through the convolution layer in the preset fully convolutional twin network, the method includes: Adjust the structure of the convolutional neural network in the initial fully convolutional twin network according to the pulse neural network structure; The trained convolutional neural network parameters are transferred to the structurally adjusted convolutional neural network to determine the preset fully convolutional twin network.

3. The method according to claim 2, characterized in that The step of adjusting the structure of the convolutional neural network in the initial fully convolutional twin network according to the spiking neural network structure includes: Performing offset value removal processing on the convolutional neural network; The maximum pooling in the convolutional neural network is replaced by average pooling, and the weights in the convolutional neural network are regularly optimized by a preset weight optimization method; The convolutional neural network after regular optimization is pruned.

4. The method according to claim 1, characterized in that The step of generating a corresponding pulse sequence from the feature graph to be detected and the template feature graph by a preset pulse frequency algorithm comprises: Inputting the pixel values ​​in the feature map to be detected and the template feature map into the preset pulse frequency algorithm to generate corresponding pulse times; A pulse sequence within a preset unit time is determined according to the pulse number.

5. The method according to claim 1, characterized in that The step of performing similarity calculation on the pulse sequence corresponding to the feature map to be detected and the template feature map through the preset fully convolutional twin network to generate a response map comprises: Acquire a corresponding pulse frequency according to a pulse sequence corresponding to the feature graph to be detected and the template feature graph; Performing similarity calculation on the pulse frequencies corresponding to the feature graph to be detected and the template feature graph to obtain a similarity score; The response graph is constructed based on the similarity scores.

6. The method according to claim 5, characterized in that The step of performing offset calculation according to the response graph to determine the target position of the target to be detected comprises: Calculate the deviation between the maximum position parameter and the center position parameter in the response graph to obtain an offset value; The target position of the target to be detected is determined according to the offset value.

7. The method according to claim 4, characterized in that The preset pulse frequency algorithm is: Among them, the S i,j is the number of pulses, S is the total number of pixels, P i,j is the pixel value of the feature map, P max , P min are the maximum pixel value and the minimum pixel value of the feature map respectively.

8. A target detection device based on a fully convolutional twin network, characterized in that: include: A feature processing unit is used to perform feature processing on the image to be detected and the preset template image through the convolution layer in the preset fully convolutional twin network to obtain a feature map to be detected and a template feature map; A sequence generating unit, used for generating a corresponding pulse sequence from the feature image to be detected and the template feature image through a preset pulse frequency algorithm; A calculation unit, used for performing similarity calculation on the pulse sequence corresponding to the feature map to be detected and the template feature map through the preset fully convolutional twin network to generate a response map; A determination unit is used to perform an offset calculation according to the response graph to determine a target position of the target to be detected.

9. A computer device, characterized in that: The computer device includes a memory and a processor, the memory stores a computer program, and the processor implements the method according to any one of claims 1 to 7 when executing the computer program.

10. A storage medium, characterized in that: The storage medium stores a computer program, wherein the computer program includes program instructions, and when the program instructions are executed by a processor, the method according to any one of claims 1 to 7 can be implemented.

Images