A target detection method and device, a storage medium and an electronic device

Abstract

The specification discloses a target detection method and device, a storage medium and an electronic device. In the embodiment of the specification, the activation function of each neuron in an original artificial neural network model is adjusted according to the required running time of a to-be-converted spiking neural network model, and an adjusted activation function is obtained. The trained model parameters obtained by training through the adjusted activation function are migrated to the spiking neural network model, and an initial spiking neural network model is obtained. In this method, since the adjusted activation function in the artificial neural network model is adjusted through the running time of the spiking neural network model, the model parameters obtained by training through the adjusted activation function are migrated to the spiking neural network model, and the spiking neural network model does not need to consume additional running time to achieve a performance close to that of the artificial neural network model, thereby reducing the calculation amount and power consumption of the spiking neural network model running.

Description

TECHNICAL FIELD

[0001] The present specification relates to the technical field of image processing, and particularly relates to a target detection method and device, a storage medium and an electronic device. BACKGROUND

[0002] Target detection in image processing technology is a relatively important research direction in the field of artificial intelligence and computer vision. With the development of deep learning, compared with deep learning technology of traditional artificial neural networks, deep learning technology of spiking neural networks has attracted widespread attention.

[0003] At present, the spiking neural network models suitable for target detection tasks mainly include two categories of direct training and indirect conversion. Among them, the indirect conversion can first train an artificial neural network to obtain a trained artificial neural network, and then modify the spiking neural network in the process of realizing indirect conversion, so that the modified spiking neural network can fire negative pulses. After normalizing the model parameters of the trained artificial neural network, the model parameters are mapped into the spiking neural network. In order to make the performance of the spiking neural network close to that of the traditional artificial neural network, it is necessary to run the spiking neural network for a long time, which increases the running time of the spiking neural network and increases the calculation amount and power consumption required for running the spiking neural network. SUMMARY

[0004] Embodiments of the present specification provide a target detection method, device, storage medium and electronic device to partially solve the problems existing in the prior art.

[0005] Embodiments of the present specification adopt the following technical solutions:

[0006] The target detection method provided by the present specification comprises:

[0007] obtaining an original artificial neural network model;

[0008] According to the running time required by the to-be-converted spiking neural network model, adjusting the activation function of each neuron in the original artificial neural network model according to a preset adjustment strategy to obtain an adjusted activation function of each neuron;

[0009] determining an adjusted artificial neural network model according to the adjusted activation function of each neuron;

[0010] training the adjusted artificial neural network model to obtain a trained model;

[0011] migrating the model parameters of the trained model to the to-be-converted spiking neural network model to determine an initial spiking neural network model;

[0012] adjust the initial spiking neural network model based on the output value of each neuron in the trained model and the frequency value of each neuron firing in the initial spiking neural network model, to obtain a final spiking neural network model;

[0013] After receiving the target detection task request, input the target image carried in the target detection task request into the final spiking neural network model, so as to perform target detection on the target image through the final spiking neural network model.

[0014] Optionally, according to the required running time of the to-be-converted spiking neural network model, the activation function of each neuron in the original artificial neural network model is adjusted according to a preset adjustment strategy to obtain an adjusted activation function of each neuron, specifically including:

[0015] determining an adjustable parameter for representing the membrane potential threshold required for each neuron in the to-be-converted spiking neural network model to fire a spike;

[0016] determining a target activation function as the target activation function according to the adjustable parameter and the required running time of the to-be-converted spiking neural network model;

[0017] replacing the activation function of each neuron in the original artificial neural network model with the target activation function according to a preset adjustment strategy to obtain an adjusted activation function of each neuron.

[0018] Optionally, the target activation function is determined according to the adjustable parameter and the required running time of the to-be-converted spiking neural network model, specifically including:

[0019] determining a first function for representing the association relationship between the input value, the running time and the adjustable parameter;

[0020] determining a second function for representing the association relationship between the first function and the running time;

[0021] determining a target activation function according to the second function and the adjustable parameter.

[0022] Optionally, the adjusted artificial neural network model is determined according to the adjusted activation function of each neuron, specifically including:

[0023] adjusting the max-pooling mode in the original artificial neural network model to an average-pooling mode;

[0024] determining an adjusted artificial neural network model according to the average-pooling mode and the adjusted activation function of each neuron.

[0025] Optionally, the adjusted artificial neural network model is trained to obtain a trained model, specifically including:

[0026] Obtain image samples;

[0027] The image sample is input into the adjusted artificial neural network model to output a target detection result for the image sample. The target detection result includes the center position and size of the predicted bounding box, whether the predicted bounding box is anchored to the target object, and the classification result of the target object.

[0028] With the goal of minimizing the difference between the target detection result and the label, the adjustable parameters contained in the adjusted activation function of each neuron in the adjusted artificial neural network model are adjusted to obtain the adjusted parameters of each neuron; wherein, the adjustable parameters are used to represent the membrane potential threshold required for each neuron of the spiking neural network model to fire a pulse;

[0029] Based on the adjusted parameters of each neuron, the trained model obtained after training the adjusted artificial neural network model is determined.

[0030] Optionally, the model parameters of the trained model are transferred to the spiking neural network model to be converted to determine the initial spiking neural network model, specifically including:

[0031] For each neuron in the trained model, the bias parameters included in the model parameters of that neuron in the trained model are adjusted according to a preset adjustment amount to obtain the adjusted bias parameters.

[0032] Based on the adjusted bias parameters, the adjusted model parameters obtained after adjusting the model parameters of the neuron in the trained model are determined.

[0033] The adjusted model parameters are used as the model parameters of the neuron in the spiking neural network model to be converted, and the adjustable parameters contained in the adjusted activation function of the neuron in the trained model are used as the membrane potential threshold required for the neuron to fire a pulse in the spiking neural network model to be converted; wherein, the model structure of the trained model is the same as the model structure of the spiking neural network model to be converted.

[0034] The initial spiking neural network model is determined based on the model parameters of each neuron and the membrane potential threshold in the spiking neural network model to be converted.

[0035] Optionally, based on the output value of each neuron in the trained model and the frequency value of the pulses fired by each neuron in the initial spiking neural network model, the initial spiking neural network model is adjusted to obtain the final spiking neural network model, specifically including:

[0036] Obtain each adjusted image sample;

[0037] For each adjusted image sample, the adjusted image sample is input into the trained model to obtain the output value of each neuron as the adjusted image sample passes through the trained model; the adjusted image sample is input into the initial spiking neural network model to obtain the frequency value of each neuron firing a pulse when the adjusted image sample passes through each neuron.

[0038] For each neuron in the initial spiking neural network model, the adjustment value for each adjusted image sample is determined based on the difference between the output value of each adjusted image sample passing through the neuron in the trained model and the frequency value of the pulse fired by the neuron when each adjusted image sample passes through the neuron in the initial spiking neural network model, as well as the runtime of the neuron in the initial spiking neural network model.

[0039] Based on the adjustment value for each adjusted image sample for that neuron, determine the average adjustment value for that neuron;

[0040] The initial membrane potential of the neuron in the initial spiking neural network model is adjusted to the average adjustment value to obtain the adjusted initial membrane potential of the neuron.

[0041] The final spiking neural network model is determined based on the adjusted initial membrane potential of each neuron.

[0042] Optionally, based on the output value of each neuron in the trained model and the frequency value of the pulses fired by each neuron in the initial spiking neural network model, the initial spiking neural network model is adjusted to obtain the final spiking neural network model, specifically including:

[0043] Obtain adjusted image samples;

[0044] The adjusted image sample is input into the trained model to obtain the output value of each neuron; the adjusted image sample is input into the initial spiking neural network model to obtain the frequency value of each neuron firing pulses.

[0045] For each neuron in the initial spiking neural network model, based on the output value of that neuron in the trained model and the frequency value of the pulses fired by that neuron in the initial spiking neural network model, the mean square error between the output value of that neuron in the trained model and the frequency value of the pulses fired by that neuron in the initial spiking neural network model is determined;

[0046] With the goal of minimizing the mean square error, the model parameters of the neuron in the initial spiking neural network model are adjusted to obtain the final spiking neural network model.

[0047] This specification provides a target detection device, comprising:

[0048] The acquisition module is used to acquire the original artificial neural network model;

[0049] The first adjustment module is used to adjust the activation function of each neuron in the original artificial neural network model according to a preset adjustment strategy based on the runtime required by the spiking neural network model to be converted, so as to obtain the adjusted activation function of each neuron.

[0050] The determination module is used to determine the adjusted artificial neural network model based on the adjusted activation function of each neuron;

[0051] The training module is used to train the adjusted artificial neural network model to obtain the trained model;

[0052] The parameter transfer module is used to transfer the model parameters of the trained model to the spiking neural network model to be converted, so as to determine the initial spiking neural network model.

[0053] The second adjustment module is used to adjust the initial spiking neural network model based on the output value of each neuron in the trained model and the frequency value of the pulses fired by each neuron in the initial spiking neural network model, so as to obtain the final spiking neural network model.

[0054] The target detection module is used to input the target image carried in the target detection task request into the final spiking neural network model after receiving the target detection task request, so as to perform target detection on the target image through the final spiking neural network model.

[0055] Optionally, the first adjustment module is specifically used to determine an adjustable parameter representing the membrane potential threshold required for each neuron in the spiking neural network model to be converted to fire a pulse; determine a target activation function based on the adjustable parameter and the required runtime of the spiking neural network to be converted; and replace the activation function of each neuron in the original artificial neural network model with the target activation function according to a preset adjustment strategy to obtain the adjusted activation function of each neuron.

[0056] Optionally, the determining module is specifically configured to determine a first function representing the relationship between the input value, the runtime, and the adjustable parameters; determine a second function representing the relationship between the first function and the runtime; and determine a target activation function based on the second function and the adjustable parameters.

[0057] Optionally, the determining module is specifically used to adjust the max pooling method in the original artificial neural network model to the average pooling method; and to determine the adjusted artificial neural network model based on the average pooling method and the adjusted activation function of each neuron.

[0058] Optionally, the training module is specifically used to acquire image samples; input the image samples into the adjusted artificial neural network model to output target detection results for the image samples through the adjusted artificial neural network model, wherein the target detection results include the center position and size of the predicted bounding box, whether the predicted bounding box is anchored to the target object, and the classification result of the target object; with minimizing the difference between the target detection results and the label as the optimization objective, the adjustable parameters contained in the adjusted activation function of each neuron in the adjusted artificial neural network model are adjusted to obtain the adjusted parameters of each neuron; wherein the adjustable parameters are used to represent the membrane potential threshold required for each neuron of the spiking neural network model to fire a pulse; based on the adjusted parameters of each neuron, the trained model obtained after training the adjusted artificial neural network model is determined.

[0059] Optionally, the parameter transfer module is specifically used to adjust the bias parameters included in the model parameters of each neuron in the trained model according to a preset adjustment amount, to obtain adjusted bias parameters; determine the adjusted model parameters obtained after adjusting the model parameters of the neuron in the trained model based on the adjusted bias parameters; use the adjusted model parameters as the model parameters of the neuron in the spiking neural network model to be converted, and use the adjustable parameters included in the adjusted activation function of the neuron in the trained model as the membrane potential threshold required for the neuron to fire a pulse in the spiking neural network model to be converted; wherein, the model structure of the trained model is the same as the model structure of the spiking neural network model to be converted; and determine the initial spiking neural network model based on the model parameters of each neuron in the spiking neural network model to be converted and the membrane potential threshold.

[0060] This specification provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the target detection method described above.

[0061] This specification provides an electronic device including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, it implements the target detection method described above.

[0062] The above-described at least one technical solution adopted in the embodiments of this specification can achieve the following beneficial effects:

[0063] In this embodiment, the activation function of each neuron in the original artificial neural network model is adjusted according to the runtime required by the spiking neural network model to be converted, resulting in an adjusted activation function for each neuron. The adjusted artificial neural network model, incorporating the adjusted activation function, is then trained to obtain a trained model. The model parameters of the trained model are then transferred to the spiking neural network model to obtain an initial spiking neural network model. Finally, the initial spiking neural network model is adjusted to obtain the final spiking neural network model. In this method, since the activation function of the artificial neural network model is adjusted based on the runtime of the spiking neural network model, transferring the model parameters obtained through training with the adjusted activation function to the spiking neural network model eliminates the need for the spiking neural network model to consume additional runtime to achieve performance close to that of the artificial neural network model, thereby reducing the computational load and power consumption of the spiking neural network model. Attached Figure Description

[0064] The accompanying drawings, which are included to provide a further understanding of this specification and form part of this specification, illustrate exemplary embodiments and are used to explain this specification, but do not constitute an undue limitation thereof. In the drawings:

[0065] Figure 1 This is a schematic flowchart of the target detection method provided in the embodiments of this specification;

[0066] Figure 2 A schematic diagram illustrating the difference between g(x) and f(x) provided in the embodiments of this specification;

[0067] Figure 3 A schematic diagram illustrating the difference between f(x) provided in the embodiments of this specification and the rewritten h(x);

[0068] Figure 4 This is a schematic diagram of the target detection device provided in the embodiments of this specification;

[0069] Figure 5 This is a schematic diagram of the structure of the electronic device provided in the embodiments of this specification. Detailed Implementation

[0070] Artificial neural networks, or spiking neural networks, can be composed of interconnected neurons. Each neuron represents a method for processing the data input to it, such as: performing convolution on the input data to obtain a convolution result; or activating the convolution result using an activation function to obtain an activation value. For some neurons, in addition to the above processing methods, pooling may also be performed on the activation values ​​to obtain a pooling result, which is then passed to the neurons in the next layer.

[0071] Next, we will introduce the feedforward calculation principle of artificial neural network models and spiking neural network models.

[0072] Taking the YOLOv3-Tiny model as an example, the feedforward calculation principle of the artificial neural network model is as follows: For each artificial neuron, the artificial neuron determines a convolution value based on the output value of the artificial neuron in the previous layer, the weight between the artificial neuron and the artificial neuron in the previous layer, and the bias parameter of the artificial neuron. The convolution value is then activated to obtain the activation value.

[0073] The specific formula is as follows:

[0074] The formula for the activation function is: x is the input value of the activation function, and leak is a constant, usually 0.1. The leak*min(0,x) part is used to preserve negative values.

[0075] The formula for calculating the input value of the activation function is: This represents the weight between the j-th artificial neuron in layer (l-1) and the i-th artificial neuron in layer l. This represents the bias parameter of the i-th artificial neuron in the l-th layer.

[0076] Taking the YOLOv3-Tiny model as an example, the feedforward calculation principle of the traditional spiking neural network model is as follows: for each spiking neuron, the spiking neuron continuously accumulates input and increases the membrane potential until the membrane potential reaches the membrane potential threshold, at which point the spiking neuron fires a pulse and resets the membrane potential.

[0077] The specific formula is as follows:

[0078] The formula for accumulating input and increasing membrane potential is: This represents the cumulative increase in membrane potential before the repositioning of the i-th spiking neuron in layer l at time t. This represents the membrane potential of the i-th spiking neuron in the l-th layer at time t-1. This represents the weight between the j-th spiking neuron in layer (l-1) and the i-th spiking neuron in layer l. This represents the pulse value emitted by the j-th spiking neuron in the (l-1)-th layer. This represents the bias parameter of the i-th spiking neuron in the l-th layer.

[0079] The pulse value of the emitted pulse is: is the membrane potential threshold of the i-th spiking neuron in the l-th layer.

[0080] The reset formula after pulse delivery is: It represents the membrane potential of the i-th spiking neuron in the l-th layer at time t after resetting.

[0081] It should be noted that in traditional artificial neural network models, neurons fire a fixed value of 1 pulse.

[0082] The principle for achieving the same or closest performance between artificial neural network (ANN) models and spiking neural network (SNN) models through indirect conversion is that the activation value output by each neuron in the ANN model needs to be closest to or equal to the firing frequency of each neuron in the SNN model. The firing frequency of each neuron in the SNN model is related to the runtime of the SNN model. The formula for calculating the firing frequency of each neuron is as follows: This can represent the frequency of the pulse fired by the i-th neuron in the l-th layer, and T can represent the runtime. This represents the pulse value fired by the i-th neuron in the l-th layer at time t. This represents the weight between the j-th neuron in layer (l-1) and the i-th neuron in layer l. This represents the frequency value of the pulse fired by the j-th neuron in the (l-1)-th layer. This represents the bias parameter of the i-th neuron in the l-th layer. This represents the error between the frequency of pulses fired by the i-th neuron in the l-th layer of a spiking neural network and the activation value output by the i-th neuron in the l-th layer of an artificial neural network through an activation function. (T) represents the membrane potential of the i-th neuron in the l-th layer of the spiking neural network after firing a pulse and resetting at time T. Theoretically, it should not change with increasing runtime. This represents the initial membrane potential of the i-th neuron in the l-th layer of a spiking neural network.

[0083] The existing technology achieves indirect conversion by approximating the performance of spiking neural networks (SNNs) to that of artificial neural networks (ANNs). Therefore, when the activation function of an ANN contains functions that produce negative values, the SNN needs to be modified to generate negative pulses to approximate the negative values ​​calculated by the activation function in the ANN. After modifying the SNN, to adapt to the pulse values ​​emitted by the SNN, the weights corresponding to each neuron in the trained ANN are normalized, and these normalized weights are then transferred to the corresponding neurons in the SNN. After transferring the normalized weights to the SNN, the SNN can be run. However, to ensure that the frequency of pulses emitted by each neuron in the SNN is the same as or closest to the activation value output by the corresponding neuron in the ANN through the activation function, the SNN takes a relatively long time to run, thus increasing the computational load and power consumption required.

[0084] From the above As the formula shows, in order to make the frequency of pulses fired by each neuron in a spiking neural network the same as or closest to the activation value output by the corresponding neuron in an artificial neural network through the activation function, it is necessary to make... To keep it as small as possible, which means increasing the runtime of the spiking neural network. The longer the runtime, the better. The smaller the value, the closer the frequency of the emitted pulses is to the activation value. Therefore, the increase in the runtime of spiking neural networks in existing technologies leads to an increase in the computational load of the spiking neural network, thereby increasing power consumption.

[0085] The target detection method provided in this specification aims to adjust the pooling method and activation function of each neuron in the artificial neural network model based on the required runtime of the spiking neural network model, so that the activation value output by each neuron of the trained artificial neural network model is as close as possible to the frequency value of the pulses emitted by each neuron of the trained spiking neural network model.

[0086] To make the objectives, technical solutions, and advantages of this specification clearer, the technical solutions of this specification will be clearly and completely described below in conjunction with specific embodiments and corresponding drawings. Obviously, the described embodiments are only a part of the embodiments of this specification, and not all of them. Based on the embodiments in this specification, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this specification.

[0087] The technical solutions provided in the various embodiments of this specification are described in detail below with reference to the accompanying drawings.

[0088] Figure 1 This is a flowchart illustrating the target detection method provided in the embodiments of this specification. This target detection method can be applied to a server and includes:

[0089] S100: Obtain the original artificial neural network model.

[0090] In the embodiments described in this specification, the server can receive a model conversion request, which carries information such as the model structure of the artificial neural network model and the required runtime of the spiking neural network model to be converted. The required runtime of the spiking neural network model to be converted can be set according to actual needs.

[0091] In this specification, the model structure of the artificial neural network model may include: YOLOv3-Tiny.

[0092] Upon receiving a model conversion request, the system can retrieve an artificial neural network (ANN) model that meets the specified model structure from a database storing ANN models, based on the model structure carried in the request. This ANN model can then be used as the original ANN model. The ANN model can be a pre-trained ANN model, and its parameters are those of the pre-trained model. These parameters may include weights, bias parameters, and other similar parameters.

[0093] Furthermore, after finding an artificial neural network model, it can be deprecated as the original model. Instead, its output dimension can be adjusted based on other parameters to obtain the adjusted output dimension. This adjusted output dimension can then be used as the original model. Other parameters can include the number of detection categories and anchor box data. Anchor box data can include the number of anchor boxes, their size, and center coordinates.

[0094] The adjusted output dimensions are (Batch, Height, Width, Channel). Batch represents the batch size of samples input to the artificial neural network model each time, Height represents the height of the output feature map, Width represents the width of the output feature map, and Channel represents the number of channels in the output feature map.

[0095] The specific formula for calculating the number of channels for the output features is: Channel = num anchor *(num class +4+1). Where, num anchor Indicates the number of anchor frames of different sizes; num class The number of detected categories is indicated by "4"; "4" represents the output four-dimensional data (x, y, w, h), where (x, y) represents the center coordinates of the anchor box, w represents the width of the anchor box, and h represents the height of the anchor box; the last item "1" represents the confidence level. The output feature map in this embodiment has two sets of width and height values, namely (13, 13) and (26, 26), num anchor and num class The values ​​are 3 and 20 respectively, meaning the value of Channel is 75.

[0096] S102: Based on the runtime required by the spiking neural network model to be converted, the activation function of each neuron in the original artificial neural network model is adjusted according to a preset adjustment strategy to obtain the adjusted activation function of each neuron.

[0097] In the embodiments of this specification, after obtaining the original artificial neural network model, the pooling method and the activation function corresponding to each neuron in the original artificial neural network model can be adjusted by combining the calculation principles of each neuron in the artificial neural network model and the spiking neural network model.

[0098] Adjust the activation function:

[0099] In the embodiments of this specification, the activation function of each neuron in the original artificial neural network model can be adjusted according to a preset adjustment strategy based on the required runtime of the spiking neural network model to be converted, thereby obtaining the adjusted activation function of each neuron. The preset adjustment strategy may include replacement. Furthermore, the spiking neural network model to be converted has the same model structure as the original artificial neural network model.

[0100] Specifically, the required runtime of the spiking neural network model to be converted can be obtained from the received model conversion request. Simultaneously, adjustable parameters representing the membrane potential threshold required for each neuron in the spiking neural network model to fire a pulse are determined. Then, based on the adjustable parameters and the required runtime of the spiking neural network model, the target activation function can be determined. Finally, according to a preset adjustment strategy, the activation function of each neuron in the original artificial neural network model is replaced with the target activation function, resulting in the adjusted activation function for each neuron. The adjustable parameters can be adjusted during the training of the artificial neural network model. The adjusted activation function is essentially the target activation function. The target activation function quantizes the data obtained after convolutional layer processing in the neurons.

[0101] When determining the target activation function, a first function can be identified to represent the relationship between the input value, runtime, and adjustable parameters. The input value, runtime, and adjustable parameters serve as input parameters to the first function. Then, a second function can be identified to represent the relationship between the first function and the runtime. The first function and the runtime serve as input parameters to the second function. Finally, based on the second function and the adjustable parameters, the target activation function is determined. The input value can refer to the value to be input to the target activation function, i.e., the data obtained after processing by the convolutional layers in the neuron.

[0102] The formula for the target activation function is: θ represents the membrane potential threshold required for each neuron in the spiking neural network model to fire a pulse; that is, it is an adjustable parameter. θ can be adjusted during the training of the artificial neural network model. round(·) represents the rounding function. The first function is defined here, and x represents the input value. clip(x′,0,1) is the clipping function, which can clip x′ to the range [0,1]. Can Cut to the range [0,1], where, Let represent the second function. T represents the runtime required for the spiking neural network model to be transformed. h(x) is the target activation function.

[0103] It should be noted that, taking the YOLOv3-Tiny artificial neural network model as an example, in the original artificial neural network model of this model structure, except for the last two convolutional layers used for output results, the output values ​​of all previous convolutional layers need to be processed by the activation function to obtain the activation values.

[0104] S104: Determine the adjusted artificial neural network model based on the adjusted activation function of each neuron.

[0105] In the embodiments of this specification, after adjusting the activation function in the original artificial neural network model to obtain the adjusted activation function, the adjusted artificial neural network model obtained after adjusting the original artificial neural network model can be determined based on the adjusted activation function of each neuron.

[0106] In artificial neural network models, some neurons require pooling of their activation values. The original artificial neural network model uses max pooling, where the maximum activation value within the pooling region is used as the input to the next layer of neurons. However, since spiking neural network models use discrete pulse sequences as input and output, max pooling is not suitable. Therefore, the max pooling method in the original artificial neural network model needs to be changed to average pooling, where the average activation value within the pooling region is used as the input to the next layer of neurons.

[0107] Therefore, after adjusting the activation function in the original artificial neural network model, the max pooling method in the original artificial neural network model can be changed to average pooling. Then, based on the average pooling method and the adjusted activation function of each neuron, the adjusted artificial neural network model is determined.

[0108] S106: Train the adjusted artificial neural network model to obtain the trained model.

[0109] In the embodiments of this specification, after determining the adjusted artificial neural network model, the adjusted artificial neural network model can be trained to obtain the trained adjusted artificial neural network model, which is used as the trained model.

[0110] When training an adjusted artificial neural network (ANN) model, if the model is a pre-trained ANN, it can be divided into a backbone and an output part. The neurons in the backbone are used to extract features from the image, while the neurons in the output part are used for object detection and classification. In this case, when training the adjusted ANN model, the model parameters of the backbone neurons can be fixed first, and the model parameters of the output neurons can be trained. Then, the model parameters of both the backbone and output neurons can be fine-tuned. If the adjusted ANN model is not a pre-trained ANN model, the entire adjusted ANN model can be trained directly.

[0111] Regardless of whether the adjusted artificial neural network model is a pre-trained artificial neural network model, the training process for the adjusted artificial neural network model is as follows:

[0112] Specifically, image samples can be obtained from open-source object detection datasets, where there can be multiple image samples. These image samples are then input into an adjusted artificial neural network model to output object detection results for the image samples. The object detection results include the center position and size of the predicted bounding box, whether the predicted bounding box anchors an object, and the object's classification result. With the optimization objective of minimizing the difference between the object detection result and the label, the adjustable parameters in the adjusted activation function of each neuron in the adjusted artificial neural network model are adjusted to obtain the adjusted parameters for each neuron. Finally, based on the adjusted parameters of each neuron, the trained model obtained after training the adjusted artificial neural network model is determined.

[0113] When adjusting the adjustable parameters in the adjusted activation function of each neuron in the adjusted artificial neural network model with the optimization objective of minimizing the difference between the target detection result and the label, the model parameters of each neuron and the adjustable parameters in the adjusted activation function of each neuron are adjusted to obtain the adjusted parameters of each neuron. The adjusted parameters may include at least one of the adjusted model parameters and the adjusted adjustable parameters.

[0114] Methods for determining the difference between the target detection result and the label may include:

[0115] The difference between the center position of the predicted bounding box and the center position of the label anchor box in the target detection result is determined as the first difference; the difference between the size of the predicted bounding box and the size of the label anchor box in the target detection result is determined as the second difference; based on the first difference and the second difference, the loss for the position and size of the predicted bounding box is determined as the first loss.

[0116] The formula for calculating the first loss is as follows: λ coord S is a constant. 2 This indicates the grid size; in this specification, the grid sizes are 13×13 and 26×26. B indicates the number of label anchor frames. This indicates that if there is an object within the bounding box at (i, j), the value is 1; otherwise, it is 0. i h represents the width of the predicted bounding box. i Indicates the height of the predicted bounding box, (x ij y ij ) represents the center position of the predicted bounding box, i.e., the center coordinates. Indicates the center coordinates of the label anchor box. This represents the width of the label anchor box. This represents the height of the label anchor frame.

[0117] Based on whether the predicted bounding box and the label anchor box are anchored to the target object in the target detection results, the prediction confidence for the predicted bounding box is determined and used as the second loss.

[0118] The formula for calculating the second loss is: λ obj , λ noobj c is a constant. ij This indicates whether the predicted bounding box is anchored to a target object. If a target object exists, c ij If it is 1, then c is the opposite. ij It is 0. This indicates that the label anchor frame is anchored to the target object. The value is 1. This indicates that if there is an object within the bounding box at (i, j), the value is 0, otherwise it is 1.

[0119] Based on the difference between the target classification results and the label classification results in the target detection results, the loss for target classification is determined as the third loss.

[0120] The formula for calculating the third loss is: λ class p is a constantij (c) represents the probability that the object in the predicted bounding box at (i, j) belongs to each category. This represents the probability that the target object in the label anchor box at (i, j) belongs to each category.

[0121] Based on the first loss, the second loss, and the third loss, the comprehensive loss is determined. That is, the first loss, the second loss, and the third loss are summed to obtain the comprehensive loss.

[0122] The formula for calculating the overall loss is: Loss = loss box +loss cls +loss obj .

[0123] When minimizing the difference between the object detection result and the label is the optimization objective, the adjusted artificial neural network model is trained with minimizing the overall loss as the optimization objective. That is, with minimizing the overall loss as the optimization objective, the model parameters of each neuron in the adjusted artificial neural network model and the adjustable parameters contained in the adjusted activation function of each neuron are adjusted to obtain the adjusted parameters of each neuron.

[0124] During backpropagation, a straight-through estimator (STE) can be used to calculate the backpropagation gradient of the floor function in the target activation function. The specific calculation is as follows: Loss represents the overall loss.

[0125] S108: Transfer the model parameters of the trained model to the spiking neural network model to be converted, so as to determine the initial spiking neural network model.

[0126] In the embodiments of this specification, after obtaining the trained model, the model parameters of the trained model can be transferred to the spiking neural network model to be converted, so as to determine the initial spiking neural network model.

[0127] In replacing the activation function of the original artificial neural network model with a target activation function, not just any function can replace the original activation function. The target activation function must satisfy at least one of these two conditions: First, the activation value output by the target activation function is the same as or closest to the activation value output by the activation function in the original artificial neural network model. Second, for each neuron, the activation value output by the target activation function for that neuron is the same as or closest to the frequency of pulses fired by that neuron in the spiking neural network model.

[0128] In determining the target activation function, to satisfy the first condition, the activation value output by the determined target activation function cannot be completely equivalent to the frequency value of the pulses emitted by the spiking neural network model, but is expected to be equivalent. In order to make the activation value output by the target activation function completely equivalent to the frequency value of the pulses emitted by the spiking neural network model, the bias parameters in the trained model can be adjusted during the transfer of the trained model to the spiking neural network model to be converted, and the adjusted bias parameters can be used as the bias parameters of the initial spiking neural network model.

[0129] Specifically, for each neuron in the trained model, the bias parameters included in the model parameters of that neuron in the trained model are adjusted according to a preset adjustment amount to obtain adjusted bias parameters. Then, based on the adjusted bias parameters, the adjusted model parameters obtained after adjusting the model parameters of that neuron in the trained model are determined. The adjusted model parameters may include: adjusted bias parameters and the weights corresponding to the neuron in the trained model. Then, the adjusted model parameters are used as the model parameters of that neuron in the spiking neural network model to be converted. The model parameters may include: weights and adjusted bias parameters. Simultaneously, the adjustable parameters included in the adjusted activation function of that neuron in the trained model are used as the membrane potential threshold required for that neuron to fire a pulse in the spiking neural network model to be converted. The adjustable parameters included in the adjusted activation function of that neuron in the trained model are adjustable parameters obtained after training the adjusted artificial neural network model. Then, based on the model parameters and membrane potential thresholds of each neuron in the spiking neural network model to be converted, the initial spiking neural network model is determined. The preset adjustment amount is... T represents the runtime of the initial spiking neural network model. This represents the membrane potential threshold of the i-th neuron in the l-th layer of the initial spiking neural network model.

[0130] The trained model has the same structure as the spiking neural network model to be converted. The spiking neural network model can refer to the Integrate-and-Fire Model (IF).

[0131] Furthermore, based on the feedforward calculation principle of the spiking neural network model, the pulse values ​​of the traditional spiking neurons in this embodiment are adjusted to obtain the spiking neural network model to be converted for target detection tasks. Because target detection tasks require outputting precise real-valued values ​​to determine the position and size of the target object, the activation value output by each neuron in the artificial neural network model and the frequency value of the pulses emitted by each neuron in the spiking neural network model must be the same or very close to each other to achieve comparable performance. However, the maximum frequency values ​​of different neurons in the spiking neural network model may be different. Therefore, in this specification, the 0 / 1 pulse is adjusted to 0 / θ to achieve scaling of the frequency value of the pulses emitted by each neuron in the spiking neural network model.

[0132] The feedforward calculation formula for the spiking neural network model is modified as follows:

[0133] The formula for accumulating input and increasing membrane potential is: This represents the cumulative increase in membrane potential before the repositioning of the i-th spiking neuron in layer l at time t. This represents the membrane potential of the i-th spiking neuron in the l-th layer at time t-1. This represents the weight between the j-th spiking neuron in layer (l-1) and the i-th spiking neuron in layer l. This represents the pulse value emitted by the j-th spiking neuron in the (l-1)-th layer. This represents the bias parameter of the i-th spiking neuron in the l-th layer.

[0134] The pulse value of the emitted pulse is: is the membrane potential threshold of the i-th spiking neuron in the l-th layer.

[0135] The reset formula after pulse delivery is: It represents the membrane potential of the i-th spiking neuron in the l-th layer at time t after resetting.

[0136] It should be noted that although the pulse value becomes a real number here, a 0 / 1 pulse can also be achieved by incorporating this value into the weights, thus reducing the computational load. In other words, the pulse value is treated as part of the weights. Among them, The weights between the j-th spiking neuron in layer (l-1) and the i-th spiking neuron in layer l are redefined.

[0137] S110: Based on the output value of each neuron in the trained model and the frequency value of the pulses emitted by each neuron in the initial spiking neural network model, the initial spiking neural network model is adjusted to obtain the final spiking neural network model.

[0138] In the embodiments of this specification, after transferring the model parameters and adjustable parameters from the trained model to the spiking neural network model, it is necessary to fine-tune the transferred spiking neural network model (i.e., the initial spiking neural network model) so that the output value of each neuron in the trained model is equal to or closest to the frequency value of the corresponding neuron firing pulses in the initial spiking neural network model. It should be noted that no additional runtime is required on top of the existing runtime during the adjustment of the initial spiking neural network model.

[0139] Specifically, based on the output value of each neuron in the trained model and the frequency value of the pulses emitted by each neuron in the initial spiking neural network model, the initial spiking neural network model is adjusted to obtain the final spiking neural network model.

[0140] When adjusting the initial spiking neural network model, at least one of the initial membrane potential corresponding to each neuron in the initial spiking neural network model and the model parameters containing bias parameters and weights can be adjusted.

[0141] Adjusting the initial membrane potential of neurons in the initial spiking neural network model:

[0142] Take an adjusted image sample as an example:

[0143] First, adjust image samples can be obtained to fine-tune the initial spiking neural network model. Then, these adjusted image samples are input into the trained model to obtain the output value of each neuron. Simultaneously, these adjusted image samples are input into the initial spiking neural network model to obtain the frequency value of each neuron's firing pulses.

[0144] Next, for each neuron in the initial spiking neural network model, an adjustment value is determined to adjust the initial membrane potential of that neuron based on the difference between the output value of that neuron in the trained model and the frequency of pulses fired by that neuron in the initial spiking neural network model, as well as the runtime of that neuron in the initial spiking neural network model. Finally, the initial membrane potential of that neuron in the initial spiking neural network model is adjusted to this adjustment value, resulting in the adjusted initial membrane potential of that neuron. Based on the adjusted initial membrane potential of each neuron, the final spiking neural network model is determined. Note that the initial membrane potential of that neuron in the initial spiking neural network model can be 0.

[0145] The formula for calculating the adjustment value is as follows: This represents the output value (i.e., activation value) of the i-th neuron in the l-th layer of the trained model after the activation function has been adjusted. This represents the frequency of pulses emitted by the i-th neuron in the l-th layer of the initial spiking neural network model when the initial membrane potential is 0.

[0146] When there are multiple adjusted image samples, for the same neuron, the adjusted values ​​corresponding to the multiple adjusted image samples can be averaged to obtain an average adjusted value. Then, the initial membrane potential of this neuron is adjusted to this average adjusted value.

[0147] Specifically, first, obtain each adjusted image sample. For each adjusted image sample, input the adjusted image sample into the trained model to obtain the output value of each neuron when the adjusted image sample passes through the trained model; input the adjusted image sample into the initial spiking neural network model to obtain the frequency value of the pulse fired by each neuron when the adjusted image sample passes through each neuron.

[0148] For each neuron in the initial spiking neural network model, the adjustment value for each adjusted image sample is determined based on the difference between the output value of the neuron when each adjusted image sample passes through the neuron in the trained model and the frequency value of the pulse emitted by the neuron when each adjusted image sample passes through the neuron in the initial spiking neural network model, as well as the runtime of the neuron in the initial spiking neural network model.

[0149] Based on the adjustment value for each adjusted image sample for that neuron, the average adjustment value for that neuron is determined. The initial membrane potential of that neuron in the initial spiking neural network model is adjusted to the average adjustment value to obtain the adjusted initial membrane potential of that neuron. Based on the adjusted initial membrane potential of each neuron, the final spiking neural network model is determined.

[0150] The formula for calculating the average adjustment value is as follows: N represents the number of image samples to be adjusted.

[0151] The model parameters of the neurons in the initial spiking neural network model are adjusted (i.e., the initial spiking neural network model is trained):

[0152] First, obtain adjusted image samples and input them into the trained model to obtain the output value of each neuron. Simultaneously, input the adjusted image samples into the initial spiking neural network model to obtain the frequency value of each neuron's firing pulses. Multiple adjusted image samples can be used.

[0153] For each neuron in the initial spiking neural network model, based on the output value of that neuron in the trained model and the frequency of pulses fired by that neuron in the initial spiking neural network model, the mean squared error between the output value of that neuron in the trained model and the frequency of pulses fired by that neuron in the initial spiking neural network model is determined. Here, the mean squared error is the loss from training the initial spiking neural network model.

[0154] With the goal of minimizing the mean squared error, the model parameters of the neuron in the initial spiking neural network model are adjusted to obtain the final spiking neural network model.

[0155] The formula for calculating the mean square error is: N represents the number of image samples to be adjusted. This represents the output value of the τth adjusted image sample at the i-th neuron in the l-th layer. This represents the frequency value of the pulse fired at the i-th neuron in the l-th layer of the τ-th adjusted image sample.

[0156] Due to the non-differentiable nature of the output function of each neuron in a spiking neural network model, an approximation gradient is needed during training to approximate the gradient of this function, enabling backpropagation. The specific formula for calculating the approximate gradient is as follows: Where |x| represents the absolute value, float(x) represents converting the logical Boolean value into a computable real value (e.g., logical "true" takes the value 1, logical "false" takes the value 0), and α is a hyperparameter related to the gradient magnitude, which can be set to 1, θ or other values ​​as needed.

[0157] Next, the weights and bias parameters of the initial spiking neural network model can be trained layer by layer using the back-propagation through time (BPTT) algorithm.

[0158] S112: After receiving the target detection task request, the target image carried in the target detection task request is input into the final spiking neural network model so as to perform target detection on the target image through the final spiking neural network model.

[0159] In the embodiments of this specification, the server can receive a target detection task request sent by a user. After receiving the target detection task request, the server can input the target image carried in the target detection task request into the final spiking neural network model to perform target detection on the target image and obtain the target detection result for the target image. Then, the target detection result for the target image is returned to the terminal device used by the user. The target detection result for the target image may include: bounding boxes anchoring the target objects in the target image, classification results of the target objects in the target image, etc.

[0160] It should be noted that all actions involving the acquisition of signal, information, or image data in this application are carried out in compliance with the relevant data protection laws and policies of the country where the application is located, and with the authorization granted by the owner of the relevant device.

[0161] Through the above Figure 1 As shown in the method, this specification adjusts the activation function of each neuron in the original artificial neural network model according to the runtime required by the spiking neural network model to be converted, obtaining the adjusted activation function for each neuron. The adjusted artificial neural network model containing the adjusted activation function is trained to obtain a trained model. The model parameters of the trained model are then transferred to the spiking neural network model to obtain an initial spiking neural network model. This initial spiking neural network model is then adjusted to obtain the final spiking neural network model. In this method, since the activation function of the artificial neural network model is adjusted based on the runtime of the spiking neural network model, transferring the model parameters obtained through training with the adjusted activation function to the spiking neural network model does not require the spiking neural network model to consume additional runtime to achieve performance close to that of the artificial neural network model, thereby reducing the computational load and power consumption of the spiking neural network model.

[0162] This specification explains the principle that the activation function of neurons in the original artificial neural network model can be adjusted to the target activation function, and the principle that the activation value output by the adjusted artificial neural network model through the target activation function is completely equivalent to the frequency value of the pulses fired by neurons in the spiking neural network model.

[0163] In the embodiments described in this specification, the feedforward calculation formula of the spiking neural network model to be converted is iteratively calculated within the runtime. For a certain neuron, the following is obtained: T represents the runtime. This represents the initial membrane potential of the i-th neuron in the l-th layer. This represents the weight between the j-th spiking neuron in layer (l-1) and the i-th spiking neuron in layer l. This represents the pulse value emitted by the j-th spiking neuron in the (l-1)-th layer. This represents the bias parameter of the i-th spiking neuron in the l-th layer.

[0164] right Dividing both sides of the equation by T, we get

[0165] Assumption That is In that case, because Since the pulse value is less than the pulse value that fires one pulse per time step, the frequency at which the neuron fires pulses cannot be changed.

[0166] Based on this, the feedforward calculation formula of the spiking neural network model can be described as follows: In order to make σ s The output value of (x) conforms to The range of values ​​for σ can be determined by... s (x) To quantify, that is,

[0167] In artificial neural network models, the output value of each neuron is not infinite in actual computation; it will definitely have a maximum value (let's assume it's λ). Therefore, we can... The activation function is rewritten as σ a (x)=f(x)=clip(x,0,λ)=min(λ,max(0,x)). λ can be set to θ.

[0168] To make the output value of the activation function in the original artificial neural network model completely equivalent to the frequency value of the pulses emitted by the spiking neural network model, f(x) in the original artificial neural network model can be replaced with g(x). That is, the activation function in the adjusted artificial neural network model is completely equivalent to the formula for calculating the frequency value of the emitted pulses in the spiking neural network model.

[0169] If g(x) is used as the target activation function, although the output value of the original artificial neural network model through the activation function is completely equivalent to the frequency value of the pulses emitted by the spiking neural network model, the difference between g(x) and f(x) is relatively large. For example... Figure 2 As shown. In Figure 2 In the diagram, assuming T is 20 and θ is 20, the dashed line represents g(x) and the solid line represents f(x).

[0170] To minimize the average error between f(x) and g(x), a constant b can be added to the bias parameters of the neurons in the spiking neural network model. shift Modify the function g(x), and use the modified g(x) as h(x).

[0171] In this way, the problem of reducing the average error between f(x) and g(x) can be transformed into finding the offset b when the theoretical error is minimized. shift The value of b, and satisfying 0 ≤ b shift T<θ.

[0172] Assuming the probability of the output value of an artificial neural network model (or the frequency value of a spiking neural network model) falling into each stage is p, then when Right now When the expected value of the error between functions f(x) and h(x) is 0, the expected value of the squared error reaches its minimum.

[0173] Rewrite h(x) as:

[0174] A diagram illustrating the difference between the rewritten h(x) and f(x) is shown below. Figure 3 As shown. In Figure 3 In the diagram, assuming T is 20 and θ is 20, the dashed line represents the rewritten h(x) and the solid line represents f(x).

[0175] In this way, the activation function f(x) in the original artificial neural network model can be replaced with the rewritten h(x).

[0176] Because the formula for calculating the frequency of the emitted pulse in the spiking neural network model was added with... This minimizes the error between f(x) and the rewritten h(x), essentially transforming the complete equivalence of the original artificial neural network model's output value through the activation function to the expected equivalence of the frequency values ​​of the pulses emitted by the spiking neural network model. In other words, the rewritten h(x) cannot make the activation value output by the adjusted artificial neural network model through the activation function the same as the frequency values ​​of the pulses emitted by the spiking neural network model.

[0177] When the target activation function is the rewritten h(x), to ensure that the target activation function in the adjusted artificial neural network model is identical to the formula for calculating the frequency of the firing pulse in the spiking neural network model, when converting the adjusted artificial neural network model to a spiking neural network model, the bias terms of the neurons in the spiking neural network model need to be added. To achieve complete equivalence between the artificial neural network model and the spiking neural network model, the bias of the final spiking neural network model is:

[0178] It should be noted that when T is changed to T′ (for example, by using a shorter runtime), h(x) and f(x) when changed to T′ still satisfy the condition that the expected error is 0 and the expected value of the squared error is minimized. Therefore, the conversion can be achieved almost without loss without retraining the artificial neural network model from scratch, which greatly saves the cost of generating the spiking neural network model.

[0179] The above are the target detection methods provided in the embodiments of this specification. Based on the same idea, this specification also provides corresponding devices, storage media and electronic devices.

[0180] Figure 4 This is a schematic diagram of a target detection device provided in an embodiment of this specification. The device includes:

[0181] Module 401 is used to acquire the original artificial neural network model;

[0182] The first adjustment module 402 is used to adjust the activation function of each neuron in the original artificial neural network model according to the running time required by the spiking neural network model to be converted and according to a preset adjustment strategy, so as to obtain the adjusted activation function of each neuron.

[0183] The determining module 403 is used to determine the adjusted artificial neural network model based on the adjusted activation function of each neuron;

[0184] Training module 404 is used to train the adjusted artificial neural network model to obtain the trained model;

[0185] The parameter transfer module 405 is used to transfer the model parameters of the trained model to the spiking neural network model to be converted, so as to determine the initial spiking neural network model.

[0186] The second adjustment module 406 is used to adjust the initial spiking neural network model based on the output value of each neuron in the trained model and the frequency value of the pulses fired by each neuron in the initial spiking neural network model, so as to obtain the final spiking neural network model.

[0187] The target detection module 407 is used to input the target image carried in the target detection task request into the final spiking neural network model after receiving the target detection task request, so as to perform target detection on the target image through the final spiking neural network model.

[0188] Optionally, the first adjustment module 402 is specifically used to: determine an adjustable parameter representing the membrane potential threshold required for each neuron in the spiking neural network model to be converted to fire a pulse; determine a target activation function based on the adjustable parameter and the required runtime of the spiking neural network model to be converted; and replace the activation function of each neuron in the original artificial neural network model with the target activation function according to a preset adjustment strategy to obtain the adjusted activation function of each neuron.

[0189] Optionally, the first adjustment module 402 is specifically used to: determine a first function representing the relationship between the input value, the runtime, and the adjustable parameters; determine a second function representing the relationship between the first function and the runtime; and determine a target activation function based on the second function and the adjustable parameters.

[0190] Optionally, the determining module 403 is specifically used to adjust the max pooling method in the original artificial neural network model to the average pooling method; and determine the adjusted artificial neural network model based on the average pooling method and the adjusted activation function of each neuron.

[0191] Optionally, the training module 404 is specifically used to: acquire image samples; input the image samples into the adjusted artificial neural network model to output target detection results for the image samples through the adjusted artificial neural network model, wherein the target detection results include the center position and size of the predicted bounding box, whether the predicted bounding box is anchored to the target object, and the classification result of the target object; with minimizing the difference between the target detection results and the label as the optimization objective, adjust the adjustable parameters contained in the adjusted activation function of each neuron in the adjusted artificial neural network model to obtain the adjusted parameters of each neuron; wherein the adjustable parameters are used to represent the membrane potential threshold required for each neuron of the spiking neural network model to fire a pulse; and based on the adjusted parameters of each neuron, determine the trained model obtained after training the adjusted artificial neural network model.

[0192] Optionally, the parameter transfer module 405 is specifically used to: for each neuron in the trained model, adjust the bias parameters included in the model parameters of that neuron in the trained model according to a preset adjustment amount to obtain adjusted bias parameters; determine the adjusted model parameters obtained after adjusting the model parameters of that neuron in the trained model according to the adjusted bias parameters; use the adjusted model parameters as the model parameters of that neuron in the spiking neural network model to be converted, and use the adjustable parameters included in the adjusted activation function of that neuron in the trained model as the membrane potential threshold required for that neuron to fire a pulse in the spiking neural network model to be converted; wherein, the model structure of the trained model is the same as the model structure of the spiking neural network model to be converted; and determine the initial spiking neural network model according to the model parameters of each neuron in the spiking neural network model to be converted and the membrane potential threshold.

[0193] Optionally, the second adjustment module 406 is specifically configured to: acquire each adjusted image sample; input the adjusted image sample into the trained model for each adjusted image sample to obtain the output value of the adjusted image sample through each neuron; input the adjusted image sample into the initial spiking neural network model to obtain the frequency value of each neuron firing pulses when the adjusted image sample passes through each neuron; for each neuron in the initial spiking neural network model, determine the adjustment value of each adjusted image sample for that neuron based on the difference between the output value of each adjusted image sample passing through that neuron in the trained model and the frequency value of each adjusted image sample firing pulses when passing through that neuron in the initial spiking neural network model, and the runtime of that neuron in the initial spiking neural network model; determine the average adjustment value for that neuron based on the adjustment value of each adjusted image sample for that neuron; adjust the initial membrane potential of that neuron in the initial spiking neural network model to the average adjustment value to obtain the adjusted initial membrane potential of that neuron; and determine the final spiking neural network model based on the adjusted initial membrane potential of each neuron.

[0194] Optionally, the second adjustment module 406 is specifically configured to: acquire an adjustment image sample; input the adjustment image sample into the trained model to obtain the output value of each neuron through the trained model; input the adjustment image sample into the initial spiking neural network model to obtain the frequency value of each neuron firing pulses through the initial spiking neural network model; for each neuron in the initial spiking neural network model, based on the output value of the neuron in the trained model and the frequency value of the neuron firing pulses in the initial spiking neural network model, determine the mean square error between the output value of the neuron in the trained model and the frequency value of the neuron firing pulses in the initial spiking neural network model; and adjust the model parameters of the neuron in the initial spiking neural network model with the goal of minimizing the mean square error to obtain the final spiking neural network model.

[0195] This specification also provides a computer-readable storage medium storing a computer program that, when executed by a processor, can be used to perform the above-described actions. Figure 1 The provided target detection method.

[0196] based on Figure 1 The target detection method shown in this specification also provides embodiments that... Figure 5 The diagram shows the structure of the electronic device. Figure 5 At the hardware level, this electronic device includes a processor, internal bus, network interface, memory, and non-volatile memory, and may also include other hardware required for business operations. The processor reads the corresponding computer program from the non-volatile memory into memory and then executes it to achieve the above. Figure 1 The target detection method described above.

[0197] Of course, in addition to software implementation, this specification does not exclude other implementation methods, such as logic devices or a combination of hardware and software. In other words, the execution subject of the following processing flow is not limited to each logic unit, but can also be hardware or logic devices.

[0198] In the 1990s, improvements to a technology could be clearly distinguished as either hardware improvements (e.g., improvements to the circuit structure of diodes, transistors, switches, etc.) or software improvements (improvements to the methodology). However, with technological advancements, many methodological improvements today can be considered direct improvements to the hardware circuit structure. Designers almost always obtain the corresponding hardware circuit structure by programming the improved methodology into the hardware circuit. Therefore, it cannot be said that a methodological improvement cannot be implemented using hardware physical modules. For example, a Programmable Logic Device (PLD) (such as a Field Programmable Gate Array (FPGA)) is such an integrated circuit whose logic function is determined by the user programming the device. Designers can program and "integrate" a digital system onto a PLD themselves, without needing chip manufacturers to design and manufacture dedicated integrated circuit chips. Furthermore, nowadays, instead of manually manufacturing integrated circuit chips, this programming is mostly implemented using "logic compiler" software. Similar to the software compiler used in program development, the original code before compilation must be written in a specific programming language, called a Hardware Description Language (HDL). There are many HDLs, such as ABEL (Advanced Boolean Expression Language), AHDL (Altera Hardware Description Language), Confluence, CUPL (Cornell University Programming Language), HDCal, JHDL (Java Hardware Description Language), Lava, Lola, MyHDL, PALASM, and RHDL (Ruby Hardware Description Language). Currently, the most commonly used are VHDL (Very-High-Speed ​​Integrated Circuit Hardware Description Language) and Verilog. Those skilled in the art should understand that by simply performing some logic programming on the method flow using one of these hardware description languages ​​and programming it into an integrated circuit, the hardware circuit implementing the logical method flow can be easily obtained.

[0199] The controller can be implemented in any suitable manner. For example, it can take the form of a microprocessor or processor and a computer-readable medium storing computer-readable program code (e.g., software or firmware) executable by the (micro)processor, logic gates, switches, application-specific integrated circuits (ASICs), programmable logic controllers, and embedded microcontrollers. Examples of controllers include, but are not limited to, the following microcontrollers: ARC 625D, Atmel AT91SAM, Microchip PIC18F26K20, and Silicon Labs C8051F320. A memory controller can also be implemented as part of the control logic of the memory. Those skilled in the art will also recognize that, in addition to implementing the controller in purely computer-readable program code form, the same functionality can be achieved by logically programming the method steps to make the controller take the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, and embedded microcontrollers. Therefore, such a controller can be considered a hardware component, and the means included therein for implementing various functions can also be considered as structures within the hardware component. Alternatively, the means for implementing various functions can be considered as both software modules implementing the method and structures within the hardware component.

[0200] The systems, devices, modules, or units described in the above embodiments can be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer. Specifically, a computer can be, for example, a personal computer, laptop computer, cellular phone, camera phone, smartphone, personal digital assistant, media player, navigation device, email device, game console, tablet computer, wearable device, or any combination of these devices.

[0201] For ease of description, the above devices are described in terms of function, divided into various units. Of course, in implementing this specification, the functions of each unit can be implemented in one or more software and / or hardware components.

[0202] Those skilled in the art will understand that embodiments of this specification can be provided as methods, systems, or computer program products. Therefore, this specification may take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this specification may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0203] This specification is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this specification. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create a machine for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0204] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0205] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes ​ The steps of the function specified in one or more boxes.

[0206] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0207] Memory may include non-persistent storage in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0208] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0209] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0210] Those skilled in the art will understand that the embodiments of this specification can be provided as methods, systems, or computer program products. Therefore, this specification may take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this specification may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0211] This specification can be described in the general context of computer-executable instructions that are executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform a specific task or implement a specific abstract data type. This specification can also be practiced in distributed computing environments, where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.

[0212] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to interchangeably. Each embodiment focuses on describing the differences from other embodiments. In particular, the system embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions in the method embodiments.

[0213] The above description is merely an embodiment of this specification and is not intended to limit this specification. Various modifications and variations can be made to this specification by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this specification should be included within the scope of the claims of this specification.

Claims

1. A target detection method characterized by, The method comprises the following steps: acquiring an original artificial neural network model; determining an adjustable parameter for representing a membrane potential threshold value required to be reached by each neuron of a pulse neural network model to be converted; determining a first function for representing an association relationship between an input value, a required runtime of the pulse neural network model to be converted, and the adjustable parameter; determining a second function for representing an association relationship between the first function and the runtime; and determining a target activation function according to the second function and the adjustable parameter; replacing, according to a preset adjustment strategy, an activation function of each neuron in the original artificial neural network model with the target activation function to obtain an adjusted activation function of each neuron; determining an adjusted artificial neural network model according to the adjusted activation function of each neuron; training the adjusted artificial neural network model to obtain a trained model; migrating model parameters of the trained model to the pulse neural network model to be converted to determine an initial pulse neural network model; adjusting the initial pulse neural network model based on an output value of each neuron in the trained model and a frequency value of pulse emission of each neuron in the initial pulse neural network model to obtain a final pulse neural network model; after receiving a target detection task request, inputting a target image carried in the target detection task request into the final pulse neural network model to perform target detection on the target image through the final pulse neural network model.

2. The method of claim 1, wherein, The method of determining an adjusted artificial neural network model according to the adjusted activation function of each neuron specifically comprises: adjusting a max-pooling mode in the original artificial neural network model to an average-pooling mode; determining an adjusted artificial neural network model according to the average-pooling mode and the adjusted activation function of each neuron.

3. The method of claim 1, wherein, The method of training the adjusted artificial neural network model to obtain a trained model specifically comprises: acquiring an image sample; inputting the image sample into the adjusted artificial neural network model to output a target detection result for the image sample through the adjusted artificial neural network model, wherein the target detection result comprises a center position and a size of a predicted bounding box, whether the predicted bounding box is anchored to a target object, and a classification result of the target object; adjusting an adjustable parameter contained in the adjusted activation function of each neuron in the adjusted artificial neural network model to obtain an adjusted parameter of each neuron, with the optimization target of minimizing a difference between the target detection result and a label; wherein the adjustable parameter is used to represent a membrane potential threshold value required to be reached by each neuron of the pulse neural network model to be converted to emit a pulse; determining a trained model obtained after training the adjusted artificial neural network model based on the adjusted parameter of each neuron.

4. The method of claim 1, wherein, The method of migrating model parameters of the trained model to the pulse neural network model to be converted to determine an initial pulse neural network model specifically comprises: For each neuron of the trained model, a preset adjustment amount is used to adjust a bias parameter included in a model parameter of the neuron in the trained model, to obtain an adjusted bias parameter; According to the adjusted bias parameter, an adjusted model parameter of the neuron in the trained model is determined after adjustment of the model parameter of the neuron in the trained model; The adjusted model parameter is used as the model parameter of the neuron in the to-be-converted spiking neural network model, and an adjustable parameter included in an adjusted activation function of the neuron in the trained model is used as a membrane potential threshold value required for the neuron in the to-be-converted spiking neural network model to fire a pulse; wherein the model structure of the trained model is the same as the model structure of the to-be-converted spiking neural network model; According to the model parameters and the membrane potential threshold value of each neuron in the to-be-converted spiking neural network model, an initial spiking neural network model is determined.

5. The method of claim 1, wherein, Based on the output value of each neuron in the trained model and the frequency value of each neuron firing a pulse in the initial spiking neural network model, the initial spiking neural network model is adjusted to obtain a final spiking neural network model, specifically including: Obtaining an adjusted image sample; For each adjusted image sample, the adjusted image sample is input into the trained model to obtain an output value of each neuron through the trained model, and the adjusted image sample is input into the initial spiking neural network model to obtain a frequency value of each neuron firing a pulse when the adjusted image sample passes through each neuron through the initial spiking neural network model; For each neuron in the initial spiking neural network model, an adjustment value of each adjusted image sample for the neuron is determined according to a difference between the output value of each adjusted image sample passing through the neuron in the trained model and the frequency value of the neuron firing a pulse when each adjusted image sample passes through the neuron in the initial spiking neural network model, and a running time length of the neuron in the initial spiking neural network model; According to the adjustment value of each adjusted image sample for the neuron, an average adjustment value for the neuron is determined; The initial membrane potential of the neuron in the initial spiking neural network model is adjusted to the average adjustment value to obtain an adjusted initial membrane potential of the neuron; According to the adjusted initial membrane potential of each neuron, a final spiking neural network model is determined.

6. The method of claim 1, wherein, Based on the output value of each neuron in the trained model and the frequency value of each neuron firing a pulse in the initial spiking neural network model, the initial spiking neural network model is adjusted to obtain a final spiking neural network model, specifically including: Obtaining an adjusted image sample; The adjusted image sample is input into the trained model to obtain an output value of each neuron through the trained model, and the adjusted image sample is input into the initial spiking neural network model to obtain a frequency value of each neuron firing a pulse through the initial spiking neural network model; For each neuron in the initial spiking neural network model, based on an output value of the neuron in the trained model and a frequency value of the neuron firing pulses in the initial spiking neural network model, a mean square error between the output value of the neuron in the trained model and the frequency value of the neuron firing pulses in the initial spiking neural network model is determined; The model parameters of the neuron in the initial spiking neural network model are adjusted to minimize the mean square error, and a final spiking neural network model is obtained.

7. A target detection apparatus characterized by comprising: Comprise: An acquisition module is configured to acquire an original artificial neural network model; A first adjustment module is configured to determine an adjustable parameter used to represent a membrane potential threshold required for each neuron of a to-be-converted spiking neural network model to fire a pulse; determine a first function used to represent an association relationship among a to-be-input value, a required runtime of the to-be-converted spiking neural network model, and the adjustable parameter; determine a second function used to represent an association relationship between the first function and the runtime; and determine a target activation function according to the second function and the adjustable parameter; According to a preset adjustment strategy, the activation function of each neuron in the original artificial neural network model is replaced by the target activation function, and an adjusted activation function of each neuron is obtained; A determination module is configured to determine an adjusted artificial neural network model according to the adjusted activation function of each neuron; A training module is configured to train the adjusted artificial neural network model to obtain a trained model; A parameter migration module is configured to migrate model parameters of the trained model to the to-be-converted spiking neural network model to determine an initial spiking neural network model; A second adjustment module is configured to adjust the initial spiking neural network model based on an output value of each neuron in the trained model and a frequency value of each neuron firing pulses in the initial spiking neural network model to obtain a final spiking neural network model; A target detection module is configured to input a target image carried in a target detection task request to the final spiking neural network model after receiving the target detection task request, so as to perform target detection on the target image by using the final spiking neural network model.

8. The apparatus of claim 7, wherein, The determination module is specifically configured to adjust a max-pooling manner in the original artificial neural network model to an average-pooling manner; and determine an adjusted artificial neural network model according to the average-pooling manner and the adjusted activation function of each neuron.

9. The apparatus of claim 7, wherein, The training module is specifically configured to: acquire an image sample; input the image sample into the adjusted artificial neural network model to output a target detection result for the image sample through the adjusted artificial neural network model, wherein the target detection result includes a center position and a size of a predicted bounding box, whether the predicted bounding box anchors a target object, and a classification result of the target object; adjust a tunable parameter contained in an adjusted activation function of each neuron in the adjusted artificial neural network model to obtain an adjusted parameter of each neuron, with minimization of a difference between the target detection result and a label as an optimization target; and determine a trained model obtained after training of the adjusted artificial neural network model based on the adjusted parameter of each neuron.

10. The apparatus of claim 7, wherein, The parameter migration module is specifically configured to: for each neuron of the trained model, adjust a bias parameter contained in a model parameter of the neuron in the trained model according to a preset adjustment amount to obtain an adjusted bias parameter; determine an adjusted model parameter obtained after adjustment of the model parameter of the neuron in the trained model based on the adjusted bias parameter; and use the adjusted model parameter as the model parameter of the neuron in the to-be-converted spiking neural network model and use a tunable parameter contained in an adjusted activation function of the neuron in the trained model as a membrane potential threshold required for the neuron in the to-be-converted spiking neural network model to fire a spike. The storage medium stores a computer program, and the computer program is executed by the processor to implement the method in any one of claims 1-6.

11. A computer readable storage medium, characterized in that, The processor executes the program to implement the method in any one of claims 1-6.

12. An electronic device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, ​

Images