Self-adaptive sensitized pulse model construction method, device and equipment and storage medium

Abstract

The invention relates to an adaptive sensitization pulse model construction method and device, equipment and a storage medium, and the method comprises the steps: recording the simulation time and excitation historical data of a self-sensitization neuron based on a neuron excitation function of a preset self-sensitization neuron; adjusting the neuron state of the self-sensitization neuron according to the simulation time and the excitation historical data so as to construct a self-sensitization neuron model; and in a preset pulse neural network model, carrying the self-sensitization neuron model, training the preset pulse neural network model, generating a self-adaptive sensitization pulse model, and obtaining a classification result of each piece of target image data by using the self-adaptive sensitization pulse model. Therefore, the problems that when model training is carried out in the prior art, resource consumption is too large, a network model does not have self-adaptive change input, and a complex network algorithm is poor in generalization ability, narrow in application range and the like are solved.

Description

Technical Field

[0001] The present application relates to the field of deep learning technology, and in particular to a method, device, equipment and storage medium for constructing an adaptive sensitized pulse model. Background Art

[0002] Existing neural networks have a data drift problem, that is, when training and deploying neural network models, the data distribution faced by the model changes in time or environment, resulting in a decline in model performance. This situation is mainly caused by the inconsistency between the data distribution seen by the model during training and the data distribution encountered in actual applications.

[0003] Currently, existing technologies can solve the above data drift problem based on data enhancement (expanding training data) or complex network structure algorithms.

[0004] However, existing technologies consume a lot of resources and energy when adding data samples for training, and complex network algorithms have weak generalization capabilities and narrow applicability. They can only temporarily alleviate the problem of data drift through a large amount of learning, but cannot fundamentally get rid of the dilemma that network models cannot autonomously adapt to changing inputs, which needs to be solved urgently. Summary of the invention

[0005] The present application provides a method, device, equipment and storage medium for constructing an adaptive sensitized pulse model to solve the problems of excessive resource consumption, the inability of the network model to autonomously adapt to changing inputs, the weak generalization ability of complex network algorithms, and the narrow scope of applicability when training models in the prior art.

[0006] The first aspect of the present application provides a method for constructing an adaptive sensitizing pulse model, which is applied to an offline training stage, wherein the method comprises the following steps: based on a neuron excitation function of a preset self-sensitizing neuron, recording a simulation time and excitation history data of the self-sensitizing neuron; adjusting the neuron state of the self-sensitizing neuron according to the simulation time and the excitation history data to construct a self-sensitizing neuron model; in a preset pulse neural network model, carrying the self-sensitizing neuron model, and training the preset pulse neural network model to generate an adaptive sensitizing pulse model, so as to obtain a classification result of each target image data using the adaptive sensitizing pulse model.

[0007] Optionally, in one embodiment of the present application, adjusting the neuron state of the autosensitized neuron according to the simulation time and the excitation history data to construct an autosensitized neuron model includes: obtaining input stimulation data of the neuron excitation function, and updating the neuron potential corresponding to the input stimulation data according to the simulation time to obtain the latest neuron potential; inputting the latest neuron potential into the neuron excitation function to determine whether the autosensitized neuron is excited.

[0008] Optionally, in an embodiment of the present application, in the preset spiking neural network model, loading the self-sensitized neuron model and training the preset spiking neural network model includes: collecting input data of the adaptive sensitized pulse model, and preprocessing the input data to obtain normalized data of the input data; within the simulation time, continuously inputting the normalized data into the adaptive sensitized pulse model, and converting the normalized data into pulse time data through a preset encoding function.

[0009] Optionally, in an embodiment of the present application, training the preset spiking neural network model to generate an adaptive sensitized pulse model includes: based on the pulse time data and the adaptive sensitized pulse model, obtaining and saving the pulse / spike data of the output layer neurons of the adaptive sensitized pulse model; calculating the firing frequency of the output layer neurons according to the pulse / spike data and the simulation time; determining the loss function of the adaptive sensitized pulse model according to the firing frequency, and optimizing the adaptive sensitized pulse model through a preset optimization algorithm and the loss function.

[0010] An embodiment of the second aspect of the present application provides a method for constructing an adaptive sensitized pulse model, which is applied to the online detection stage. The method includes the following steps: collecting at least one target image data in the current environment; performing image size transformation, data type conversion, and normalization operations on the at least one target image data to construct a classification data set for the adaptive sensitized pulse model; inputting the classification data in the classification data set into a pre-trained adaptive sensitized pulse model to generate a classification result for each target image data.

[0011] An embodiment of the third aspect of the present application provides an apparatus for constructing an adaptive sensitized pulse model, which is applied to the offline training stage. The apparatus includes: a recording module, configured to record the simulation time and the firing history data of the self-sensitized neuron based on the neuron firing function of the preset self-sensitized neuron; an adjustment module, configured to adjust the neuron state of the self-sensitized neuron according to the simulation time and the firing history data to construct a self-sensitized neuron model; a training module, configured to load the self-sensitized neuron model in a preset spiking neural network model and train the preset spiking neural network model to generate an adaptive sensitized pulse model, so as to obtain a classification result for each target image data by using the adaptive sensitized pulse model.

[0012] Optionally, in an embodiment of the present application, the adjustment module includes: an update unit, configured to obtain the input stimulus data of the neuron activation function, and update the neuron potential corresponding to the input stimulus data according to the simulation time to obtain the latest neuron potential; a determination unit, configured to input the latest neuron potential into the neuron activation function to determine whether the self-sensitized neuron is activated.

[0013] Optionally, in an embodiment of the present application, the training module includes: a generation unit, configured to collect the input data of the adaptive sensitization pulse model, and perform preprocessing on the input data to obtain the normalized data of the input data; a conversion unit, configured to continuously input the normalized data into the adaptive sensitization pulse model within the simulation time, and convert the normalized data into pulse time data through a preset encoding function.

[0014] Optionally, in an embodiment of the present application, the training module further includes: a saving unit, configured to obtain and save the pulse / spike data of the output layer neurons of the adaptive sensitization pulse model based on the pulse time data and the adaptive sensitization pulse model; a calculation unit, configured to calculate the firing frequency of the output layer neurons according to the pulse / spike data and the simulation time; an optimization unit, configured to determine the loss function of the adaptive sensitization pulse model according to the firing frequency, and optimize the adaptive sensitization pulse model through a preset optimization algorithm and the loss function.

[0015] An embodiment of the fourth aspect of the present application provides an adaptive sensitization pulse model construction device, which is applied to the online detection stage, and includes: a collection module, configured to collect at least one target image data in the current environment; a preprocessing module, configured to perform image size transformation, data type conversion, and normalization operations on the at least one target image data to construct a classification data set of the adaptive sensitization pulse model; a classification module, configured to input the classification data in the classification data set into a pre-trained adaptive sensitization pulse model to generate a classification result for each target image data.

[0016] An embodiment of the fifth aspect of the present application provides an electronic device, including: a memory, a processor, and a computer program stored on the memory and executable on the processor, where the processor executes the program to implement the adaptive sensitization pulse model construction method as described in the above embodiment.

[0017] An embodiment of the sixth aspect of the present application provides a computer-readable storage medium, where the computer-readable storage medium stores a computer program, and when the program is executed by a processor, it implements the adaptive sensitization pulse model construction method as described above.

[0018] Accordingly, the embodiments of the present application have the following beneficial effects:

[0019] Embodiments of the present application can record the simulation time and firing history data of self-sensitizing neurons through a neuron firing function based on a preset self-sensitizing neuron; adjust the neuron state of the self-sensitizing neuron according to the simulation time and firing history data to construct a self-sensitizing neuron model; load the self-sensitizing neuron model in a preset spiking neural network model, and train the preset spiking neural network model to generate an adaptive sensitizing spiking model, so as to obtain the classification result of each target image data by using the adaptive sensitizing spiking model. Through the novel adaptive sensitizing spiking neural network model, the present application does not need to consume a large amount of computing power for data training, effectively solves the data drift problem, and greatly improves the learning and training efficiency of the model. Accordingly, it solves the problems in the prior art that when performing model training, the resource consumption is too large, the network model does not have the ability to autonomously adapt to changing inputs, and the generalization ability of complex network algorithms is weak and the applicable range is narrow.

[0020] Some aspects and advantages of the present application will be given in part in the following description, some will become obvious from the following description, or will be understood through the practice of the present application. Description of the Drawings

[0021] The above and / or additional aspects and advantages of the present application will become obvious and easy to understand from the following description of the embodiments in conjunction with the drawings, where:

[0022] Figure 1 It is a flowchart of a method for constructing an adaptive sensitizing spiking model applied to the offline training stage according to an embodiment of the present application;

[0023] Figure 2 It is a schematic diagram of the structure of a spiking convolutional neural network provided by an embodiment of the present application;

[0024] Figure 3 It is a schematic diagram of the structure of a spiking fully-connected network provided by an embodiment of the present application;

[0025] Figure 4 It is a schematic diagram of the effect of using the standard general Fashion MNIST dataset provided by an embodiment of the present application;

[0026] Figure 5 It is a schematic diagram of the effect of using self-built vehicles in a changing environment for vehicle classification provided by an embodiment of the present application;

[0027] Figure 6 It is a flowchart of a method for constructing an adaptive sensitizing spiking model applied to the online decision-making stage according to an embodiment of the present application;

[0028] Figure 7 It is an exemplary diagram of an adaptive sensitized pulse model construction device applied to the offline training stage according to an embodiment of the present application;

[0029] Figure 8 It is an exemplary diagram of an adaptive sensitized pulse model construction device applied to the online decision-making stage according to an embodiment of the present application;

[0030] Figure 9 It is a schematic structural diagram of an electronic device provided by an embodiment of the present application.

[0031] Among them, 10 - an adaptive sensitized pulse model construction device applied to the offline training stage, 20 - an adaptive sensitized pulse model construction device applied to the online detection stage; 101 - a recording module, 102 - an adjustment module, 103 - a training module; 201 - a collection module, 202 - a preprocessing module, 203 - a classification module; 901 - a memory, 902 - a processor, 903 - a communication interface. Detailed implementation manners

[0032] The embodiments of the present application will be described in detail below. The examples of the embodiments are shown in the accompanying drawings, where the same or similar reference numerals denote the same or similar elements or elements with the same or similar functions from beginning to end. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to explain the present application, but should not be construed as limiting the present application.

[0033] The adaptive sensitized pulse model construction method, device, equipment and storage medium of the embodiments of the present application will be described below with reference to the accompanying drawings. In view of the problems mentioned in the above background art, the present application provides an adaptive sensitized pulse model construction method. In this method, based on the neuron excitation function of a preset self-sensitized neuron, the simulation time and excitation history data of the self-sensitized neuron are recorded; according to the simulation time and excitation history data, the neuron state of the self-sensitized neuron is adjusted to construct a self-sensitized neuron model; in a preset pulse neural network model, the self-sensitized neuron model is loaded, and the preset pulse neural network model is trained to generate an adaptive sensitized pulse model, so as to obtain the classification result of each target image data by using the adaptive sensitized pulse model. The present application uses a novel adaptive sensitized pulse neural network model, which does not require a large amount of computing power for data training, effectively solves the data drift problem, and greatly improves the learning and training efficiency of the model. Thus, the problems in the prior art such as excessive resource consumption during model training, the network model not being able to adapt to changing inputs autonomously, and the weak generalization ability and narrow application range of complex network algorithms are solved.

[0034] Specifically, Figure 1 It is a flowchart of an adaptive sensitized pulse model construction method applied to the offline training stage provided by an embodiment of the present application.

[0035] As Figure 1 shown, the method for constructing the adaptive sensitization pulse model includes the following steps:

[0036] In step S101, based on the neuron excitation function of the preset self-sensitization neuron, record the simulation time and excitation history data of the self-sensitization neuron.

[0037] In the embodiments of the present application, first, according to the neuron excitation function of the preset self-sensitization neuron, the simulation time of the self-sensitization neuron and the excitation history data of the neuron can be recorded by setting corresponding variables, so as to provide reliable data support for the construction of the self-sensitization neuron model.

[0038] In step S102, according to the simulation time and excitation history data, adjust the neuron state of the self-sensitization neuron to construct a self-sensitization neuron model.

[0039] After recording the simulation time and excitation history data of the self-sensitization neuron, further, in the embodiments of the present application, the neuron state of the self-sensitization neuron can also be adjusted by combining the simulation time information and the excitation history data each time the excitation function is called, so as to generate a self-sensitization neuron model, thereby realizing the construction of the subsequent adaptive sensitization pulse model.

[0040] Optionally, in an embodiment of the present application, adjusting the neuron state of the self-sensitization neuron according to the simulation time and excitation history data to construct a self-sensitization neuron model includes: obtaining the input stimulus data of the neuron excitation function, and updating the neuron potential corresponding to the input stimulus data according to the simulation time to obtain the latest neuron potential; inputting the latest neuron potential into the neuron excitation function to determine whether the self-sensitization neuron is excited.

[0041] It should be noted that in the single calculation step of the embodiments of the present application, the neuron can regard the input data as the stimulus intensity, calculate the latest neuron potential under the action of the corresponding intensity, and judge whether the neuron is excited according to the excitation function of the neuron.

[0042] Specifically, according to the current simulation time (or simulation time step), the update of the neuron potential corresponding to the input stimulus is completed, and the calculation formula of the neuron potential is:

[0043]

[0044] Among them, represents the change rate of the membrane potential with time, τ in represents the input time constant, τ lk represents the leakage time constant, and V inject represents the input voltage; where τ in, τ lk It is determined by the physical properties of the neuron material, and its specific value can be obtained through actual tests. The specific calculation formula is as follows:

[0045]

[0046]

[0047] Among them, C m is the capacitance value, representing the capacitance of the circuit or system. A capacitor is an element that stores electric charge and plays an important role in the transmission of input signals and the response speed of the system; R d , R s is the resistance value, representing the series resistance and parallel resistance respectively. A resistor is an element that controls the flow of current and affects the damping and leakage characteristics of the system.

[0048] Subsequently, the embodiments of the present application can use the corresponding neuron potential as the input, calculate the firing probability function of the current corresponding neuron state, obtain the firing probability of the neuron at the current simulation time step, and use this firing probability as the input to determine whether the final neuron fires by using the torch.bernoulli() function. When the output is 1, the corresponding neuron fires; when the output is 0, the corresponding neuron does not fire.

[0049] After completing the calculation of all corresponding neurons in the Tensor, the embodiments of the present application can output the calculation results and accumulate the pulse firing results of all neurons to obtain the total number of firing pulses up to the current simulation time step.

[0050] It should be noted that during the calculation process, it is important to adjust the state of the self-sensitizing neurons. Self-sensitizing neurons can reduce the firing difficulty of neurons when continuously receiving stimuli without firing pulses. In self-sensitizing neurons, different neuron states correspond to different neuron firing difficulties, that is, different neuron "thresholds".

[0051] Specifically, in the simulation time step calculation loop, after completing the accumulation of firing pulses, the embodiments of the present application can judge the current simulation time step according to the updated simulation time step information in the self-sensitizing neuron model to determine whether the current simulation time is a preset checkpoint.

[0052] As a feasible implementation, in the embodiments of the present application, the checkpoints can be set as the 2nd, 3rd, 6th, 12th, 18th, and 50th simulation time steps. If the checkpoint requirements are met, the determination is made according to the current neuron historical firing data:

[0053] If the historical firing pulse number of the neuron is 0, then change the state of the corresponding neuron in the neuron layer.

[0054] Among them, when the checkpoint is 1 and the simulation time step is 2, if the change condition is satisfied, the neuron state switches to state 1;

[0055] When the checkpoint is 2 and the simulation time step is 3, if the change condition is satisfied, the neuron state switches to state 2;

[0056] When the checkpoint is 3 and the simulation time step is 6, if the change condition is satisfied, the neuron state switches to state 3;

[0057] When the checkpoint is 4 and the simulation time step is 12, if the change condition is satisfied, the neuron state switches to state 4;

[0058] When the checkpoint is 5 and the simulation time step is 18, if the change condition is satisfied, the neuron state switches to state 5;

[0059] When the checkpoint is 6 and the simulation time step is 50, if the change condition is satisfied, the neuron state switches to the ground state.

[0060] It should be noted that the above data and the firing functions corresponding to different neuron states are all obtained by fitting the test data of the self-sensitizing material. The firing functions of the neuron ground state and states 1-5 are as follows:

[0061] Ground state:

[0062] State 1:

[0063] State 2:

[0064] State 3:

[0065] State 4:

[0066] State 5:

[0067] Thus, the embodiments of the present application can effectively avoid the interference of noise data by switching the neuron state at different checkpoints and resetting the ground state of the neuron state at a certain checkpoint.

[0068] In step S103, in the preset spiking neural network model, a self-sensitizing neuron model is loaded, and the preset spiking neural network model is trained to generate an adaptive sensitizing spiking model, so as to obtain the classification result of each target image data by using the adaptive sensitizing spiking model.

[0069] After constructing the self-sensitized neuron model, further, the embodiments of the present application can also combine the self-sensitized neuron model and a preset spiking neural network model to construct an adaptive sensitized spiking model and train the model to improve its accuracy and real-time performance in classifying target images.

[0070] Optionally, in an embodiment of the present application, in the preset spiking neural network model, the self-sensitized neuron model is loaded and the preset spiking neural network model is trained, including: collecting the input data of the adaptive sensitized spiking model and preprocessing the input data to obtain the normalized data of the input data; within the simulation time, continuously inputting the normalized data into the adaptive sensitized spiking model and converting the normalized data into spike time data through a preset encoding function.

[0071] It should be noted that after the self-sensitized neuron model is constructed, the embodiments of the present application can load the adaptive sensitized neuron model in the spiking neural network model to construct an adaptive sensitized spiking model, where the spiking convolutional network model can be based on the Alex Net architecture and implemented using the PyTorch framework.

[0072] Specifically, after the input data is input into the adaptive sensitized spiking model, it is first necessary to perform a preprocessing operation on the input data to obtain the normalized data of the input data. The specific preprocessing process is as follows:

[0073] 1. Use the torchvision.transforms.Resize() function to redefine the size of the input data as 224x224;

[0074] 2. Through the torchvision.transforms.ToTensor() function, convert the data type of the input data into Tensor;

[0075] 3. According to the torchvision.transforms.Normalize() function, perform normalization processing on the input data, where the specific parameters of the function are mean = [0.485, 0.456, 0.406] and std = [0.229, 0.224, 0.2].

[0076] In the embodiments of the present application, the adaptive sensitized spiking model is mainly composed of two parts: a convolutional module and a fully connected module. The convolutional module shares the same convolutional kernel shape and number of feature channels as the Alex Net architecture and includes five convolutional layers, as Figure 2 shown:

[0077] 1. First Convolutional Layer (Conv1): 96 convolutional kernels of size 11×11, number of channels of the output feature map: 96, size of the output feature map: 54×54;

[0078] 2. Second Convolutional Layer (Conv2): 256 convolutional kernels of size 5×5, number of channels of the output feature map: 256, size of the output feature map: 26×26;

[0079] 3. Third Convolutional Layer (Conv3): 384 convolutional kernels of size 3×3, number of channels of the output feature map: 384, size of the output feature map: 12×12;

[0080] 4. Fourth Convolutional Layer (Conv4): 384 convolutional kernels of size 3×3, number of channels of the output feature map: 384, size of the output feature map: 12×12;

[0081] 5. Fifth Convolutional Layer (Conv5): 256 convolutional kernels of size 3×3, number of channels of the output feature map: 256, size of the output feature map: 12×12.

[0082] In addition, there is a normalization layer after each convolutional layer, which is used to preprocess the data input to the neuron layer. Pooling operations are also performed after the first two convolutional layers and the last convolutional layer, where the pooling window size is 3×3 (kernel_size = 3) and the stride is 2 (stride = 2); each convolutional layer extracts different features from the input image, gradually capturing higher-level abstract information; after completing the calculation of the convolutional module, the network generates a feature map with 512 channels, and the size of each channel is 5×5. Subsequently, before the obtained feature map enters the fully connected module, it can be flattened into a one-dimensional input by using the torch.view() function.

[0083] In the fully connected module, there are two linear calculation operation layers, two normalization layers and two neuron layers, as Figure 3 shown. Among them, the linear calculation layers (linear1 and linear1) are custom linear layers, which can be used to perform MAC (multiplication-accumulation) operations. Through the torch.matmul() function, the matrix multiplication calculation between the input and the built-in parameters and the weights of the linear layer is completed to realize the crossbar array of pulse electronic devices for linear transformation; the custom normalization layers (BatchNorm1 and BatchNorm2) are called TIA_Norm, which can normalize the output data to the range of -2.0 to 2.0V to ensure that the values are kept within a specific range. The specific normalization implementation process is as follows:

[0084] (1) Subtract the minimum value in the data from the Tensor (obtained using torch.min());

[0085] (2) Divide by the maximum value in the Tensor (obtained using torch.max()), and remap the Tensor within the range [0, 1];

[0086] (3) Multiply the Tensor by an appropriate scaling factor to obtain an appropriate neuron firing frequency;

[0087] (4) Use the torch.clamp_() function to limit the Tensor within the appropriate range [-2, +2].

[0088] In addition, the neuron layer in the fully connected module is the self-sensitizing neuron model constructed above. Finally, the number of neurons in the specific output layer can be determined according to the specific classification task.

[0089] During the training process of the adaptive sensitization pulse model, the embodiments of the present application can convert the preprocessed standardized data into spike time data through the Poisson_encoder() function, and continuously input it in the simulated time step loop (the standardized data is re-converted after a single calculation). The process of converting the standardized data into spike time data is as follows:

[0090] (1) Calculate and output through the torch.rand_like(x).le(x).float() function to generate a tensor with the same shape as the training data tensor x, and each element therein can be compared with the corresponding element in the input tensor x according to a randomly generated value;

[0091] (2) Represent the comparison result as a floating-point number of 0.0 or 1.0, and the larger the element value, the greater the probability that the corresponding element in the output tensor outputs 1.0.

[0092] It should be noted that although the parameters of the convolutional layer in the Alex net network model are used in the embodiments of the present application, in the actual execution process, those skilled in the art can also use other convolutional network models, such as VGGNet, LeNet-5, etc., and the same technical effects can be achieved, without affecting the superiority of the performance of the self-sensitizing neuron model.

[0093] Thus, the embodiments of the present application preprocess the input data and convert the standardized data into spike time data, improving the quality of the data and greatly improving the learning and training efficiency of the model.

[0094] Optionally, in an embodiment of the present application, training a preset spiking neural network model to generate an adaptive sensitization spiking model includes: based on the spike time data and the adaptive sensitization spiking model, obtaining and saving the spike / spike data of the output layer neurons of the adaptive sensitization spiking model; calculating the firing frequency of the output layer neurons according to the spike / spike data and the simulation time; determining the loss function of the adaptive sensitization spiking model according to the firing frequency, and optimizing the adaptive sensitization spiking model through a preset optimization algorithm and the loss function.

[0095] After converting the preprocessed normalized data into spike time data, an embodiment of the present application can calculate the spike time data to obtain and save the output of the adaptive sensitization spiking model. At the same time, variables are set to accumulate the results of each simulation time step, obtaining the total number of firing spikes of the output layer neurons in the simulation time step calculation, and dividing the total number of firing spikes by the simulation time step to obtain the firing frequency of the output layer neurons.

[0096] Furthermore, an embodiment of the present application can define the difference between the label of the input data and the firing frequency of the output layer neurons in a preset training set as the loss function of the adaptive sensitization spiking model, and optimize this loss function through an optimization algorithm in the PyTorch framework. For example, the Adam optimization algorithm can be selected, and the training period is set to 10 and the simulation time step is set to 100 to optimize the above loss function. In the specific implementation process, those skilled in the art can also select other optimization algorithms to optimize the above loss function, which is not specifically limited herein.

[0097] It can be understood that in the adaptive sensitization spiking model based on material characteristics in the embodiments of the present application, through the training of specific scenario data, an accurate understanding of relevant changing scenarios is achieved, effectively solving the problem of data drift in traditional models, that is, the problem of performance degradation of the model when the input distribution changes, reducing the algorithm complexity of the model, and improving the generalization ability of the model.

[0098] The following will analyze the execution effect of the construction method of the adaptive sensitization spiking model of the present application in conjunction with the accompanying drawings.

[0099] The present application conducts effect tests using a general dataset and a self-built dataset respectively (the general dataset is applied to Fashion MNIST; the self-built dataset is applied to vehicle classification in a changing scenario).

[0100] In Fashion MNIST, the present application can simulate different lighting conditions by adjusting the brightness of the input pictures. Specifically, after reading the grayscale values of the pictures, the present application can multiply all values by an intensity factor in the range of [0.2, 1.0]. Figure 4Schematic diagram of the effect of using the standard general Fashion MNIST dataset, as Figure 4 shown, it can be seen that with the help of the adaptive sensitization neuron, the spiking neural network has a more accurate understanding of the changing input and accurately completes the image classification task;

[0101] Furthermore, the present application can build a self-built vehicle dataset through the AI painting model Stable Diffusion. The self-built vehicles include four types: sedans, sport utility vehicles (SUVs), light trucks (pickups), and airplanes. This self-built dataset contains a total of 3,400 images, with 1,700 images for each vehicle type under bright and dark conditions. Specifically, there are 500 images each for sedans, sport utility vehicles (SUVs), and light trucks (pickups), and 200 images for airplanes. The above images are all three-channel color images with a size of 512x512 pixels to simulate the working conditions in the real world. Figure 5 Schematic diagram of the effect of using the self-built vehicles in a changing environment for vehicle classification, as Figure 5 shown, the adaptive sensitization spiking model can autonomously reconstruct feature saliency, distinguish the differences between the main body and the environment in a changing environment, and thus autonomously focus on the classification main body during the classification process. It directly breaks through the limitation that traditional networks cannot adapt to environmental changes in a simple way, and there is a significant improvement in the classification accuracy. It always maintains an accuracy of 90% in a changing environment, greatly improving the detection and classification performance.

[0102] According to the method for constructing an adaptive sensitization spiking model applied to the offline training stage proposed in the embodiment of the present application, by using the neuron firing function based on a preset self-sensitization neuron, record the simulation time and firing history data of the self-sensitization neuron; according to the simulation time and firing history data, adjust the neuron state of the self-sensitization neuron to construct a self-sensitization neuron model; in a preset spiking neural network model, load the self-sensitization neuron model and train the preset spiking neural network model to generate an adaptive sensitization spiking model, so as to obtain the classification result of each target image data by using the adaptive sensitization spiking model. The present application effectively solves the data drift problem through the new adaptive sensitization spiking neural network model without consuming a large amount of computing power for data training, and greatly improves the learning and training efficiency of the model.

[0103] Figure 6 Flowchart of a method for constructing an adaptive sensitization spiking model applied to the online detection stage provided by an embodiment of the present application.

[0104] As Figure 6 shown, the method for constructing the adaptive sensitization spiking model includes the following steps:

[0105] In step S601, at least one target image data in the current environment is collected.

[0106] In step S602, image size transformation, data type conversion, and normalization operations are performed on at least one target image data to construct a classification dataset for the adaptive sensitization pulse model.

[0107] In step S603, the classification data in the classification dataset is input into the pre-trained adaptive sensitization pulse model to generate the classification result of each target image data.

[0108] According to the method for constructing an adaptive sensitization pulse model applied to the online detection stage proposed in the embodiments of the present application, at least one target image data in the current environment is collected; image size transformation, data type conversion, and normalization operations are performed on at least one target image data to construct a classification dataset for the adaptive sensitization pulse model; the classification data in the classification dataset is input into the pre-trained adaptive sensitization pulse model to generate the classification result of each target image data. Through the novel adaptive sensitization pulse neural network model, the present application can maintain good classification accuracy for the same classification task and different test input distribution scenarios, and does not require a large amount of computing power for data training, effectively solving the data drift problem and greatly improving the learning and training efficiency of the model.

[0109] Secondly, an apparatus for constructing an adaptive sensitization pulse model according to an embodiment of the present application is described with reference to the accompanying drawings.

[0110] Figure 7 is a block diagram of an apparatus for constructing an adaptive sensitization pulse model applied to the offline training stage according to an embodiment of the present application.

[0111] As Figure 7 shown, the apparatus 10 for constructing an adaptive sensitization pulse model applied to the offline training stage includes: a recording module 101, an adjustment module 102, and a training module 103.

[0112] Among them, the recording module 101 is configured to record the simulation time and firing history data of the self-sensitizing neuron based on the neuron firing function of the preset self-sensitizing neuron.

[0113] The adjustment module 102 is configured to adjust the neuron state of the self-sensitizing neuron according to the simulation time and firing history data to construct a self-sensitizing neuron model.

[0114] The training module 103 is configured to load the self-sensitizing neuron model in the preset pulse neural network model and train the preset pulse neural network model to generate an adaptive sensitization pulse model, so as to obtain the classification result of each target image data by using the adaptive sensitization pulse model.

[0115] Optionally, in an embodiment of the present application, the adjustment module 102 includes: an update unit and a determination unit.

[0116] Among them, the update unit is configured to obtain the input stimulus data of the neuron excitation function, and update the neuron potential corresponding to the input stimulus data according to the simulation time to obtain the latest neuron potential.

[0117] The determination unit is configured to input the latest neuron potential into the neuron excitation function to determine whether the self-sensitized neuron is excited.

[0118] Optionally, in an embodiment of the present application, the training module 103 includes: a generation unit and a conversion unit.

[0119] Among them, the generation unit is configured to collect the input data of the adaptive sensitization pulse model, and preprocess the input data to obtain the normalized data of the input data.

[0120] The conversion unit is configured to continuously input the normalized data into the adaptive sensitization pulse model within the simulation time, and convert the normalized data into pulse time data through a preset coding function.

[0121] Optionally, in an embodiment of the present application, the training module 103 further includes: a saving unit, a calculation unit, and an optimization unit.

[0122] Among them, the saving unit is configured to obtain and save the pulse / spike data of the output layer neurons of the adaptive sensitization pulse model based on the pulse time data and the adaptive sensitization pulse model.

[0123] The calculation unit is configured to calculate the firing frequency of the output layer neurons according to the pulse / spike data and the simulation time.

[0124] The optimization unit is configured to determine the loss function of the adaptive sensitization pulse model according to the firing frequency, and optimize the adaptive sensitization pulse model through a preset optimization algorithm and the loss function.

[0125] An adaptive sensitization pulse model construction device applied to the offline training stage according to an embodiment of the present application includes a recording module, configured to record the simulation time and firing history data of a self-sensitizing neuron based on the neuron firing function of a preset self-sensitizing neuron; an adjustment module, configured to adjust the neuron state of the self-sensitizing neuron according to the simulation time and firing history data to construct a self-sensitizing neuron model; and a training module, configured to load the self-sensitizing neuron model into a preset spiking neural network model and train the preset spiking neural network model to generate an adaptive sensitization pulse model, so as to obtain the classification result of each target image data by using the adaptive sensitization pulse model. Through the novel adaptive sensitization spiking neural network model, the present application does not need to consume a large amount of computing power for data training, effectively solves the data drift problem, and greatly improves the learning and training efficiency of the model.

[0126] Figure 8 FIG. is a schematic block diagram of an adaptive sensitization pulse model construction device applied to the online detection stage according to an embodiment of the present application.

[0127] As Figure 8 shown, the adaptive sensitization pulse model construction device 20 applied to the online detection stage includes: a collection module 201, a preprocessing module 202, and a classification module 203.

[0128] Among them, the collection module 201 is configured to collect at least one target image data in the current environment.

[0129] The preprocessing module 202 is configured to perform operations such as image size transformation, data type conversion, and normalization on at least one target image data to construct a classification data set for the adaptive sensitization pulse model.

[0130] The classification module 203 is configured to input the classification data in the classification data set into a pre-trained adaptive sensitization pulse model to generate the classification result of each target image data.

[0131] It should be noted that the foregoing explanation of the embodiment of the adaptive sensitization pulse model construction method is also applicable to the adaptive sensitization pulse model construction device of this embodiment, and will not be elaborated here.

[0132] An adaptive sensitized pulse model construction device applied to the online detection stage according to an embodiment of the present application includes an acquisition module for acquiring at least one target image data in the current environment; a preprocessing module for performing image size transformation, data type conversion, and normalization operations on the at least one target image data to construct a classification data set for the adaptive sensitized pulse model; and a classification module for inputting the classification data in the classification data set into a pre-trained adaptive sensitized pulse model to generate a classification result for each target image data. Through the novel adaptive sensitized pulse neural network model, the present application does not require a large amount of computing power for data training, effectively solves the data drift problem, and greatly improves the learning and training efficiency of the model.

[0133] Figure 9 A schematic structural diagram of an electronic device provided by an embodiment of the present application. The electronic device may include:

[0134] A memory 901, a processor 902, and a computer program stored on the memory 901 and executable on the processor 902.

[0135] When the processor 902 executes the program, it implements the adaptive sensitized pulse model construction method provided in the above embodiment.

[0136] Further, the electronic device further includes:

[0137] A communication interface 903 for communication between the memory 901 and the processor 902.

[0138] The memory 901 is used to store a computer program executable on the processor 902.

[0139] The memory 901 may include a high-speed RAM memory, and may also include a non-volatile memory, such as at least one disk memory.

[0140] If the memory 901, the processor 902, and the communication interface 903 are implemented independently, the communication interface 903, the memory 901, and the processor 902 may be interconnected through a bus and communicate with each other. The bus may be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. The bus may be divided into an address bus, a data bus, a control bus, etc. For the sake of convenience of representation, Figure 9It is represented only by a thick line, but it does not mean that there is only one bus or one type of bus.

[0141] Optionally, in a specific implementation, if the memory 901, the processor 902, and the communication interface 903 are integrated on a single chip, the memory 901, the processor 902, and the communication interface 903 can communicate with each other through an internal interface.

[0142] The processor 902 may be a central processing unit (CPU for short), or an application specific integrated circuit (ASIC for short), or one or more integrated circuits configured to implement the embodiments of the present application.

[0143] The embodiments of the present application further provide a computer-readable storage medium, on which a computer program is stored, and when the program is executed by a processor, the above-mentioned method for constructing an adaptive sensitized pulse model is implemented.

[0144] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples", etc. means that the specific features, structures, materials, or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the present application. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described can be combined in any one or N embodiments or examples in a suitable manner. In addition, without contradiction, those skilled in the art can combine and combine the different embodiments or examples described in this specification and the features of different embodiments or examples.

[0145] In addition, the terms "first" and "second" are only used for descriptive purposes, and cannot be understood as indicating or implying relative importance or implicitly specifying the quantity of the indicated technical features. Thus, the features defined with "first" and "second" may explicitly or implicitly include at least one of the features. In the description of the present application, the meaning of "N" is at least two, such as two, three, etc., unless otherwise specifically defined.

[0146] Any process or method description represented in a flowchart or otherwise described herein can be understood to represent a module, segment, or portion of code including one or N executable instructions for implementing a customized logical function or process. The scope of the preferred embodiments of this application includes additional implementations, where functions can be executed in a manner that is not shown or discussed, including in a substantially simultaneous manner according to the functions involved or in a reverse order, which should be understood by those skilled in the technical field to which the embodiments of this application pertain.

[0147] Logic and / or steps represented in a flowchart or otherwise described herein, for example, can be considered a sequenced list of executable instructions for implementing a logical function, and can be specifically implemented in any computer-readable medium for use by an instruction execution system, apparatus, or device (such as a computer-based system, a system including a processor, or other systems that can fetch and execute instructions from the instruction execution system, apparatus, or device), or in conjunction with these instruction execution systems, apparatus, or devices. For the purposes of this specification, a "computer-readable medium" can be any device that can contain, store, communicate, propagate, or transport a program for use by or in conjunction with an instruction execution system, apparatus, or device. More specific examples (non-exhaustive list) of computer-readable media include the following: an electrical connection portion with one or N wirings (electronic device), a portable computer diskette (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disc read-only memory (CDROM). Additionally, a computer-readable medium can even be paper or other suitable media on which the program can be printed, because the program can be obtained electronically by optically scanning the paper or other media, followed by editing, interpretation, or otherwise processing as appropriate, and then stored in a computer memory.

[0148] It should be understood that various parts of this application can be implemented using hardware, software, firmware, or a combination thereof. In the above embodiments, the N steps or methods can be implemented using software or firmware stored in a memory and executed by a suitable instruction execution system. If implemented using hardware, as in another embodiment, any one or a combination of the following techniques well known in the art can be used: discrete logic circuits having logic gate circuits for implementing logical functions on data signals, application-specific integrated circuits having appropriate combinational logic gate circuits, programmable gate arrays (PGAs), field programmable gate arrays (FPGAs), etc.

[0149] Those of ordinary skill in the art can understand that all or part of the steps carried out in the method of the above embodiments can be completed by instructing relevant hardware through a program. The program can be stored in a computer-readable storage medium. When the program is executed, it includes one or a combination of the steps of the method embodiments.

[0150] In addition, in each of the embodiments of the present application, the functional units can be integrated into one processing module, or each unit can exist physically alone, or two or more units can be integrated into one module. The above integrated module can be implemented in the form of hardware or in the form of a software functional module. When the above integrated module is implemented in the form of a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium.

[0151] The storage medium mentioned above can be a read-only memory, a magnetic disk, an optical disc, etc. Although the embodiments of the present application have been shown and described above, it can be understood that the above embodiments are exemplary and should not be construed as limiting the present application. Those of ordinary skill in the art can make changes, modifications, substitutions, and variations to the above embodiments within the scope of the present application.

Claims

1. An adaptive sensitization pulse model construction method, characterized in that Applied to the offline training stage, including the following steps: Based on the neuron firing function of the preset self-sensitizing neuron, record the simulation time and firing history data of the self-sensitizing neuron; According to the simulation time and the firing history data, adjust the neuron state of the self-sensitizing neuron to construct a self-sensitizing neuron model; In the preset spiking neural network model, load the self-sensitizing neuron model and train the preset spiking neural network model to generate an adaptive sensitizing pulse model, so as to obtain the classification result of each target image data by using the adaptive sensitizing pulse model.

2. The method according to claim 1, wherein The adjusting the neuron state of the self-sensitizing neuron according to the simulation time and the firing history data to construct a self-sensitizing neuron model includes: Obtain the input stimulus data of the neuron firing function, and update the neuron potential corresponding to the input stimulus data according to the simulation time to obtain the latest neuron potential; Input the latest neuron potential into the neuron firing function to determine whether the self-sensitizing neuron fires.

3. The method according to claim 1, characterized in that, The loading the self-sensitizing neuron model in the preset spiking neural network model and training the preset spiking neural network model includes: Collect the input data of the adaptive sensitizing pulse model, and preprocess the input data to obtain the normalized data of the input data; Within the simulation time, continuously input the normalized data into the adaptive sensitizing pulse model, and convert the normalized data into pulse time data through a preset encoding function.

4. The method according to claim 3, characterized in that, The training the preset spiking neural network model to generate an adaptive sensitizing pulse model includes: Based on the pulse time data and the adaptive sensitizing pulse model, obtain and save the pulse / spike data of the output layer neurons of the adaptive sensitizing pulse model; According to the pulse / spike data and the simulation time, calculate the firing frequency of the output layer neurons; According to the firing frequency, determine the loss function of the adaptive sensitizing pulse model, and optimize the adaptive sensitizing pulse model through a preset optimization algorithm and the loss function.

5. An adaptive sensitization pulse model construction method, characterized in that Applied to the online detection stage, including the following steps: Collect at least one target image data in the current environment; Perform image size transformation, data type conversion and normalization operations on the at least one target image data to construct a classification data set of the adaptive sensitizing pulse model; Input the classification data in the classification data set into the pre-trained adaptive sensitizing pulse model to generate the classification result of each target image data.

6. An adaptive sensitization pulse model construction device, characterized in that, Applied to the offline training stage, including the following steps: A recording module, configured to record the simulation time and firing history data of the self-sensitizing neuron based on the neuron firing function of the preset self-sensitizing neuron; An adjusting module, configured to adjust the neuron state of the self-sensitizing neuron according to the simulation time and the firing history data to construct a self-sensitizing neuron model; A training module, which is used to load the self-sensitizing neuron model in a preset spiking neural network model and train the preset spiking neural network model to generate an adaptive sensitizing spiking model, so as to obtain the classification result of each target image data by using the adaptive sensitizing spiking model.

7. The device according to claim 6, characterized in that, The adjustment module includes: An update unit, which is used to obtain the input stimulus data of the neuron activation function and update the neuron potential corresponding to the input stimulus data according to the simulation time to obtain the latest neuron potential; A determination unit, which is used to input the latest neuron potential into the neuron activation function to determine whether the self-sensitizing neuron is activated.

8. A method for constructing an adaptive sensitization pulse model, characterized in that, Applied to the online detection stage, it includes: An acquisition module, which is used to acquire at least one target image data in the current environment; A preprocessing module, which is used to perform image size transformation, data type conversion and normalization operations on the at least one target image data to construct a classification data set of the adaptive sensitizing spiking model; A classification module, which is used to input the classification data in the classification data set into a pre-trained adaptive sensitizing spiking model to generate the classification result of each target image data.

9. An electronic device, characterized in that, It includes: A memory, a processor, and a computer program stored on the memory and executable on the processor. The processor executes the program to implement the method for constructing an adaptive sensitizing spiking model according to any one of claims 1-4 or claim 5.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, The program is executed by the processor to implement the method for constructing an adaptive sensitizing spiking model according to any one of claims 1-4 or claim 5.

Images