Data processing method, device, apparatus and computer storage medium

Abstract

The present application discloses a data processing method, device, apparatus, and computer storage medium. The data processing method disclosed in the present application includes: processing an image to be processed to obtain first data; wherein the first data represents pulse data corresponding to the image to be processed; performing feature extraction and pooling operations on the first data through multiple pulse neurons carrying synaptic weight information in at least one feature extraction layer to obtain first feature data; processing the first feature data to obtain second data; wherein the second data represents the type of object in the image to be processed. The data processing method provided in the present application can improve the accuracy of object recognition in the image to be processed, and can also overcome the disadvantage of convolutional neural networks lacking rotation invariance.

Description

Technical Field

[0001] The present application relates to the field of information technology, and in particular to a data processing method, device, apparatus, and computer-readable storage medium. Background Art

[0002] In practical applications, in order to identify the type of target object, such as the type of vehicle, ultrasonic equipment or laser infrared equipment is usually used.

[0003] When using laser infrared devices for vehicle identification, arrays of these devices are typically placed on both sides of the lane to detect vehicle wheel count, axle count, front height, and side profile, enabling vehicle identification and classification. Ultrasonic and laser infrared devices offer advantages for target identification, such as vehicle type recognition, due to their high directivity, concentrated energy, strong transmission capability, and high recognition efficiency. However, their disadvantages are also obvious: they carry high maintenance costs.

[0004] To reduce the cost of vehicle type recognition using ultrasonic or laser infrared devices, related technologies often employ convolutional neural networks (CNNs) or backpropagation (BP) algorithms for vehicle type recognition. However, BP algorithms suffer from low recognition accuracy, require a large number of parameters to adjust, have a slow convergence rate, and require long training times. CNNs, on the other hand, require no significant improvement in the number of parameters to adjust, and fail to consider the spatial hierarchy between the various features of a target object, such as a vehicle, resulting in insufficient ability to extract rotational invariance. Summary of the Invention

[0005] The present application provides a data processing method, device, apparatus, and computer-readable storage medium.

[0006] The data processing method provided in this application can solve the problem of low recognition accuracy of the BP algorithm in the related art, and can also overcome the shortcoming of CNN's lack of rotation invariance.

[0007] The technical solution provided by this application is as follows:

[0008] The present application provides a data processing method, the method comprising:

[0009] Processing the image to be processed to obtain first data; wherein the first data represents pulse data corresponding to the image to be processed;

[0010] Performing feature extraction and pooling processing on the first data using a plurality of spiking neurons carrying synaptic weight information in at least one feature extraction layer to obtain first feature data;

[0011] The first feature data is processed to obtain second data; wherein the second data represents the type of the object in the image to be processed.

[0012] In some embodiments, each of the feature extraction layers includes a pulse data processing layer and a pooling layer; and the performing feature extraction and pooling operations on the first data includes:

[0013] The pulse data processing layer performs feature extraction on the input data of each feature extraction layer based on the synaptic weight information to obtain the second feature data; wherein the pulse data processing layer includes a plurality of the pulse neurons;

[0014] The pooling operation is performed on the second feature data through the plurality of pulse neurons in the pooling layer.

[0015] In some embodiments, the steps of processing the image to be processed to obtain first data, performing feature extraction and pooling operations on the first data through multiple pulse neurons carrying synaptic weight information in at least one feature extraction layer to obtain first feature data, and processing the first feature data to obtain second data are implemented by a neural network; the neural network is trained by image sample data.

[0016] In some embodiments, the neural network is trained using image sample data, including:

[0017] Processing the image sample data to obtain third data; wherein the third data represents pulse data corresponding to the image sample data;

[0018] Processing the third data through the at least one feature extraction layer to obtain third feature data; wherein the third feature data is used to represent the feature data output by the last feature extraction layer;

[0019] Processing the third feature data to obtain fourth data; wherein the fourth data represents a type corresponding to any one of the image samples;

[0020] Based on the third feature data and the fourth data, parameter information of the neural network is adjusted; wherein the parameter information at least includes the synaptic weight information.

[0021] In some embodiments, adjusting the parameter information of the neural network based on the third feature data and the fourth data includes:

[0022] Acquire a first rule; wherein the first rule at least includes a rule for adjusting the synaptic weight information;

[0023] The parameter information is adjusted based on the first rule, the third feature data, and the fourth data.

[0024] In some embodiments, the parameter information further includes parameters of the pooling operation.

[0025] In some embodiments, processing the first feature data to obtain second data includes:

[0026] Obtaining a second rule; wherein the second rule includes a rule for performing statistics on the first feature data;

[0027] Based on the second rule, statistics are collected on each characteristic value in the first characteristic data to obtain the second data.

[0028] The present application also provides a data processing device, the device comprising a processor and a memory; wherein:

[0029] The memory is used to store calculations and programs that can be run on the processor;

[0030] The processor is configured to execute any of the above-mentioned data processing methods when running the computer program.

[0031] The present application also provides a data processing device, which includes: an acquisition module and a processing module; wherein:

[0032] The acquisition module is configured to process the image to be processed to obtain first data; wherein the first data represents pulse data corresponding to the image to be processed;

[0033] The processing module is configured to perform feature extraction and pooling processing on the first data using a plurality of spiking neurons carrying synaptic weight information in at least one feature extraction layer to obtain first feature data;

[0034] The processing module is further configured to process the first feature data to obtain second data; wherein the second data represents the type of the object in the image to be processed.

[0035] The present application also provides a computer-readable storage medium, which stores one or more programs. The one or more programs can be executed by one or more processors to implement the data processing method as described in any one of the above.

[0036] In the data processing method provided by the present application, each feature extraction layer in at least one feature extraction layer can perform feature extraction and pooling operations on the input data through multiple pulse neurons carrying synaptic weight information. As a result, multiple pulse neurons carrying synaptic weight information can efficiently and accurately identify the feature information in the image to be processed, which can overcome the disadvantage of CNN's lack of rotation invariance. On the other hand, through the pooling operation, it can also reduce the redundant information in the output data of each feature extraction layer. Therefore, each feature extraction layer in the data processing method provided by the present application can achieve efficient feature extraction of the input data. In this way, the first feature data obtained by feature extraction in sequence by at least one feature extraction layer is processed, and the second data representing the type of object in the image to be processed can more flexibly and accurately reflect the type information of the image to be processed.

[0037] From the above, it can be seen that in the data processing method provided by the present application, the image to be processed is processed to obtain pulse data corresponding to the image to be processed, that is, the first data carries sufficient effective information in the image to be processed, and when the conditions for generating the first data are determined, the amount of calculation generated by processing each pixel of the image to be processed in the related technology can be reduced.

[0038] In the data processing method provided in the present application, the pulse data processing layer and the pooling layer can be set continuously and adjacently in an interlaced manner. As a result, the receptive field of each feature extraction layer will become larger as the number of feature extraction layers increases, and the selectivity of subsequent feature extraction layers for complex features will gradually increase.

[0039] The data processing method provided in the present application adopts a spiking neural network (Spiking Neuron Networks, SNN) that carries synaptic weight information as a spiking data processing layer of a feature extraction layer, which can realize accurate feature extraction of input spiking data and overcome the shortcoming of the CNN network that lacks rotation invariance; a pooling layer is adopted to add a pooling operation to the feature extraction layer, thereby realizing redundancy removal of the spiking data processing layer, thereby greatly reducing the redundancy of the data processed in each feature extraction layer; on this basis, according to the second rule, the output first feature data is statistically analyzed to obtain second data representing the type of the image to be processed, which can not only improve the shortcoming of the CNN that lacks rotation invariance, but also improve the accuracy of type recognition of objects in the image to be processed. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 A flowchart of the first data processing method provided in this application;

[0041] Figure 2 A flowchart of the second data processing method provided in this application;

[0042] Figure 3 A flowchart of the neural network training process provided for this application;

[0043] Figure 4 This is an example diagram of some data in the image sample data in the data processing method provided by this application;

[0044] Figure 5 A diagram of the neural network structure provided for this application;

[0045] Figure 6 A structural diagram of a data processing device provided in this application;

[0046] Figure 7 This is a structural diagram of a data processing device provided in this application. DETAILED DESCRIPTION

[0047] The technical solutions in the embodiments of the present application will be described clearly and completely below in conjunction with the drawings in the embodiments of the present application.

[0048] It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0049] The data processing method provided in this application relates to the field of information technology, and in particular to a data processing method, device, apparatus, and computer-readable storage medium.

[0050] SNNs are considered the third generation of artificial neural networks. The neurons in SNNs, known as spiking neurons, are not activated every time they receive a pulse signal, but only when their membrane potential reaches a certain threshold. Only when a spiking neuron is activated does it generate a pulse signal and transmit it to the synapse. Once the synapse receives the pulse signal and its membrane potential exceeds the membrane potential threshold, a pulse signal can be sent to the next neuron. Therefore, the most significant characteristic of SNNs is the discontinuous nature of information transmission, which is the biggest difference from traditional artificial neural networks. Furthermore, the discontinuous activation of spiking neurons in SNNs also gives SNNs excellent sparsity.

[0051] In practical applications, there is no need to set up a large number of complex connections between the various pulse neurons of SNN. During the SNN training process, with the continuous input of sample data, the connections between the pulse neurons will be established automatically, and the connection strength between neurons will also be adaptively adjusted, so the SNN training process is also simpler.

[0052] When using SNNs for target recognition, such as vehicle recognition, the characteristic information of the target object is converted into pulses and then input into spiking neurons. Each spiking neuron in the SNN will only send a pulse signal if the input pulse strength is sufficient. Therefore, the SNN's recognition results for the target features are more closely aligned with the actual characteristics of the target object. Furthermore, in practical applications, the SNN employs an unsupervised learning mechanism, enabling each spiking neuron in the SNN to adaptively adjust its synaptic weight parameters based on the characteristics of various types of sample data, thereby enabling the SNN to overcome the CNN's lack of rotational invariance.

[0053] The data processing method provided in the embodiment of the present application is based on the idea of ​​deep learning and organically combines CNN and SNN, that is, the pulse neurons in SNN replace each deep learning computing unit in CNN. Therefore, the data processing method provided in the embodiment of the present application can not only reduce the number of parameters that need to be adjusted during the CNN training process, but also overcome the disadvantage of CNN's lack of rotational invariance.

[0054] In the data processing method provided in the embodiment of the present application, for SNN, a spike timing dependent synaptic plasticity (STDP) model is established based on the pre- and post-synaptic trajectory learning rules, and is optimized using a deep learning algorithm. According to the sample data set, the synaptic weight information of the spiking neurons with connection relationships is trained and adjusted, thereby reducing the energy consumption in the CNN deep learning process.

[0055] The data processing method provided in the embodiment of the present application can be implemented by a processor of a data processing device.

[0056] like Figure 1 As shown, the data processing method provided in the embodiment of the present application can be implemented by the following steps:

[0057] Step 101: Process the image to be processed to obtain first data.

[0058] The first data represents pulse data corresponding to the image to be processed.

[0059] In one embodiment, the image to be processed may be an image containing the target object captured in real time by an image capture device.

[0060] In one embodiment, the image to be processed may be an image containing a target object obtained through a network or other means.

[0061] In one embodiment, the image to be processed may be an image carrying at least one specific target object.

[0062] In one embodiment, the number of images to be processed may be multiple.

[0063] In one embodiment, the first data may be pulse data corresponding to a first region of the image to be processed, wherein the first region may be used to represent any region in the image to be processed.

[0064] In one embodiment, the first data may be pulse data corresponding to a first feature of the image to be processed, wherein the first feature may be used to represent a feature of an edge region in the image to be processed.

[0065] In one embodiment, the first data may be pulse data corresponding to a second feature of the image to be processed, wherein the second feature may be used to represent a pixel feature in the image to be processed.

[0066] For example, a second feature in the image to be processed can be converted into first data according to a first algorithm. Specifically, the pixel value of each point in the image to be processed can be obtained to obtain a first pixel value, and the first pixel value can be processed according to the first algorithm to obtain first data corresponding to the intensity of the first pixel value. The first algorithm can be a pulse coding rule.

[0067] Exemplarily, the first data may represent a single pulse having an intensity corresponding to the first pixel value.

[0068] Exemplarily, the first pixel values ​​in the image to be processed that meet the first condition are converted using the first algorithm, and a pulse sequence corresponding to each first pixel value can be obtained.

[0069] Accordingly, the first data may represent a plurality of pulses having intensities corresponding to the first pixel value, ie, a pulse sequence.

[0070] In one embodiment, the first data may be pulse data corresponding to a first pixel value that satisfies a first condition. The first condition may indicate that the first pixel value is above a specified threshold. In this manner, pixels with lower pixel values ​​may not be processed, thereby reducing the amount of first data.

[0071] In one embodiment, the frequency of occurrence of the first data may represent a changing trend of pixel values ​​in the image to be processed. When the first data appears relatively densely, it may indicate that there are more pixels with high pixel values ​​in the image to be processed, or that the pixels with relatively high pixel values ​​are relatively dense. Conversely, when the first data appears relatively sparsely, it may indicate that there are fewer pixels with relatively high pixel values ​​in the image to be processed, or that the distribution of pixels with relatively high pixel values ​​is relatively sparse.

[0072] Step 102: Perform feature extraction and pooling operations on the first data using a plurality of spiking neurons carrying synaptic weight information in at least one feature extraction layer to obtain first feature data.

[0073] In one embodiment, a synapse can be used to represent a unit that connects spiking neurons in an SNN.

[0074] In one embodiment, the connection between spiking neurons may be unidirectional, and two spiking neurons may be connected via only one synapse.

[0075] In one embodiment, synaptic weight information can be used to indicate the nature of a synapse between connected spiking neurons, for example, whether the synapse is inhibitory or excitatory. For excitatory synapses, the corresponding synaptic weight information may be a positive number; for inhibitory synapses, the corresponding synaptic weight information may not be a positive number.

[0076] In one embodiment, synaptic weight information may be used to represent the weight of a synapse between connected spiking neurons when processing a received pulse. For example, if the amplitude of a pulse output by a synapse to a presynaptic neuron is increased or decreased by N times, the synaptic weight information may be N, where N is a number not equal to 0.

[0077] In one embodiment, the synaptic weight information is determined based on the SNN structure and the STDP model.

[0078] For example, the STDP model is used to adjust the strength of the connection relationship between spiking neurons according to the order in which the spiking neurons learn. If the time t at which the Kth spiking neuron outputs a pulse is k Later than the time t when the K-1th pulse neuron outputs a pulse k-1 , then the connection between the K-1th pulse neuron and the Kth pulse neuron will be strengthened, that is, the synaptic weight information of the K-1th pulse will be strengthened; conversely, if the time t at which the Kth pulse neuron outputs a pulse k The time t before the K-1th pulse neuron outputs a pulse k-1, then the connection between the K-1th spiking neuron and the Kth spiking neuron will be weakened, that is, the synaptic weight information of the K-1th pulse will be weakened. Wherein, K is an integer greater than 1.

[0079] In one embodiment, the plurality of spiking neurons carrying synaptic weight information may be spiking neurons in an SNN.

[0080] In one embodiment, the plurality of spiking neurons carrying synaptic weight information may be spiking neurons in a certain layer when the SNN structure is determined.

[0081] In one embodiment, the first feature data may be the result of the last feature extraction layer processing its input data.

[0082] In one embodiment, by performing a pooling operation, redundant information may be removed from the second feature data.

[0083] In one embodiment, when the number of feature extraction layers is greater than or equal to two, at least one feature extraction layer may be adjacently arranged.

[0084] Accordingly, in two adjacent feature extraction layers, the P spiking neurons in the Mth feature extraction layer that carry synaptic weight information can be connected to the L spiking neurons in the M+1th feature extraction layer that carry synaptic weight information via synapses. P and L can be integers greater than 1, can be different, and the P spiking neurons in the Mth feature extraction layer do not need to establish a connection relationship with each spiking neuron in the M+1th feature extraction layer.

[0085] Step 103: Process the first feature data to obtain second data.

[0086] The second data indicates the type of the object in the image to be processed.

[0087] In one embodiment, the second data may be used to indicate the type of the target object in the image to be processed.

[0088] In one embodiment, the second data may be used to indicate the type of a specific target object in the image to be processed.

[0089] In one embodiment, the second data may be used to indicate the type of a specific component of the target object in the image to be processed, such as the type of the wheel of the target object.

[0090] In one embodiment, the output data of the last feature extraction layer may be processed to obtain second data.

[0091] It should be noted that the above-mentioned processor can be at least one of an application-specific integrated circuit (ASIC), a digital signal processor (DSP), a digital signal processing device (DSPD), a programmable logic device (PLD), a field programmable gate array (FPGA), a central processing unit (CPU), a controller, a microcontroller, and a microprocessor.

[0092] In the data processing method provided in the embodiment of the present application, each feature extraction layer in at least one feature extraction layer can extract features from the input data through multiple pulse neurons carrying synaptic weight information. Therefore, the data processing method provided in the embodiment of the present application can not only efficiently and accurately identify the feature information in the image to be processed, but also overcome the disadvantage of CNN's lack of rotation invariance. On the other hand, through the pooling operation, it can also reduce the redundant information in the output data of each feature extraction layer. Therefore, each feature extraction layer in the data processing method provided in the embodiment of the present application can achieve efficient feature extraction of the input data. In this way, the first feature data obtained by feature extraction in sequence by at least one feature extraction layer is processed, and the second data representing the type of the object in the image to be processed is obtained, which can more flexibly and accurately reflect the type information of the object in the image to be processed.

[0093] From the above, it can be seen that in the data processing method provided in the embodiment of the present application, the image to be processed is processed to obtain pulse data corresponding to the image to be processed, that is, the first data carries sufficient effective information in the image to be processed, and when the generation condition of the first data is determined, the amount of calculation generated by processing each pixel of the image to be processed in the related technology can be reduced.

[0094] Based on the above embodiments, the present application provides a data processing method, such as Figure 2 The data processing method may include the following steps:

[0095] Step 201: Process the image to be processed to obtain first data.

[0096] The first data represents pulse data corresponding to the image to be processed.

[0097] Each feature extraction layer in the aforementioned embodiment includes a pulse data processing layer and a pooling layer.

[0098] Step 202: The pulse data processing layer performs feature extraction on the input data of each feature extraction layer based on the synaptic weight information to obtain second feature data.

[0099] Among them, the pulse data processing layer includes multiple pulse neurons carrying synaptic weight information.

[0100] Step 203: Perform a pooling operation on the second feature data through multiple spiking neurons in the pooling layer.

[0101] In the embodiment of the present application, each feature extraction layer may include a pulse data processing layer as described in step 202 and a pooling layer as described in step 203.

[0102] In one embodiment, the second feature data may be used to represent a certain local feature carried in the first data, such as an edge feature.

[0103] In one embodiment, the number of second feature data may be multiple. For example, the Mth feature extraction layer processes its input data to obtain M1 second feature data. Similarly, M1 second feature data may serve as input data for the M+1th feature extraction layer. Here, M and M1 are both integers greater than or equal to 1. For example, when M is 1, the input data of the first feature extraction layer is the first data.

[0104] In one embodiment, each feature extraction layer can perform feature extraction on the input data to obtain second feature data, and can also remove redundant information carried in the second feature data, thereby saving time and resource costs for subsequent feature extraction operations of other feature extraction layers.

[0105] In one embodiment, the pulse data processing layer may represent a level for processing input pulse data.

[0106] In one embodiment, the spike data processing layer may represent a layer in the SNN.

[0107] In one embodiment, each pulse data processing layer can serve as a data input processing layer for each feature extraction layer.

[0108] In one embodiment, the i-th input data is input to the i-th spike data processing layer, and features are extracted from each spike data in the i-th input data using the synaptic weight information carried by the spike neurons in the i-th spike data processing layer to obtain the i-th output data. Where i is an integer greater than or equal to 1. When i is 1, the first input data may be the first data.

[0109] Exemplarily, the number of pulse data in the i-th input data may be less than or equal to the number of pulse neurons in the i-th pulse data processing layer.

[0110] In one embodiment, the pooling layer may be a layer disposed adjacent to the pulse data processing layer.

[0111] In one embodiment, a pooling layer may be used to provide translation invariance. For example, translation invariance may be achieved by allowing pulses output earlier in a pulse data processing layer to be transmitted.

[0112] In one embodiment, the pooling layer can also be used to compress visual information. For adjacent pulse neurons with overlapping receptive fields, the pulse signals they output will carry redundant information. Therefore, the input windows of the neurons in the pooling layer can be set to be non-overlapping. In this way, the input information is compressed by reducing redundancy, and the data processing amount of the pulse neurons in the subsequent pulse data processing layer can also be reduced.

[0113] In one embodiment, the pooling layer may perform a pooling operation on the output data of the spike data processing layer through the spike neurons contained therein.

[0114] In one embodiment, the i-th pooling layer may perform a pooling operation on the i-th output data to obtain the i-th feature data. Exemplarily, the i-th feature data may serve as input data for the i+1-th pulse data processing layer.

[0115] In the data processing method provided in the embodiment of the present application, the pulse data processing layer and the pooling layer can be set continuously and adjacently in an interlaced manner. As a result, the receptive field of each feature extraction layer will become larger as the number of feature extraction layers increases, and the selectivity of subsequent feature extraction layers for complex features will gradually increase.

[0116] In one embodiment, after step 203, steps A1 and A2 may be further performed:

[0117] Step A1: Obtain the second rule.

[0118] The second rule includes a rule for performing statistics on the first feature data.

[0119] In one embodiment, the second rule may include a condition for performing statistics on the data in the first feature data, such as starting to perform statistics on the first feature data only when the amount of data in the first feature data reaches a first threshold. The first threshold may be an integer greater than 0.

[0120] In one embodiment, the second rule may include an interval division rule for statistically analyzing the data in the first characteristic data. For example, several intervals are divided in advance, such as a first interval, a second interval, ..., a Yth interval; and the data range of the first interval is from the first threshold to the second threshold; the data range of the second interval is from the second threshold to the third threshold, ..., and the data range of the Yth interval is from the Yth threshold to the Y+1th threshold. When the first characteristic data is detected, the pulse value of the first characteristic data is obtained, and the relationship between the pulse value and each of the above thresholds is used to determine which interval the first characteristic data belongs to. Wherein, Y is an integer greater than 2, and each of the above thresholds is a number greater than 0.

[0121] In one embodiment, each of the above thresholds may also represent a specific time value corresponding to the appearance time of each feature data in the first feature data.

[0122] Step A2: Based on the second rule, statistics are performed on each characteristic value in the first characteristic data to obtain second data.

[0123] In one embodiment, the characteristic value can be used to represent the actual value of any characteristic data in the first characteristic data.

[0124] In one embodiment, the second data may be obtained in the following manner: based on the second rule, the feature values ​​in the first feature data that are greater than the first threshold are screened to obtain the second data.

[0125] In one embodiment, the second data can be used to represent the probability of the type corresponding to the image to be processed. For example, if the first feature data includes R pulse data, Q of the pulses have amplitudes within the first interval corresponding to the first type, and L of the pulses have amplitudes within the second interval corresponding to the second type, and Q is significantly greater than L, then the image to be processed can be determined to belong to the first type corresponding to the Q pulses. R, Q, and L are integers greater than 1, and R = Q + L.

[0126] The data processing method provided in the embodiment of the present application adopts a pulse neural network carrying synaptic weight information in an SNN as a pulse data processing layer of a feature extraction layer, which can realize accurate feature extraction of input pulse data and overcome the disadvantage of the CNN network's lack of rotation invariance; a pooling layer is adopted to add a pooling operation to the feature extraction layer, thereby realizing redundancy removal of the pulse data processing layer, thereby greatly reducing the redundancy of the data processed in each feature extraction layer; on this basis, according to the second rule, the output first feature data is statistically analyzed to obtain second data representing the type of object in the image to be processed, which can not only improve the disadvantage of the CNN's lack of rotation invariance, but also improve the accuracy of object type recognition in the image to be processed.

[0127] Based on the aforementioned embodiments, embodiments of the present application provide a data processing method, wherein the steps of processing an image to be processed to obtain first data, performing feature extraction and pooling operations on the first data using multiple spiking neurons carrying synaptic weight information in at least one feature extraction layer to obtain first feature data, and processing the first feature data to obtain second data are implemented by a neural network. The neural network is trained using image sample data.

[0128] Based on the above embodiments, the present application provides a data processing method, which can realize the training of a neural network. For example, Figure 3 As shown, the process of training the neural network in the data processing method may include the following steps:

[0129] Step 301: Process the image sample data to obtain third data.

[0130] The third data represents pulse data corresponding to the image sample data.

[0131] In one embodiment, the image sample data may include multiple types of image samples.

[0132] In one embodiment, each type of image sample in the image sample data may include multiple images.

[0133] In one embodiment, the image sample data may be an image sample including a specific target object type.

[0134] In one embodiment, each sample image in the image sample data may be an image of a complete target object. For example, the sample image above may be a complete picture of a car.

[0135] In one embodiment, each sample image in the image sample data may be an image that only includes a partial area of ​​the target object, for example, may be a picture that only displays an area of ​​a car tire.

[0136] In one embodiment, taking vehicle model recognition as an example, the image sample data can be a Comprehensive Car (CompCars) vehicle recognition database. The CompCars dataset includes image data of various car brands in various scenarios. The above image data includes images of the entire vehicle as well as images of parts of the vehicle body.

[0137] In one embodiment, taking the image sample data as the CompCars dataset as an example, the pixel intensity of the image in the CompCars dataset can be obtained, and a pulse sequence can be triggered based on the pixel intensity, so that the triggering rate of the pulse sequence is proportional to the intensity of the pixel in the image of the CompCars dataset, that is, the triggering rate of the pulse intensity and the pulse intensity carry the pixel information of the image in the CompCars dataset.

[0138] In one embodiment, after obtaining the corresponding raw pulse data based on the pixel intensity in the CompCars dataset, the raw pulse data is then encoded to obtain the third data. In some embodiments, the pulse data can be encoded using a Difference of Gaussian (DOG) function.

[0139] In one embodiment, the third data exhibits a Poisson distribution.

[0140] Figure 4 The following are some sample images of the image sample data, including body images of some common car models. The above images contain information such as the height, length, and wheel size of the vehicle.

[0141] In one embodiment, the third data may be obtained by processing pixel values ​​in the image sample data using the first algorithm.

[0142] Step 302: Process the third data through at least one feature extraction layer to obtain third feature data.

[0143] The third feature data is used to represent the feature data output by the last feature extraction layer.

[0144] Before step 302, step B may also be performed:

[0145] Step B: Determine the structure of the neural network.

[0146] In one embodiment, the structure of the neural network may include the number of layers of the neural network.

[0147] In one embodiment, the structure of the neural network may include the connection relationship between the layers of the neural network, such as whether the W-1 layer needs to provide feedback input to the W layer, where W is an integer greater than 1.

[0148] In one embodiment, the structure of the neural network may include a connection relationship between a spike data processing layer and a pooling layer in each feature extraction layer of the neural network.

[0149] In one embodiment, step B can be implemented through steps C1 to C4:

[0150] Step C1: Determine the first structure.

[0151] Among them, the first structure can be used to represent the structure of SNN.

[0152] In one embodiment, the first structure can be used to represent a multi-layer feed-forward SNN structure.

[0153] In one embodiment, the first structure can be used to represent the connection structure between spiking neurons and synapses in an SNN. For example, the connection structure between spiking neurons and synapses in an SNN can be determined by an equation for changes in membrane potentials of spiking neurons and synapses.

[0154] For example, the equations for the change of the spiking neuron and synaptic membrane potential over time in SNN are shown in formula (1):

[0155]

[0156] In formula (1), V is the membrane voltage of the pulse neuron, E is r is the resting membrane potential of the spiking neuron, E e and E i are the equilibrium potentials of excitatory and inhibitory synapses, respectively, c e and c i are the conductance coefficients of excitatory and inhibitory synapses, respectively, and τ is the time constant.

[0157] Among them, c in formula (1) e It can be determined by formula (2):

[0158]

[0159] In formula (2), τ ce is the time constant of the excitatory postsynaptic potential.

[0160] Among them, c in formula (1) ican be determined by formula (3):

[0161]

[0162] In formula (3), τ ci is the time constant of the inhibitory postsynaptic potential.

[0163] Through formulas (1)-(3), the temporal change trend and calculation method of the membrane potential of the spiking neurons and synapses in the SNN after receiving the pulse can be determined, laying the foundation for the subsequent adjustment of the synaptic weight information.

[0164] Step C2: Determine the second structure.

[0165] Among them, the second structure can be used to represent the structure of the pooling layer.

[0166] In one embodiment, the second structure can be used to represent the number of spiking neurons in the pooling layer.

[0167] In one embodiment, the second structure can be used to represent the distribution structure of spiking neurons in the pooling layer.

[0168] In one embodiment, the second structure can be used to represent the dimensional distribution of spiking neurons in the pooling layer.

[0169] Step C3: Determine the structure of the feature extraction layer based on the first structure and the second structure.

[0170] In one embodiment, the first structure and the second structure may be arranged adjacent to each other and cross-arranged to determine the structure of the feature extraction layer.

[0171] In one embodiment, a plurality of first structures may be disposed adjacent to a single second structure to determine the structure of the feature extraction layer.

[0172] In one embodiment, it is possible to determine how neurons in the pooling layer obtain spike data output by spike neurons in the spike processing layer, thereby determining the structure of the feature extraction layer.

[0173] Step C4: Determine the structure of the neural network based on the structure of the feature extraction layer.

[0174] In one embodiment, the structure of the neural network may include the number of feature extraction layers.

[0175] In one embodiment, the number of feature extraction layers can be determined based on image sample data, and then the structure of the neural network can be determined based on the number of feature extraction layers.

[0176] In one embodiment, the structure of the neural network may include data input and output relationships between various feature extraction layers.

[0177] In one embodiment, the structure of the neural network may include whether there are feedback connections between each feature extraction layer.

[0178] In one embodiment, the structure of the neural network may represent a structure in which each feature extraction layer is adjacently arranged.

[0179] Exemplarily, step B may also be implemented through steps D1-D3:

[0180] Step D1: Establish a CNN and determine the connection relationship between the convolution layer and the pooling layer.

[0181] In step D2, the SNN structure determined by equations (1)-(3) is used to replace the convolutional layer in the CNN, which becomes the pulse data processing layer in the feature extraction layer of the neural network.

[0182] Step D3: Determine the pulse data processing relationship between the pooling layer and the pulse data processing layer, that is, determine the output data selectivity of the pooling layer to the pulse data processing layer.

[0183] In some implementations, after step D3, the following operations may be performed:

[0184] A merging layer is established to compress the redundant visual information carried by adjacent spiking neurons with overlapping inputs.

[0185] In some implementations, a global pooling layer may be established to judge the output results of the feature extraction layer to determine their categories.

[0186] In some embodiments, the third data may be input into at least one feature extraction layer and processed by the at least one feature extraction layer to obtain third feature data.

[0187] In one embodiment, the number of third feature data may be multiple. For example, the Mth feature extraction layer processes its input data to obtain M1 third feature data. Similarly, M1 third feature data may serve as input data for the M+1th feature extraction layer. Here, M and M1 are both integers greater than or equal to 1. For example, when M is 1, the input data of the first feature extraction layer is the third data.

[0188] Step 303: Process the third characteristic data to obtain fourth data.

[0189] The fourth data represents the type corresponding to any image sample.

[0190] In one embodiment, the fourth data may be used to indicate the type of a specific target object in any image sample.

[0191] In one embodiment, the fourth data may be used to indicate the type of a specific component of the target object in any image sample, such as the type of a wheel of the target object.

[0192] In one implementation, the output data of the last feature extraction layer may be processed to obtain fourth data.

[0193] Step 304: Adjust the parameter information of the neural network based on the third feature data and the fourth data.

[0194] The parameter information at least includes synaptic weight information.

[0195] Step 304 may be implemented through steps E1 and E2:

[0196] Step E1: Obtain a first rule.

[0197] The first rule at least includes a rule for adjusting synaptic weight information.

[0198] In one embodiment, the first rule may indicate whether synaptic weight information needs to be adjusted based on labels in the image sample data. For example, the first rule may indicate an unsupervised learning rule, a supervised learning rule, or a semi-supervised transfer learning rule. The labels in the image sample data may indicate the type of the image sample data or the type of the target object in the image sample data.

[0199] In one embodiment, the first rule may be used to represent a change rule of a model on which synaptic weight information in an SNN is adjusted.

[0200] In one embodiment, the first rule may be a rule for adjusting synaptic weight information in an STDP model.

[0201] For example, in order to adjust the synaptic weight coefficient using the STDP model, it is first necessary to determine the first structure through step B1. In the embodiment of the present application, the feedforward SNN structure is used as an example for explanation.

[0202] For example, in order to adjust the synaptic weight coefficient using the STDP model, it is also necessary to establish the STDP model.

[0203] The STDP model is exemplarily used to adjust the synaptic weight information connecting spiking neurons. Furthermore, the STDP model applies an adaptive adjustment rule for synaptic weight information, i.e., when the presynaptic spiking neuron generates a pulse later than the postsynaptic spiking neuron, the strength of the connection between the presynaptic spiking neuron and the postsynaptic spiking neuron is weakened, i.e., the synaptic weight is weakened; conversely, when the presynaptic spiking neuron generates a pulse earlier than the postsynaptic spiking neuron, the strength of the connection between the presynaptic spiking neuron and the postsynaptic spiking neuron is strengthened, i.e., the synaptic weight is strengthened.

[0204] Exemplarily, the STDP model may adjust synaptic weight information in the following two ways:

[0205] The first method is to adjust the synaptic weight information according to the trace value of the postsynaptic pulse. The adjustment process is shown in formula (4):

[0206] △w=-η pre x post (4)

[0207] In formula (4), △w is the synaptic weight information, η pre is the learning rate when the presynaptic spike occurs, x post is the postsynaptic spike trace value.

[0208] The first method is to adjust the synaptic weight information according to the trace value of the presynaptic pulse. The adjustment process is shown in formula (5):

[0209] △w=η post (x pre -xtar) (5)

[0210] In formula (4), △w is the synaptic weight information, η post is the learning rate when the postsynaptic spike occurs, xpre is the presynaptic spike trace value, and x tar is the target mean of the presynaptic trace when the postsynaptic spike occurs.

[0211] Step E2: Adjust parameter information based on the first rule, the third characteristic data, and the fourth data.

[0212] The parameter information at least includes synaptic weight information.

[0213] In one embodiment, adjusting the parameter information of the neural network can be achieved by:

[0214] The third data is input into the neural network, and the pulse neurons in the neural network trigger or inhibit each pulse data in the third data, and adjust the synaptic weight information related to each pulse neuron in the neural network according to the first rule.

[0215] For example, when the amount of image sample data input into the neural network reaches a preset threshold, the fourth data may be matched with the type labels corresponding to the image sample data to obtain a matching result. If the proportion of correct type recognition in the matching result satisfies a training termination condition, the training of the neural network may be stopped. Conversely, if the proportion of correct type recognition in the matching result does not meet the training termination condition, the image sample data may continue to be input into the neural network to achieve continuous training of the neural network. The training termination condition may be used to indicate that the proportion of correct type recognition by the neural network reaches a preset recognition rate threshold.

[0216] In one embodiment, adjusting the parameter information of the neural network can be achieved by:

[0217] The parameter information is adjusted in an unsupervised manner based on the first rule, the third feature data, and the fourth data.

[0218] In one embodiment, the parameter information may be adjusted in the following manner:

[0219] First, a layer-by-layer recursive approach is used to establish connections between adjacent feature extraction layers.

[0220] For example, the third data can be input into the first pulse data processing layer in the first feature extraction layer, the first feature extraction layer outputs a first pulse sequence signal, and the number of pulse neurons excited in the first feature extraction layer within one cycle is counted. The first pulse sequence signal is input into the second pulse data processing layer in the second feature extraction layer, and the number of pulse neurons excited in the second feature extraction layer within one cycle is counted. Similarly, the activated pulse neurons in all feature extraction layers can be obtained, and connections can be established in sequence between the activated pulse neurons in each of the above pulse data processing layers, thereby establishing connections between adjacent feature extraction layers.

[0221] Secondly, a pooling layer is added between adjacent spike data processing layers.

[0222] In some embodiments, the pooling layer may be disposed after the impulse data processing layer and adjacent to the impulse data processing layer.

[0223] The next sample is selected from the image sample data, and the above process is repeated until all image sample data are traversed, resulting in a trained neural network; or until the type recognition accuracy represented by the output reaches an expected level, resulting in a trained neural network. The expected level can mean that the type recognition accuracy reaches a preset threshold.

[0224] For example, whether the type recognition accuracy reaches a preset recognition rate threshold can be determined by matching the fourth data output by the neural network with the type label carried by the current image sample. If the two match, it can be indicated that the type recognition accuracy has reached the preset recognition rate threshold; conversely, if the two fail to match, it can be indicated that the type recognition accuracy has not reached the preset recognition rate threshold.

[0225] In one embodiment, the parameter information further includes parameters of the pooling operation, that is, the training of the parameter information may further include a process of adjusting the parameters of the pooling operation.

[0226] In one embodiment, the parameters of the pooling operation may be parameters set according to the features of the image sample data, and the parameters of the pooling operation of the pooling layers of each feature extraction layer may be kept consistent.

[0227] In one embodiment, the parameters of the pooling operation may be the weights of the neurons in the pooling layer. In some embodiments, the neurons in the pooling layer may also be spiking neurons.

[0228] In one embodiment, the parameter of the pooling operation may be pooled synaptic weight information. The pooled synaptic weight information represents the weight associated with multiple spiking neurons in the pooling layer. In some embodiments, the pooled synaptic weight information may be set to a uniform number, such as 1.

[0229] Figure 5 This is a structural diagram of the neural network of the data processing method provided in the embodiment of the present application. Figure 5 , 501 represents image sample data, 502 represents pulse data corresponding to the image sample data, i.e., third data, 503 represents a first pulse data processing layer in the first feature extraction layer, 504 represents a first pooling layer in the first feature extraction layer, 505 and 506 represent a second pulse data processing layer in the second feature extraction layer and a second pooling layer in the second feature extraction layer, respectively, 507 and 508 represent a third feature extraction layer in the third feature extraction layer and a third pooling layer in the third feature extraction layer; Used to represent the synaptic weight information in the m-th pulse data processing layer, Parameters used to represent the pooling operation in the mth pooling layer; Represents the pulse data after DOG encoding of the image sample data. At the end of the neural network, a global pooling layer can be added to count each pulse data in the third feature data output by the last feature extraction layer. Where m is an integer greater than or equal to 1.

[0230] For example, before statistics are performed, pulse amplitude intervals can be pre-set. Based on the amplitude of the actual output pulse data, the pulses are divided into different amplitude intervals. If the Zth amplitude interval has the largest number of pulses, the type corresponding to the Zth amplitude interval is used as the type corresponding to the image sample data. Here, Z is an integer greater than 1.

[0231] Through the above operations, it is possible to effectively identify several types of vehicles in the image sample data, such as multi-purpose vehicles (MPVs), sport utility vehicles (SUVs), sedans, vans, passenger and cargo vehicles, sports cars, convertibles, etc.

[0232] Table 1 shows the accuracy of the data processing method provided in the embodiment of the present application compared with CNN in identifying vehicles such as MPVs, SUVs, sedans, minibuses, light trucks, and convertibles. The accuracy is calculated in percentages.

[0233] As can be seen from Table 1, the neural network provided in the embodiment of the present application has an accuracy rate of 5% higher for MPV recognition than that of CNN, an accuracy rate of 2.8% higher for SUV recognition than that of CNN, an accuracy rate of 4.4% higher for box sedan recognition than that of CNN, an accuracy rate of 7.2% higher for minibus recognition than that of CNN, an accuracy rate of 3.6% higher for light truck recognition than that of CNN, and an accuracy rate of 5.8% higher for convertible recognition than that of CNN.

[0234] MPV SUV box car China-Pakistan light truck Convertible CNN 83.3 82.6 81.4 81.9 89.3 84.1 The neural network provided by this application 88.3 85.4 85.8 88.1 92.9 89.9

[0235] Table 1

[0236] That is to say, the neural network in the data processing method provided in the embodiment of the present application has a certain degree of improvement in the accuracy of identifying various types of vehicles compared with CNN.

[0237] From the above, it can be seen that the data processing method provided in the embodiment of the present application obtains the pulse data corresponding to the image to be processed, that is, the first data, and then extracts features from each pulse data through the pulse data processing layer to obtain second feature data, and then performs a pooling operation on the second feature data through the pooling layer, thereby making the layering of the neural network clear, and through the collaboration of each layer, accurate type recognition of objects in the image to be processed can be achieved.

[0238] Based on the above embodiments, an embodiment of the present application provides a data processing device 6, which includes a processor 61 and a memory 62, wherein the memory 62 is used to store the computer program running on the processor 61; the processor 61 is used to execute the data processing method described in any of the above embodiments when running the above computer program.

[0239] Exemplarily, the processor 61 may be implemented by a processor. The processor may be at least one of an application-specific integrated circuit (ASIC), a DSP, a DSPD, a PLD, an FPGA, a CPU, a controller, a microcontroller, and a microprocessor. It is understood that the electronic device used to implement the aforementioned processor functions may also be other electronic devices, and this is not specifically limited in the present embodiment.

[0240] The memory 62 may be a volatile memory, such as RAM; or a non-volatile memory, such as ROM, flash memory, hard disk drive (HDD) or solid-state drive (SSD); or a combination of the above types of memory, and provides instructions and data to the processor.

[0241] In the data processing device provided in the embodiment of the present application, each feature extraction layer in at least one feature extraction layer can extract features from the input data through multiple pulse neurons carrying synaptic weight information to obtain corresponding second feature data. Therefore, the data processing device provided in the embodiment of the present application can not only efficiently and accurately identify the feature information in the image to be processed, but also overcome the disadvantage of CNN's lack of rotation invariance. On the other hand, through the pooling operation, it can also reduce the redundant information in the output data of each feature extraction layer. Therefore, each feature extraction layer in the data processing device provided in the embodiment of the present application can achieve efficient feature extraction of the input data. In this way, the first feature data obtained by feature extraction in sequence by at least one feature extraction layer is processed, and the second data representing the type of the object in the image to be processed is obtained, which can more flexibly and accurately reflect the type information of the object in the image to be processed.

[0242] From the above, it can be seen that in the data processing device provided in the embodiment of the present application, the image to be processed is processed to obtain pulse data corresponding to the image to be processed, that is, the first data carries sufficient effective information in the image to be processed, and when the conditions for generating the first data are determined, the amount of calculation generated by processing each pixel of the image to be processed in the related technology can be reduced.

[0243] Based on the above embodiments, the present application provides a data processing device 7, such as Figure 7 As shown, the data processing device 7 includes: an acquisition module 71 and a processing module 72; wherein:

[0244] An acquisition module 71 is configured to process the image to be processed to obtain first data, wherein the first data represents pulse data corresponding to the image to be processed;

[0245] a processing module 72 configured to perform feature extraction and pooling operations on the first data using a plurality of spiking neurons carrying synaptic weight information in at least one feature extraction layer to obtain first feature data;

[0246] The processing module 72 is further configured to process the first feature data to obtain second data; wherein the second data represents the type of the object in the image to be processed.

[0247] In some embodiments, the processing module 72 is further configured to perform feature extraction on the input data of each feature extraction layer based on the synaptic weight information through a spike data processing layer to obtain second feature data; wherein the spike data processing layer includes a plurality of spike neurons carrying the synaptic weight information;

[0248] The processing module 72 is configured to perform a pooling operation on the second feature data through a plurality of spiking neurons in a pooling layer, wherein each feature extraction layer includes a spiking data processing layer and a pooling layer.

[0249] In some embodiments, the processing module 72 processes the image to be processed to obtain first data, processes the first data through multiple pulse neurons carrying synaptic weight information in at least one feature extraction layer to obtain first feature data, and performs feature extraction and pooling operations on the first feature data to obtain second data, which are implemented by a neural network; the neural network is obtained by training image sample data.

[0250] In some embodiments, the processing module 72 is further configured to process the image sample data to obtain third data; wherein the third data represents pulse data corresponding to the image sample data; and process the third data through at least one feature extraction layer to obtain third feature data; wherein the third feature data represents feature data output by the last feature extraction layer.

[0251] The processing module 72 is further configured to process the third feature data to obtain fourth data; wherein the fourth data represents a type corresponding to any image sample;

[0252] The processing module 72 is further configured to adjust parameter information of the neural network based on the third feature data and the fourth data; wherein the parameter information includes at least synaptic weight information.

[0253] In some embodiments, the processing module 72 is further configured to obtain a first rule; wherein the first rule at least includes a rule for adjusting synaptic weight information;

[0254] The processing module 72 is further configured to adjust the parameter information based on the first rule, the third characteristic data, and the fourth data.

[0255] In some embodiments, the processing module 72 is further configured to obtain a second rule; wherein the second rule includes a rule for performing statistics on the first feature data;

[0256] The processing module 72 is further configured to perform statistics on each characteristic value in the first characteristic data based on a second rule to obtain second data.

[0257] In the data processing device 7, the acquisition module 71 and the processing module 72 can be implemented by a processor. Specifically, the processor can be at least one of an application-specific integrated circuit (ASIC), a DSP, a DSPD, a PLD, an FPGA, a CPU, a controller, a microcontroller, and a microprocessor. It is understood that the electronic device used to implement the functions of the processor can also be other electronic devices, and this embodiment of the present application is not specifically limited thereto.

[0258] In the data processing device provided in the embodiment of the present application, each feature extraction layer in at least one feature extraction layer can extract features from the input data through multiple pulse neurons carrying synaptic weight information. In this way, the multiple pulse neurons carrying synaptic weight information can efficiently and accurately identify the feature information in the image to be processed, which can overcome the shortcoming of CNN's lack of rotation invariance. On the other hand, the pooling operation can also reduce the redundant information in the output data of each feature extraction layer. Therefore, each feature extraction layer in the data processing device provided in the embodiment of the present application can achieve efficient feature extraction of the input data. In this way, the first feature data obtained by feature extraction in sequence by at least one feature extraction layer is processed, and the second data representing the type corresponding to the image to be processed is obtained, which can more flexibly and accurately reflect the type information of the object in the image to be processed.

[0259] From the above, it can be seen that in the data processing device provided in the embodiment of the present application, the image to be processed is processed to obtain pulse data corresponding to the image to be processed, that is, the first data carries sufficient effective information in the image to be processed, and when the conditions for generating the first data are determined, the problem of excessive computational complexity caused by processing each pixel of the image to be processed in the related technology can be reduced.

[0260] An embodiment of the present application also provides a computer-readable storage medium, which stores one or more programs. The one or more programs can be executed by one or more processors to implement the steps of the data processing method described in any of the aforementioned embodiments.

[0261] In some embodiments, the functions or modules included in the device provided in the embodiments of the present application can be used to execute the method described in the above method embodiments. The specific implementation can refer to the description of the above method embodiments. For the sake of brevity, it will not be repeated here.

[0262] The above description of the various embodiments tends to emphasize the differences between the various embodiments. The same or similar aspects can be referenced with each other and will not be repeated herein for the sake of brevity.

[0263] The methods disclosed in the various method embodiments provided in this application can be arbitrarily combined without conflict to obtain new method embodiments.

[0264] The features disclosed in the various product embodiments provided in this application can be arbitrarily combined without conflict to obtain new product embodiments.

[0265] The features disclosed in the various method or device embodiments provided in this application can be arbitrarily combined without conflict to obtain new method embodiments or device embodiments.

[0266] It should be noted that the above-mentioned computer-readable storage medium can be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), a magnetic random access memory (FRAM), a flash memory (Flash Memory), a magnetic surface storage, an optical disc, or a compact disc read-only memory (CD-ROM); it can also be various electronic devices that include one or any combination of the above-mentioned memories, such as mobile phones, computers, tablet devices, personal digital assistants, etc.

[0267] It should be noted that, in this document, the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, article, or apparatus comprising a series of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such process, method, article, or apparatus. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of other identical elements in the process, method, article, or apparatus comprising the element.

[0268] The serial numbers of the above embodiments of the present invention are for description only and do not represent the advantages or disadvantages of the embodiments.

[0269] Through the description of the above embodiments, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus the necessary general hardware platform, and of course, by hardware, but in many cases the former is a better embodiment. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product. The computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes a number of instructions for enabling a terminal device (which can be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in the various embodiments of the present invention.

[0270] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowcharts and / or block diagrams, as well as combinations of processes and / or blocks in the flowcharts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowcharts and / or block diagrams. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0271] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0272] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0273] The above are only preferred embodiments of the present invention and are not intended to limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made using the contents of the present invention description and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. A data processing method, characterized in that: The method comprises: Processing the image to be processed to obtain first data; wherein the first data represents pulse data corresponding to features of an edge region of the image to be processed, and the pulse data includes a pulse sequence corresponding to a first pixel value in the image to be processed that is greater than a specified threshold; the image to be processed includes an image of a partial region of a vehicle; Performing feature extraction and pooling processing on the first data using a plurality of spiking neurons carrying synaptic weight information in at least one feature extraction layer to obtain first feature data; The first feature data is processed to obtain second data; wherein the second data represents the model of the car in the image to be processed.

2. The method according to claim 1, characterized in that Each of the feature extraction layers includes a pulse data processing layer and a pooling layer; the feature extraction and pooling operation processing on the first data includes: The pulse data processing layer performs feature extraction on the input data of each feature extraction layer based on the synaptic weight information to obtain second feature data; wherein the pulse data processing layer includes the plurality of pulse neurons carrying the synaptic weight information; The pooling operation is performed on the second feature data through the plurality of pulse neurons in the pooling layer.

3. The method according to claim 1, characterized in that The steps of processing the image to be processed to obtain first data, performing feature extraction and pooling operations on the first data through multiple pulse neurons carrying synaptic weight information in at least one feature extraction layer to obtain first feature data, and processing the first feature data to obtain second data are implemented by a neural network; the neural network is obtained by training image sample data.

4. The method according to claim 3, characterized in that The neural network is trained by image sample data, including: Processing the image sample data to obtain third data; wherein the third data represents pulse data corresponding to the image sample data; Processing the third data through the at least one feature extraction layer to obtain third feature data; wherein the third feature data is used to represent the feature data output by the last feature extraction layer; Processing the third feature data to obtain fourth data; wherein the fourth data represents a type corresponding to any one of the image samples; Based on the third feature data and the fourth data, parameter information of the neural network is adjusted; wherein the parameter information at least includes the synaptic weight information.

5. The method according to claim 4, characterized in that The adjusting the parameter information of the neural network based on the third feature data and the fourth data includes: Acquire a first rule; wherein the first rule at least includes a rule for adjusting the synaptic weight information; The parameter information is adjusted based on the first rule, the third feature data, and the fourth data.

6. The method according to claim 4, characterized in that The parameter information also includes parameters of the pooling operation.

7. The method according to claim 1, characterized in that The processing of the first feature data to obtain second data includes: Obtaining a second rule; wherein the second rule includes a rule for performing statistics on the first feature data; Based on the second rule, statistics are collected on each characteristic value in the first characteristic data to obtain the second data.

8. A data processing device, characterized in that: The device comprises a processor and a memory; wherein: The memory is used to store a computer program that can be run on the processor; The processor is configured to execute the data processing method according to any one of claims 1 to 7 when running the computer program.

9. A data processing device, characterized in that: The data processing device includes: an acquisition module and a processing module; wherein: The acquisition module is configured to process the image to be processed to obtain first data; wherein the first data represents pulse data corresponding to features of an edge region of the image to be processed, and the pulse data includes a pulse sequence corresponding to a first pixel value in the image to be processed that is greater than a specified threshold; and the image to be processed includes an image of a partial region of a vehicle; The processing module is configured to perform feature extraction and pooling processing on the first data using a plurality of spiking neurons carrying synaptic weight information in at least one feature extraction layer to obtain first feature data; The processing module is further configured to process the first feature data to obtain second data; wherein the second data represents the model of the car in the image to be processed.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to implement the data processing method according to any one of claims 1 to 7.

Images