An image processing method and related device

Abstract

An embodiment of the present application discloses an image processing method and related devices. The method is applied to the field of image processing technology and includes: obtaining the cumulative processed image values of each network layer in a target image processing model; if it is compared and the cumulative processed image value of the first network layer is greater than or equal to a preset processed image threshold, triggering the network layer at the next level of the first network layer to send a pulse to the network layer at the next-next level of the first network layer, where the pulse carries the image data corresponding to the image, triggering the network layer at the next-next level corresponding to the first network layer to process the image data, and outputting a processing result for the image according to the processing results of each network layer on the image data. In this way, the image data is transmitted between network layers in the form of pulses, which can not only reduce the amount of calculation, but also improve the transmission rate of the image data, thereby improving the processing efficiency of the target image processing model.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular, to an image processing method and related device. Background Art

[0002] Currently, traditional deep learning neural network models such as Convolutional Neural Networks (CNN) models and Deep Belief Networks (DBN) models are usually used for image processing. However, in traditional deep learning neural network models, traditional neurons such as Sigmoid, Tanh, ReLU, etc. are used, and a specific floating-point number is transmitted between network layers. There are problems such as high computational complexity, high power consumption, high cost, and too large model size. Especially when the computing power is insufficient, the processing speed of the model will also be slower and the processing efficiency is low. Summary of the Invention

[0003] Embodiments of the present application provide an image processing method and related device, which can improve the processing efficiency of the target image processing model.

[0004] In a first aspect, embodiments of the present application provide an image processing method, which includes:

[0005] When it is detected that an image is input into a pre-constructed target image processing model, obtain the cumulative processed image values of each network layer in at least one network layer included in the target image processing model, and each network layer includes at least one spiking neuron for transmitting image data;

[0006] Compare the cumulative processed image values of each network layer with a preset processed image threshold respectively. If it is compared that the cumulative processed image value of the first network layer is greater than or equal to the preset processed image threshold, trigger the network layer at the next level of the first network layer to send a pulse to the network layer at the next-next level of the first network layer, and the pulse carries the image data corresponding to the image. The first network layer is any one of the at least one network layer;

[0007] Trigger the network layer at the next-next level corresponding to the first network layer to process the image data;

[0008] Output the processing result for the image according to the processing results of each network layer on the image data.

[0009] In one embodiment, before obtaining the cumulative processed image values of each network layer in at least one network layer included in the target image processing model, an initial image processing model may also be obtained. The initial image processing model includes at least one initial network layer and connection weight values corresponding to each initial network layer in the at least one initial network layer. Each initial network layer includes at least one neuron for transmitting image data. A training image is obtained, and the initial image processing model is trained according to the training image to obtain an image processing model, where the image processing model includes connection weight values corresponding to each initial network layer after training. Based on the connection weight values corresponding to each initial network layer after training and a preset spiking neuron function, a target image processing model corresponding to the image processing model is constructed. The target image processing model includes at least one network layer, and each network layer includes at least one spiking neuron for transmitting image data.

[0010] In one embodiment, the cumulative processed image value of the first network layer is the sum value of at least one processed image value corresponding to the first network layer before the system time. Each processed image value is the product of the image value input to the first network layer at the system time and the connection weight value corresponding to the first network layer.

[0011] In one embodiment, the image values input to each network layer may also be compared with a preset image reference value respectively. If it is compared that the image value input to the i-th network layer is greater than or equal to the preset image reference value, the image value is increased based on a preset function, and the product of the increased image value and the connection weight value of any network layer is used as the target image value of the next-level network layer input to any network layer. The i-th network layer is any network layer among the each network layer. If it is compared that the image value input to the i-th network layer is less than the preset image reference value, the image value is decreased based on a preset function, and the product of the decreased image value and the connection weight value of any network layer is used as the target image value of the next-level network layer input to any network layer.

[0012] In one embodiment, the preset function includes a preset exponential decay function and the connection weight value corresponding to the i-th network layer.

[0013] In one embodiment, before comparing the image values input to each network layer with the preset image reference value respectively, the image values corresponding to the images input to the target image processing model n times may also be counted, where n is a positive integer. The average value of the image values input n times is calculated, and the calculated average value is determined as the preset image reference value.

[0014] In one embodiment, after comparing the cumulative processed image values of each network layer with the preset processed image threshold respectively, if it is obtained by comparison that the cumulative processed image value of the first network layer is greater than or equal to the preset processed image threshold, the input of the next image value to the first network layer can be waited for until the cumulative processed image value corresponding to the first network layer is greater than or equal to the preset processed image threshold, and then the network layer at the next level of the first network layer is triggered to send a pulse to the network layer at the next-next level of the first network layer.

[0015] In a second aspect, an embodiment of the present application provides an image processing device, which includes a module for executing the method in the first aspect above.

[0016] In a third aspect, an embodiment of the present application provides a server, which includes a processor, a network interface, and a memory. The processor, the network interface, and the memory are interconnected. Among them, the network interface is controlled by the processor to send and receive messages, the memory is used to store a computer program that supports the server to execute the above method, the computer program includes program instructions, and the processor is configured to call the program instructions to execute the method in the first aspect above.

[0017] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, which stores a computer program. The computer program includes program instructions, and when the program instructions are executed by a processor, the processor is enabled to execute the method in the first aspect above.

[0018] In the embodiment of the present application, when the server detects that an image is input into a pre-constructed target image processing model, the cumulative processed image values of each network layer in at least one network layer included in the target image processing model can be obtained. Each network layer includes at least one pulse neuron for transmitting image data. Further, the server can compare the cumulative processed image values of each network layer with the preset processed image threshold respectively. If it is obtained by comparison that the cumulative processed image value of the first network layer is greater than or equal to the preset processed image threshold, the network layer at the next level of the first network layer is triggered to send a pulse to the network layer at the next-next level of the first network layer. The pulse carries the image data corresponding to the image, and the first network layer is any one of the at least one network layer. Further, the server can trigger the network layer at the next-next level corresponding to the first network layer to process the image data, and output a processing result for the image according to the processing results of each network layer on the image data. In this way, the image data is transmitted between network layers in the form of pulses, which can not only reduce the amount of calculation, but also improve the transmission rate of the image data, thereby improving the processing efficiency of the target image processing model. Description of the Drawings

[0019] To more clearly illustrate the technical solutions of the embodiments of the present application, the following will briefly introduce the accompanying drawings required for the description of the embodiments. Obviously, the accompanying drawings in the following description are some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other accompanying drawings can also be obtained based on these drawings.

[0020] Figure 1 is a schematic flowchart of an image processing method provided by an embodiment of the present application;

[0021] Figure 2 is a schematic structural diagram of a target graphics processing model provided by an embodiment of the present application;

[0022] Figure 3 is a schematic flowchart of another image processing method provided by an embodiment of the present application;

[0023] Figure 4 is a schematic diagram of replacing the siegert neurons in the DBN model with LIF neurons provided by an embodiment of the present application;

[0024] Figure 5 is a schematic block diagram of an image processing device provided by an embodiment of the present application;

[0025] Figure 6 is a schematic block diagram of a server provided by an embodiment of the present application. Detailed Embodiments

[0026] The following will clearly and completely describe the technical solutions in the embodiments of the present application in conjunction with the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are some, but not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts belong to the scope of protection of the present invention.

[0027] See Figure 1 , Figure 1 is a schematic flowchart of an image processing method provided by an embodiment of the present application. This method is applied to a server and can be executed by the server. As shown in the figure, this image processing method may include:

[0028] S101: When it is detected that an image is input into a pre-constructed target image processing model, obtain the cumulative processed image values of each network layer in at least one network layer included in the target image processing model. Each network layer includes at least one pulse neuron for transmitting image data.

[0029] Among them, the target image processing model can be an image recognition model for recognizing objects included in an image, such as recognizing the sky, grassland, human face, etc. in the image; or it can be an image classification model for determining the type of objects included in the image, such as dogs, cats, humans, etc.

[0030] In practical applications, the field to which the target image processing model is applied is related to the training samples for training and optimizing the target image processing model. Exemplarily, if the target image processing model is used for face recognition, then a face image annotated with face feature information can be determined as the training image for the target image processing; if the target image processing model is used for image classification, then an image annotated with category information can be determined as the training image for the target image processing model. For example, if the target image processing model is used to recognize dogs, then an image that has been determined to be of the dog category can be used as the dog training image for the target image processing model.

[0031] In one embodiment, the target image processing model includes at least one network layer, and each network layer corresponds to a connection weight value. The network layer of the first level corresponds to the connection weight value W I , the network layer of the second level corresponds to the connection weight value W II , the network layer of the third level corresponds to the connection weight value W III , and so on. When an image is input into the pre-constructed target image processing model, in practical applications, it can be understood that the image value corresponding to the image is input into the network layer of the first level of the target image processing model, and this network layer of the first level can be called the input layer. Exemplarily, for an image, the image value corresponding to the image is the grayscale value of the image.

[0032] In one embodiment, the cumulative processed image value of the above first network layer is the sum value of at least one processed image value corresponding to the first network layer before the system time, and each processed image value is the product of the image value input into the first network layer at the system time and the connection weight value corresponding to the first network layer.

[0033] Exemplarily, assume that at 17:00 on August 6, 2019, the server detects that the target image processing model has its first image input. The first input image is P1, and the image value input into the network layer of the first level of the target image processing model is the grayscale value of this image P1 (hereinafter referred to as the first grayscale value). Then, the processed image value of the network layer of the first level at 17:00 on August 6, 2019 is the first grayscale value (i.e., the image value input into the network layer of the first level) * W I (i.e., the connection weight value of the network layer of the first level). Since it is the first time there is an image input, the cumulative image value of the network layer of the first level at 17:00 on August 6, 2019 is also the first grayscale value * WI Furthermore, if the server detects that the target image processing model has another image P2 input for the first time at 17:01 on August 6, 2019, the grayscale value corresponding to the image P2 is called the second grayscale value. The server can set the processed image value of the first-level network layer at 17:01 on August 6, 2019 as the second grayscale value*W I In this case, before 17:01 on August 6, 2019, the sum of at least one processed image value corresponding to the first-level network layer, that is, the cumulative processed image value of the first-level network layer is the first gray value * W I + second gray value*W I .

[0034] In one embodiment, each time each network layer has an image value input, the server will continuously accumulate the image value input each time for each network layer, and record the accumulated image value of each network layer (i.e., the accumulated processed image value). In this case, the server can obtain the accumulated processed image value input to each network layer according to a preset period. The preset period is pre-set based on experimental measurement data and can be adjusted based on actual needs later.

[0035] S102: Compare the cumulative processed image value of each network layer with the preset processed image threshold respectively. If the cumulative processed image value of the first network layer is greater than or equal to the preset processed image threshold, trigger the network layer of the next level of the first network layer to send a pulse to the network layer of the next level of the first network layer. The pulse carries image data corresponding to the image. The first network layer is any one of the at least one network layer.

[0036] Exemplarily, assuming that the target image processing model includes at least one network layer including an input layer, a first hidden layer, a second hidden layer and an output layer, the structural diagram of the target image processing model is as follows: Figure 2 As shown in the figure, the input layer contains 784 neurons, two hidden layers, each layer contains 500 neurons, and the output layer contains 10 neurons, corresponding to the numbers 0 to 9. The image value input to the input layer can be the gray value of a 28*28 handwritten digital picture, and the connection weight value corresponding to the input layer is W. I , the connection weight value corresponding to the first hidden layer is W II , the connection weight value corresponding to the second hidden layer is W IIIAmong them, the first network layer can be any one of the input layer, the first hidden layer, the second hidden layer, and the output layer. In this case, the server can first detect whether the cumulative processed image value of the input layer is greater than or equal to a preset processed image threshold. If so, since the cumulative processed image value of the input layer is used as the input of the first hidden layer, a pulse can be triggered from the first hidden layer to the second hidden layer to transmit image data. Here, the first hidden layer is the network layer at the next level of the input layer, and the second hidden layer is the network layer at the next-next level of the input layer.

[0037] In one embodiment, the server compares the cumulative processed image values of each network layer with the preset processed image threshold. If it is found that the cumulative processed image value of the first network layer is less than the preset processed image threshold, it can wait for the next image value to be input into the first network layer until the cumulative processed image value corresponding to the first network layer is greater than or equal to the preset processed image threshold, and then trigger the network layer at the next level of the first network layer to send a pulse to the network layer at the next-next level of the first network layer.

[0038] S103: Trigger the network layer at the next-next level corresponding to the first network layer to process the image data, and output the processing result for the image according to the processing results of each network layer on the image data.

[0039] Among them, the processing result is related to the application field of the target image processing model. Exemplarily, if the target image processing model is used for face recognition, the processing result for the image indicates whether the image is a face. If the target image processing model is used for image classification, the processing result for the image indicates the category of the object in the image, such as dogs, cats, etc. The embodiments of the present application do not make specific limitations on this.

[0040] In the embodiments of the present application, when the server detects that an image is input into a pre-constructed target image processing model, it can obtain the cumulative processed image values of each network layer in at least one network layer included in the target image processing model. Each network layer includes at least one spiking neuron for transmitting image data. Further, the server can compare the cumulative processed image values of each network layer with a preset processed image threshold respectively. If it is compared that the cumulative processed image value of the first network layer is greater than or equal to the preset processed image threshold, the server triggers the network layer at the next level of the first network layer to send a spike to the network layer at the next-next level of the first network layer. The spike carries the image data corresponding to the image. The first network layer is any one of the at least one network layer. Further, the server can trigger the network layer at the next-next level corresponding to the first network layer to process the image data, and output a processing result for the image according to the processing results of each network layer on the image data. In this way, the image data is transmitted between network layers in the form of spikes, which can not only reduce the amount of calculation, but also improve the transmission rate of the image data, thereby improving the processing efficiency of the target image processing model.

[0041] See Figure 3 , Figure 3 FIG. is a schematic flowchart of another image processing method provided by the embodiments of the present application. This method is applied to a server and can be executed by the server. As shown in the figure, the image processing method may include:

[0042] S301: Obtain an initial image processing model, where the initial image processing model includes at least one initial network layer and connection weight values corresponding to each initial network layer in the at least one initial network layer. Each initial network layer includes at least one neuron for transmitting image data.

[0043] Among them, the initial image processing model is a traditional deep learning neural network model, such as CNN and DBN. Exemplarily, when the initial image processing model is a DBN model, the neurons included in the initial network layer corresponding to the DBN model are sigert neurons.

[0044] S302: Obtain a training image, and train the initial image processing model according to the training image to obtain an image processing model, where the image processing model includes the connection weight values corresponding to each trained initial network layer.

[0045] Among them, the above training images are related to the application field of the target image processing model. If the target image processing model is used for face recognition, a face image labeled with face feature information can be determined as the training image of the initial image processing model; if the target image processing model is used for image classification, an image labeled with category information can be determined as the training image of the initial image processing model. For example, if the target image processing model is used to identify dogs, an image that has been determined to be of the dog category can be used as the dog training image of the initial image processing model.

[0046] In one embodiment, the server can use the contrastive divergence algorithm to train the above initial image processing model. Among them, each network layer corresponds to a connection weight value. Before training the initial image processing model, the connection weight value is randomly generated. During the training process, the server will fine-tune the connection weight values of each network layer, and then obtain the connection weight values corresponding to each network layer after training.

[0047] S303: Based on the connection weight values corresponding to each initial network layer after training and the preset spiking neuron function, construct the target image processing model corresponding to the image processing model. The target image processing model includes at least one network layer, and each network layer includes at least one spiking neuron for transmitting image data.

[0048] Exemplarily, the above preset spiking neuron function can be the Leaky Integrate-and-fire (LIF) neuron function, and the above image processing model is a DBN model. The neuron function included in the DBN model is siegert. In this case, while ensuring that the network structure and the connection weight values of each network layer of the trained image processing model remain unchanged, the server can construct the LIF model corresponding to the DBN model based on the connection weight values corresponding to each initial network layer after training and the preset spiking neuron function, so as to replace the siegert neurons in the DBN model with LIF neurons.

[0049] Exemplarily, the specific process of replacing the siegert neurons in the DBN model with LIF neurons to obtain the target image processing model, the LIF model, can be seen in Figure 4 as shown. Among them, W I 、W II and W III are the connection weights corresponding to each network layer in the DBN model (i.e., the image processing model). Since the target image processing model is established based on the connection weight values corresponding to each network layer in the image processing model, the connection weights corresponding to each network layer in the target image processing model are also W I 、W II and WIII 。

[0050] S304: When it is detected that an image is input into a pre - constructed target image processing model, obtain the cumulative processed image values of each network layer in at least one network layer included in the target image processing model. Each network layer includes at least one spiking neuron for transmitting image data.

[0051] S305: Compare the cumulative processed image values of each network layer with a preset processed image threshold respectively. If it is compared that the cumulative processed image value of the first network layer is greater than or equal to the preset processed image threshold, then trigger the network layer at the next level of the first network layer to send a pulse to the network layer at the next - next level of the first network layer. The pulse carries the image data corresponding to the image.

[0052] Exemplarily, assume that the target image processing includes a network layer L1 at the first level, a network layer L2 at the second level, a network layer L3 at the third level, and a network layer L4 at the fourth level. The connection weight value corresponding to the network layer L1 at the first level is W I 、the connection weight value corresponding to the network layer L2 at the second level is W II 、the connection weight value corresponding to the network layer L3 at the third level is W III 、the connection weight value corresponding to the network layer L4 at the fourth level is W IV 。The server detects that an image is first input into the target image processing model. The first - input image is an image P1 of 28*28. Since it is the first time an image is input, the processed image value of L1 can be equivalent to the cumulative processed image value of L1 (hereinafter referred to as the first cumulative processed image value). Among them, the first cumulative processed image value is the image value input into L1 (i.e., the gray value of image P1) multiplied by the connection weight value W I of. If the first cumulative processed image value is the image value input into the network layer L2 at the second level, then, in this case, the processed image value of the L2 layer can be equivalent to the cumulative processed image value of L2 (hereinafter referred to as the second cumulative processed image value), which is the first cumulative processed image value * W II 。

[0053] Further, the server can determine whether the cumulative processed image value of L1 is greater than or equal to the preset processed image threshold. If so, trigger the spiking neuron in L2 to send a pulse to the spiking neuron in L3. Numerically, this pulse can be expressed as a fixed value α, and this phenomenon of sending a pulse is called "firing". In this case, this α can be understood as the image value input by the spiking neuron in L2 to the spiking neuron input into L3. Then, since it is the first input, the processed image value of the L3 layer can be equivalent to the cumulative processed image value of L3 (hereinafter referred to as the third cumulative processed image value), which is α * W III 。

[0054] Further, after the spiking neuron in L2 sends a spike to the spiking neuron in L3, the server can continue to detect whether the cumulative processed image value in L2 is greater than or equal to a preset processed image threshold. If so, it triggers the spiking neuron in L3 to transmit a spike α to the spiking neuron in L4. Then, since it is the first input, the processed image value of this L4 layer can be equivalent to the cumulative processed image value of L4 (hereinafter referred to as the third cumulative processed image value), which is α*W IV .

[0055] On the contrary, when the cumulative processed image value input in the current network layer is less than the preset processed image threshold, it can continue to wait for the input of the next image value until the cumulative processed image value of the current network layer is greater than or equal to the preset processed image threshold, and then perform the above-mentioned "discharging" to the network layer connected behind it, so as to realize the transmission of image data.

[0056] S306: Trigger the network layer corresponding to the next lower level of the first network layer to process the image data, and output the processing result for the image according to the processing results of each network layer on the image data.

[0057] Although the neuron type is changed in the target image processing model, and the neuron is changed to a spiking neuron, which speeds up the transmission rate of each network layer to a certain extent, but if it is necessary to increase the adaptive adjustment ability of the target image processing model, the internal plasticity of the neuron can also be added, so that the target image processing model can automatically adjust the internal firing rate of the neuron correspondingly whether the output is very weak or very strong, so that the network can quickly respond to the input. When the input is weak, increase the internal firing rate of the neuron, so that more spikes are transmitted to the next layer, accelerating the network transmission. When the input is strong, correspondingly reduce the internal firing rate of the neuron, so that the network discharge reaches an optimal balanced effect. It is equivalent to the pupil of the human eye. When the light is strong, the pupil shrinks to reduce the amount of light entering. When the light is weak, the pupil is correspondingly enlarged to increase the amount of light entering, so that the amount of light entering is at a balanced level.

[0058] In one embodiment, the server can compare the image values input to each network layer with the preset image reference value respectively. If it is compared that the image value input to the i-th network layer is greater than or equal to the preset image reference value, the image value is increased based on a preset function, and the product of the increased image value and the connection weight value of the i-th network layer is used as the target image value of the network layer of the next lower level of the i-th network layer, where the i-th network layer is any one of the network layers.

[0059] Alternatively, if it is determined through comparison that the image value of the i-th network layer of the input is less than a preset image reference value, the image value is decreased based on a preset function, and the product of the decreased image value and the connection weight value of the i-th network layer is used as the target image value of the next network layer of the i-th network layer of the input.

[0060] In one embodiment, the server may count the image values corresponding to the images input to the target image processing model n times, calculate the average value of the image values input n times, and then determine the obtained average value as the preset image reference value, where n is a positive integer. Alternatively, the specific values of the preset image reference value and n may be preset according to experimental measurement data, and the embodiments of the present application do not make specific limitations thereto.

[0061] Exemplarily, assume that the target image processing includes a network layer L1 of the first level, a network layer L2 of the second level, and a network layer L3 of the third level, and the connection weight value corresponding to the network layer L1 of the first level is W I , the connection weight value corresponding to the network layer L2 of the second level is W II , and the connection weight value corresponding to the network layer L3 of the third level is W III . When the server detects that the image value input to L1 (hereinafter referred to as the input value) is less than the preset image reference value, the input value may be increased through a preset function to obtain an increased input value y, and the product of the y and the connection weight value W I corresponding to L1 is used as the image value input to the network layer L2 of the second level, thereby increasing the input value of the network layer L2 of the second level. Correspondingly, more neurons of the network layer L2 of the second level generate discharge phenomena, and more pulses are transmitted to the network layer L3 of the third level.

[0062] Conversely, when the image value input to the network layer L1 of the first level is greater than or equal to the preset image reference value, the input value may be decreased through a preset function to obtain a decreased input value y, and the product of the decreased y and the connection weight value W I corresponding to L1 is used as the image value input to the network layer L2 of the second level, thereby decreasing the input value of the network layer L2 of the second level. Correspondingly, fewer neurons of the network layer L2 of the second level generate discharge phenomena, reducing the firing rate of the neurons in the network layer L2 of the second level, thereby achieving balance.

[0063] In one embodiment, the preset function includes a preset exponential decay function and the connection weight value corresponding to the i-th network layer.

[0064] Exemplarily, the above preset function may be as shown in Formula 1:

[0065]

[0066] Among them, W i is the connection weight value corresponding to the network layer of the i-th level in the target image processing model, r is the image value input to the network layer of the i-th level, and β is the above-mentioned preset image reference value. The server can increase or decrease the image value output from the network layer of the i-th level based on the preset function corresponding to Formula 1, that is, increase or decrease the image value output from the network layer of the (i + 1)-th level.

[0067] In this way, by balancing the magnitudes of the image values input to each network layer of the target image processing model, the transmission of image data between each network layer can be balanced, the adaptive adjustment ability of the standard image processing model can be increased, and it is ensured that the processing efficiency of the standard image processing model is not affected by the magnitude of the input image value.

[0068] In the embodiment of the present application, the server can train the initial image processing model according to the training images to obtain an image processing model, and the image processing model includes the connection weight values corresponding to each initial network layer after training. Further, the server can construct a target image processing model corresponding to the image processing model based on the connection weight values corresponding to each initial network layer after training and the preset pulse neuron function. When the server detects that an image is input to the pre-constructed target image processing model, it can obtain the cumulative processed image values of each network layer in at least one network layer included in the target image processing model, and each network layer includes at least one pulse neuron for transmitting image data. Further, the server can compare the cumulative processed image values of each network layer with the preset processed image threshold respectively. If it is compared that the cumulative processed image value of the first network layer is greater than or equal to the preset processed image threshold, then trigger the network layer at the next level of the first network layer to send a pulse to the network layer at the next-next level of the first network layer, and the pulse carries the image data corresponding to the image. The first network layer is any one of the at least one network layer. Further, the server can trigger the network layer at the next-next level corresponding to the first network layer to process the image data, and output the processing result for the image according to the processing results of each network layer for the image data. In this way, the image data is transmitted between each network layer in the form of pulses, which can not only reduce the amount of calculation, but also improve the transmission rate of the image data, thereby improving the processing efficiency of the target image processing model.

[0069] The embodiment of the present application also provides an image processing device. The device includes modules for executing the foregoing Figure 1 or Figure 3 the method described above, and is configured in the server. Specifically, referring to Figure 5 , it is a schematic block diagram of the image processing device provided by the embodiment of the present application. The image processing device of this embodiment includes:

[0070] A processing module 50, configured to, when detecting that an image is input into a pre-constructed target image processing model, obtain the cumulative processed image values of each network layer in at least one network layer included in the target image processing model, where each network layer includes at least one spiking neuron for transmitting image data;

[0071] The processing module 50 is further configured to compare the cumulative processed image values of each network layer with a preset processed image threshold respectively. If it is compared that the cumulative processed image value of a first network layer is greater than or equal to the preset processed image threshold, trigger the network layer at the next level of the first network layer to send a pulse to the network layer at the next-next level of the first network layer, where the pulse carries the image data corresponding to the image, and the first network layer is any one of the at least one network layer;

[0072] The processing module 50 is further configured to trigger the network layer at the next-next level corresponding to the first network layer to process the image data, and according to the processing results of each network layer on the image data;

[0073] An output module 51, configured to output a processing result for the image.

[0074] In one embodiment, the processing module 50 is further configured to obtain an initial image processing model, where the initial image processing model includes at least one initial network layer and connection weight values corresponding to each initial network layer in the at least one initial network layer, and each initial network layer includes at least one neuron for transmitting image data; obtain training images, and train the initial image processing model according to the training images to obtain an image processing model, where the image processing model includes connection weight values corresponding to each trained initial network layer; based on the connection weight values corresponding to each trained initial network layer and a preset spiking neuron function, construct a target image processing model corresponding to the image processing model, where the target image processing model includes at least one network layer, and each network layer includes at least one spiking neuron for transmitting image data.

[0075] In one embodiment, the cumulative processed image value of the first network layer is the sum value of at least one processed image value corresponding to the first network layer before the system time, and each processed image value is the product of the image value input into the first network layer at the system time and the connection weight value corresponding to the first network layer.

[0076] In one embodiment, the processing module 50 is further configured to compare the image values input to each network layer with a preset image reference value respectively; if it is compared that the image value input to the i-th network layer is greater than or equal to the preset image reference value, the image value is increased based on a preset function, and the product of the increased image value and the connection weight value of any network layer is used as the target image value of the next-level network layer input to any network layer; if it is compared that the image value input to the i-th network layer is less than the preset image reference value, the image value is decreased based on a preset function, and the product of the decreased image value and the connection weight value of any network layer is used as the target image value of the next-level network layer input to any network layer.

[0077] In one embodiment, the preset function includes a preset exponential decay function and the connection weight value corresponding to the i-th network layer.

[0078] In one embodiment, the processing module 50 is further configured to count the image values corresponding to the images input to the target image processing model n times, where n is a positive integer; calculate the average value of the image values input n times, and determine the obtained average value as the preset image reference value.

[0079] In one embodiment, the processing module 50 is further configured to, if it is compared that the cumulative processed image value of the first network layer is greater than or equal to the preset processed image threshold, wait for the next image value to be input to the first network layer, and trigger the next-level network layer of the first network layer to send a pulse to the next-next-level network layer of the first network layer until the cumulative processed image value corresponding to the first network layer is greater than or equal to the preset processed image threshold.

[0080] It should be noted that the functions of the functional modules of the image processing device described in the embodiments of the present application can be specifically implemented according to Figure 1 or Figure 3 the method in the method embodiments described above, and the specific implementation process can refer to Figure 1 or Figure 3 the relevant descriptions of the method embodiments, which will not be elaborated here.

[0081] Please refer to Figure 6 , Figure 6 which is a schematic block diagram of a server provided by an embodiment of the present application. As Figure 6 shown, the server includes a processor 601, a memory 602, and a network interface 603. The above-mentioned processor 601, memory 602, and network interface 603 can be connected through a bus or other means. In the embodiments of the present application shown Figure 6Take the bus connection as an example. Among them, the network interface 603 is controlled by the processor to send and receive messages, the memory 602 is used to store computer programs, and the computer programs include program instructions. The processor 601 is used to execute the program instructions stored in the memory 602. Among them, the processor 601 is configured to call the program instructions to execute: when it is detected that an image is input into a pre-built target image processing model, obtain the cumulative processed image values of each network layer in at least one network layer included in the target image processing model. Each of the network layers includes at least one spiking neuron for transmitting image data; compare the cumulative processed image values of the respective network layers with a preset processed image threshold respectively. If it is compared that the cumulative processed image value of the first network layer is greater than or equal to the preset processed image threshold, trigger the network layer at the next level of the first network layer to send a pulse to the network layer at the next-next level of the first network layer. The pulse carries the image data corresponding to the image. The first network layer is any one of the at least one network layers; trigger the network layer at the next-next level corresponding to the first network layer to process the image data; output the processing result for the image through the network interface 603 according to the processing results of the respective network layers on the image data.

[0082] In one embodiment, the processor 601 is further configured to obtain an initial image processing model, where the initial image processing model includes at least one initial network layer and connection weight values corresponding to each initial network layer in the at least one initial network layer. Each of the initial network layers includes at least one neuron for transmitting image data; obtain training images, and train the initial image processing model according to the training images to obtain an image processing model, where the image processing model includes connection weight values corresponding to the respective trained initial network layers; based on the connection weight values corresponding to the respective trained initial network layers and a preset spiking neuron function, construct a target image processing model corresponding to the image processing model. The target image processing model includes at least one network layer, and each of the network layers includes at least one spiking neuron for transmitting image data.

[0083] In one embodiment, the cumulative processed image value of the first network layer is the sum value of at least one processed image value corresponding to the first network layer before the system time. Each of the processed image values is the product of the image value input to the first network layer at the system time and the connection weight value corresponding to the first network layer.

[0084] In one embodiment, the processor 601 is further configured to compare the image values input to each network layer with a preset image reference value respectively; if it is compared that the image value input to the i-th network layer is greater than or equal to the preset image reference value, increase the image value based on a preset function, and use the product of the increased image value and the connection weight value of any network layer as the target image value of the next network layer input to any network layer; if it is compared that the image value input to the i-th network layer is less than the preset image reference value, decrease the image value based on a preset function, and use the product of the decreased image value and the connection weight value of any network layer as the target image value of the next network layer input to any network layer.

[0085] In one embodiment, the preset function includes a preset exponential decay function and the connection weight value corresponding to the i-th network layer.

[0086] In one embodiment, the processor 601 is further configured to count the image values corresponding to the images input to the target image processing model for n times, where n is a positive integer; calculate the average value of the image values input for n times, and determine the obtained average value as the preset image reference value.

[0087] In one embodiment, the processor 601 is further configured to, if it is compared that the cumulative processed image value of the first network layer is greater than or equal to the preset processed image threshold, wait for the next image value to be input to the first network layer until the cumulative processed image value corresponding to the first network layer is greater than or equal to the preset processed image threshold, and then trigger the next network layer of the first network layer to send a pulse to the next-next network layer of the first network layer.

[0088] It should be understood that in the embodiments of the present application, the so-called processor 601 may be a central processing unit (CPU), and this processor 601 may also be other general-purpose processors, digital signal processors (DSPs), application specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or this processor may also be any conventional processor, etc.

[0089] The memory 602 may include a read-only memory and a random access memory, and provide instructions and data to the processor 601. A part of the memory 602 may also include a non-volatile random access memory. For example, the memory 602 may also store information about the device type.

[0090] In a specific implementation, the processor 601, the memory 602, and the network interface 603 described in the embodiments of the present application may execute the Figure 1 Or Figure 3 The implementation method described in the method embodiments described above may also execute the implementation method of the image processing device described in the embodiments of the present application, which will not be elaborated here.

[0091] In another embodiment of the present invention, a computer-readable storage medium is provided. The computer-readable storage medium stores a computer program, and the computer program includes program instructions. When executed by a processor, the program instructions implement: when it is detected that an image is input into a pre-constructed target image processing model, obtaining cumulative processed image values of each network layer in at least one network layer included in the target image processing model, where each network layer includes at least one spiking neuron for transmitting image data; comparing the cumulative processed image values of the respective network layers with a preset processed image threshold respectively. If it is compared that the cumulative processed image value of the first network layer is greater than or equal to the preset processed image threshold, triggering the network layer at the next level of the first network layer to send a pulse to the network layer at the next-next level of the first network layer, where the pulse carries the image data corresponding to the image, and the first network layer is any one of the at least one network layer; triggering the network layer at the next-next level corresponding to the first network layer to process the image data; and outputting a processing result for the image according to the processing results of the respective network layers on the image data.

[0092] The computer-readable storage medium may be an internal storage unit of the server described in any of the foregoing embodiments, such as the hard disk or memory of the server. The computer-readable storage medium may also be an external storage device of the server, such as a plug-in hard disk equipped on the server, a Smart Media Card (SMC), a Secure Digital (SD) card, a Flash Card, etc. Further, the computer-readable storage medium may also include both the internal storage unit of the server and the external storage device. The computer-readable storage medium is used to store the computer program and other programs and data required by the server. The computer-readable storage medium may also be used to temporarily store data that has been output or is to be output.

[0093] Those of ordinary skill in the art can understand that all or part of the processes in the methods of the above embodiments can be completed by instructing relevant hardware through a computer program. The program can be stored in a computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above various methods. Among them, the storage medium can be a magnetic disk, an optical disk, a read-only memory (ROM), or a random access memory (RAM), etc.

[0094] The above-disclosed are only some embodiments of the present invention. Of course, the scope of the rights of the present invention cannot be limited thereby. Those of ordinary skill in the art can understand all or part of the processes of the above embodiments, and the equivalent changes made according to the claims of the present invention still fall within the scope covered by the invention.

Claims

1. An image processing method, characterized in that, The method includes: When it is detected that an image is input into a pre-constructed target image processing model, obtaining the cumulative processed image values of each network layer in at least one network layer included in the target image processing model, where each network layer includes at least one spiking neuron for transmitting image data; Comparing the cumulative processed image values of each network layer with a preset processed image threshold respectively. If it is obtained through comparison that the cumulative processed image value of the first network layer is greater than or equal to the preset processed image threshold, triggering the network layer at the next level of the first network layer to send a pulse to the network layer at the next-next level of the first network layer, where the pulse carries the image data corresponding to the image, and the first network layer is any one of the at least one network layer; Triggering the network layer at the next-next level corresponding to the first network layer to process the image data; Outputting a processing result for the image according to the processing results of each network layer on the image data.

2. The method according to claim 1, characterized in that, Before obtaining the cumulative processed image values of each network layer in at least one network layer included in the target image processing model, the method further includes: Obtaining an initial image processing model, where the initial image processing model includes at least one initial network layer and the connection weight values corresponding to each initial network layer in the at least one initial network layer, and each initial network layer includes at least one neuron for transmitting image data; Obtaining a training image and training the initial image processing model according to the training image to obtain an image processing model, where the image processing model includes the connection weight values corresponding to each initial network layer after training; Based on the connection weight values corresponding to each initial network layer after training and a preset spiking neuron function, constructing a target image processing model corresponding to the image processing model, where the target image processing model includes at least one network layer, and each network layer includes at least one spiking neuron for transmitting image data.

3. The method according to claim 2, characterized in that, The cumulative processed image value of the first network layer is the sum value of at least one processed image value corresponding to the first network layer before the system time, and each processed image value is the product of the image value input into the first network layer at the system time and the connection weight value corresponding to the first network layer.

4. The method according to claim 1, characterized in that, The method further includes: Comparing the image values input into each network layer with a preset image reference value respectively; If it is obtained through comparison that the image value input into the i-th network layer is greater than or equal to the preset image reference value, increasing the image value based on a preset function, and taking the product of the increased image value and the connection weight value of the i-th network layer as the target image value input into the network layer at the next level of the i-th network layer, where the i-th network layer is any one of the each network layer; If it is obtained through comparison that the image value input into the i-th network layer is less than the preset image reference value, decreasing the image value based on a preset function, and taking the product of the decreased image value and the connection weight value of the i-th network layer as the target image value input into the network layer at the next level of the i-th network layer.

5. The method according to claim 4, characterized in that, The preset function includes a preset exponential decay function and the connection weight value corresponding to the i-th network layer.

6. The method according to claim 4, characterized in that, Before comparing the image values input to each network layer with a preset image reference value respectively, the method further includes: Counting the image values corresponding to the images input to the target image processing model n times, where n is a positive integer; Calculating the average value of the image values input n times, and determining the obtained average value as the preset image reference value.

7. The method according to claim 1, characterized in that, After comparing the cumulative processed image values of each network layer with a preset processed image threshold respectively, the method further includes: If it is compared that the cumulative processed image value of the first network layer is greater than or equal to the preset processed image threshold, then wait for the next image value to be input to the first network layer until the cumulative processed image value corresponding to the first network layer is greater than or equal to the preset processed image threshold, and trigger the operation that the network layer at the next level of the first network layer sends a pulse to the network layer at the next-next level of the first network layer.

8. An image processing apparatus, characterized in that, The device includes: A processing module, configured to, when detecting that an image is input to a pre-constructed target image processing model, obtain the cumulative processed image values of each network layer in at least one network layer included in the target image processing model, and each network layer includes at least one pulse neuron for transmitting image data; The processing module is further configured to compare the cumulative processed image values of each network layer with a preset processed image threshold respectively. If it is compared that the cumulative processed image value of the first network layer is greater than or equal to the preset processed image threshold, then trigger the network layer at the next level of the first network layer to send a pulse to the network layer at the next-next level of the first network layer, and the pulse carries the image data corresponding to the image, and the first network layer is any one of the at least one network layer; The processing module is further configured to trigger the network layer at the next-next level corresponding to the first network layer to process the image data, and according to the processing results of each network layer on the image data; An output module, configured to output the processing result for the image.

9. A server, characterized in that, It includes a processor and a memory, the processor and the memory are connected to each other. Wherein, the memory is used to store a computer program, the computer program includes program instructions, and the processor is configured to call the program instructions to execute the method according to any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, and the computer program is executed by a processor to implement the method according to any one of claims 1-7.

Images