A method and apparatus for edge imaging of real-time image pulsing

Abstract

The application provides a kind of real-time image pulsing edge imaging method and device, by obtaining input biasing electric signal, the electric signal is amplified, and the electric signal to be sampled is obtained, for each pixel unit corresponding to the to-be-processed signal included in the to-be-sampled electric signal, the to-be-processed signal is repeatedly sampled for a predetermined number of times, and the probabilistic pulse signal is obtained, the probabilistic pulse signal is spliced, the spliced pulse signal is obtained, and based on the spliced pulse signal, and the pixel coordinates obtained in advance, generate the image to be identified. In this way, the electric signal corresponding to the pixel array can be converted in real time, so as to reduce the delay in the image transmission process, solve the defect that the image data is pulsed, improve the user's experience.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of data processing, and particularly relates to a real-time image pulsing edge imaging method and device. BACKGROUND

[0002] In the background of the big data era, the classic architecture of separating memory and processor in the traditional data processing method brings the Von Neumann bottleneck problem, and this processing method has gradually been unable to meet people's growing data processing needs. Compared with it, the information activity of the human brain nervous system has the characteristics of large-scale parallelism, distributed storage and processing, and self-organization, self-adaptation and self-learning, and the information storage and processing has no obvious boundary. As the third generation of neural networks designed to simulate the mechanism of biological neurons, the spiking neural network (SNN, Spiking Neuron Networks) has more brain-like characteristics in connection mode, information processing mechanism and synaptic weight learning method compared with the current popular numerical deep learning (Deep Learning) neural network. The simulated neurons in the spiking neural network are closer to the actual characteristics: neurons will only be activated when the membrane potential reaches the threshold, and the spiking neural network considers the pulse generation time, enhances the ability to process space-time data, and theoretically can obtain stronger computing and recognition ability than the second generation of neural networks.

[0003] The spiking neural network processes pulse time series, and there are two ways to obtain the pulse sequence: the first way is to convert the image into a pulse sequence according to a certain encoding method (for example: threshold encoding method, frequency encoding method, Gaussian difference method, etc.). The second way is directly derived from a special neuromorphic vision sensor. However, the principle and process of the neuromorphic vision sensor are not mature enough, and the price is high, so its application is not universal. Therefore, the currently used pulse time series is mainly derived from the encoding generation method.

[0004] The current technical means for converting images into pulse sequences has problems such as serious information loss and low coding efficiency, which is one of the reasons for the low recognition rate of the pulse neural network. Taking the above-mentioned several encoding methods as an example, the threshold encoding method first sets a threshold, and encodes according to the principle that the pixels exceeding the threshold excite pulses and the pixels below the threshold do not excite pulses. Because the encoding mechanism is too simple, the pulse sequence after encoding loses a large amount of necessary image information such as texture and color compared with the original image. The frequency encoding method first sets the frequency of the pulse excited by each pixel to be proportional to the intensity value of the pixel, and restores the intensity of the original pixel by counting the number of pulses excited by each pixel in a unit time. This encoding method needs a long time, which causes the low recognition efficiency of the overall pulse neural network. The Gaussian difference method first detects the local contrast of the image, and determines the excitation order of the pulse according to the local contrast strength. This method introduces a large amount of Gaussian noise, which further affects the recognition effect of the pulse neural network.

[0005] In addition, even if the pulse neural network is the third generation neural network, a large amount of time is still needed for model construction and training. For example, in specific application scenarios such as automatic driving, images need to be generated in time and sent to the computing system for processing. If the image is obtained first and then sent to the back-end computing platform for pulse processing, the computing power of the computing platform is consumed, and the recognition speed of the pulse neural network is reduced. It can be seen that there is a delay in the process of pulse processing of image data at present, which cannot meet the use demand of users. SUMMARY

[0006] The present application provides a kind of real-time image pulse edge imaging method, to solve the defect that there is delay in the process of pulse processing of image data in prior art, cannot meet the use demand of user, improve the use experience of user.

[0007] The present application provides a kind of real-time image pulse edge imaging method, the method comprises:

[0008] obtain the electrical signal after input biasing;

[0009] amplify the electrical signal to obtain a to-be-sampled electrical signal;

[0010] for each pixel unit corresponding to the to-be-sampled electrical signal, repeat sampling the to-be-processed signal a preset number of times to obtain a probabilistic pulse signal;

[0011] splice the probabilistic pulse signal to obtain a spliced pulse signal, and generate a to-be-recognized image based on the spliced pulse signal and a pre-acquired pixel coordinate.

[0012] According to the real-time image pulsing edge imaging method provided by the application, the step of obtaining a probabilistic pulse signal by repeatedly sampling a preset number of times on a to-be-processed signal corresponding to each pixel unit included in the to-be-sampled electrical signal comprises:

[0013] In each sampling process of the to-be-processed signal, a voltage corresponding to the to-be-processed signal, a driving voltage of a transistor and a voltage divider voltage are used to determine a level signal corresponding to the sampling process, wherein the voltage divider voltage is a voltage obtained based on a power supply voltage and a magnetic tunnel junction voltage.

[0014] According to the real-time image pulsing edge imaging method provided by the application, the step of obtaining a probabilistic pulse signal by repeatedly sampling a preset number of times on a to-be-processed signal corresponding to each pixel unit included in the to-be-sampled electrical signal comprises:

[0015] Based on the characteristics of the transistor, the driving voltage and the voltage corresponding to the to-be-processed signal, it is determined whether the transistor is turned on or not;

[0016] In the case that the transistor is turned on, a level signal is output based on the voltage divider voltage;

[0017] In the case that the transistor is not turned on, a low-level signal is output.

[0018] According to the real-time image pulsing edge imaging method provided by the application, the step of outputting a level signal based on the voltage divider voltage comprises:

[0019] In the case that the voltage divider voltage is a high level, a low-level signal is output;

[0020] In the case that the voltage divider voltage is a high level, a high-level signal is output.

[0021] According to the real-time image pulsing edge imaging method provided by the application, the probabilistic pulse signal comprises a plurality of pulse signals corresponding to to-be-processed signals, and each to-be-processed signal corresponds to a preset number of pulse signals;

[0022] The step of splicing the probabilistic pulse signal to obtain a spliced pulse signal comprises:

[0023] For each to-be-processed signal, the preset number of pulse signals corresponding to the to-be-processed signal are spliced to obtain a spliced pulse signal.

[0024] According to the real-time image pulsing edge imaging method provided by the application, the step of generating a to-be-identified image based on the spliced pulse signal and a pixel coordinate obtained in advance comprises:

[0025] For each to-be-processed signal, the pixel coordinates corresponding to the to-be-processed signal are added to the spliced pulse signal corresponding to the to-be-processed signal.

[0026] The spliced pulse signal after adding the pixel coordinates is sorted to generate a to-be-identified image.

[0027] According to the edge imaging method for real-time image pulsing provided by the application, after the step of generating a to-be-identified image based on the spliced pulse signal and the pre-acquired pixel coordinates, the method further comprises:

[0028] The to-be-identified image is input into a pre-trained pulse neural network for image recognition to obtain an identification result.

[0029] The application further provides an edge imaging device for real-time image pulsing, which comprises:

[0030] An acquisition unit is configured to acquire an electrical signal after input biasing.

[0031] An amplification and gating unit is configured to perform amplification processing on the electrical signal to obtain a to-be-sampled electrical signal.

[0032] A repeated sampling unit is configured to perform repeated sampling on a to-be-processed signal corresponding to each pixel unit included in the to-be-sampled electrical signal for a preset number of times to obtain a probabilistic pulse signal.

[0033] A generation unit is configured to perform splicing processing on the probabilistic pulse signal to obtain a spliced pulse signal, and generate a to-be-identified image based on the spliced pulse signal and pre-acquired pixel coordinates.

[0034] According to the edge imaging device for real-time image pulsing provided by the application, the repeated sampling unit is specifically configured to determine a level signal corresponding to each sampling process of the to-be-processed signal based on a voltage corresponding to the to-be-processed signal, a driving voltage of a transistor and a divided voltage in the sampling process, wherein the divided voltage is a voltage acquired based on a power supply voltage and a magnetic tunnel junction voltage.

[0035] According to the edge imaging device for real-time image pulsing provided by the application, the repeated sampling unit comprises a transistor, a magnetic tunnel junction, an inverter and a power supply.

[0036] The voltage corresponding to the to-be-processed signal of the time is an input voltage of a gate of the transistor, a drain of the transistor is connected with one end of the magnetic tunnel junction, the other end of the magnetic tunnel junction is connected with a negative electrode of the power supply, a positive electrode of the power supply is connected with a source of the transistor, an input end of the inverter is connected with the source of the transistor, and the supply voltage is a voltage provided by the power supply.

[0037] For each time of sampling the to-be-processed signal, the corresponding level signal of the time is output through the inverter based on the voltage corresponding to the to-be-processed signal of the time, the driving voltage of the transistor and the voltage division voltage.

[0038] The application provides a real-time image pulse edge imaging method and device, which comprises the following steps: obtaining an input bias electric signal, amplifying the electric signal to obtain a to-be-sampled electric signal, repeatedly sampling a to-be-processed signal corresponding to each pixel unit included in the to-be-sampled electric signal for a preset number of times to obtain a probabilistic pulse signal, splicing the probabilistic pulse signal to obtain a spliced pulse signal, and generating a to-be-identified image based on the spliced pulse signal and a pixel coordinate obtained in advance.

[0039] In this way, the electric signal corresponding to the pixel array can be converted in real time, thereby reducing the delay in the image transmission process, solving the defect that the image data is pulse processed with delay, and meeting the use requirements of users. BRIEF DESCRIPTION OF DRAWINGS

[0040] In order to more clearly illustrate the technical solutions in the application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiments or the prior art description. Obviously, the drawings in the following description are some embodiments of the application, and other drawings can also be obtained by those skilled in the art without creative labor.

[0041] Figure 1 is one of the flowcharts of the real-time image pulse edge imaging method provided by the application;

[0042] Figure 2 is one of the structural schematic diagrams of the real-time image pulse edge imaging device provided by the application;

[0043] Figure 3 is the structural schematic diagram of the random circuit provided by the application;

[0044] Figure 4 is the schematic diagram of the probabilistic pulse signal provided by the application;

[0045] Figure 5Fig. 2 is a structural schematic diagram of the edge imaging device for real-time image pulsing provided by the present application;

[0046] Figure 6 Fig. 3 is a structural schematic diagram of the edge imaging device for real-time image pulsing provided by the present application.

[0047] Reference signs:

[0048] 310: transistor; 320: magnetic tunnel junction; 330: inverter; 501: color filter; 502: pixel sensor; 503: bias generation circuit; 504: amplification circuit; 505: gating switch; 506: timing generation circuit; 507: pre-inverter; 508: random circuit; 509: buffer. DETAILED DESCRIPTION

[0049] In order to make the objects, technical solutions and advantages of the present application clearer, the technical solutions in the present application will be described clearly and completely below with reference to the drawings in the present application. Obviously, the described embodiments are some but not all of the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative work fall within the protection scope of the present application.

[0050] In order to reduce the delay caused by the pulsing processing of image data to meet the use requirements of users, the present application provides an edge imaging method and device for real-time image pulsing, which will be described below with reference to the drawings Figure 1 The edge imaging method for real-time image pulsing provided by the present application will be described below:

[0051] As shown in Fig. 1, the present application provides an edge imaging method for real-time image pulsing, which comprises the following steps. Figure 1

[0052] S101, acquiring an electrical signal after input bias.

[0053] In order to eliminate the non-uniformity existing in the generation process of each pixel in the image sensor and improve the imaging quality, the input bias can be input to the image acquisition device, so that the electrical signal after input bias can be acquired.

[0054] The image acquisition device can be a CMOS (Complementary Metal Oxide Semiconductor) image sensor, and of course other image sensors (pixel sensors) can also be set according to actual use requirements, which are not limited here.

[0055] ​In an embodiment, the bias can be generated by a bias generation circuit, and the inputted bias corresponds to different signals according to different image sensors. For example, in the case of a CCD (Charge Coupled Device) camera as the image sensor, a corresponding current signal can be provided as the inputted bias. For another example, in the case of a CMOS image sensor as the image sensor, a corresponding voltage signal can be provided as the inputted bias. This is reasonable and is not specifically limited herein.

[0056] The step S101 is described below by taking a CMOS image sensor as an example of the image acquisition device: Since the CMOS image sensor does not have color resolution capability, in order to obtain the electrical signal after the inputted bias, the visible light can be filtered so that the pixel units corresponding to the CMOS image sensor distinguish three basic colors of red, green and blue, so as to obtain the electrical signal after the inputted bias.

[0057] In an embodiment, the CMOS image sensor can be compatible with a color filter arrangement device of a common image format, so that the color filter arrangement device (color filter) can be used to filter the visible light, for example, the color filter arrangement device can be arranged in front of the CMOS image sensor. The common image format can include Bayer, RGBE, RGBW, CYYM, CYGM, etc., which are reasonable and are not specifically limited herein.

[0058] That is, the CMOS image sensor can be connected with the color filter arrangement device, and the CMOS image sensor can be connected with the bias generation circuit, so that the electrical signal after the inputted bias can be obtained, wherein the electrical signal after the inputted bias is the electrical signal corresponding to the pixel array of the CMOS image sensor, and the pixel array includes a plurality of pixel units.

[0059] As an embodiment, the electrical signal corresponding to the current image after the inputted bias can be obtained in real time during the process of acquiring the current image by the image acquisition device, and then the corresponding electrical signal is selected from the electrical signal corresponding to the current image after the inputted bias by using a preset strobe mode as the electrical signal after the inputted bias. The preset strobe mode includes a row-by-row selection mode and a column-by-column selection mode.

[0060] In this embodiment, the electrical signal after the inputted bias is the electrical signal corresponding to a row of pixel units in the pixel array, or the electrical signal after the inputted bias is the electrical signal corresponding to a column of pixel units in the pixel array.

[0061] As another implementation, the electrical signal corresponding to the current image of the input bias can be obtained in real time during the image acquisition device acquires the current image, and the obtained electrical signal corresponding to the current image of the input bias is taken as the electrical signal after the input bias. In this implementation, the electrical signal after the input bias is the electrical signal corresponding to the pixel array,

[0062] S102, amplifying the electrical signal to obtain a to-be-sampled electrical signal.

[0063] After obtaining the electrical signal after the input bias, the electrical signal corresponding to the pixel unit is small and cannot meet the preset voltage requirement, wherein the preset voltage requirement is the voltage requirement for subsequent repeated sampling.

[0064] Therefore, in order to facilitate subsequent repeated sampling to obtain the probabilistic pulse signal, the electrical signal can be amplified to obtain a to-be-sampled electrical signal. In one implementation, when the electrical signal after the input bias is the electrical signal corresponding to a row of pixel units in the pixel array, or the electrical signal after the input bias is the electrical signal corresponding to a column of pixel units in the pixel array, the electrical signal after the input bias can be amplified to obtain the to-be-sampled electrical signal.

[0065] In another implementation, when the electrical signal after the input bias is the electrical signal corresponding to the pixel array, the electrical signal after the input bias can be amplified and gated. Specifically, the electrical signal is amplified to obtain a to-be-gated electrical signal, and then a corresponding electrical signal is selected from the to-be-gated electrical signal as a to-be-sampled electrical signal using a preset gating method, wherein the preset gating method includes a row-by-row selection method and a column-by-column selection method.

[0066] The to-be-sampled electrical signal is the amplified electrical signal corresponding to a row of pixel units in the pixel array, or the to-be-sampled electrical signal is the amplified electrical signal corresponding to a column of pixel units in the pixel array.

[0067] S103, for each pixel unit included in the to-be-sampled electrical signal, a to-be-processed signal corresponding to the pixel unit is repeatedly sampled a preset number of times to obtain a probabilistic pulse signal.

[0068] After obtaining the to-be-sampled electrical signal, for each pixel unit included in the to-be-sampled electrical signal, a to-be-processed signal corresponding to the pixel unit is repeatedly sampled a preset number of times to obtain a probabilistic pulse signal. Through the preset number of repeated samplings, the fidelity of the to-be-identified image can be improved to facilitate subsequent acquisition of a to-be-identified image with higher fidelity.

[0069] The probability pulse signal includes preset number of pulse signals corresponding to the to-be-sampled electrical signal. For example, the preset number is 8, the to-be-sampled electrical signal includes electrical signals corresponding to L pixel units, that is, the number of to-be-processed signals is L, then there are 8 pulse signals for each to-be-processed signal, and the probability pulse signal includes 8xL pulse signals.

[0070] In S104, the probability pulse signal is spliced to obtain a spliced pulse signal, and a to-be-recognized image is generated based on the spliced pulse signal and a pixel coordinate obtained in advance.

[0071] After the probability pulse signal is obtained, the probability pulse signal can be spliced to obtain a spliced pulse signal, and a to-be-recognized image is generated based on the spliced pulse signal and a pixel coordinate obtained in advance. The edge imaging method for real-time image pulse provided in the embodiment can convert the electrical signal corresponding to the pixel array in real time, thereby reducing the delay in the image transmission process, solving the defect that the pulse processing of the image data has a delay, so as to meet the use requirement of the user.

[0072] As an embodiment of the embodiment, the step of sampling the to-be-processed signal corresponding to each pixel unit included in the to-be-sampled electrical signal for a preset number of times to obtain a probability pulse signal can include:

[0073] In each sampling process of the to-be-processed signal, a level signal corresponding to the sampling is determined based on the voltage corresponding to the to-be-processed signal, the driving voltage of the transistor and the voltage divider voltage. The voltage divider voltage is a voltage obtained based on the supply voltage and the magnetic tunnel junction voltage.

[0074] The transistor can be an NMOS (Negative channel Metal Oxide Semiconductor, N-type metal oxide semiconductor) transistor, or a PMOS (Positive channel Metal Oxide Semiconductor, P-type metal oxide semiconductor) transistor.

[0075] A magnetic tunnel junction (MTJ) is a sandwich structure device, including a reference layer, a tunnel barrier layer and a free layer from top to bottom. Among them, the free layer and the fixed layer are two layers of magnetic thin film, forming a nanometer magnet, and the insulating layer is an oxide film, which separates the electron migration in the free layer and the fixed layer as a tunnel barrier. By applying an external electromagnetic field control, the magnetic moment direction of the free layer can be switched to parallel or anti-parallel direction with the fixed layer. If it is parallel state, the resistance of the magnetic tunnel junction is low, and the resistance is high in anti-parallel state, so as to use the resistance high and low as the logic state. When the tunnel barrier in the magnetic tunnel junction is low, the parallel state and the anti-parallel state appear random jump phenomenon due to thermal noise, generating random output characteristics.

[0076] It can be seen that in the embodiment, the corresponding level signal can be determined based on the voltage corresponding to the to-be-processed signal, the driving voltage of the transistor and the voltage division voltage each time, so that the pulse signal of the preset number of segments corresponding to each to-be-processed signal can be obtained, so as to obtain the to-be-identified image with high fidelity.

[0077] As an embodiment of the present application, in the process of sampling the to-be-processed signal each time, the step of determining the corresponding level signal each time based on the voltage corresponding to the to-be-processed signal, the driving voltage of the transistor and the voltage division voltage can include:

[0078] Based on the characteristics of the transistor, the driving voltage and the voltage corresponding to the to-be-processed signal each time, it is determined whether the transistor is turned on.

[0079] In the case of NMOS transistor, if the voltage corresponding to the to-be-processed signal each time is greater than the driving voltage, it is determined that the transistor is turned on, and if the voltage corresponding to the to-be-processed signal each time is not greater than the driving voltage, it is determined that the transistor is not turned on.

[0080] In the case of PMOS transistor, if the voltage corresponding to the to-be-processed signal each time is not greater than the driving voltage, it is determined that the transistor is turned on, and if the voltage corresponding to the to-be-processed signal each time is greater than the driving voltage, it is determined that the transistor is not turned on.

[0081] In the case that the transistor is turned on, the level signal is output based on the voltage division voltage.

[0082] In the case that the transistor is determined to be turned on, the level signal can be output based on the voltage division voltage.

[0083] In one embodiment, in the case that the voltage division voltage is high, a low level signal is output. In the case that the voltage division voltage is low, a high level signal is output.

[0084] In a case where it is determined that the transistor is not turned on, a low-level signal is output.

[0085] It can be seen that in the embodiment, whether the transistor is turned on can be determined first, and different ways are used to determine the level signal according to the on-off state of the transistor, so that the probabilistic pulse signal is obtained.

[0086] As an implementation of the embodiment of the present application, the probabilistic pulse signal can include pulse signals corresponding to a plurality of to-be-processed signals, and each to-be-processed signal corresponds to a preset number of pulse signals.

[0087] The step of splicing the probabilistic pulse signal to obtain the spliced pulse signal can include:

[0088] For each to-be-processed signal, the preset number of pulse signals corresponding to the to-be-processed signal are spliced to obtain the spliced pulse signal.

[0089] In an implementation, the preset number of pulse signals corresponding to the to-be-processed signal can be spliced in an order corresponding to repeated sampling of the to-be-processed signal to obtain the spliced pulse signal.

[0090] In another implementation, the preset number of pulse signals corresponding to the to-be-processed signal can be spliced in an order corresponding to sampling times of the pulse signals to obtain the spliced pulse signal.

[0091] As an implementation of the embodiment of the present application, the step of generating a to-be-recognized image based on the spliced pulse signal and the pixel coordinates obtained in advance can include:

[0092] For each to-be-processed signal, the pixel coordinates corresponding to the to-be-processed signal are added to the spliced pulse signal corresponding to the to-be-processed signal.

[0093] In an implementation, after obtaining the spliced pulse signal corresponding to each to-be-processed signal, the pixel coordinates corresponding to the to-be-processed signal can be added to the spliced pulse signal corresponding to the to-be-processed signal to obtain the spliced pulse signal after adding the pixel coordinates.

[0094] For example, the pixel coordinates corresponding to the to-be-processed signal A are (256, 256), and (000011111111, 000011111111) can be added in front of the spliced pulse signal corresponding to the to-be-processed signal A, where (000011111111, 000011111111) is the pixel address of the spliced pulse signal corresponding to the to-be-processed signal A. In (000011111111, 000011111111), the binary number at the front position represents a row, and the binary number at the rear position represents a column, and each coordinate corresponds to 12 bits, which can be compatible with the size of mainstream image sensors. Of course, the number of bits of the binary number in the coordinate can be set according to actual conditions, which is not specifically limited here.

[0095] The spliced pulse signal after adding the pixel coordinates is sorted to generate a to-be-recognized image.

[0096] After obtaining the spliced pulse signal after adding the pixel coordinates, the spliced pulse signal after adding the pixel coordinates can be sorted according to the position corresponding to the pixel coordinates, so that a to-be-recognized image can be generated.

[0097] As an embodiment of the present application, after the above step of generating a to-be-recognized image based on the spliced pulse signal and the pixel coordinates obtained in advance, the above method can further include:

[0098] The to-be-recognized image is input into a pre-trained pulse neural network for image recognition to obtain a recognition result.

[0099] In an embodiment, the to-be-recognized image can be input into the pulse neural network based on a wired network or a wireless network, so that the pulse neural network can perform image recognition on the to-be-recognized image to obtain a recognition result.

[0100] A real-time image pulsing edge imaging device provided by an embodiment of the present application is described below. The real-time image pulsing edge imaging device described below can be correspondingly referred to the real-time image pulsing edge imaging method described above.

[0101] As shown in Figure 2 The present application provides a real-time image pulsing edge imaging device, which comprises:

[0102] The acquisition unit 210 is configured to acquire the input biasing electrical signal.

[0103] The amplification and gating unit 220 is configured to amplify the electrical signal to obtain a to-be-sampled electrical signal.

[0104] The repeated sampling unit 230 is configured to repeatedly sample a to-be-processed signal corresponding to each pixel unit included in the to-be-sampled electrical signal for a preset number of times, to obtain a probabilistic pulse signal.

[0105] The generating unit 240 is configured to splice the probabilistic pulse signal to obtain a spliced pulse signal, and generate a to-be-identified image based on the spliced pulse signal and a pixel coordinate obtained in advance.

[0106] As an implementation of an embodiment of the present application, the repeated sampling unit 230 is specifically configured to determine a level signal corresponding to each sampling of the to-be-processed signal based on a voltage corresponding to the to-be-processed signal, a driving voltage of a transistor, and a voltage divided by a voltage divider during the sampling.

[0107] As an implementation of an embodiment of the present application, as shown in Figure 3 The repeated sampling unit 230 can be a random circuit, which can include a transistor 310, a magnetic tunnel junction 320, an inverter 330, and a power supply (not labeled in the figure).

[0108] The voltage corresponding to the to-be-processed signal during the sampling is an input voltage of a gate of the transistor 310, a drain of the transistor 310 is connected to one end of the magnetic tunnel junction 320, the other end of the magnetic tunnel junction 320 is connected to a negative electrode (-VDD1) of the power supply, a positive electrode (VDD1) of the power supply is connected to a source of the transistor 310, an input end of the inverter 330 is connected to the source of the transistor 310, and the input end of the inverter 330 is also connected to the positive electrode of the power supply. The voltage supplied by the power supply is the voltage supplied by the power supply.

[0109] In Figure 3 the positive electrode of the power supply is connected to the source of the transistor 310 through a matching resistor. That is, one end of the matching resistor is connected to the positive electrode of the power supply, the other end of the matching resistor is connected to the source of the transistor 310, and the input end of the inverter 330 is connected to the other end of the matching resistor.

[0110] The matching resistor can function as a voltage divider together with the magnetic tunnel junction 320, that is, the voltage divided by the voltage divider can be a voltage obtained based on the supply voltage, the voltage of the magnetic tunnel junction 320, and the voltage of the matching resistor. As an implementation, the matching resistor is an adjustable resistor, and the non-uniformity introduced by the magnetic tunnel junction 320 during manufacturing can be eliminated by adjusting the resistance value of the matching resistor.

[0111] During each sampling of the signal to be processed, based on the voltage of the signal to be processed, the driving voltage of the transistor 310, and the voltage divider voltage, the inverter 330 outputs the corresponding level signal.

[0112] The logic of the random circuit can be as follows: Based on the characteristics of transistor 310, the voltage corresponding to the current signal to be processed, and the driving voltage, it is determined whether transistor 310 is turned on. Transistor 310 acts as a switch in the random circuit; specifically, transistor 310 can determine whether to turn on based on the voltage corresponding to the current signal to be processed input to its gate.

[0113] When transistor 310 is turned on, the voltage input to the input terminal of inverter 330 is a voltage divider, therefore inverter 330 can output a level signal based on the voltage divider. When the voltage divider is high, inverter 330 outputs a low-level signal; when the voltage divider is low, inverter 330 outputs a high-level signal.

[0114] When transistor 310 is not turned on, the voltage input to the input terminal of inverter 330 is the supply voltage. Therefore, inverter 330 can output a low-level signal based on the supply voltage.

[0115] The control function of the random circuit is manifested as follows: when the voltage corresponding to the signal to be processed input to the gate of transistor 310 is less than the driving voltage, the random circuit outputs a low-level signal. When the voltage corresponding to the signal to be processed input to the gate of transistor 310 is not less than the driving voltage, the random circuit outputs a random-level signal.

[0116] The above-mentioned random circuit can still achieve self-oscillation by relying on the magnetic tunnel junction 320 without external triggering, thus completely solving the problem of data misalignment caused by trigger signal mismatch.

[0117] Probabilized pulse signals can be like Figure 4 As shown, in Figure 4 In the case where the input voltage of the random circuit is relatively small, that is, when the voltage corresponding to the signal to be processed at this time, input to the gate of transistor 310 is relatively small, the average value of the probabilistic pulse signal output by the random circuit is 0.22. Figure 4 (as shown in (a)). As the voltage corresponding to the signal to be processed input to the gate of transistor 310 gradually increases, the average value of the probabilistic pulse signal output by the random circuit can increase from 0.22 to 0.49. Figure 4(b) shown). In the case where the voltage of the signal to be processed corresponding to this time input to the gate of the transistor 310 continues to increase, the average of the probabilistic pulse signal output from the stochastic circuit can rise from 0.49 to 0.75 (c) shown). Figure 4 (c) shown).

[0118] wherein, Figure 4 is merely illustrative, and in Figure 4 the value corresponding thereto is the average of the probabilistic pulse signal in 100 μs of sampling time, Figure 5 the ordinate is the normalized value.

[0119] As an embodiment of the present application, the acquisition unit 210 can include a color filter arrangement device, an image acquisition device, and a bias generation circuit unit. The color filter arrangement device is used to filter the visible light (ambient light) into monochromatic light.

[0120] The image acquisition device can be an image sensor. Since the CMOS type image sensor can only respond to light intensity and cannot respond to color, a color filter can be provided so that the corresponding electrical signal of the pixel array of the image sensor has a positive correlation with the intensity of the monochromatic light, so as to subsequently obtain the image to be recognized.

[0121] The image sensor can acquire the signal of the image to be processed. The pixel array corresponding to the image sensor can generate a corresponding electrical signal under light excitation. The image acquisition unit 410 is specifically used for: the visible light is decomposed into monochromatic light by the color filter and enters the pixel unit of the pixel array corresponding to the image sensor, and then photoelectric effect can occur to generate a corresponding electrical signal, and the electrical signal after inputting the bias by the bias generation circuit unit is obtained.

[0122] In the case where the image sensor is a COMS image sensor, the COMS image sensor has the ability to distinguish each pixel unit, and the COMS image sensor can realize one-to-one correspondence between the electrical signal corresponding to each pixel unit and the pixel coordinate.

[0123] As an embodiment of the present application, the probabilistic pulse signal includes a plurality of pulse signals corresponding to the to-be-processed signals, and each to-be-processed signal corresponds to a preset number of pulse signals.

[0124] The generation unit 240 can include a buffer. The buffer is used to splice the preset number of pulse signals corresponding to each to-be-processed signal to obtain spliced pulse signals.

[0125] That is, the buffer can temporarily store the pulse signals.

[0126] As an implementation of the embodiment of the present application, the buffer is specifically configured to add the pixel coordinates corresponding to each to-be-processed signal to the spliced pulse signal corresponding to the to-be-processed signal; and the spliced pulse signal after adding the pixel coordinates is sorted to generate a to-be-identified image.

[0127] The buffer can add the pixel coordinates to the spliced pulse signal after splicing the preset number of pulse signals of one to-be-processed signal, the spliced pulse signal after adding the pixel coordinates can be taken as a data block, the buffer can transmit the data block out, and the buffer can be cleared to wait for the data corresponding to the next to-be-processed signal. In this way, the efficiency of data transmission can be improved.

[0128] The to-be-identified image is composed of a plurality of pixel units, and the coordinate information of the pixel units can determine the overall performance of the to-be-identified image. The to-be-identified image can be obtained by restoring the coordinate information of each pixel unit.

[0129] For example, the preset number is 8, the duration corresponding to the sampling process of each time is 40 clock cycles, the spliced pulse signal obtained by the buffer is S, and S is taken as a high-level signal in 40 clock cycles if there is a high-level signal in any clock cycle corresponding to the 8 pulse signals. S includes 40 bits. After adding the pixel coordinates to S, S is a 64-bit data block.

[0130] As an implementation of the embodiment of the present application, the device can further include an identification unit configured to input the to-be-identified image to a pre-trained pulse neural network for image identification to obtain an identification result after generating the to-be-identified image based on the spliced pulse signal and pre-acquired pixel coordinates.

[0131] As an implementation of the embodiment of the present application, the amplification and gating unit 220 includes a gating switch and an amplification circuit unit. The real-time image pulsing edge imaging device can further include a pre-inverter timing generation circuit unit.

[0132] The timing generation circuit unit is configured to provide timing signals to the gating switch, the buffer and the image acquisition device, and specifically to generate timing signals in a row-by-row gating mode and a column-by-column gating mode. The gating switch is configured to gate the pixel signal corresponding to a specific position when receiving the timing signal sent by the timing generation circuit unit.

[0133] For example, the preset number is A, the image sensor includes a DxL pixel array, where D is the number of rows, L is the number of columns, and D and L are positive integers. The trigger frequency of the image sensor is f, the trigger frequency of the gating switch is not less than AxDxLxf, and the trigger frequency of the buffer is not less than DxLxf.

[0134] In one embodiment, the timing generation circuit unit can encode the row / column information of the pixel array into the number of high level signals or the number of low level signals and store into the timing signal. Correspondingly, the gating switch can count the high level signals or the low level signals, so that the parameter information corresponding to the electrical signal of the pixel unit which needs to be gated and amplified can be obtained, wherein the parameter information can correspond to the number of high level signals or the number of low level signals. Further, the position of the pixel unit connected to the input end of the random circuit can be obtained.

[0135] The above-mentioned amplification circuit unit comprises an amplifier, a first capacitor and a second capacitor, wherein one end of the first capacitor is connected with the image acquisition device, the other end of the first capacitor is connected with the negative input end of the amplifier, the other end of the first capacitor is also connected with one end of the second capacitor, the other end of the second capacitor is connected with the output end of the amplifier, and the positive input end of the amplifier is connected with a conventional voltage.

[0136] The above-mentioned amplification circuit unit can amplify the obtained electrical signal after biasing by C1 / C2 times, wherein C1 is the capacitance value of the first capacitor and C2 is the capacitance value of the second capacitor. Since the above-mentioned amplification circuit unit inverts the electrical signal, the level signal inputted by the input end of the pre-inverter can be inverted by the above-mentioned pre-inverter, that is, the to-be-sampled electrical signal is inverted.

[0137] The following will take Figure 5 as an example to introduce the edge imaging device for real-time image pulsing provided by the embodiment of the present application:

[0138] As shown in Figure 6 , the edge imaging device for real-time image pulsing provided by the embodiment of the present application comprises a color filter 501, a pixel sensor 502, a bias generation circuit 503, an amplification circuit 504, a gating switch 505, a timing generation circuit 506, a pre-inverter 507, a random circuit 508 and a buffer 509.

[0139] Among them, the color filter 501 is a color filter arrangement device, the pixel sensor 502 is an image sensor (image acquisition device), the bias generation circuit 503 is a bias generation circuit unit, the amplification circuit 504 is an amplification circuit unit, and the timing generation circuit 506 is a timing generation circuit unit.

[0140] The color filter 501 is connected with the pixel sensor 502, the pixel sensor 502 is connected with the bias generation circuit 503, the pixel sensor 502 is also connected with the timing generation circuit 506, and the pixel sensor 502 is also connected with the input end of the amplification circuit 504, that is, one end of the first capacitor.

[0141] The output of amplifier circuit 504 is connected to gating switch 505, which is also connected to timing generation circuit 506. Gating switch 505 is also connected to the input of pre-inverter 507, whose output is connected to the input of random circuit 508, specifically to the gate of the transistor included in random circuit 508. The output of random circuit 508 is connected to buffer 509, meaning the output of the inverter is connected to buffer 509. Buffer 509 is also connected to timing generation circuit 506.

[0142] As can be seen, the edge imaging device for real-time image pulse generation provided by this invention can achieve edge imaging with real-time image pulse generation in a compact manner. It not only meets the requirements of spiking neural networks for the image data to be recognized, but also avoids the computational pressure caused by using pure algorithmic methods to pulse the image. At the same time, this device transmits the data immediately after pulse generation of a single pixel unit, thus improving the efficiency of network transmission.

[0143] like ​ The diagram shown is a schematic of an edge imaging device for real-time image pulse generation provided in an embodiment of the present invention, including a row gating circuit 601, a timing generation circuit 602, a pixel array 603, a bias circuit 604, a column gating circuit 605, an amplifier 606, a random circuit 607, a buffer 608, and an output circuit 609.

[0144] The row selection circuit 601, timing generation circuit 602, pixel array 603, bias 604, column selection circuit 605, and amplifier 606 are used to acquire the input biased electrical signal, amplify the electrical signal, and obtain the electrical signal to be sampled.

[0145] The output circuit 609 is used to input the image to be recognized into a pre-trained spiking neural network for image recognition and obtain the recognition result.

[0146] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method of edge imaging of real-time image pulsing, characterized by, The method comprises: acquiring an input biased electrical signal; amplifying the electrical signal to obtain a to-be-sampled electrical signal; for each pixel unit corresponding to a to-be-processed signal included in the to-be-sampled electrical signal, repeatedly sampling the to-be-processed signal a preset number of times to obtain a probabilistic pulse signal; splicing the probabilistic pulse signal to obtain a spliced pulse signal, and generating a to-be-identified image based on the spliced pulse signal and a pixel coordinate obtained in advance; the step of repeatedly sampling the to-be-processed signal a preset number of times to obtain a probabilistic pulse signal for each pixel unit corresponding to a to-be-processed signal included in the to-be-sampled electrical signal comprises: in each sampling process of the to-be-processed signal, determining a level signal corresponding to this sampling process based on a voltage corresponding to the to-be-processed signal, a driving voltage of a transistor, and a voltage dividing voltage, wherein the voltage dividing voltage is a voltage obtained based on a supply voltage and a magnetic tunnel junction voltage.

2. The method of claim 1, wherein, the step of determining a level signal corresponding to each sampling process of the to-be-processed signal based on a voltage corresponding to the to-be-processed signal, a driving voltage of a transistor, and a voltage dividing voltage comprises: determining whether the transistor is turned on based on the characteristics of the transistor, the driving voltage, and the voltage corresponding to the to-be-processed signal in this sampling process; in the case that the transistor is turned on, outputting a level signal based on the voltage dividing voltage; in the case that the transistor is not turned on, outputting a low level signal.

3. The method of claim 2, wherein, the step of outputting a level signal based on the voltage dividing voltage comprises: in the case that the voltage dividing voltage is a high level, outputting a low level signal; in the case that the voltage dividing voltage is a low level, outputting a high level signal.

4. The method of claim 1, wherein, The probabilistic pulse signal comprises pulse signals corresponding to a plurality of to-be-processed signals, and each to-be-processed signal corresponds to a preset number of pulse signals; the step of splicing the probabilistic pulse signal to obtain a spliced pulse signal comprises: for each to-be-processed signal, splicing the preset number of pulse signals corresponding to the to-be-processed signal to obtain a spliced pulse signal.

5. The method of claim 4, wherein, the step of generating a to-be-identified image based on the spliced pulse signal and a pixel coordinate obtained in advance comprises: for each to-be-processed signal, adding the pixel coordinate corresponding to the to-be-processed signal to the spliced pulse signal corresponding to the to-be-processed signal; sorting the spliced pulse signal after adding the pixel coordinate to generate a to-be-identified image.

6. The method of edge imaging of real-time image pulsing of any of claims 1-5, wherein, after the step of generating a to-be-identified image based on the spliced pulse signal and a pixel coordinate obtained in advance, the method further comprises: inputting the to-be-identified image into a pre-trained pulse neural network for image recognition to obtain a recognition result.

7. An edge imaging device for real-time image pulsing, characterized by The device comprises: an acquisition unit configured to acquire an input biased electrical signal; an amplification and gating unit configured to amplify the electrical signal to obtain a to-be-sampled electrical signal; The repeated sampling unit is configured to perform repeated sampling on a to-be-processed signal corresponding to each pixel unit included in the to-be-sampled electrical signal for a preset number of times, to obtain a probabilistic pulse signal. The generating unit is configured to perform splicing processing on the probabilistic pulse signal to obtain a spliced pulse signal, and generate a to-be-identified image based on the spliced pulse signal and a pixel coordinate obtained in advance. The repeated sampling unit is specifically configured to determine, in each sampling process of the to-be-processed signal, a level signal corresponding to the sampling process based on a voltage corresponding to the to-be-processed signal in the sampling process, a driving voltage of a transistor, and a voltage division voltage, wherein the voltage division voltage is a voltage obtained based on a supply voltage and a magnetic tunnel junction voltage.

8. The real-time image pulsing edge imaging apparatus of claim 7, wherein, The repeated sampling unit includes a transistor, a magnetic tunnel junction, an inverter, and a power supply. The voltage corresponding to the to-be-processed signal in the sampling process is an input voltage of a gate of the transistor, a drain of the transistor is connected with one end of the magnetic tunnel junction, the other end of the magnetic tunnel junction is connected with a negative electrode of the power supply, a positive electrode of the power supply is connected with a source of the transistor, an input end of the inverter is connected with the source of the transistor, and the supply voltage is a voltage provided by the power supply. In each sampling process of the to-be-processed signal, the level signal corresponding to the sampling process is output by the inverter based on the voltage corresponding to the to-be-processed signal in the sampling process, the driving voltage of the transistor, and the voltage division voltage.

Images