A method and system for perceiving object motion trends based on spiking neural networks

Abstract

The present invention relates to a method and system for perceiving object motion trends based on a spiking neural network. The method comprises obtaining an index of the region in which an object is currently located; binary encoding the region index to generate pulse information; an FPGA converting the pulse information into frame information and sending it to PAICORE 2.0 via a PCIe interface; and the PAICORE 2.0 parsing the frame information and performing inference to generate the object's position information at the next moment. The PAICORE 2.0 is a brain-like chip equipped with a spiking neural network model, which is used to implement spiking neural network functions. This method addresses the problems of traditional algorithms with lengthy computation times, high hardware system power consumption, and low utilization of information temporal and spatial correlations.

Description

Technical Field

[0001] The present invention relates to the field of novel computer technology, and in particular to a method and system for perceiving object motion trends based on a pulse neural network. Background Art

[0002] Most existing object motion trend perception systems are implemented by combining traditional target detection and tracking algorithms. Traditional target detection algorithms primarily include background subtraction, inter-frame subtraction, and optical flow for static backgrounds, and block-based motion estimation and compensation for dynamic backgrounds. Their primary task is to extract changing regions from the background image within a sequence of images. Traditional tracking algorithms first effectively represent the target, then employ similarity metrics to match the moving target with the frame image, and finally employ search algorithms to find the optimal position. Common prediction algorithms include Kalman filtering and particle filtering.

[0003] Existing technologies have the following shortcomings: 1. Traditional target detection and prediction algorithms have lengthy computation times. Hardware implementations of these algorithms can take hundreds of milliseconds, making them unsuitable for predicting and sensing the trends of high-speed moving targets. 2. Hardware systems consume high power. Traditional algorithm deployments do not utilize dedicated chips, resulting in high computational resources and power consumption. 3. They poorly utilize the spatiotemporal correlation of information. Traditional systems for sensing the movement trends of objects rely on continuous, single-shot calculations, disregarding the temporal correlation of information. Summary of the Invention

[0004] In order to solve at least one of the problems mentioned in the above background technology, the present invention proposes a method and system for perceiving the motion trend of an object based on a pulse neural network, in which the PC determines the index of the area where the object is currently located through event information, and generates pulse information through binary encoding. FPGA acts as a converter between pulse information and frames required by PAICORE2.0, and sends the frames to PAICORE2.0 through the PCIe interface (2.5GT / s). PAI CORE2.0 parses the frame information, sends the pulse information to the internally built pulse neural network for inference, and obtains the position information of the object at the next moment. This information is sent to the FPGA through a frame and then to the PC. The corresponding position index is parsed to realize the perception of the object's motion trend.

[0005] To achieve the above objectives, the technical solutions adopted by the present invention include:

[0006] A method for sensing object motion trends based on a spiking neural network, comprising:

[0007] Get the index of the area where the object is currently located;

[0008] Binary-encode the region index to generate pulse information;

[0009] FPGA converts the pulse information into frame information and sends it to PAICORE2.0 via the PCIe interface;

[0010] The PAICORE 2.0 parses the frame information and performs inference to generate the position information of the object at the next moment.

[0011] Furthermore, the PAICORE2.0 is a brain-like chip deployed with a pulse neural network model, which is used to realize the pulse neural network function.

[0012] Furthermore, the obtaining of the index of the area where the object is currently located includes: the PC determining the index of the area where the object is currently located through event information, wherein the method for obtaining the event information includes obtaining it using a sensor.

[0013] Furthermore, the method further includes: transmitting the position information back to the PC via the FPGA, and parsing to obtain a position index corresponding to the position information.

[0014] Furthermore, the method also includes: using DQN to train the neural network and converting the neural network into the pulse neural network.

[0015] Furthermore, the PAICORE 2.0 supports the LIF neuron model.

[0016] The present invention also relates to an object motion trend perception system based on a pulse neural network, comprising:

[0017] Input module, used to obtain the index of the area where the object is currently located;

[0018] Binary-encode the region index to generate pulse information;

[0019] FPGA converts the pulse information into frame information and sends it to PAICORE2.0 via the PCIe interface;

[0020] The processing module is used for the PAICORE 2.0 to parse the frame information and perform inference to generate the position information of the object at the next moment.

[0021] The present invention also relates to a computer-readable storage medium having a computer program stored thereon, and the computer program implements the above method when executed by a processor.

[0022] The present invention also relates to an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the above-mentioned method when executing the computer program.

[0023] The present invention also relates to a computer program product, comprising a computer program and / or instructions, characterized in that the computer program and / or instructions implement the steps of the above method when executed by a processor.

[0024] The beneficial effects of the present invention are:

[0025] 1. Compared with the calculation delay of hundreds of milliseconds of traditional algorithms, the response speed of 10ms / each feedback is achieved by utilizing the characteristics of PAICORE2.0 and event information processing.

[0026] 2. Low hardware power consumption. When the object motion trend perception system is running, the power consumption of PAICORE2.0 is less than 1W.

[0027] 3. Utilizing the event correlation of information, similar to the way biological organisms process information, the event stream is binary-encoded into pulse information. The pulse neural network uses the temporal and spatial correlation of pulses for reasoning, thereby accelerating perception. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] Figure 1 The figure is a flow chart of the object motion trend perception method based on pulse neural network of the present invention.

[0029] Figure 2 Schematic diagram of a simulation scenario according to a preferred embodiment of the present invention.

[0030] Figure 3 This is a flowchart of the table tennis interception experiment procedure in a preferred embodiment of the present invention.

[0031] Figure 4 Schematic diagram of the constraints of the interception motion mode of the slider in the preferred embodiment of the present invention.

[0032] Figure 5 This is a neural network structure diagram of a preferred embodiment of the present invention.

[0033] Figure 6 Schematic diagram of the DQN algorithm principle of the preferred embodiment of the present invention.

[0034] Figure 7 This is a flowchart of neural network training according to a preferred embodiment of the present invention.

[0035] Figure 8 This is a schematic diagram of the abstract process from biological neurons to spiking neurons in a preferred embodiment of the present invention.

[0036] Figure 9 This is the abstracted spiking neuron model of the preferred embodiment of the present invention.

[0037] Figure 10 Schematic diagram comparing the calculation modes of ANN and SNN in the preferred embodiment of the present invention.

[0038] Figure 11 This is a schematic diagram of the structure of the object motion trend perception system based on the pulse neural network of the present invention.

[0039] Figure 12 This is a simplified wiring diagram of the input module of the preferred embodiment of the present invention.

[0040] Figure 13 This is a flowchart of the input module program of the preferred embodiment of the present invention.

[0041] Figure 14 This is a flowchart of the processing module program of the preferred embodiment of the present invention.

[0042] Figure 15 This is a flowchart of the output module program of the preferred embodiment of the present invention.

[0043] Figure 16 This is the final measured system parameter diagram of the present invention. DETAILED DESCRIPTION

[0044] To more clearly illustrate the technical solutions of the embodiments of this specification, the following briefly describes the drawings required for describing the embodiments. Obviously, the drawings described below are merely examples or embodiments of this specification. Those skilled in the art can apply this specification to other similar scenarios based on these drawings without inventive effort. Unless otherwise apparent from the context or otherwise noted, the same reference numerals in the figures represent the same structure or operation.

[0045] It should be understood that the terms "system," "device," "unit," and / or "module" used herein are a method for distinguishing different components, elements, parts, portions, or assemblies at different levels. However, if other terms can achieve the same purpose, the terms may be replaced by other expressions.

[0046] As used in this specification and claims, unless the context clearly indicates otherwise, the words "a," "an," "an," and / or "the" do not refer to the singular but also include the plural. Generally speaking, the terms "comprises" and "include" only indicate the inclusion of the steps and elements specifically identified, and these steps and elements do not constitute an exclusive list. A method or apparatus may also include other steps or elements.

[0047] Flowcharts are used throughout this specification to illustrate the operations performed by systems according to embodiments of this specification. It should be understood that preceding or following operations do not necessarily need to be performed in exact order. Instead, the steps may be processed in reverse order or simultaneously. Furthermore, other operations may be added to these processes, or one or more operations may be removed from these processes.

[0048] The first aspect of the present invention relates to a process of steps as follows Figure 1 The object motion trend perception method based on the pulse neural network shown includes:

[0049] Get the index of the area where the object is currently located;

[0050] Binary-encode the region index to generate pulse information;

[0051] FPGA converts the pulse information into frame information and sends it to PAICORE2.0 via the PCIe interface;

[0052] Specifically, in an embodiment of the present application, the PC determines the index of the area where the object is currently located through event information, where the event information can be obtained by a sensor, such as an event camera. The area where the object is located is determined by the number of events in a certain area, and the area index is recorded in real time, such as (6,5) and (4,12). The area index is binary-encoded to generate pulse information. The FPGA acts as a converter between the pulse information and the frames required by PAICORE2.0, and sends the frames to PAICORE2.0 through the PCIe interface (2.5GT / s).

[0053] The PAICORE 2.0 parses the frame information and performs inference to generate the position information of the object at the next moment.

[0054] Specifically, in the embodiment of the present application, Python programming is used to construct a scene to simulate intercepting a ping-pong ball in reality. The ping-pong ball is shot into the acrylic plate from the outside at any angle and any speed. The slider predicts the movement trend and final arrival position of the ping-pong ball, and intercepts the ping-pong ball at the corresponding predicted position. Figure 2 shown.

[0055] In this process, pygame.init() is mainly used to initialize pygame; pygame.display.set_mode([1280,800]) sets the background screen size to 1280×800, which is consistent with the event camera resolution when the hardware is deployed; pygame.Rect(1280 / 2-80,800-160,160,160) creates a square ping-pong ball with a width and height of 160; pygame.Rect(-80,0,320,40) creates a rectangular slider with a width of 320 and a height of 40; pygame.font.Font() sets the font; pygame.time.Clock( ) sets the clock; pygame.draw.ellipse() and pygame.draw.rect() draw the ping-pong ball and slider on the background respectively. The ellipse() command can draw the ping-pong ball as a circle, and the rect() command can draw the slider as a bar; pygame.display.flip() updates the text texture; pygame.image.save() saves the current screen; pygame.display.update() updates the entire scene; cv2.imread() reads image information.

[0056] After building the basic scene, we need to set the ping-pong ball and slider in motion, completing the task of using the slider to intercept the ping-pong ball. First, we need to set a reward rule. If the slider reaches the predicted position ahead of time and successfully intercepts the ping-pong ball, the agent receives a significant reward. If the slider fails to intercept the ball or fails to predict its movement, the agent receives a penalty. This rule provides the basis for DQN reinforcement learning and consists of three elements: state, action, and reward.

[0057] Use the pygame library and cv2 library to program the above-mentioned table tennis and slider movements and reward and penalty rules. The program flow chart is as follows Figure 3 shown.

[0058] In the ping-pong ball interception task, this embodiment gives the player 100 chances: 100 ping-pong balls are launched from the outside world at the slider. Each time the slider intercepts the ball, it's considered a success, which facilitates future calculations of the interception success rate. Initially, the ball is launched at a random angle and speed. The ball and slider then move sequentially according to the constraints. After each movement, rewards and penalties are applied using the following rules: First, determine whether the ball collides with the slider or the bottom edge. If no collision occurs, calculate the current distance between the ball and the slider using the statement reward_1 = 1 - (abs(ball.centerx - paddle.centerx) + abs(ball.centery - paddle.centery)) / 1820. The closer the distance, the greater the reward_1. The maximum reward_1 value is 1, and the minimum reward_1 value is 0. If a collision occurs, we need to determine whether the ping-pong ball collided with the slider or the bottom edge. If the ping-pong ball collided with the slider, the agent receives a bonus point. This bonus point is calculated using the statement reward = -1 / 24 * abs(ball.centerx - paddle.centerx) + 10. The maximum reward value is 10 and the minimum value is 0. This means that when the slider and the ping-pong ball collide, the closer their centers are, the better the collision. We hope that the slider and the ping-pong ball will collide in this manner in the future. If the ping-pong ball collided with the bottom edge, the agent receives a deduction point, calculated using the statement reward = -5. The total score is then calculated using reward_all = reward / 10.0 * 0.7 + reward_1 * 0.3. This completes the acquisition of a set of <state, action, reward>.

[0059] The main constraints of the model are concentrated on the movement of the ping-pong ball. In addition, this embodiment also imposes some constraints on the interception movement of the slider. First, the process of each ping-pong ball ejection must be constrained. At the beginning, the ping-pong ball is ejected at a random angle and speed, but it must be ensured that the ping-pong ball does not move outside the scene during the entire process. Therefore, the ejection angle and speed are restricted here. The constraints are as follows: Figure 4As shown, each time a ping-pong ball is launched, regardless of whether the slider successfully intercepts it, it will be refreshed to a random initial position. These positions are arbitrarily selected for balls 1, 2, and 3. This process is constrained by the statements ball.centerx = random.randint(80,1200) and ball.centery = 720. For example, if ball 2 is refreshed to position 2, the allowed range of the launch angle is between 2→4 and 2→6. The minimum launch angle is zeta1, and the maximum is zeta2. The random angle selection process is described by the statements zeta1 = math.atan2((80-ball.centery),(80-ball.centerx)), zeta2 = math.atan2((80-ball.centery),(1200-ball.centerx)), and speed_direction = random.uniform(zeta1,zeta2). To suit the deployment of the table tennis interception system, the ejection speed is arbitrarily selected between 6 and 18, which is mapped to 24 cm / s to 128 cm / s in the real world, and is constrained by the statement speed = random.uniform(6,18).

[0060] After the ping pong ball is launched, it will move in a straight line on the acrylic board. In reality, the ping pong ball will be affected by friction during its movement on the acrylic board, and its speed will continue to decrease slightly. However, since the movement distance of the ping pong ball on the acrylic board is very small, the reduction in the speed of the ping pong ball is limited. Therefore, this patent believes that the movement form of the ping pong ball can be regarded as uniform linear motion. According to the above discussion, ball.centerx + = math.cos (speed_direction) * speed and ball.centery + = math.sin (speed_direction) * speed are used to constrain the movement form of the ping pong ball to ensure that the movement of the ping pong ball is uniform linear motion. During the movement, the movement angle and speed of the ping pong ball on the background curtain board are consistent with the angle and speed of each ejection.

[0061] In reality, the movement of the slider is a combination of three types of motion. The entire movement process (controlled by a servo drive) undergoes accelerated linear motion, uniform linear motion, and decelerated linear motion, ultimately moving from one position to another. However, since the slider's movement speed is much faster than that of a table tennis ball, it needs to move from one position to another at an extremely fast speed. Therefore, this embodiment regards this motion as a one-dimensional jump motion. The instantaneous displacement of the slider is 160° when it moves once. Because the width of the screen is 1280°, the slider can jump back and forth in 8 horizontal positions. This embodiment calls this type of motion "octant jump."

[0062] In DQN reinforcement learning, a neural network needs to be used instead of the Q-table, so a neural network needs to be constructed. The neural network structure of this embodiment is as follows: Figure 5 shown.

[0063] The entire network consists of 7 fully connected layers and 6 activation layers. ReLu() is the activation function; QuantTrainLinear() is the quantized fully connected layer, where the two parameters are the input feature quantity and the output feature quantity.

[0064] The network input is pulse information converted from the index. The index is converted from binary code into 11 bits of binary horizontal coordinates for the ping-pong ball, 10 bits of binary vertical coordinates for the slider, and 11 bits of binary horizontal coordinates for the slider, for a total of 32 bits of pulse information. This pulse information is replicated four times, generating 128 bits of pulse information as the network input. The network output is three feature quantities, representing one rightward movement of the slider, one stationary movement of the slider, and one leftward movement of the slider.

[0065] The DQN algorithm is used for neural network training. The principle of the DQN algorithm is as follows Figure 6 shown.

[0066] Input the current pulse sequence and the next pulse sequence into the prediction network and the real network respectively, calculate the Q value of the three actions respectively, take the maximum Q value of the prediction network and the real network as the mean square error, use this mean square error as the loss function for back propagation, update the parameters of the prediction network, and update the parameters of the prediction network to the real network at regular intervals. Write a program to train the neural network according to the principle of the DQN algorithm. The program flow chart is as follows Figure 7 shown.

[0067] Each DQN neural network training session involves 5 million state updates. This state update refers to the process of updating all parameters involved in the movement of the slider and the table tennis ball. To fully explore various states, not only is the neural network required to determine the slider's action after each table tennis movement (here, the action refers to the slider's three different states: one leftward movement, one stationary movement, and one rightward movement), but random actions are also required in the early stages of training to randomly cause the slider to move left, remain stationary, or move right. This prevents DQN reinforcement learning from sometimes becoming stuck in a local optimal solution. Of course, this embodiment intends that random assignment of slider actions only occurs during the initial training phase. Therefore, the condition random.random(0,100)<100*(decline**(learn_time / 1000)) is used as a criterion to determine whether random slider actions are required during training. If random slider actions are not required, the current state is fed into the neural network, and the slider's action is derived through inference. This action is then used to manipulate the slider and the table tennis ball according to the rules, generating the following states for future use. Finally, we need to organize and store the current state, next state, action, and reward into an array. This information will be used later when iterating the neural network parameters. Finally, we need to continuously determine whether the current score is the highest. If a new highest score is found, we need to store the current model and use it as the local optimal model.

[0068] In this embodiment, the steps for updating the parameters of the prediction network are as follows: First, take any segment of data in the array (including the current state, next state, action, and benefit). This segment of data contains 32 sets of <current state, next state, action, and benefit>. That is, the neural network training process is carried out with 32 sets of <current state, next state, action, and benefit> as the minimum unit. Then, the Q values ​​of the 32 sets of prediction networks and the actual network are calculated, which are respectively referred to as q and q_truth, where q = net1(b_s).gather(1,b_a), q_next = net2(b_s_).detach().max(1)[0].reshape(b_size,1), q_truth = b_r+gama*q_next*(1-b_done). The loss function is the minimum mean square error of q and q_truth, and then backpropagation is performed to update the prediction network parameters. After the training is completed, the trained prediction network model is saved as the current optimal model. During the training process, the parameters of the prediction network must be assigned to the real network every once in a while. The real network does not participate in the training process. Only the prediction network uses the gradient descent method to perform back propagation to update the network parameters.

[0069] After the network training is completed, the neural network needs to be converted into a spiking neural network. The spiking neural network is composed of spiking neurons. The spiking neuron model is abstracted from biological neurons. The biological neuron dendrites receive neurotransmitters released by the synapses of the previous neuron, generating electrical signal stimulation. After being transmitted to the cell body, an action potential is generated. The action potential is transmitted to the synapse through the axon and continues to release neurotransmitters to the next neuron, completing the transmission process of mutual conversion from electrical signals to chemical signals. The abstract process from biological neurons to spiking neurons is as follows: Figure 8 shown.

[0070] The abstracted spike neuron model is as follows Figure 9 As shown in the figure, the input is a pulse sequence. Multiple pulse inputs are accumulated with corresponding weights to generate membrane potential in the neuron cell body. When this membrane potential exceeds a certain threshold, the cell body emits a pulse.

[0071] PAICORE2.0 supports the LIF neuron model, which can be expressed by the following formula:

[0072]

[0073]

[0074]

[0075]

[0076]

[0077] Where t represents the time step, n represents the layer of the network, and w ij represents the weight from the jth neuron in the front layer to the ith neuron in the back layer, o j Represents the output of the jth neuron, a value of 1 indicates that a pulse is emitted, and a value of 0 indicates that no pulse is emitted. i represents the membrane potential of the i-th neuron. f(·) is the decay function, and g(·) is the activation function, representing the leakage and spike firing processes, respectively. After a spike is fired, the membrane potential returns to zero. If no spikes are fired, the membrane potential leaks naturally over time.

[0078] Comparison of the calculation modes of ANN (artificial neural network) and SNN (spiking neural network) Figure 10As shown, the ANN and SNN have the same network structure, weights, and biases. The difference is that the input and output of the ANN are single floating-point numbers, while the input and output of the SNN are sequences of pulses. Therefore, if an ANN needs to be converted to an SNN, appropriate timestep and Vthr need to be selected for the SNN. The larger the timestep, the closer the accuracy of the converted SNN is to the original ANN. However, considering efficiency issues, the timestep cannot be too large. Vthr can be selected as the maximum value that the membrane potential has ever reached, or a (0 < a < 1) times the maximum value of the membrane potential. The best Vthr can also be selected through multiple attempts.

[0079] In the conversion scheme of the embodiment, the timestep is selected as 64, and Vthr is determined by an iterative method. Vthr is the value when the inference result of each layer of the ANN and the inference result of each layer of the SNN are the most similar. Then, the SNN is finally generated through the processes of creating a quantizer, creating a converter, parsing the ANN using the init_dag method of the transformer, quantizing using the quantize method of the transformer, and generating the SNN using the generate_snn method of the transformer. Converting the SNN into a computational graph and then into a network-on-chip can obtain the specific mapping of the network on PAICORE2.0.

[0080] To complete the network configuration by deploying the mapping to PAICORE2.0, a configuration frame needs to be sent to PAICORE2.0. After completing the network configuration, input data needs to be sent to PAICORE2.0. The input accepted by PAICORE2.0 is in the form of a work frame. The result after each inference needs to be converted into a frame and sent to the FPGA, and then sent to PAICORE2.0 through the PCIe interface. After completing one input, PAICORE2.0 parses the frame and performs inference, and then outputs the work frame back through the FPGA. In this way, the operation of the motion trend perception system can be realized by looping.

[0081] This application can be applied to various scenarios of object motion trend perception. Taking the table tennis interception scenario as an example, with an event camera as the sensor and inputting event information, considering generality, the spiking neural network can be deployed on existing common platforms, such as Jetson Xavier NX.

[0082] On the other hand, this invention also relates to an object motion trend perception system based on a spiking neural network, the structure of which is as Figure 2 shown, including:

[0083] An input module, used to obtain the index of the current area where the object is located;

[0084] Generating pulse information by binary encoding the area index;

[0085] FPGA converts the pulse information into frame information and sends it to PAICORE2.0 via the PCIe interface;

[0086] The processing module is used for the PAICORE 2.0 to parse the frame information and perform inference to generate the position information of the object at the next moment.

[0087] The input module is further configured to allow the PC to determine the index of the area where the object is currently located through event information, wherein the method for obtaining the event information includes obtaining it using a sensor.

[0088] The system further includes an output module for transmitting the position information back to the PC via the FPGA, and parsing the information to obtain a position index corresponding to the position information.

[0089] Specifically, in this embodiment, the entire system is divided into three modules, which sequentially transmit and process information: the input module, the processing module, and the output module. The system is used to predict the movement of a table tennis ball (a table tennis ball is shot from the outside toward the slide rail and then moves in a two-dimensional uniform linear motion on the acrylic plate) and intercept the moving table tennis ball with the slider.

[0090] The input module is an event camera. The input module is mainly used to obtain the real-time position of the ping-pong ball and the slider on the acrylic board, and input the event stream information to the AI ​​edge computing device; the processing module is an AI edge computing device. The processing module is mainly used to process the event stream input by the input module and convert it into a pulse stream. Through the pulse neural network calculation deployed on it, it predicts the movement trajectory of the ping-pong ball and the interception position of the slider, and sends the slider interception position instruction to the output module; the output module consists of a single-chip microcomputer, a servo driver, a servo motor, a slide rail and a slider, and a switching power supply. The single-chip microcomputer receives the slider interception position instruction sent by the processing module, forms a control signal and sends it to the servo driver in real time. The servo driver drives the servo motor to move the slider to the corresponding interception position. The simplified wiring diagram of the system is as follows Figure 12 shown.

[0091] Input module program flow chart as follows Figure 13 shown.

[0092] Processing module program flow chart as follows Figure 14 shown.

[0093] Output module program flow chart as follows Figure 15 shown.

[0094] The final measured system parameters are as follows Figure 16 shown.

[0095] By using this system, the above-mentioned calculation and processing method can be executed and the corresponding technical effects can be achieved.

[0096] An embodiment of the present invention also provides a computer-readable storage medium capable of implementing all steps of the object motion trend perception method based on a pulse neural network in the above-mentioned embodiment. The computer-readable storage medium stores a computer program, which, when executed by a processor, implements all steps of the object motion trend perception method based on a pulse neural network in the above-mentioned embodiment.

[0097] An embodiment of the present invention also provides an electronic device for executing the above-mentioned method. As an implementation device of the method, the electronic device has at least a processor and a memory, and in particular, the memory stores the data and related computer programs required for executing the method, and the processor calls the data and programs in the memory to execute all the steps of the implementation method and obtains the corresponding technical effect.

[0098] Preferably, the electronic device may include a bus architecture, which may include any number of interconnected buses and bridges, and the bus will include various circuits linked together by one or more processors and memories. The bus may also link together various other circuits such as peripheral devices, voltage regulators, and power management circuits, which are all well known in the art and, therefore, will not be described further herein. The bus interface provides an interface between the bus and the receiver and transmitter. The receiver and transmitter can be the same component, namely a transceiver, which provides a unit for communicating with various other systems over a transmission medium. The processor is responsible for managing the bus and general processing, while the memory can be used to store data used by the processor when performing operations.

[0099] Additionally, the electronic device may further include components such as a communication module, an input unit, an audio processor, a display, and a power supply. The processor (or controller, operating control) employed may include a microprocessor or other processor device and / or logic device, which receives input and controls the operation of various components of the electronic device. The memory may be one or more of a cache, flash memory, a hard drive, removable media, volatile memory, non-volatile memory, or other suitable devices, and may store the aforementioned data and information. It may also store programs for executing the relevant information, and the processor may execute the programs stored in the memory to implement information storage or processing. The input unit is used to provide input to the processor, and may, for example, be a keypad or touch input device. The power supply is used to provide power to the electronic device. The display is used to display objects such as images and text, and may, for example, be an LCD display. The communication module is a transmitter / receiver that sends and receives signals via an antenna. The communication module (transmitter / receiver) is coupled to the processor to provide input signals and receive output signals, similar to the case of a conventional mobile communication terminal. Based on different communication technologies, multiple communication modules can be provided in the same electronic device, such as a cellular network module, a Bluetooth module, and / or a wireless local area network module. The communication module (transmitter / receiver) is also coupled to a speaker and a microphone via an audio processor to provide audio output via the speaker and receive audio input from the microphone, thereby implementing common telecommunications functions. The audio processor may include any suitable buffer, decoder, amplifier, etc. In addition, the audio processor is also coupled to a central processing unit, enabling local recording via the microphone and playback of stored audio via the speaker.

[0100] It will be understood by those skilled in the art that embodiments of the present invention may be provided as methods, systems, or computer program products. Thus, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0101] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowcharts and / or block diagrams, as well as combinations of processes and / or blocks in the flowcharts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowcharts and / or block diagrams. Figure 1 a process or multiple processes and / or boxes Figure 1 A system that specifies the functions of a box or boxes.

[0102] These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture including an instruction system that is implemented in the process. Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0103] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 The present invention is described in detail below. ...

[0104] The above description is merely a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in the present invention are intended to be covered by the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be based on the scope of protection of the claims.

Claims

1. A method for perceiving object motion trends based on a pulse neural network, characterized in that: include: Get the index of the area where the object is currently located; Binary-encode the region index to generate pulse information; FPGA converts the pulse information into frame information and sends it to PAICORE2.0 through the PCIe interface; The PAICORE 2.0 parses the frame information and performs inference to generate the position information of the object at the next moment.

2. The method according to claim 1, wherein The PAICORE2.0 is a brain-like chip deployed with a pulse neural network model, which is used to realize the pulse neural network function.

3. The method according to claim 1, wherein The obtaining of the index of the area where the object is currently located includes: the PC determining the index of the area where the object is currently located through event information, wherein the method for obtaining the event information includes obtaining it using a sensor.

4. The method according to claim 2, wherein The method further includes: transmitting the position information back to the PC via the FPGA, and parsing to obtain a position index corresponding to the position information.

5. The method according to any one of claims 1 to 4, characterized in that The method further includes: using DQN to train a neural network, and converting the neural network into the pulse neural network.

6. The method according to claim 2, wherein PAICORE 2.0 supports the LIF neuron model.

7. An object motion trend perception system based on a pulse neural network, characterized in that: include: Input module, used to obtain the index of the area where the object is currently located; Binary-encode the region index to generate pulse information; FPGA converts the pulse information into frame information and sends it to PAICORE2.0 through the PCIe interface; The processing module is used for the PAICORE 2.0 to parse the frame information and perform inference to generate the position information of the object at the next moment.

8. A computer-readable storage medium, characterized in that The storage medium stores a computer program, which implements the method according to any one of claims 1 to 6 when executed by a processor.

9. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the method according to any one of claims 1 to 6 is implemented.

10. A computer program product comprising a computer program and / or instructions, characterized in that When the computer program and / or instructions are executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images