Hardware acceleration system and method for parallel computing data stream

By using a parallel computing data stream hardware acceleration system, data is collected by hand sensors and radar, and then processed in parallel and fed back in real time. This solves the problem that existing equipment cannot remotely control instruments, and enables efficient and timely data processing and safe operation.

CN117369640BActive Publication Date: 2026-11-17ARTIFICIAL INTELLIGENCE RES INST OF HEFEI COMPREHENSIVE NAT SCI CENT (ANHUI ARTIFICIAL INTELLIGENCE LAB)
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
CN202311348935.4
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2023-10-17
Publication Date
2026-11-17
Estimated Expiration
2043-10-17

AI Technical Summary

Technical Problem

Existing equipment cannot effectively control instruments remotely, feedback is not strong or timely enough, hardware resources are wasted, and data processing efficiency is low.

Method used

The system employs a parallel computing data stream hardware acceleration system, which includes a sensor module, a filtering module, a chip-embedded identifier module, an LSTM neural network computing module, a central control module, a storage module, a conversion module, and a display module. It collects data through a sensor that detects the bending of two fingers and radar, and performs parallel processing and real-time feedback.

Benefits of technology

It improves data processing efficiency, enables timely feedback and wide applicability, avoids resource waste, and ensures operational safety and efficient control.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN117369640B_ABST
    Figure CN117369640B_ABST
Patent Text Reader

Abstract

This invention provides a hardware acceleration system and method for parallel computing data streams, comprising: a sensor module for collecting finger data and hand position data from both hands, and sending a start signal to a central control module; a filtering module for removing unwanted data; a chip-embedded identification module for corresponding identification of different data; a chip-embedded LSTM neural network computing module for data preprocessing, parallel processing of the data through two sets of arithmetic units, and then matrix multiplication and convolution calculations; a central control module for controlling the data flow and computing process; a storage module for storing the data before and after processing, acting as a buffer; a conversion module with a built-in chip decoder for classifying the processed data and changing its data type; and external devices including a robotic arm for controlling the robotic arm to perform corresponding operations and interacting with the wearer for feedback. This invention effectively ensures personnel safety, promptly detects operational errors, and makes adjustments.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This invention relates to the fields of integrated circuit design and big data processing technology, specifically to a hardware acceleration system and method for parallel computing data streams. Background Technology

[0002] Equipment explosions have various causes, but the fact that all such equipment requires on-site operation by personnel is a major cause of injuries and fatalities. Currently, no equipment exists that can effectively solve this problem; either it's inconvenient to operate the equipment remotely, or the feedback is not strong or timely enough.

[0003] While some devices on the market can process this type of data, these devices are general-purpose, leading to a waste of hardware resources and failing to handle data processing efficiently and quickly. Therefore, it is necessary to propose a dedicated chip solution to achieve accelerated processing of data stream acquisition, storage, and computation. Due to numerous limitations, these chips still primarily process data sequentially and provide results, with much of the processing time wasted on waiting. Summary of the Invention

[0004] To address the shortcomings of existing technologies, this invention provides a hardware acceleration system and method for parallel computing data streams.

[0005] According to the present invention, a hardware acceleration system and method for parallel computing data streams are provided, the solution of which is as follows:

[0006] In a first aspect, a hardware acceleration system for parallel computing data streams is provided, the system comprising:

[0007] Sensor module: Collects data on the fingers and positions of both hands, and sends a start signal to the central control module;

[0008] Filtering module: Removes unwanted data from the data set;

[0009] The chip has a built-in identification module: it identifies different data accordingly;

[0010] The chip has a built-in LSTM neural network operation module: first, the data is preprocessed, then the data is processed by parallel arithmetic units and memory, the data is processed in parallel by two sets of arithmetic units, and then matrix multiplication and convolution calculation are performed.

[0011] Central control module: controls data flow and processing;

[0012] Storage module: Stores data before and after processing, acting as a buffer;

[0013] Conversion module: Built-in chip decoder, which classifies the processed data and changes its data type;

[0014] Display module: Shows the data comparison results to the wearer to facilitate subsequent corrections;

[0015] External devices include robotic arms, which are controlled to perform corresponding operations and provide interactive feedback to the wearer;

[0016] First, data is collected, then filtered, labeled, processed and stored, and then passed to a conversion module for data type conversion. Finally, the robot arm is controlled to move and its posture is fed back to the wearer.

[0017] Preferably, the system incorporates an identifier decoding module into the chip, enabling data to be uniformly processed in parallel by the computing module after being identified, thus avoiding the need for chip resources and accelerating data processing.

[0018] Preferably, the sensor module has finger bending sensors installed on the fingers of the wearer's hands, and a radar installed in the palm of each hand for positioning the hands. The finger sensors collect force signals from the finger areas in real time, while the radar in the palm continuously emits signals to locate the spatial position of the hands.

[0019] Preferably, signals are collected by a hand finger bending sensor to detect the degree of bending of each part of the fingers, and the collected finger bending information and the spatial position of the hands are transmitted to the chip's built-in identification module to identify the finger bending signals and the spatial position of the hands respectively; the filtering module filters the collected data and sends a signal to the central control module to prepare for the corresponding calculation and processing of the data.

[0020] Preferably, the collected finger bending signals and the spatial position of both hands are processed as a data stream;

[0021] The chip's built-in LSTM neural network processing module first preprocesses the data. When the data before and after transmission are within the set error range, it will directly send a signal to the robot arm to maintain the robot arm's current state and clear the data at this time.

[0022] Preferably, the chip's built-in LSTM neural network operation module includes a full adder, a subtractor, a multiplier, and a divider, which are processed in parallel.

[0023] There are two full adders, two subtractors, two multipliers, and two dividers, and the weights and biases of the neural network will be stored in separate RAM memories.

[0024] After data preprocessing, the data will be divided into two types: hand finger data and hand spatial position data. Each type will be processed by a set of arithmetic units, and the data will be processed by the corresponding arithmetic units in a set order.

[0025] Preferably, the data passes through a storage module before entering the chip's built-in LSTM neural network operation module. The storage module is divided into four parts, which respectively store the collected hand data, the copied hand data, the processed data, and the weights and biases of the neural network, thus buffering the data flow.

[0026] Preferably, the conversion module converts the data type, and the built-in chip decoder distinguishes the previously labeled data, separates the data on the degree of finger bending and the spatial position of the hands collected by the sensor module, removes the labeled parts from the data, and transmits the data to the robotic arm.

[0027] Preferably, the data processed by the conversion module will be input into the robotic arm, the data on the bending of the fingers of both hands will control the bending of the fingers of the robotic arm to make the same movement, and the data on the spatial position of both hands will control the robotic arm to move in the same direction;

[0028] Finger bending sensors, pressure sensors, and positioning radar are also installed on the robotic arm. When the robotic arm makes the same movement, data is collected. The finger bending data and position data are compared with the data collected by the wearer to calibrate the movement of the robotic arm. At the same time, the data collected by the pressure sensor is fed back to the wearer's hand to stimulate the corresponding parts of the wearer's hand.

[0029] Specifically, while pressure sensors are installed on both hands of the robotic arm, coils that can be controlled to retract are installed in the same positions on the wearer's hands. When the robotic arm touches an object, the pressure sensors will send a signal to control the coils in the same positions on the wearer's hands to retract, thus providing feedback to the wearer about what the robotic arm has touched.

[0030] Secondly, a hardware acceleration method for parallel computing data streams is provided, the method comprising:

[0031] Step S1: Data is collected by the sensor module, and after different data are identified by the chip's built-in identification module, the data is sent to the filtering module;

[0032] Step S2: The data is processed by the filtering module. After processing, the data is fed into the storage module as an input stream and copied and stored in the data acquisition memory. The data acquisition memory is located inside the storage module, occupies part of the storage module, and is used to store the filtered data.

[0033] Step S3: Once the robotic arm's feedback data arrives, compare the processed data with the feedback data, and display the comparison result on the display module so that the operator can adjust the posture in a timely manner.

[0034] Step S4: After the comparison is completed, the compared data is deleted. At the same time, the central control module controls the input stream to enter the chip's built-in LSTM neural network operation module to process the data according to the set process.

[0035] Step S5: The processed data is transmitted as an output stream to the storage module, and then passed to the conversion module to change the data type. The data is then distinguished according to the degree of finger bending and the spatial position of the hands, and the marking part is removed.

[0036] Step S6: Finally, output the data to the external robotic arm and provide feedback to the wearer.

[0037] Compared with the prior art, the present invention has the following beneficial effects:

[0038] 1. This invention processes data collected by finger bending sensors and positioning radar on both hands in the form of a data stream. The neural network chip processes the data directly according to a pre-defined process, without relying on timing, which can effectively improve the chip's working efficiency.

[0039] 2. This invention installs a finger bending sensor and a positioning radar at the same position on the robotic hand, feeds back the real-time behavior of the robotic hand to the chip, and compares it with the data on the degree of finger bending and spatial position of the wearer's hands collected. If there is an error, it will be corrected in time.

[0040] 3. This invention incorporates pressure sensors on both hands of the robotic arm and controllable retractable coils at the same locations on the wearer's hands. When the robotic arm touches an object, the pressure sensors send a signal to control the retraction of the coils at the same locations on the wearer's hands, allowing the wearer to sense where the robotic arm has touched the object and provide feedback. Because data is processed via a data stream, transmission time is short. The time from when the wearer's hands make a movement to when the robotic arm touches the object and provides feedback is negligible, enabling timely feedback and improving work efficiency.

[0041] 4. This invention identifies the data of both hands and distinguishes the collected data of both hands, regardless of whether the hands belong to the same person, making the applicable scenarios wider.

[0042] 5. This invention can bypass timing constraints and allow data to flow into and out of the chip like water, thus avoiding the waste of a lot of time while data is waiting to be processed and effectively improving the efficiency of data processing.

[0043] Other beneficial effects of the present invention will be explained in detail through the introduction of specific technical features and technical solutions in specific embodiments. Those skilled in the art should be able to understand the beneficial technical effects brought about by these technical features and technical solutions through the introduction of these technical features and technical solutions. Attached Figure Description

[0044] Other features, objects, and advantages of the present invention will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings:

[0045] Figure 1 This is a schematic diagram of the overall operation of the present invention;

[0046] Figure 2 This is a schematic diagram of the chip operation of the present invention;

[0047] Figure 3 This is a schematic diagram of the neural network module processing data according to the present invention;

[0048] Figure 4 This is a schematic diagram of the parallel data stream operation of the present invention;

[0049] Figure 5 This is a flowchart demonstrating the actual use scenario of the present invention. Detailed Implementation

[0050] The present invention will now be described in detail with reference to specific embodiments. These embodiments will help those skilled in the art to further understand the present invention, but do not limit the invention in any way. It should be noted that those skilled in the art can make several changes and improvements without departing from the concept of the present invention. These all fall within the scope of protection of the present invention.

[0051] This invention provides a hardware acceleration system for parallel computing data streams, referring to... Figure 1 As shown, the system specifically includes the following:

[0052] Sensor module: Collects data on the fingers and positions of both hands, and sends a start signal to the central control module;

[0053] Filtering module: Removes unwanted data from the data set;

[0054] The chip has a built-in identification module: it identifies different data accordingly;

[0055] The chip has a built-in LSTM neural network operation module: first, the data is preprocessed, then the data is processed by parallel arithmetic units and memory, the data is processed in parallel by two sets of arithmetic units, and then matrix multiplication and convolution calculation are performed.

[0056] Central control module: controls data flow and processing;

[0057] Storage module: Stores data before and after processing, acting as a buffer;

[0058] Conversion module: Built-in chip decoder, which classifies the processed data and changes its data type;

[0059] Display module: Shows the data comparison results to the wearer to facilitate subsequent corrections;

[0060] External devices include robotic arms, which are controlled to perform corresponding operations and provide interactive feedback to the wearer;

[0061] First, data is collected, then filtered, labeled, processed and stored, and then passed to a conversion module for data type conversion. Finally, the robot arm is controlled to move and its posture is fed back to the wearer.

[0062] The system also incorporates an identifier decoding module into the chip, enabling data to be uniformly processed in parallel by the computing module after being identified, thus avoiding the need for chip resources and accelerating data processing.

[0063] Specifically, the sensor module has finger bending sensors installed on the fingers of the wearer's hands, and a radar installed in the palm of each hand to locate the hands. The finger sensors collect force signals in the fingers in real time, while the radar in the palm continuously emits signals to locate the spatial position of the hands.

[0064] Signals are collected by the finger bending sensors of both hands to detect the degree of bending of each part of the fingers. The collected finger bending information and the spatial position of the hands are transmitted to the chip’s built-in identification module to identify the finger bending signals and the spatial position of the hands respectively. The filtering module filters the collected data and sends a signal to the central control module to prepare for the corresponding calculation and processing of the data.

[0065] The collected finger bending signals and the spatial positions of both hands are processed as a data stream. The chip's built-in LSTM neural network module first preprocesses the data. If the data before and after transmission is within a set error range, a signal is directly sent to the robotic arm to maintain its current state, and the data at that time is cleared. The chip's built-in LSTM neural network module includes a full adder, subtractor, multiplier, and divider. To minimize data processing time, parallel processing is employed.

[0066] There are two full adders, two subtractors, two multipliers, and two dividers. The weights and biases of the neural network will be stored in separate RAM memories. When the data is preprocessed, it will be divided into two sets of finger data and two sets of spatial position data, which will be processed by a set of arithmetic units. The data will be processed by the corresponding arithmetic units in a set order.

[0067] The data passes through the storage module before entering the chip's built-in LSTM neural network processing module. The storage module is divided into four parts, which respectively store the collected hand data, the copied hand data, the processed data, and the weights and biases of the neural network, thus buffering the data flow.

[0068] The conversion module transforms the data type and incorporates a built-in chip decoder to differentiate previously labeled data. It distinguishes between data on finger flexion and hand spatial position collected by the sensor module, removes the labeled portions, and transmits the data to the corresponding module in the robotic arm. Hardware acceleration is achieved through the labeling module, neural network module, and converter module.

[0069] The data processed by the conversion module will be input to the corresponding parts of the robotic arm. Data on the bending of the fingers of both hands will control the bending of the fingers of the robotic arm to make the same movements. Data on the spatial position of both hands will control the robotic arm to move in the same direction. Finger bending sensors, pressure sensors, and positioning radar are also installed on the robotic arm. When the robotic arm makes the same movements, data is collected. The finger bending data and position data are compared with the data collected by the wearer to calibrate the robotic arm's movements. At the same time, the data collected by the pressure sensors is fed back to the wearer's hands to stimulate the corresponding parts of the wearer's hands. Specifically, while pressure sensors are installed on the robotic arm's hands, controllable coils that can be contracted are installed at the same positions on the wearer's hands. When the robotic arm touches an object, the pressure sensor will send a signal to control the coils at the same positions on the wearer's hands to contract, feeding back the object touched by the robotic arm to the wearer.

[0070] This invention also provides a hardware acceleration method for parallel computing data streams, the method specifically including:

[0071] Step S1: Data is collected by the sensor module, and after different data are identified by the chip's built-in identification module, the data is sent to the filtering module;

[0072] Step S2: The data is processed by the filtering module. After processing, the data is fed into the storage module as an input stream and copied and stored in the data acquisition memory. The data acquisition memory is located inside the storage module, occupies part of the storage module, and is used to store the filtered data.

[0073] Step S3: Once the robotic arm's feedback data arrives, compare the processed data with the feedback data, and display the comparison result on the display module so that the operator can adjust the posture in a timely manner.

[0074] Step S4: After the comparison is completed, the compared data is deleted. At the same time, the central control module controls the input stream to enter the chip's built-in LSTM neural network operation module to process the data according to the set process.

[0075] Step S5: The processed data is transmitted as an output stream to the storage module, and then passed to the conversion module to change the data type. The data is then distinguished according to the degree of finger bending and the spatial position of the hands, and the marking part is removed.

[0076] Step S6: Finally, output the data to the external robotic arm and provide feedback to the wearer.

[0077] The present invention will now be described in more detail.

[0078] This invention provides a parallel computing data stream hardware acceleration system, comprising: a sensor module for collecting finger data and hand position data from both hands, and sending a start signal to a central control module; a filtering module for removing unwanted data; a chip-built-in identification module for correspondingly identifying different data collected from the wearer's hands; a chip-built-in LSTM neural network module for preprocessing the data, followed by parallel processing using two sets of processors (parallel arithmetic units and memory) to improve data processing speed and perform matrix multiplication, convolution, and other calculations; a central control module for controlling the data flow and computation process; a storage module for storing data before and after processing, acting as a buffer; and a conversion module with a built-in chip decoder for classifying the processed data and changing its data type.

[0079] Data is first collected by sensors, then labeled by the identification module and sent to the filtering module. After processing, the data is input to the storage module (which consists of four parts: storing input streams, storing output streams, storing collected data, and storing neural network weights and biases). The data is then copied and stored in the data acquisition memory. When the data from the robotic arm arrives, the two sets of data are compared, and the comparison result is output. After the comparison, the compared data is deleted. Simultaneously, the central control module controls the input stream to enter the neural network module, which processes the data according to a pre-defined process. The processed data is then output to the corresponding part of the memory and then sent to the converter module to change the data type. The data is then differentiated according to the degree of finger bending and the spatial position of the hands, and the labeled parts are removed. Finally, the data is output to the robotic arm, and the relevant information is fed back.

[0080] like Figure 2As shown in the diagram, the chip works by sending signals collected by sensors on the wearer's hand to the chip, which then processes the data and outputs it to the robotic arm (the chip structure and data processing method are as follows). Figure 1 As shown, the movement of the robotic arm is controlled, and the robotic arm feeds back the data after the movement to the chip, which compares it with the data of the wearer's hands and outputs the results. At the same time, the robotic arm will also feed back the part of the object it touches to the wearer, forming a human-computer interaction.

[0081] like Figure 3 As shown, after the data enters the neural network module, it will first undergo data preprocessing. If the input data is within the allowable error range, a signal will be directly output to the robotic arm to maintain its current state, and the data at this time will be cleared. After the data passes the preprocessing, it will be divided into hand finger data and hand spatial position data, which will be processed separately using an LSTM neural network (the operation flowchart is shown below). Figure 4 As shown, each set of arithmetic units is used to process the data, and the corresponding weights and biases are input. The processed data will be output together.

[0082] like Figure 4 As shown, the neural network module's computation section mainly includes adders, subtractors, multipliers, and dividers, employing parallel computing. Each arithmetic unit has two units, which can be divided into two groups. Each arithmetic unit has a data ready signal, indicating that the previous data has been processed and is ready to process the next data. Simultaneously, it has a data output signal, indicating that the data in the arithmetic unit is ready for the next operation. If the data has completed all operations, it is directly output into memory for the next operation. The chip selector is used to input data one by one, and the neural network weights are stored in a weight memory and input along with the data.

[0083] Specific implementation principle:

[0084] like Figure 5 As shown, the operator's hand posture data is collected by sensors. After being labeled by the identification module, the data is sent to the filtering module. After processing by the filtering module, the data is used as input to the storage module (the storage module consists of four parts: storing input stream, storing output stream, storing collected data, and storing neural network weights and biases). The data is also copied and stored in the collected data memory. When the data from the robotic arm arrives, the two sets of data are compared. The comparison result is displayed on the display module so that the operator can adjust the posture in time. After the comparison is completed, the compared data is deleted. For a detailed structural diagram, please refer to [reference needed]. Figure 1 Simultaneously, the central control module controls the input stream to enter the neural network module, processing the data according to a pre-defined process. The data processing units in the neural network module are divided into two groups, each containing a full adder, subtractor, multiplier, and divider, such as... Figure 4 As shown.

[0085] The data undergoes preprocessing in the neural network module. It is classified using an identifier to separate the data into finger flexion data and spatial data. Simultaneously, the data is compared before and after the classification. If the data falls within a specified error range, a command is directly output to the robotic arm to maintain the current posture. Each of the two classified data sets is processed by a separate processor. The same method is used to process the same set of data, which accelerates data processing. Figure 3 As shown, the same set of data will quickly enter the corresponding arithmetic unit according to a predetermined process, and immediately enter the next arithmetic unit after the calculation is completed. The processed data is immediately transmitted to the outgoing memory. Then it is sent to the converter module to change the data type, distinguish the data according to the degree of finger bending and the spatial position of the hands, and remove the marking part. Finally, the data is output to the external robotic arm, which will operate the instrument synchronously. In this way, the marking module, neural network module, and conversion module jointly complete the hardware acceleration of parallel computing data flow. Due to the short data processing time, the results displayed by the display module and the feedback from the robotic arm allow the operator to perceive the status of the robotic arm in real time and make timely adjustments. The addition of a marking and decoding module to the chip allows the data to enter the computing module for parallel processing after being marked, reducing the chip's resource requirements, reducing costs, and accelerating data processing.

[0086] This invention provides a hardware acceleration system and method for parallel computing data streams, allowing operators to directly control a robotic arm with their hands to perform the same actions on instruments. This not only keeps operators away from the instruments, protecting their safety, but also enables precise, fast, and effective instrument control. Furthermore, the high data processing speed provides timely feedback, allowing for immediate correction of any errors. Through hand-controlled robotic arm movement and interactive feedback, the wearer can assist in operating hazardous instruments, such as chemical apparatus. The wearer controls the robotic arm with their hands to perform desired operations, and the robotic arm provides interactive feedback on the parts touched during the process, allowing the wearer to determine whether the robotic arm has achieved precise control at each stage. This effectively ensures personnel safety, is highly convenient, provides strong feedback, and allows for timely detection and adjustment of operational errors.

[0087] Those skilled in the art will understand that, besides implementing the system and its various devices, modules, and units provided by this invention in the form of purely computer-readable program code, the same functions can be achieved entirely through logical programming of the method steps, making the system and its various devices, modules, and units of this invention function in the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, and embedded microcontrollers. Therefore, the system and its various devices, modules, and units provided by this invention can be considered as a hardware component, and the devices, modules, and units included therein for implementing various functions can also be considered as structures within the hardware component; alternatively, the devices, modules, and units for implementing various functions can be considered as both software modules implementing the method and structures within the hardware component.

[0088] Specific embodiments of the present invention have been described above. It should be understood that the present invention is not limited to the specific embodiments described above, and those skilled in the art can make various changes or modifications within the scope of the claims, which do not affect the essence of the present invention. Unless otherwise specified, the embodiments and features described in this application can be arbitrarily combined with each other.

Claims

1. A parallel arithmetic dataflow hardware acceleration system, characterized by, include: Sensor module: Collects data on the fingers and positions of both hands, and sends a start signal to the central control module; Filtering module: Removes unwanted data from the data set; The chip has a built-in identification module: it identifies different data accordingly; The chip has a built-in LSTM neural network operation module: first, the data is preprocessed, then the data is processed by parallel arithmetic units and memory, the data is processed in parallel by two sets of arithmetic units, and then matrix multiplication and convolution calculation are performed. Central control module: controls data flow and processing; Storage module: Stores data before and after processing, acting as a buffer; Conversion module: Built-in chip decoder, which classifies the processed data and changes its data type; Display module: Shows the data comparison results to the wearer to facilitate subsequent corrections; External devices include robotic arms, which are controlled to perform corresponding operations and provide interactive feedback to the wearer; First, data is collected, then filtered, the filtered data is labeled, then processed and stored, the processed data is sent to the conversion module for data type conversion, and finally the robot arm is controlled to move and the robot arm's posture is fed back to the wearer. The sensor module has finger bending sensors installed on the fingers of the wearer's hands, and a radar installed in the palm of each hand to locate the hands. The finger sensors collect force signals from the finger parts in real time, while the radar in the palm continuously emits signals to locate the spatial position of the hands. Signals are collected by the finger bending sensors of both hands to detect the degree of bending of each part of the fingers. The collected finger bending information and the spatial position of the hands are transmitted to the chip’s built-in identification module to identify the finger bending signals and the spatial position of the hands respectively. The filtering module filters the collected data and sends a signal to the central control module to prepare for the corresponding calculation and processing of the data.

2. The parallel operation dataflow hardware acceleration system of claim 1, wherein, The system incorporates an identifier decoding module into the chip, enabling data to be uniformly processed in parallel by the computing module after being identified, thus avoiding the need for chip resources and accelerating data processing.

3. The parallel operation dataflow hardware acceleration system of claim 1, wherein, The collected finger bending signals and the spatial position of both hands are processed as a data stream; The chip's built-in LSTM neural network processing module first preprocesses the data. When the data before and after transmission are within the set error range, it will directly send a signal to the robot arm to maintain the robot arm's current state and clear the data at this time.

4. The parallel operation dataflow hardware acceleration system of claim 1, wherein, The chip's built-in LSTM neural network operation module includes a full adder, a subtractor, a multiplier, and a divider, which are processed in parallel. There are two full adders, two subtractors, two multipliers, and two dividers, and the weights and biases of the neural network will be stored in separate RAM memories. After data preprocessing, the data will be divided into two types: hand finger data and hand spatial position data. Each type will be processed by a set of arithmetic units, and the data will be processed by the corresponding arithmetic units in a set order.

5. The parallel operation dataflow hardware acceleration system of claim 4, wherein, The data passes through the storage module before entering the chip's built-in LSTM neural network processing module. The storage module is divided into four parts, which respectively store the collected hand data, the copied hand data, the processed data, and the weights and biases of the neural network, thus buffering the data flow.

6. The parallel operation dataflow hardware acceleration system of claim 1, wherein, The conversion module converts the data type. At the same time, the built-in chip decoder in the conversion module distinguishes the previously labeled data, separates the data on the degree of finger bending and the spatial position of the hands collected by the sensor module, removes the labeled parts of the data, and transmits the data to the robotic arm.

7. The parallel operation dataflow hardware acceleration system of claim 6, wherein, The data processed by the conversion module will be input into the robotic arm. The data on the bending of the fingers of both hands will control the bending of the fingers of the robotic arm to make the same movement. The data on the spatial position of both hands will control the robotic arm to move in the same direction. Finger bending sensors, pressure sensors, and positioning radar are also installed on the robotic arm. When the robotic arm makes the same movement, data is collected. The finger bending data and position data are compared with the data collected by the wearer to calibrate the movement of the robotic arm. At the same time, the data collected by the pressure sensor is fed back to the wearer's hand to stimulate the corresponding parts of the wearer's hand. Specifically, while pressure sensors are installed on both hands of the robotic arm, coils that can be controlled to retract are installed in the same positions on the wearer's hands. When the robotic arm touches an object, the pressure sensors will send a signal to control the coils in the same positions on the wearer's hands to retract, thus providing feedback to the wearer about what the robotic arm has touched.

8. A hardware acceleration method for parallel computing data streams, based on the hardware acceleration system for parallel computing data streams according to any one of claims 1-7, characterized in that, include: Step S1: Data is collected by the sensor module, and after different data are identified by the chip's built-in identification module, the data is sent to the filtering module; Step S2: The data is processed by the filtering module. After processing, the data is fed into the storage module as an input stream and copied and stored in the data acquisition memory. The data acquisition memory is located inside the storage module, occupies part of the storage module, and is used to store the filtered data. Step S3: Once the robotic arm's feedback data arrives, compare the processed data with the feedback data, and display the comparison result on the display module so that the operator can adjust the posture in a timely manner. Step S4: After the comparison is completed, the compared data is deleted. At the same time, the central control module controls the input stream to enter the chip's built-in LSTM neural network operation module to process the data according to the set process. Step S5: The processed data is transmitted as an output stream to the storage module, and then passed to the conversion module to change the data type. The data is then distinguished according to the degree of finger bending and the spatial position of the hands, and the marking part is removed. Step S6: Finally, output the data to the external robotic arm and provide feedback to the wearer.

Citation Information

Patent Citations

  • Multi-modal data fusion and classification method based on SVM model parameter optimization

    CN111340061A

  • System and method for processing human related data including physiological signals to formulate context aware decisions with distributed machine learning of edges and clouds

    CN114830132A