Data processing and transmission method and device thereof
By using an overflow-prevention buffer queue and an adaptive batch transmission method based on queue pressure indicators, the problems of insufficient burst data processing capability and control command blocking in the acquisition and transmission of robot tactile sensor data are solved, achieving efficient data transmission and system stability.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- YUANSHENGXIANDA TECHNOLOGY (SHENZHEN) CO LTD
- Filing Date
- 2026-02-13
- Publication Date
- 2026-06-02
Smart Images
Figure CN122132340A_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of embedded data acquisition and communication, and particularly relates to a processing method and device for high-speed acquisition, buffering, scheduling and transmission of bus data (such as CAN / CANFD and other multiplexed buses). Background Technology
[0002] In existing technologies, tactile sensors, as key devices for robots to acquire environmental contact information in scenarios such as embodied intelligent robots, dexterous hand manipulation, and human-computer interaction, are developing towards higher resolution, larger-area arrays, and multi-dimensional force detection. For example: In high-frequency slip detection and grasping scenarios of robotic dexterous hands, when grasping fragile or smooth objects, sensors need to monitor the micro-vibration signals of the contact surface between the fingertip and the object at extremely high sampling rates (typically >1kHz) to determine whether micro-slippage has occurred. Traditional transmission methods often cause bus congestion when slippage signals are detected due to a sudden surge in data volume. This results in the loss of slippage characteristic signals, preventing the robot controller from adjusting the grasping force in time, leading to the object falling.
[0003] In collision safety monitoring scenarios using full-body electronic skin (E-Skin) on humanoid robots, the entire robot may be covered with thousands of tactile sensing units. When the robot experiences an accidental collision or fall, the sensors in the impact area will simultaneously generate large bursts of data, which can easily lead to data gaps due to buffer overflows. This also makes it impossible to perform post-accident analysis (black box function) and robot compliance control.
[0004] In the application of dynamic parameter variation and online calibration of tactile sensors, robots in complex industrial grinding or assembly scenarios need to adjust the sensitivity (gain) of the tactile sensors or perform zero-point drift calibration in real time according to the hardness of the workpiece material. In existing technologies, when the sensor is uploading force control data at full speed, the issued configuration commands are easily blocked or discarded. It cannot be guaranteed that the control commands from the host computer (such as "reduce gain" or "stop data acquisition") will be processed preferentially and deterministically, thus compromising the controllability and safety of the system.
[0005] In the field of embodied AI dataset acquisition, training deep learning models for tactile perception (such as CNNs or Transformers) requires acquiring massive, continuous, frame-free sequences of tactile images. When communication is subject to transient errors due to electromagnetic interference (EMI), the training data cannot be quickly retransmitted, significantly impacting the temporal continuity and integrity of the acquired training dataset, ultimately leading to model training bias due to frame drops.
[0006] In summary, existing tactile data acquisition and transmission solutions generally suffer from the following problems: 1. Insufficient burst data processing capability: When the robot grasps or makes contact with the object, the effective signal in the tactile array will increase explosively (data surge). If a traditional single-frame forwarding or simple buffering mechanism is used, it is very easy to lose critical contact force tactile feature data due to buffer overflow at the moment of contact.
[0007] 2. The contradiction between real-time performance and throughput: Robot force control algorithms typically require extremely low communication latency, while high-precision tactile texture recognition requires extremely high data throughput. Existing fixed-period or fixed-batch transmission strategies struggle to dynamically balance latency and throughput requirements based on the current contact state (e.g., idle, light touch, heavy pressure).
[0008] 3. Risk of Control Command Blocking: When sensors are uploading data at full speed (e.g., during slip detection), the communication bandwidth is occupied by a large amount of tactile data. If the host computer issues configuration commands at this time (such as dynamically adjusting gain, zeroing calibration, or switching operating modes), the commands are very likely to be blocked by the upstream data stream, causing sensor response delays or even system control logic freezes, seriously affecting the robot's operational safety. Summary of the Invention
[0009] In order to overcome at least one technical problem existing in the prior art, the present invention provides a data processing and transmission method.
[0010] The present invention also provides a device based on the above-described data processing and transmission method.
[0011] The present invention also provides a computer-readable medium based on the above-described data processing and transmission method.
[0012] This invention provides a data processing and transmission method, the method comprising: The system receives bus messages via at least one bus interface and encapsulates the bus messages into data units of a predetermined format. The data unit is written into a first cache queue, which has an enqueue index and a dequeue index. When the first cache queue is detected to have reached congestion conditions, overflow information is recorded and overflow protection is performed to avoid out-of-bounds or blocking. Obtain the queue pressure metrics for the first cache queue; The batch sending parameters are adaptively determined based on the queue pressure index, and the batch sending parameters include at least the batch size and the sending waiting time / timeout. Multiple data units are retrieved from the first buffer queue according to the batch transmission parameters and transmitted in batches via the communication interface; When the communication interface returns to a busy state, fast retry or backoff processing is performed based on the queue pressure index.
[0013] Furthermore, the method also includes the following steps: The configuration / control commands issued by the host computer are written into a second queue independent of the first cache queue, and are asynchronously retrieved and forwarded to the bus interface in the main loop or an independent task; when the bus interface returns to a busy state, the configuration / control commands are re-enqueued and the current processing loop is exited.
[0014] Furthermore, the data unit contains at least two or more of the following fields: frame type, channel identifier, identifier type, data length, identifier, data area, and timestamp.
[0015] Furthermore, the data unit is a fixed-length data unit, with a data area length of 64 bytes and a total length of 76 bytes, and adopts a 1-byte alignment method to reduce the difference in structure padding.
[0016] Furthermore, the first cache queue is a circular buffer queue with a capacity of 2^n data units; enqueueing and dequeueing are implemented using bitmasks or modulo operations to achieve index wrapping.
[0017] Furthermore, the first cache queue has a capacity of 2048 data units and is equipped with an overflow counter to count the number of overflows.
[0018] Furthermore, the overflow protection process includes at least one of the following: discarding the latest data unit, discarding the oldest data unit, overwriting the oldest data unit, selective discarding based on priority, and load reduction based on sampling rate.
[0019] Furthermore, the queue pressure indicators include at least queue occupancy rate and number of pending requests.
[0020] Furthermore, the queue pressure metrics also include one or a combination of the following: arrival rate, service rate, continuous busy count, and end-to-end delay estimate.
[0021] Furthermore, when the queue pressure index increases, the batch size is increased and the sending waiting time / timeout is shortened; when the queue pressure index decreases, the batch size is decreased and the sending waiting time / timeout is extended.
[0022] Furthermore, the batch transmission includes packet concatenation or multi-frame aggregation; the communication interface includes one or a combination of the following: USB, Ethernet, serial port, and wireless communication interface.
[0023] Furthermore, when the queue pressure index is higher than the first threshold, a short delay is performed followed by an immediate retry; when the continuous busy count is higher than the second threshold, a long delay is performed to back off and the continuous busy count is reduced or the batch size is reduced.
[0024] Furthermore, the depth of the second queue is less than the capacity of the first cache queue, and the second queue is a FIFO queue or a priority queue.
[0025] The present invention also provides a device including a processor, a memory, at least one bus interface and a communication interface, characterized in that the processor executes a program stored in the memory to implement the above-described data processing and transmission method.
[0026] The present invention also provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the above-described data processing and transmission method.
[0027] Compared with the prior art, the beneficial effects of the present invention using the above solution are as follows: (1) By using an anti-overflow cache queue and an overflow recording mechanism, we can avoid out-of-bounds and blocking under sudden high loads, thereby reducing the risk of packet loss and anomalies; (2) Improve the effective throughput of the link and reduce end-to-end latency jitter by using batch scheduling and Busy hierarchical strategy with queue pressure closed loop; (3) By using an independent asynchronous queue and busy re-entry mechanism for configuration commands, configuration / control commands can maintain a predictable response even under data surges. Attached Figure Description
[0028] Figure 1 A schematic diagram illustrating the steps of a data processing and transmission method; Figure 2 This is a schematic diagram of the overall process of bus reception, buffering, and batch transmission. Figure 3 This is a schematic diagram of the overflow detection and overflow policy handling process when the first cache queue is enqueued. Figure 4 This is a schematic diagram of the stress-aware fast retry and backoff process in the Busy state of the link. Figure 5 This is a schematic diagram of the batch transmission scheduling and Busy processing flow based on queue pressure. Figure 6 This is a schematic diagram illustrating the process of asynchronously transmitting configuration / control commands to the bus side via an independent queue. Detailed Implementation
[0029] To better understand the above-mentioned objectives, features, and advantages of the present invention, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be noted that, unless otherwise specified, the embodiments and features described in these embodiments can be combined with each other.
[0030] Many specific details are set forth in the following description in order to provide a full understanding of the invention. However, the invention may also be practiced in other ways different from those described herein, and therefore the scope of protection of the invention is not limited to the specific embodiments disclosed below. Example
[0031] like Figure 1 , Figure 2 As shown, the present invention provides a data processing and transmission method, including: Receiving steps: Receive bus messages via at least one bus interface and encapsulate the bus messages into data units of a predetermined format; Caching steps: The data unit is written into the first cache queue, which has an enqueue index and a dequeue index; when the first cache queue is detected to have reached the congestion condition, overflow information is recorded and overflow protection is performed to avoid out-of-bounds or blocking. Stress assessment steps: Obtain the queue stress index of the first cache queue; Scheduling steps: Adaptively determine batch sending parameters based on the queue pressure index, wherein the batch sending parameters include at least the batch size and the sending waiting time / timeout; Batch transmission steps: Take multiple data units from the first buffer queue according to the batch transmission parameters and transmit them in batches via the communication interface; Busy state handling steps: When the communication interface returns to a busy state, perform fast retry or backoff processing according to the queue pressure index; The asynchronous command forwarding steps are as follows: the configuration / control commands issued by the host computer are written into a second queue that is independent of the first cache queue, and are asynchronously retrieved and forwarded to the bus interface in the main loop or an independent task; when the bus interface returns to a busy state, the configuration / control commands are re-enqueued and the current processing loop is exited.
[0032] In specific implementation, such as Figure 3 As shown, the device acquires the bus message during the bus receive interrupt / callback and encapsulates the message into a data unit. The data unit can be of fixed or variable length; preferably, it includes at least the frame type, channel identifier, identifier type, data length, identifier, data area, and timestamp fields. The data unit is then written to a first buffer queue. The first buffer queue can be a circular buffer queue or other FIFO structure; preferably, it has a capacity of 2^n and uses bitmasks to implement index wrapping to reduce computational overhead.
[0033] When a full queue or congestion is detected, overflow protection is implemented. Overflow protection can choose to discard the latest data unit, discard the oldest data unit, overwrite the oldest data unit, or perform downsampling based on frame type / priority. Overflow information (e.g., overflow count, occurrence time) is also recorded for diagnostics and quality assessment.
[0034] One specific implementation uses a fixed-length structure as the data unit and constructs a circular queue with a capacity of 2048 (2^11). When next_head equals tail and the queue is full, overflow_count is incremented and the current packet is discarded. The above values are only examples; the queue capacity, data unit length, and overflow strategy can all be adjusted according to the application scenario.
[0035] In specific implementation, such as Figure 4 , Figure 5 As shown, the sending side periodically reads the queue pressure metrics of the first buffer queue. Queue pressure metrics include at least: queue occupancy and pending data; they can also be expanded to include arrival rate, service rate, continuous busy count, end-to-end latency estimate, etc. The sending side adaptively selects batch sending parameters based on queue pressure, such as batch size and sending wait time / timeout. When queue pressure increases, batch size is increased and timeout is shortened; when queue pressure decreases, batch size is decreased and a longer timeout is allowed to reduce latency fluctuations.
[0036] When the link returns to a busy state, a pressure-aware hierarchical processing method is adopted: when the pressure is high, a short delay is performed followed by a rapid retry to avoid the queue from continuing to accumulate; when the busy state persists or the busy count exceeds the threshold, a longer backoff is performed to avoid CPU idling, and the busy count can be reduced or the scheduling intensity can be decreased to restore system stability.
[0037] One specific strategy divides the queue occupancy rate into multiple threshold intervals (e.g., >75%, >50%, >25%), selecting different maximum batch sizes and timeouts for each. When the link returns to a busy state, a short NOP delay is applied immediately afterward during periods of high pressure, and a long NOP is applied to back off and the count is reduced when the busy count is too high. The aforementioned thresholds and parameters can be replaced with any piecewise or continuous function.
[0038] In specific implementation, such as Figure 6 As shown, the host computer issues configuration / control commands through the communication interface. The device writes the commands into a separate second queue, and asynchronously retrieves and forwards them to the bus transmission interface in the main loop or an independent task. The second queue can be a FIFO, a priority queue, or a queue structure divided by command type; its depth can be much smaller than the first queue to emphasize the timeliness of the control channel.
[0039] When the bus transmit interface returns to a busy state, the device will re-enqueue the command to be sent and exit the current processing loop, so that the control processing will not get stuck in a busy dead loop; at the same time, it can feed back the error code / status to the host computer so as to realize command retransmission or status synchronization.
[0040] In ENSCAN_V3.0.1 (analyzer box software), the USB OUT callback enqueues configuration packets that meet a preset length (76 bytes in the example), and the second queue depth is 8 bytes in the example. The asynchronous processing function re-enqueues the command and exits the loop when it returns a busy status. The above is just an example; the command format and queue depth can be expanded.
[0041] The technical solution of this embodiment is as follows: Receiving side: Obtain the message during a bus interrupt / callback → encapsulate it into a data unit (fixed / variable length, containing necessary fields such as ID / length / timestamp, etc.) → write it into the first buffer queue; when the queue is full or congestion conditions are met, record the overflow and execute overflow protection. Sending side: Periodically calculate queue pressure indicators → adaptively determine batch sending parameters (at least batch_size and timeout) → dequeue in batches and send them to the host computer via the communication interface; when the link returns to a busy state, execute short-delay fast retry or long-delay backoff based on the pressure. Control side: Host computer configuration / control commands are written to an independent second queue → asynchronously forwarded to the bus in the main loop; when the bus is busy, re-enqueue and exit the loop.
[0042] The technical principle of this embodiment is as follows: producer-consumer decoupling is achieved through FIFO caching (preferably a 2^n circular queue) to avoid interruption-side blocking; overflow protection strategy is used to avoid out-of-bounds access and diagnose packet loss; closed-loop scheduling of "queue pressure → batch parameters" achieves an adaptive trade-off between throughput and latency; Busy hierarchical fast retries and backoff are used to avoid CPU idling and improve stability; and independent command queues isolate the control channel from the data channel to ensure deterministic response of configuration commands.
[0043] Those skilled in the art will understand that the parameters mentioned in the above embodiments are merely exemplary selections and are not intended to limit the present invention. Without departing from the core concept of the present invention, those skilled in the art have reason to reasonably adjust and extend the above parameters according to the accuracy requirements, hardware resource limitations, or transmission efficiency requirements of actual application scenarios (multi-level priority queues, adaptive downsampling by frame type, bandwidth estimation and congestion control, dynamic MTU adaptation under different links (USB / Ethernet), etc.), and such parameter adjustments should all fall within the protection scope of the present invention.
Claims
1. A data processing and transmission method, characterized in that, include: The system receives bus messages via at least one bus interface and encapsulates the bus messages into data units of a predetermined format. The data unit is written into a first cache queue, which has an enqueue index and a dequeue index. When the first cache queue is detected to have reached congestion conditions, overflow information is recorded and overflow protection is performed to avoid out-of-bounds or blocking. Obtain the queue pressure metrics for the first cache queue; The batch sending parameters are adaptively determined based on the queue pressure index, and the batch sending parameters include at least the batch size and the sending waiting time / timeout. Multiple data units are retrieved from the first buffer queue according to the batch transmission parameters and transmitted in batches via the communication interface; When the communication interface returns to a busy state, fast retry or backoff processing is performed based on the queue pressure index.
2. The data processing and transmission method according to claim 1, characterized in that, The method further includes the following steps: The configuration / control commands issued by the host computer are written into a second queue independent of the first cache queue, and are asynchronously retrieved and forwarded to the bus interface in the main loop or an independent task; when the bus interface returns to a busy state, the configuration / control commands are re-enqueued and the current processing loop is exited.
3. The data processing and transmission method according to claim 2, characterized in that, The data unit contains at least two or more of the following fields: frame type, channel identifier, identifier type, data length, identifier, data area, and timestamp.
4. The data processing and transmission method according to claim 3, characterized in that, The data unit is a fixed-length data unit, with a data area length of 64 bytes and a total length of 76 bytes. It adopts a 1-byte alignment method to reduce the difference in structure padding.
5. The data processing and transmission method according to any one of claims 1-4, characterized in that, The first cache queue is a circular cache queue with a capacity of 2^n data units; enqueueing and dequeueing are implemented using bitmasks or modulo operations to achieve index wrapping.
6. The data processing and transmission method according to claim 5, characterized in that, The first cache queue has a capacity of 2048 data units and is equipped with an overflow counter to count the number of overflows.
7. The data processing and transmission method according to claim 6, characterized in that, The overflow protection process includes at least one of the following: discarding the latest data unit, discarding the oldest data unit, overwriting the oldest data unit, selective discarding based on priority, and load reduction based on sampling rate.
8. The data processing and transmission method according to claim 7, characterized in that, The queue pressure indicators include at least the queue occupancy rate and the number of pending requests.
9. The data processing and transmission method according to claim 8, characterized in that, The queue pressure metrics also include one or a combination of the following: arrival rate, service rate, continuous busy count, and end-to-end delay estimate.
10. The data processing and transmission method according to claim 9, characterized in that, When the queue pressure index increases, increase the batch size and shorten the sending wait time / timeout; when the queue pressure index decreases, decrease the batch size and extend the sending wait time / timeout.
11. The data processing and transmission method according to claim 10, characterized in that, The batch transmission includes packet concatenation or multi-frame aggregation; the communication interface includes one or a combination of the following: USB, Ethernet, serial port, and wireless communication interface.
12. The data processing and transmission method according to claim 11, characterized in that, When the queue pressure index is higher than the first threshold, a short delay is performed and the process is immediately retried; when the continuous busy count is higher than the second threshold, a long delay is performed to back off and the continuous busy count is reduced or the batch size is reduced.
13. The data processing and transmission method according to claim 12, characterized in that, The depth of the second queue is less than the capacity of the first cache queue, and the second queue is a FIFO queue or a priority queue.
14. A device comprising a processor, a memory, at least one bus interface, and a communication interface, characterized in that, The processor executes a program stored in the memory to implement the method according to any one of claims 1 to 13.
15. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method described in any one of claims 1 to 13.