Starburst-based intelligent mechanical arm VLA end-to-end control system and method

Abstract

The application provides a star flash-based intelligent mechanical arm VLA end-to-end control system, which comprises: a man-machine interaction module for an intelligent mechanical arm, which is used for executing a motion instruction sequence of an intelligent computing module to complete a user instruction; a perception information module, which is used for collecting environment and intelligent mechanical arm state data in real time to constitute multi-modal perception data, receiving a user instruction, and transmitting the multi-modal perception data and the user instruction to the intelligent computing module through a wireless transmission module; an intelligent computing module, which is an edge server configured with a star flash communication module, and is provided with a VLA model inference engine, and is used for generating a motion instruction sequence according to the multi-modal perception data and the user instruction; and a wireless transmission module, which is configured as a star flash communication module, and is used for data communication between the man-machine interaction module and the intelligent computing module; wherein the wireless transmission module performs full-link end-to-end time synchronization operation to ensure that all links use a unified global timestamp.

Description

Technical Field

[0001] This invention relates to the field of robot intelligent control and wireless communication technology, specifically to the field of vision-language-action model (VLA) end-to-end control technology for intelligent robotic arms, and more specifically, to the design of a star-flash-based intelligent robotic arm VLA end-to-end control system and method. Background Technology

[0002] Embodied Intelligence (EI), as a strategic core technology driving industrial upgrading and enhancing technological competitiveness, has received high attention from industry and academia. With the rapid iteration of embodied intelligence and industrial robot technology, the Vision Language Action (VLA) end-to-end control paradigm is gradually replacing the traditional hierarchical control architecture, becoming the core technological support for the flexible operation of precision robotic arms. The VLA system relies on a multimodal large model to achieve end-to-end unified modeling of visual perception, natural language understanding, and robot motion control. It can directly convert environmental perception information and human-machine language commands into sequences of robotic arm joint movements, overcoming the technical bottlenecks of traditional robots that rely on manually preset rules and have poor scene generalization capabilities. It demonstrates strong application value in industrial precision assembly, flexible sorting, and intelligent collaboration scenarios. Compared to the traditional hierarchical architecture where perception, planning, and control are independent, the VLA end-to-end architecture simplifies the task execution process, significantly improving the intelligent operation capabilities and scene adaptability of robotic arms, making it one of the core research directions for the current intelligent upgrading of industrial robots.

[0003] The core of the VLA end-to-end operation mode is a closed-loop real-time iterative mechanism of perception, transmission, computation, and control. This mechanism imposes stringent constraints on the performance of the communication link. During actual operation, the robotic arm is equipped with high-resolution vision, force, and other sensors that continuously collect massive amounts of high-precision environmental data. All perceived data must be aggregated in real-time to the edge computing node via the communication link to complete real-time inference calculations of the VLA large model, ultimately generating a continuous sequence of joint movement commands, which are then synchronously sent to the controller for closed-loop control. This inference control cycle is typically in the millisecond range, and the multi-sensor data fusion and multi-axis collaborative control place extremely high demands on data alignment accuracy and transmission stability. This leads to the need for communication capabilities with ultra-low deterministic latency, precise time synchronization, ultra-high transmission reliability, and strong anti-interference capabilities. Specifically, extremely low transmission latency and jitter are the foundation for avoiding motion jamming, trajectory deviation and attitude instability of the robotic arm; precise time synchronization can ensure accurate matching of multi-channel sensor data and multi-axis control commands, supporting multi-joint collaborative precision operation; highly reliable transmission can completely avoid equipment misoperation and operation failure caused by data packet loss and transmission errors; and the ability to resist interference in complex industrial electromagnetic environments is the key prerequisite for ensuring the long-term stable operation of the robotic arm.

[0004] Currently, the mainstream industrial robotic arms use wired communication solutions to carry closed-loop control data transmission. However, their hardware characteristics and transmission mechanisms are difficult to adapt to the stringent communication requirements of VLA end-to-end systems, resulting in many inherent technical shortcomings. Wired communication relies on physical media such as cables and connectors for data transmission. During long-term, high-frequency operation, problems such as connector contact oxidation, wear, and loosening, as well as cable bending and breakage, are prone to occur, significantly reducing the long-term reliability of the system and increasing the probability of equipment failure and maintenance costs. At the same time, the fixed topology of wired cabling severely limits the working space and range of motion of the robotic arm, making it unsuitable for the production needs of flexible production lines, such as rapid changeover, flexible equipment deployment, and dynamic workstation reconfiguration. This greatly restricts the implementation of intelligent flexible manufacturing and has become a major bottleneck for the large-scale industrial deployment of VLA robotic arms.

[0005] To address the physical constraints of wired solutions, the industry has gradually explored the introduction of wireless short-range communication technologies to replace wired transmission. However, existing mainstream wireless technologies all suffer from performance deficiencies. Specifically, WiFi technology employs a carrier sense random access mechanism, resulting in highly random transmission latency and significant jitter. It lacks deterministic scheduling capabilities, making it difficult to meet the stable latency and multi-sensor concurrent transmission requirements of VLA systems. Bluetooth technology has inherent limitations in transmission bandwidth and coverage. Traditional piconet networks only support a small number of active devices, failing to simultaneously meet the demands for high-bandwidth data transmission and ultra-low latency response. Zigbee technology focuses on low-power, low-speed sensor data transmission, resulting in a relatively low data transmission rate and weak real-time performance and node capacity, making it unsuitable for the high-speed, real-time, and high-precision transmission requirements of robotic arm VLA closed-loop control.

[0006] In summary, existing industrial robotic arm communication solutions have the following problems:

[0007] 1. Wired communication methods are not suitable for the stringent communication requirements of VLA end-to-end systems. They rely on cables and are prone to problems such as connector contact oxidation, wear, loosening, cable bending, and breakage, which significantly reduce the long-term reliability of the system, increase the probability of equipment failure and maintenance costs, and cannot adapt to the production needs of flexible production lines for rapid changeover, flexible equipment deployment, and dynamic workstation reconfiguration.

[0008] 2. The WIFI communication scheme adopts a carrier sense random access mechanism, which has strong randomness in transmission delay and large jitter amplitude. It lacks deterministic scheduling capability and is difficult to meet the requirements of stable delay and concurrent transmission of multiple sensors in the VLA system.

[0009] 3. Bluetooth technology has inherent limitations in transmission bandwidth and coverage. Traditional piconet can only support a small number of active devices and cannot simultaneously meet the requirements of high-bandwidth data transmission and ultra-low latency response.

[0010] 4. Zigbee technology focuses on low-power and low-speed sensor data transmission. Its data transmission rate is relatively low, and its real-time performance and node carrying capacity are weak. It cannot meet the high-speed, real-time, and high-precision transmission requirements of VLA closed-loop control of robotic arms.

[0011] Therefore, there is an urgent need for a solution that can meet the requirements of stable latency, concurrent transmission of multiple sensors, high bandwidth data transmission, and ultra-low latency response in VLA end-to-end systems.

[0012] It should be noted that the background information presented here is only for illustrating relevant information about the present invention to aid in understanding the technical solution of the present invention, and does not imply that the relevant information is necessarily prior art. The relevant information was submitted and disclosed together with the present invention, and should not be considered prior art unless there is evidence that the relevant information was disclosed before the filing date of the present invention. Summary of the Invention

[0013] Therefore, the purpose of this invention is to overcome the shortcomings of the prior art and provide a new VLA end-to-end control system and method for a smart robotic arm based on starlight.

[0014] The objective of this invention is achieved through the following technical solution:

[0015] According to a first aspect of the present invention, a VLA end-to-end control system for a smart robotic arm based on StarSpark is provided. The system includes: a human-machine interaction module, a sensing information module, an intelligent computing module, and a wireless transmission module. The human-machine interaction module is configured as a smart robotic arm, which includes a robotic arm main control unit and a robotic arm execution unit. Under the control of the robotic arm main control unit, the robotic arm execution unit executes a sequence of action instructions from the intelligent computing module to complete user commands. The robotic arm execution unit includes multiple joint servo controllers deployed at the joints of the robotic arm, each joint servo controller being configured with a time synchronization interface. The sensing information module is used to collect environmental and smart robotic arm status data in real time to form multimodal sensing data, and to receive user commands, transmitting the multimodal sensing data and user commands to the intelligent computing module via the wireless transmission module. It includes a depth camera configured at the end of the smart robotic arm, a force sensor configured at the force application position of the smart robotic arm, a joint encoder configured at the joints of the smart robotic arm, and a third-view depth camera and a status monitoring sensor configured in the working environment of the smart robotic arm. The intelligent computing module, an edge server equipped with a StarFlash communication module, houses a VLA model inference engine and a time synchronization interface. It generates action command sequences based on multimodal sensing data transmitted from the sensing information module and user commands, transmitting these sequences wirelessly to the human-machine interface module to control the intelligent robotic arm. The wireless transmission module, also a StarFlash communication module, facilitates data communication between the human-machine interface module and the intelligent computing module. It performs end-to-end time synchronization across the entire link through the time synchronization interfaces of each servo controller and the VLA model inference engine, ensuring that a unified global timestamp is used in all stages of sensor sampling, model inference, and command execution.

[0016] Preferably, the wireless transmission module includes a StarFlash management node and multiple StarFlash terminal nodes, wherein: the StarFlash management node is deployed in the intelligent computing module; the multiple StarFlash terminal nodes are deployed one-to-one in the depth camera, the third-view depth camera, and the robotic arm main control unit in the perception information module; wherein, the StarFlash management node is used to align the multimodal perception data uploaded by the multiple StarFlash terminal nodes according to time, and to schedule and allocate communication time slots and forward action command sequences; each StarFlash terminal node is used to upload the data sensed by its component to the StarFlash management node and to receive control action command sequences.

[0017] Preferably, the wireless transmission module is configured to perform end-to-end time synchronization upon system startup as follows: the StarSpark management node broadcasts network beacon frames to complete full-channel scanning and interference detection of the operating frequency band, and selects the channel with the highest signal-to-noise ratio as the operating channel; all StarSpark terminal nodes send access requests to the StarSpark management node to complete two-way authentication and device function registration; the StarSpark management node broadcasts timestamp synchronization beacon frames to all connected StarSpark terminal nodes at preset time intervals; each StarSpark terminal node corrects its local clock in real time based on the received synchronization beacon and feeds back the local clock drift to the StarSpark management node; the StarSpark management node runs a real-time clock drift compensation mechanism based on the received clock drift from the StarSpark terminal nodes, and can dynamically adjust the transmission time of the next synchronization beacon frame according to the historical drift data of each StarSpark terminal node; the StarSpark management node triggers the time synchronization interface of the VLA model inference engine on the edge service and each joint servo controller to complete calibration with the global clock of the StarSpark management node to ensure that all aspects of sensor sampling, model inference, and instruction execution use a unified global timestamp.

[0018] Preferably, the StarSpark management node is configured to: after time synchronization is completed, automatically load and configure a time-sensitive hierarchical scheduling mechanism according to the system's business characteristics, so as to divide the data into different time-sensitive levels and map them to different StarSpark transmission modes to allocate differentiated transmission resources for different data.

[0019] Preferably, in the time-sensitive hierarchical scheduling mechanism, the time-sensitive levels include: L1 level, which is the control command level; L2 level, which is the force data level; and L3 level, which is the visual or state data level. The star-flash transmission mode includes a deterministic transmission mode and an adaptive transmission mode. Specifically, L1 level maps to the deterministic transmission mode and is allocated exclusive periodic time slots, employing an ultra-short frame structure and forward error correction coding; L2 level maps to the adaptive transmission mode and is allocated a dynamically shared time slot pool, with a higher priority than L3 level data; L3 level maps to the adaptive transmission mode and is allocated a dynamically shared time slot pool. Furthermore, during actual transmission, the star-flash management node is configured to dynamically adjust the time slot ratio of L2 and L3 level data to maximize channel resource utilization.

[0020] Preferably, the StarSpark basic service layer is configured with a VLA StarSpark collaborative control module. After the time-sensitive hierarchical scheduling mechanism is configured, the VLA StarSpark collaborative control module is configured to establish a status information channel in the following ways: establish a local communication channel with the VLA model inference engine to obtain the status information of the inference task in real time; receive the expected inference completion time pushed by the VLA model inference engine, and pre-trigger the StarSpark management node to reserve a dedicated L1-level time slot for the control command to be generated based on the expected inference completion time; and dynamically adjust the communication parameters based on the robotic arm status information uploaded by the robotic arm control unit.

[0021] According to a second aspect of the present invention, a VLA end-to-end control method for an intelligent robotic arm based on the system described in the first aspect of the present invention is provided. The method includes: S1, powering on the system and performing end-to-end time synchronization operation on the system; S2, after the time synchronization operation is completed, configuring a time-sensitive hierarchical scheduling mechanism; S3, after the time-sensitive hierarchical scheduling mechanism is configured, establishing a status information channel; S4, a sensing information module collects status information and user instructions and uploads them to an edge server via a wireless transmission module. The VLA model inference engine on the edge server begins inference. After inference is completed, an action instruction sequence is generated and transmitted to a human-machine interaction module via the wireless transmission module to control the intelligent robotic arm to execute the action sequence.

[0022] Preferably, step S1 includes: S11, the StarSpark management node on the edge server side broadcasts network beacon frames to complete full-channel scanning and interference detection of the working frequency band, and selects the channel with the highest signal-to-noise ratio as the working channel; S12, each StarSpark terminal node sequentially sends an access request to the StarSpark management node to complete two-way authentication and device function registration; S13, the StarSpark management node broadcasts a timestamp synchronization beacon frame to all connected StarSpark terminal nodes at a preset time interval. After receiving the synchronization beacon, each StarSpark terminal node corrects its local clock in real time and feeds back the current clock drift amount to the StarSpark management node; S14, the StarSpark management node performs real-time clock drift compensation to dynamically adjust the next synchronization beacon transmission time based on the historical drift data of each node; S15, the VLA model inference engine and the joint servo controller of the robotic arm execution unit complete calibration with the global clock of the StarSpark terminal nodes and the StarSpark management node through their respective time synchronization interfaces to ensure that the sensor information collected by the terminal nodes, model inference, and instructions issued by the management node use a unified global timestamp.

[0023] Preferably, step S2 includes: after completing the time synchronization operation, the StarSpark management node loads a time-sensitive hierarchical scheduling mechanism to divide the data transmitted in the system into three time-sensitive levels, wherein the time-sensitive levels include: L1 level, which is the control command level; L2 level, which is the force data level; and L3 level, which is the visual or status data level; the StarSpark transmission mode includes a deterministic transmission mode and an adaptive transmission mode; the StarSpark management node maps different time-sensitive levels to different StarSpark transmission modes, wherein L1 level is mapped to the deterministic transmission mode and is allocated exclusive periodic time slots and adopts an ultra-short frame structure and forward error correction coding; L2 level is mapped to the adaptive transmission mode and is allocated a dynamic shared time slot pool with a higher priority than L3 level data; L3 level is mapped to the adaptive transmission mode and is allocated a dynamic shared time slot pool; during actual transmission, the StarSpark management node dynamically adjusts the time slot ratio of L2 level and L3 level data to maximize channel resource utilization.

[0024] Preferably, step S3 includes: S31, the VLA Star-Spark Cooperative Control Module and the VLA Model Inference Engine establish a local communication channel to obtain the status information of the inference task in real time; S32, when the VLA Model Inference Engine receives new aligned multimodal perception data and starts inference, it immediately sends an "Inference Start" signal to the VLA Star-Spark Cooperative Control Module, and simultaneously pushes the estimated inference completion time obtained from training based on historical data; S33, the VLA Star-Spark Cooperative Control Module, according to the estimated inference completion time, triggers the Star-Spark Management Node in advance to reserve a dedicated L1-level time slot for the control command to be generated, so as to ensure that the control command can immediately enter the transmission channel after the inference is completed, without waiting for resource allocation.

[0025] Compared with the prior art, the advantages of the present invention are as follows: The present invention introduces a wireless communication method to realize the wireless connection of control information of the embodied intelligent body represented by the robotic arm, which improves the system deployment flexibility; The present invention proposes a star-flash hierarchical scheduling mechanism based on time sensitivity level to solve the resource conflict problem of VLA heterogeneous multimodal data and realize ultra-low latency and high reliability deterministic transmission of control commands; The star-flash network driven end-to-end time synchronization system constructed by the present invention can realize microsecond-level precise alignment of the entire process of perception, reasoning and control; (4) Design an integrated closed-loop collaborative architecture of communication, computing and control to compress the VLA end-to-end closed-loop cycle and improve the system response speed. Attached Figure Description

[0026] The embodiments of the present invention will be further described below with reference to the accompanying drawings, wherein:

[0027] Figure 1 This is a schematic diagram of the integrated communication, computing, and control architecture of the StarSpark-based intelligent robotic arm VLA end-to-end control system according to an embodiment of the present invention.

[0028] Figure 2 This is a schematic diagram of the information flow of the VLA end-to-end control system for a star-flash-based intelligent robotic arm according to an embodiment of the present invention. Detailed Implementation

[0029] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are merely illustrative and are not intended to limit the invention.

[0030] As mentioned in the background section, existing industrial robotic arm communication solutions suffer from the following problems:

[0031] 1. Wired communication methods are not suitable for the stringent communication requirements of VLA end-to-end systems. They rely on cables and are prone to problems such as connector contact oxidation, wear, loosening, cable bending, and breakage, which significantly reduce the long-term reliability of the system, increase the probability of equipment failure and maintenance costs, and cannot adapt to the production needs of flexible production lines for rapid changeover, flexible equipment deployment, and dynamic workstation reconfiguration.

[0032] 2. The WIFI communication scheme adopts a carrier sense random access mechanism, which has strong randomness in transmission delay and large jitter amplitude. It lacks deterministic scheduling capability and is difficult to meet the requirements of stable delay and concurrent transmission of multiple sensors in the VLA system.

[0033] 3. Bluetooth technology has inherent limitations in transmission bandwidth and coverage. Traditional piconet can only support a small number of active devices and cannot simultaneously meet the requirements of high-bandwidth data transmission and ultra-low latency response.

[0034] 4. Zigbee technology focuses on low-power and low-speed sensor data transmission. Its data transmission rate is relatively low, and its real-time performance and node carrying capacity are weak. It cannot meet the high-speed, real-time, and high-precision transmission requirements of VLA closed-loop control of robotic arms.

[0035] Through research on SparkLink technology, the inventors discovered that SparkLink, as a representative of next-generation short-range wireless communication technology, is bringing new opportunities to wireless communication in embodied intelligent body domains. The system architecture of SparkLink technology consists of three parts: the SparkLink access layer, the basic service layer, and the basic application layer. The access layer provides two communication interfaces: SparkLink Basic (SLB) and SparkLink Low Energy (SLE). The SLB interface employs multiple technologies such as ultra-short frames, multi-point synchronization, two-way authentication, fast interference coordination, two-way authentication encryption, and cross-layer scheduling optimization to support service scenarios with transmission requirements such as low latency, high reliability, precise synchronization, high concurrency, and high security. SLE uses Polar channel coding to improve transmission reliability, reduce retransmissions, and save power, while supporting a maximum transmission bandwidth of 4 MHz and a maximum of 8 phase shift keying modulation, fully considering energy-saving factors to support service scenarios with low power consumption and high reliability requirements.

[0036] However, it must be acknowledged that although Starflash technology has significant performance advantages in short-range wireless communication, its research and application are currently concentrated in consumer electronics fields such as audio and interactive media. There are no concrete designs that systematically apply this new short-range wireless communication technology to the end-to-end closed-loop architecture of VLA (Virtual Link) systems for robotic arms. Directly using existing standard interfaces for VLA systems will face challenges such as how to deterministically mix and schedule multimodal data streams and multichannel control streams within the same wireless domain, how to coordinate end-to-end time synchronization with model inference, and how to achieve system-level end-to-end optimization between communication, computation, and control.

[0037] In view of this, this invention proposes an end-to-end control scheme for a smart robotic arm using a star-flash-based VLA (Virtual Linkage) architecture. In this invention, specific designs are implemented to address the characteristics of the smart robotic arm's VLA end-to-end scenario: First, this invention proposes a star-flash-based hierarchical scheduling mechanism based on time sensitivity levels to resolve resource conflicts in heterogeneous multimodal data; second, this invention constructs a star-flash network-driven end-to-end time synchronization mechanism to achieve precise alignment of perception, inference, and control stages; third, this invention designs an integrated closed-loop collaborative architecture for communication, computation, and control to reduce unnecessary system latency.

[0038] Specifically, the time-sensitive hierarchical scheduling mechanism proposed in this invention refers to the extension of time-sensitive hierarchical scheduling at the star-light access layer. This mechanism divides VLA data into three strict time-sensitive levels and maps them to different transmission modes, namely:

[0039] L1 level (control command level): a deterministic transmission mode mapped to the StarSpark basic access interface, allocating exclusive periodic time slots, prohibiting any low-priority data preemption, and adopting an ultra-short frame structure to ensure ultra-low latency and ultra-high reliability communication capabilities.

[0040] Level L2 (force sensing data level): Mapped to the star-flash adaptive transmission mode, allocated with dynamic shared time slots, and has a higher priority than Level L3 data;

[0041] Level 3 (Vision / Status Data Level): Mapped to the Starflash adaptive transmission mode and employing an adaptive switching mechanism, based on fragmented transmission and out-of-order reassembly mechanism to achieve data transmission.

[0042] The end-to-end time synchronization mechanism driven by the StarSpark network constructed in this invention refers to building a full-link time synchronization system for sensors, edge nodes, and controllers with the StarSpark management node (G node) as the global time reference. A time synchronization interface has been added to the VLA inference module and joint controller to ensure that all links of sensor sampling, model inference, and instruction execution use a unified global timestamp.

[0043] The communication, computing, and control integrated closed-loop collaborative architecture designed in this invention refers to the addition of a VLA StarSpark collaborative control module to the StarSpark basic service layer. When the VLA model starts inference, it sends the estimated inference completion time to the VLA StarSpark collaborative control module. The VLA StarSpark collaborative control module reserves dedicated transmission resources for the control commands to be generated in advance, so that the command is transmitted immediately after the inference ends, thus reducing the delay in command transmission.

[0044] To better understand the present invention, the present invention will be described in detail below with reference to the accompanying drawings and embodiments.

[0045] According to one embodiment of the present invention, the present invention provides a VLA end-to-end control system for a smart robotic arm based on starlight, such as... Figure 1As shown, the system includes: a human-machine interaction module, a perception information module, an intelligent computing module, and a wireless transmission module. The human-machine interaction module is configured as an intelligent robotic arm, which includes a robotic arm main control unit and a robotic arm execution unit. Under the control of the robotic arm main control unit, the robotic arm execution unit executes the action command sequence of the intelligent computing module to complete user commands. The robotic arm execution unit includes multiple joint servo controllers deployed at the joints of the robotic arm, each equipped with a time synchronization interface. The perception information module is used to collect environmental (visual information) and intelligent robotic arm state data (robotic arm state information) in real time to form multimodal perception data, and to receive user commands (i.e., natural language command information). It transmits the multimodal perception data and user commands to the intelligent computing module via the wireless transmission module. It includes a depth camera configured at the end effector of the intelligent robotic arm, a force sensor configured at the force application position of the intelligent robotic arm, a joint encoder configured at the joints of the intelligent robotic arm, and a third-view depth camera and a state monitoring sensor configured in the working environment of the intelligent robotic arm. The intelligent computing module, an edge server equipped with a StarFlash communication module, houses a VLA model inference engine and a time synchronization interface. It generates action command sequences based on multimodal sensing data transmitted from the sensing information module and user commands, transmitting these sequences wirelessly to the human-machine interface module to control the intelligent robotic arm. The wireless transmission module, also a StarFlash communication module, facilitates data communication between the human-machine interface module and the intelligent computing module. It performs end-to-end time synchronization across the entire link through the time synchronization interfaces of each servo controller and the VLA model inference engine, ensuring that a unified global timestamp is used in all stages of sensor sampling, model inference, and command execution.

[0046] Since the physical layer logic of each module of StarSpark and the inference logic of the VLA model inference engine are technologies known to those skilled in the art, the embodiments of this invention will not elaborate on the principle of StarSpark itself or the specific price of the VLA model, but will focus on the wireless control implemented by StarSpark in the VLA end-to-end system.

[0047] The wireless transmission module includes: a StarFlash management node (G node, deployed on the edge server side), multiple StarFlash terminal nodes (T nodes, deployed on the main control unit of the robotic arm, various depth camera components, etc.), and a VLA collaborative control module is also configured in the StarFlash base layer.

[0048] The VLA end-to-end control system and its data flow constructed using the above modules are as follows: Figure 2As shown, the star-flash terminal nodes in the main control unit of the robotic arm constitute a star-flash communication module (T1), the star-flash terminal nodes of the depth camera at the end of the robotic arm constitute a star-flash communication module (T2), and the star-flash terminal nodes of the third-view depth camera constitute a star-flash communication module (T3). From Figure 2 As can be seen, the multimodal sensors in the perception information module collect environmental and robotic arm status data in real time, and transmit them wirelessly to the StarSpark G node on the edge server side via the StarSpark T node. Among them, the StarSpark communication module (T1) transmits the robotic arm motion status information, the StarSpark communication module (T2) transmits the video stream from the first-person perspective of the robotic arm, and the StarSpark communication module (T3) transmits the video stream from the third-person perspective. The edge server inputs the multimodal perception data and natural language task instructions into the VLA model for end-to-end inference to generate a sequence of robotic arm joint actions. The action sequence is wirelessly transmitted to the StarSpark T node of the robotic arm main control unit via the StarSpark G node. After parsing, it drives each joint controller to execute the corresponding action, forming a closed loop of perception, transmission, calculation, and control.

[0049] In the actual operation of the system, end-to-end time synchronization, time-sensitive hierarchical scheduling configuration, and the activation of the integrated collaboration mechanism are required before finally entering the normal working state of end-to-end inference. Each stage is explained below.

[0050] I. StarNetwork Initialization and End-to-End Time Synchronization

[0051] After the system powers on and starts up, it first performs the StarScan network setup and end-to-end time synchronization operation to provide a unified high-precision time reference for subsequent data transmission, including the following steps:

[0052] Step 101: The StarScan G node on the edge server side starts as the network coordinator, broadcasts network beacon frames, completes full-channel scanning and interference detection of the working frequency band, and selects the channel with the highest signal-to-noise ratio as the working channel.

[0053] Step 102: The StarShine T nodes on the robotic arm main control unit, depth camera and other devices send access requests to the StarShine G nodes in sequence to complete two-way identity authentication and device function registration.

[0054] Step 103: Activate the end-to-end time synchronization mechanism driven by the StarSpark network: StarSpark G node serves as the globally unique time reference and broadcasts a synchronization beacon frame with a timestamp to all connected StarSpark T nodes every 8ms (or other time intervals, which can be set according to the use case); After receiving the synchronization beacon, each StarSpark T node corrects its local clock in real time and feeds back the clock drift amount to the StarSpark G node.

[0055] Step 104: The StarShock G node runs a real-time clock drift compensation algorithm to dynamically adjust the transmission time of the next synchronization beacon based on the historical drift data of each node. Since the clock drift compensation algorithm in StarShock is a technology known to those skilled in the art, it will not be described in detail in this embodiment of the invention.

[0056] Step 105: The VLA model inference engine of the edge server and the joint servo controller of the robotic arm execution unit are calibrated with the global always-on StarSpark G node through the newly added time synchronization interface to ensure that all environments, including sensor sampling, model inference, and instruction execution, use a unified global timestamp.

[0057] II. Starlight Tiered Scheduling Configuration Based on Time Sensitivity Level

[0058] After network synchronization is complete, the StarShine G node automatically loads and configures hierarchical scheduling rules based on the service characteristics of VLA, allocating differentiated transmission resources for different types of data, including the following steps:

[0059] Step 201: The StarScan G node loads a time-sensitive hierarchical scheduling algorithm at the access layer, dividing the data transmitted by the VLA system into three time-sensitive levels and mapping them to the corresponding StarScan transmission modes:

[0060] Level L1 (control command level) maps to the deterministic transmission mode of the StarSpark basic access interface, allocates exclusive periodic time slots, and prohibits any low-priority data preemption at the protocol level; it adopts an ultra-short frame structure and forward error correction coding to ensure ultra-low latency and ultra-high reliability transmission of control commands.

[0061] Level L2 (force sensor data level) is mapped to the star-flash adaptive transmission mode, and a dynamically shared time slot pool is allocated, with higher priority than Level L3 data. The number of time slots is dynamically adjusted according to the sampling rate of the force sensor, and adaptive modulation and coding technology is used to balance transmission efficiency and reliability.

[0062] Level L3 (visual / state data level) is mapped to the star-flash adaptive transmission mode, and the interface is dynamically switched according to the channel quality and data volume. For high-resolution visual data, a fragmented transmission and out-of-order reassembly mechanism is adopted to split large frame data into multiple small fragments for parallel transmission. The receiving end reassembles and restores the data according to the global timestamp.

[0063] Step 202: The StarShine G node monitors the traffic changes of each level of data in real time, runs the traffic-aware dynamic resource adjustment algorithm, and dynamically adjusts the time slot ratio of L2 and L3 levels to maximize channel resource utilization while ensuring the absolute priority of L1 level time slots.

[0064] III. Activation of the Integrated Collaborative Mechanism for Communication, Computing, and Control

[0065] After the hierarchical scheduling configuration is completed, the VLA StarSpark collaborative control module deployed in the StarSpark basic service layer is started, establishing status information channels between the communication layer, computing layer, and control layer, including:

[0066] Step 301: The VLA Star Flash Collaborative Control Module establishes a dedicated local communication channel with the VLA model inference engine of the edge server to obtain the status information of the inference task in real time.

[0067] Step 302: When the VLA model inference engine receives new aligned multimodal perception data and starts inference, it immediately sends an "inference start" signal to the VLA Star Flash Cooperative Control Module and pushes the estimated inference completion time based on historical data training.

[0068] Step 303: The VLA StarSpark Cooperative Control Module reserves a dedicated L1 time slot in advance in the scheduling table of the StarSpark G node for the control commands to be generated based on the expected inference completion time, so as to ensure that the control commands can enter the transmission channel immediately after the inference is completed without waiting for resource allocation.

[0069] Step 304: The main control unit of the robotic arm is connected to the execution unit of the robotic arm via a wired connection. It collects status information such as joint angle, angular velocity, and preset motion trajectory in real time and sends it to the VLA Star Flash Cooperative Control Module through the Star Flash T node. The VLA Star Flash Cooperative Control Module dynamically adjusts the communication parameters according to the motion status of the robotic arm.

[0070] IV. VLA end-to-end closed-loop information transmission process;

[0071] After the system completes all initialization and configuration, it enters normal working state, with its complete information transmission and control process.

[0072] Step 401: The depth camera acquires environmental visual images, the force sensor acquires three-dimensional force and torque information at the end effector, and the joint encoder acquires angle data for each joint. While acquiring data, all sensors add a precise global timestamp to each set of data. The force data and joint encoder data are uploaded as L2-level data through the Star Flash T node in the allocated dynamic shared time slot. The visual data is uploaded as L3-level data in segments.

[0073] Step 402: After receiving all uploaded data, the StarSpark G node sorts and aligns the data according to the global timestamp, and sends the aligned multimodal perception data to the VLA model inference engine of the edge server.

[0074] Step 403: The VLA model inference engine receives the aligned multimodal perception data and the natural language task instructions input by the user, and performs end-to-end inference.

[0075] Step 404: When inference begins, the VLA model inference engine sends the estimated inference completion time to the StarFlash VLA StarFlash Cooperative Control Module. Based on the estimated completion time, the VLA StarFlash Cooperative Control Module reserves an exclusive L1 time slot in advance for the control commands that will be generated.

[0076] Step 405: After inference is completed, the VLA model inference engine generates a sequence of action instructions containing the target angles, velocities, and execution timestamps of each joint, and immediately sends it to the StarShutter G node.

[0077] Step 406: After receiving the control command, the Star Flash G node sends the command to the Star Flash T node of the robotic arm main control unit through the reserved L1 level time slot;

[0078] Step 407: After receiving the instruction, the main control unit of the robotic arm distributes the control instructions of each joint to the corresponding joint servo controller at the precise time point according to the global timestamp in the instruction.

[0079] Step 408: The joint servo controller executes control commands to drive the robotic arm body to complete the corresponding actions;

[0080] Step 409: During the execution of the action, each sensor continuously collects and uploads status data, and the next closed-loop cycle begins.

[0081] Compared with the existing technology, the present invention introduces wireless communication to realize wireless connection of control information of embodied intelligent agents represented by robotic arms, which improves the system deployment flexibility; the present invention proposes a star-flash hierarchical scheduling mechanism based on time sensitivity level to solve the resource conflict problem of VLA heterogeneous multimodal data and realize ultra-low latency and high reliability deterministic transmission of control commands; the star-flash network driven end-to-end time synchronization system constructed by the present invention can realize microsecond-level precise alignment of the entire process of perception, reasoning and control; (4) design an integrated closed-loop collaborative architecture of communication, computing and control to compress the VLA end-to-end closed-loop cycle and improve the system response speed.

[0082] It should be noted that although the steps are described in a specific order above, it does not mean that the steps must be executed in the above specific order. In fact, some of these steps can be executed concurrently, or even in a different order, as long as the required function can be achieved.

[0083] This invention can be a system, method, electronic device, computing device, computer-readable medium, and / or computer program product. A computer program product primarily refers to a software product that implements this solution through a computer program.

[0084] A computer-readable storage medium can be a tangible device that holds and stores instructions for use by an instruction execution device. Computer-readable storage media can include, for example, but not limited to, electrical storage devices, magnetic storage devices, optical storage devices, electromagnetic storage devices, semiconductor storage devices, or any suitable combination thereof. More specific examples (a non-exhaustive list) of computer-readable storage media include: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), portable compact disc read-only memory (CD-ROM), digital multifunction disc (DVD), memory sticks, floppy disks, mechanical encoding devices, such as punch cards or recessed protrusions storing instructions thereon, and any suitable combination thereof.

[0085] The various embodiments of the present invention have been described above. These descriptions are exemplary and not exhaustive, nor are they limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is chosen to best explain the principles, practical application, or technical improvements to the embodiments in the market, or to enable others skilled in the art to understand the embodiments disclosed herein.

Claims

1. A VLA end-to-end control system for an intelligent robotic arm based on star-flash technology, characterized in that, The system includes: a human-computer interaction module, a sensing information module, an intelligent computing module, and a wireless transmission module, wherein: The human-computer interaction module is configured as an intelligent robotic arm. The intelligent robotic arm includes a robotic arm main control unit and a robotic arm execution unit. Under the control of the robotic arm main control unit, the robotic arm execution unit executes the action instruction sequence of the intelligent computing module to complete the user's instructions. The robotic arm execution unit includes multiple joint servo controllers deployed at the joints of the robotic arm, and each joint servo controller is configured with a time synchronization interface. The perception information module is used to collect environmental and intelligent robotic arm status data in real time to form multimodal perception data, and to receive user commands and transmit the multimodal perception data and user commands to the intelligent computing module through the wireless transmission module; it includes a depth camera configured at the end of the intelligent robotic arm, a force sensor configured at the force application position of the intelligent robotic arm, a joint encoder configured at the joint of the intelligent robotic arm, and a third-view depth camera and status monitoring sensor configured in the working environment of the intelligent robotic arm; The intelligent computing module is an edge server equipped with a StarFlash communication module. It is equipped with a VLA model inference engine and a time synchronization interface. It is used to generate a sequence of action instructions based on the multimodal perception data transmitted by the perception information module and user instructions, and transmit it to the human-machine interaction module through the wireless transmission module to control the intelligent robotic arm to execute. The wireless transmission module, configured as a StarFlash communication module, is used for data communication between the human-computer interaction module and the intelligent computing module. The wireless transmission module performs end-to-end time synchronization through the time synchronization interface of each servo controller and the time synchronization interface of the VLA model inference engine to ensure that all aspects of sensor sampling, model inference, and instruction execution use a unified global timestamp.

2. The VLA end-to-end control system for a star-flash-based intelligent robotic arm according to claim 1, characterized in that, The wireless transmission module includes one StarScan management node and multiple StarScan terminal nodes, wherein: The StarShine management node is deployed within the intelligent computing module; Multiple StarFlash terminal nodes are deployed one-to-one in the depth camera, third-view depth camera and robotic arm main control unit of the perception information module; Among them, the StarFlash management node is used to align the multimodal sensing data uploaded by multiple StarFlash terminal nodes according to time, as well as to schedule and allocate communication time slots and forward action instruction sequences; Each StarSpark terminal node is used to upload the data sensed by its component to the StarSpark management node and to receive control action command sequences.

3. The VLA end-to-end control system for a star-flash-based intelligent robotic arm according to claim 2, characterized in that, The wireless transmission module is configured to perform end-to-end time synchronization at system startup as follows: The Starflash management node broadcasts network beacon frames to complete full-channel scanning and interference detection of the operating frequency band, and selects the channel with the highest signal-to-noise ratio as the operating channel; All StarShine terminal nodes send access requests to the StarShine management node to complete two-way authentication and device function registration; The StarSpark management node broadcasts timestamp synchronization beacon frames to all connected StarSpark terminal nodes at preset time intervals. Each StarSpark terminal node corrects its local clock in real time based on the received synchronization beacon and feeds back the local clock drift to the StarSpark management node. The StarSpark management node operates a real-time clock drift compensation mechanism based on the clock drift received from the StarSpark terminal nodes, and can dynamically adjust the transmission time of the next synchronization beacon frame according to the historical drift data of each StarSpark terminal node. The StarSpark management node triggers the VLA model inference engine on the edge service and the time synchronization interface of each joint servo controller to complete the calibration with the global clock of the StarSpark management node, so as to ensure that all aspects of sensor sampling, model inference and instruction execution use a unified global timestamp.

4. The VLA end-to-end control system for a star-flash-based intelligent robotic arm according to claim 3, characterized in that, The StarSpark management node is configured to automatically load and configure a time-sensitive hierarchical scheduling mechanism based on the system's business characteristics after time synchronization is completed. This mechanism divides data into different time-sensitive levels and maps them to different StarSpark transmission modes to allocate differentiated transmission resources to different data.

5. The VLA end-to-end control system for a star-flash-based intelligent robotic arm according to claim 4, characterized in that: In the time-sensitive hierarchical scheduling mechanism, the time-sensitive levels include: L1 level, which is the control command level; L2 level, which is the force data level; and L3 level, which is the visual or state data level. The star flash transmission mode includes deterministic transmission mode and adaptive transmission mode. Specifically, L1 level is mapped to deterministic transmission mode and is allocated exclusive periodic time slots and uses an ultra-short frame structure and forward error correction coding; L2 level is mapped to adaptive transmission mode and is allocated a dynamic shared time slot pool with higher priority than L3 level data; L3 level is mapped to adaptive transmission mode and is allocated a dynamic shared time slot pool. Furthermore, during actual transmission, the StarScan management node is configured to dynamically adjust the time slot ratio of L2 and L3 data to maximize channel resource utilization.

6. The VLA end-to-end control system for a star-flash-based intelligent robotic arm according to claim 5, characterized in that, The StarShine basic service layer is configured with a VLA StarShine collaborative control module. After the time-sensitive hierarchical scheduling mechanism is configured, the VLA StarShine collaborative control module is configured to establish a status information channel in the following way: Establish a local communication channel with the VLA model inference engine to obtain the status information of the inference task in real time; Receive the estimated inference completion time pushed by the VLA model inference engine, and pre-trigger the StarFlash management node to reserve a dedicated L1 time slot for the control commands to be generated based on the estimated inference completion time; The communication parameters are dynamically adjusted based on the robotic arm status information uploaded by the robotic arm control unit.

7. A VLA end-to-end control method for an intelligent robotic arm based on the system described in any one of claims 1-6, characterized in that, The method includes: S1. Power on the system and start the system to perform end-to-end time synchronization operation across the entire link; S2. After the time synchronization operation is completed, configure a time-sensitive hierarchical scheduling mechanism; S3. After the time-sensitive hierarchical scheduling mechanism is configured, the status information channel is opened. S4. The perception information module collects status information and user commands and uploads them to the edge server via the wireless transmission module. The VLA model inference engine on the edge server starts inference. After the inference is completed, it generates a sequence of action commands and transmits them to the human-machine interaction module via the wireless transmission module to control the intelligent robotic arm to execute the action sequence.

8. The intelligent robotic arm VLA end-to-end control method according to claim 7, characterized in that, S1 includes: S11. The star-flash management node on the edge server side broadcasts network beacon frames to complete full-channel scanning and interference detection of the working frequency band, and selects the channel with the highest signal-to-noise ratio as the working channel. S12. Each StarShine terminal node sends an access request to the StarShine management node in sequence to complete two-way identity authentication and device function registration; S13. The StarSpark management node broadcasts a synchronization beacon frame with a timestamp to all connected StarSpark terminal nodes at a preset time interval. After receiving the synchronization beacon, each StarSpark terminal node corrects its local clock in real time and feeds back the clock drift amount to the StarSpark management node. S14. Real-time compensation for clock drift in the Star Flash management node to dynamically adjust the next synchronization beacon transmission time based on the historical drift data of each node; The S15, VLA model inference engine, and the joint servo controller of the robotic arm execution unit are calibrated with the global clocks of the StarFlash terminal node and StarFlash management node through their respective time synchronization interfaces to ensure that the sensor information collected by the terminal node, model inference, and instructions issued by the management node use a unified global timestamp.

9. The intelligent robotic arm VLA end-to-end control method according to claim 8, characterized in that, S2 includes: After completing the time synchronization operation, the StarShock management node loads the time-sensitive hierarchical scheduling mechanism, which divides the data transmitted in the system into three time-sensitive levels. The time-sensitive levels include: L1 level, which is the control command level; L2 level, which is the force data level; and L3 level, which is the visual or status data level. The StarShock transmission mode includes deterministic transmission mode and adaptive transmission mode. The StarScan management node maps different time sensitivity levels to different StarScan transmission modes. Specifically, L1 level is mapped to the deterministic transmission mode and is allocated exclusive periodic time slots with an ultra-short frame structure and forward error correction coding. L2 level is mapped to the adaptive transmission mode and is allocated a dynamic shared time slot pool with higher priority than L3 level data. L3 level is also mapped to the adaptive transmission mode and is allocated a dynamic shared time slot pool. In actual transmission, the time slot ratio of L2 and L3 data is dynamically adjusted by the Star Flash management node to maximize channel resource utilization.

10. The intelligent robotic arm VLA end-to-end control method according to claim 9, characterized in that, S3 includes: S31, the VLA Star Flash Cooperative Control Module establishes a local communication channel with the VLA Model Inference Engine to obtain the status information of the inference task in real time; S32. When the VLA model inference engine receives new aligned multimodal perception data and starts inference, it immediately sends an "inference start" signal to the VLA Star Flash Cooperative Control Module and pushes the estimated inference completion time based on historical data training. Based on the estimated inference completion time, the S33 and VLA Star Flash Cooperative Control Modules pre-trigger the Star Flash Management Node to reserve a dedicated L1-level time slot for the control commands to be generated, ensuring that the control commands can immediately enter the transmission channel after the inference is completed, without waiting for resource allocation.

11. A computer program product comprising a computer program / instructions that, when executed by a processor, implement the steps of the method as claimed in any one of claims 7-10.

Images