Action queue control method and device, equipment and medium

Abstract

The invention relates to the technical field of artificial intelligence, can be applied to business scenes of financial science and technology, medical health and the like, and discloses an action queue control method, device, equipment and medium, and the method comprises the steps: obtaining initial observation data, sending the initial observation data to a strategy server, and receiving a generated initial action queue; executing the action in the action queue; when the residual amount of the queue is lower than a threshold value, obtaining new observation data; judging whether action prediction is triggered or not based on the similarity between the data and historical observation data; if yes, the new observation data are sent to the strategy server in a non-blocking mode, and a new action block is generated; and receiving a new action block, integrating the new action block with the action queue to form an updated action queue, and repeatedly executing the updated action queue as a new action queue until all actions are completed. According to the method, the action blocks are generated in parallel in the action execution process in an asynchronous reasoning mode, control blind areas and waiting delay are reduced, response timeliness and continuous control capacity are improved, and the requirement of edge equipment for resource efficiency is met.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, and in particular to an action queue control method, apparatus, device, and storage medium. Background Technology

[0002] In current robot control systems, motion decision-making commonly employs a synchronous inference model. This means that the control system only acquires observation data and generates the next set of motion instructions after all actions in the entire action block have been executed. While this serial control method has a clear structure, it exposes many technical limitations during actual deployment, especially in tasks requiring responses to complex environmental dynamics, easily leading to severe control lag and resource waste.

[0003] During the long action block delay between the raw observation data and the action prediction, the system is in an open-loop state that does not accept new environmental inputs and cannot adjust the control strategy in real time according to environmental changes. This execution process, lacking feedback loop, results in a severely inadequate ability to respond to emergencies and environmental disturbances, with obvious control blind spots, especially in highly dynamic or unstructured environments.

[0004] In the fintech business, motion control for smart terminals has been applied to task scenarios such as self-service teller robots and intelligent inspection systems. If synchronous inference mode is adopted, once the execution instruction queue is exhausted, the system will enter a state of waiting for new prediction results, resulting in unavoidable idle lag, which directly affects the user response speed and interaction experience, and reduces the throughput and continuity of the business system.

[0005] In the healthcare sector, scenarios such as service robots used for assisted operations and ward delivery robots place particular emphasis on task continuity and environmental feedback processing capabilities. However, the serial latency introduced by synchronous inference can cause robots to pause and wait during critical tasks (such as obstacle avoidance and path adjustment), thereby affecting patient safety and healthcare efficiency, and making it difficult to meet the clinical requirements that balance real-time performance and safety.

[0006] Furthermore, although there are existing control strategies employing step-by-step inference to improve the system's adaptability to dynamic environments, these methods often require high-frequency calls to the policy network, consuming significant computational resources. In edge computing devices or low-power terminal deployment scenarios, this approach struggles to balance computational efficiency and responsiveness, further limiting its widespread application in complex real-world tasks. Summary of the Invention

[0007] The main objective of this invention is to provide an action queue control method, apparatus, device, and storage medium, aiming to solve the technical problem in the prior art where the action control adopts a synchronous reasoning mode, resulting in the system being unable to acquire new observation data during action execution, leading to control blind spots and idle delays.

[0008] To achieve the above objectives, the present invention provides an action queue control method, comprising:

[0009] Acquire initial observation data and send the initial observation data to the policy server, and receive the initial action queue generated by the policy server;

[0010] Obtain an action from the initial action queue and execute the action;

[0011] When the remaining amount in the action queue is lower than a preset threshold, new observation data is acquired;

[0012] Based on the similarity between the new observation data and the historical observation data, determine whether to trigger action prediction;

[0013] When it is determined that an action prediction is triggered, the new observation data is sent to the policy server in a non-blocking manner, and a new action block is generated by the policy server.

[0014] Receive a new action block generated by the policy server, and integrate the new action block with the action queue to generate an updated action queue;

[0015] The updated action queue is used as a new action queue, and the process of action acquisition and execution, queue monitoring, similarity judgment, prediction triggering, queue integration and queue update is repeated until all actions are completed.

[0016] Furthermore, to achieve the above objectives, the present invention provides an action queue control device, comprising:

[0017] The observation data acquisition module is used to acquire initial observation data and send the initial observation data to the policy server, and to receive the initial action queue generated by the policy server.

[0018] An action execution module is used to retrieve actions from the initial action queue and execute the actions;

[0019] The queue monitoring module is used to acquire new observation data when the remaining amount in the action queue is lower than a preset threshold.

[0020] The similarity judgment module is used to determine whether to trigger action prediction based on the similarity between the new observation data and the historical observation data;

[0021] An asynchronous prediction triggering module is used to send the new observation data to the policy server in a non-blocking manner when it is determined that a trigger action prediction is needed, and to generate a new action block through the policy server.

[0022] The action queue integration module is used to receive new action blocks generated by the policy server and integrate the new action blocks with the action queue to generate an updated action queue.

[0023] The action flow control module is used to treat the updated action queue as a new action queue and repeatedly execute the action acquisition and execution, queue monitoring, similarity judgment, prediction triggering, queue integration and queue update process until all actions are completed.

[0024] Furthermore, to achieve the above objectives, the present invention also provides a computer device, the computer device including a memory, a processor, and an action queue control program stored in the memory and executable on the processor, wherein the action queue control program, when executed by the processor, implements the steps of the action queue control method as described above.

[0025] Furthermore, to achieve the above objectives, the present invention also provides a computer-readable storage medium storing an action queue control program, which, when executed by a processor, implements the steps of the action queue control method described above.

[0026] Beneficial Effects: This invention relates to the field of artificial intelligence technology and can be applied to business scenarios such as fintech and healthcare. It discloses an action queue control method, apparatus, device, and medium, comprising: acquiring initial observation data and sending it to a policy server; receiving an initial action queue generated by the policy server; acquiring and executing actions from the initial action queue; acquiring new observation data when the remaining amount in the action queue is lower than a preset threshold; determining whether action prediction is triggered based on the similarity between the new observation data and historical observation data; if action prediction is triggered, sending the new observation data to the policy server in a non-blocking manner and generating a new action block; receiving the new action block and integrating it with the action queue to generate an updated action queue; using the updated action queue as a new action queue and repeatedly executing the action acquisition, execution, and queue update process until the action is completed. This invention, by employing an asynchronous inference control mechanism, generates new action blocks and integrates queues in parallel before the action queue is exhausted, avoiding control blind spots and idle waiting during execution, effectively improving the real-time performance of action response and the continuity of system operation. It is suitable for intelligent control tasks in scenarios with limited computing resources. Attached Figure Description

[0027] The present invention will be further described below with reference to the accompanying drawings and embodiments. In the accompanying drawings:

[0028] Figure 1 This is a schematic diagram of an application environment for the action queue control method in one embodiment of the present invention;

[0029] Figure 2 This is a flowchart illustrating an embodiment of the action queue control method of the present invention;

[0030] Figure 3 This is a schematic diagram of the functional modules of a preferred embodiment of the action queue control device of the present invention;

[0031] Figure 4 This is a schematic diagram of the structure of a computer device according to an embodiment of the present invention;

[0032] Figure 5 This is another structural schematic diagram of a computer device according to one embodiment of the present invention. Detailed Implementation

[0033] It should be understood that the specific embodiments described herein are for illustrative purposes only and are not intended to limit the scope of the invention.

[0034] The action queue control method provided in this embodiment of the invention can be applied to, for example... Figure 1 In this application environment, the user terminal communicates with the server via a network. The server can obtain initial observation data from the user terminal and send it to the policy server, receiving the initial action queue generated by the policy server; it then retrieves and executes actions from the initial action queue; when the remaining amount in the action queue is lower than a preset threshold, it acquires new observation data; based on the similarity between the new observation data and historical observation data, it determines whether to trigger action prediction; if action prediction is triggered, it sends the new observation data to the policy server in a non-blocking manner and generates a new action block; it receives the new action block and integrates it with the action queue to generate an updated action queue; it uses the updated action queue as the new action queue and repeats the action acquisition, execution, and queue update process until the action is completed. This invention employs an asynchronous inference control mechanism, generating new action blocks and integrating queues in parallel before the action queue is exhausted, avoiding control blind spots and idle waiting during execution, effectively improving the real-time performance of action response and the continuity of system operation, and is suitable for intelligent control tasks in scenarios with limited computing resources. The user terminal can be, but is not limited to, various personal computers, laptops, smartphones, tablets, and portable wearable devices. The server can be implemented using a standalone server or a server cluster composed of multiple servers. The present invention will now be described in detail through specific embodiments.

[0035] Please see Figure 2 , Figure 2This is a flowchart illustrating an embodiment of the action queue control method provided by the present invention. It should be noted that although the logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than that shown here.

[0036] like Figure 2 As shown, the action queue control method proposed in this invention includes the following steps:

[0037] S10, acquire initial observation data and send the initial observation data to the policy server, and receive the initial action queue generated by the policy server;

[0038] In this embodiment, the initial observation data acquisition process relies on the perception of the environmental state, which is typically collected through integrated multi-source sensing devices. These devices include, but are not limited to, RGB cameras, depth cameras, inertial measurement units (IMUs), lidar, and infrared sensors, which can be combined to form a multimodal sensor array. The purpose of constructing this array is to improve the comprehensiveness and robustness of environmental perception through the temporal and spatial complementarity of heterogeneous data sources.

[0039] The raw observation data acquired after acquisition exhibits differences in resolution, sampling frequency, and encoding format, necessitating spatiotemporal alignment. Alignment in the temporal dimension can be achieved through a unified timestamp mechanism, such as time stamping based on a master clock signal. Spatial alignment relies on calibration parameters or fusion algorithms to perform coordinate system transformations, thereby integrating data from multiple sensor sources within a unified spatial reference framework. Once aligned, a data frame with consistent structure and standardized fields can be generated, serving as standard-format observation data.

[0040] To ensure the integrity and security of the transmission process, a secure communication link needs to be established. This link can achieve end-to-end encryption through TLS / SSL-based encryption protocols and employ authentication mechanisms such as token issuance and two-way handshakes to prevent man-in-the-middle attacks or forged data injection. Data transmission can be based on lightweight transmission protocols such as gRPC, MQTT, or ZeroMQ, combined with data compression and batch sending mechanisms to reduce communication overhead.

[0041] Standard-formatted observation data, upon arriving at the policy server, is fed into the model's inference process. The policy server, which can be deployed on edge computing nodes or cloud servers, primarily functions by invoking pre-trained action policy models for data inference. These pre-trained models are typically built using deep reinforcement learning or behavior cloning techniques, with input being a state-space representation and output being a structured sequence of consecutive actions. This sequence undergoes validity checks, conflict resolution, and semantic expansion by the model's post-processing module before being encapsulated into initial action blocks.

[0042] The initial action block is not used directly as the final control sequence, but is further constructed into an initial action queue. This queue has a first-in, first-out (FIFO) structure, and its internal elements are sets of action instructions. Each set contains information such as the action target, execution parameters, and expected duration. The action queue construction process involves data encapsulation, sequential arrangement, and cache write operations, which can be implemented based on a memory cache or a low-latency database.

[0043] Multimodal sensor arrays can be configured for on-demand activation, activating specific sensor types only upon startup or detection of environmental changes, thus saving power. Integrated acquisition modules can also be used to synchronize cameras and IMUs, simplifying the alignment process. For generating standard-format observation data, JSON, protobuf, or custom binary structures can be used to encapsulate multidimensional data, improving parsing and transmission efficiency.

[0044] To improve the stability of secure communication links, a QoS assessment can be performed after the link is established, automatically adjusting packet size and transmission frequency based on network latency and bandwidth conditions. For policy server deployment, local embedded computing units or cloud-based elastic container services can be selected based on real-time requirements. If deployed on edge computing nodes, it is recommended to use FPGAs or GPUs to optimize model inference speed. When building the initial action queue, expiration times or maximum queue lengths can be set to implement automatic cleanup and dynamic update mechanisms, avoiding execution delays or conflicts.

[0045] Example: In the field of healthcare, this can be used to enable surgical robots to fully perceive the surgical area environment in the initial stage and construct multi-step operation paths, such as completing initial action queues for instrument delivery and light source adjustment, thereby reducing manual intervention during surgery.

[0046] In the fintech business, it can be used by service robots to perceive the environment and customer distribution during the initialization phase of branch outlets, and generate initial action sequences such as greeting, display, and interactive prompts, so as to achieve seamless process initiation and continuous behavior response.

[0047] This embodiment, through the steps described above, enables an asynchronous closed-loop control process from observation data acquisition to the initial action queue construction. This process significantly improves the robot's real-time response capability to environmental conditions while ensuring the continuity and execution safety of the action queue. Standardized data formats and efficient communication mechanisms effectively reduce the interface adaptation complexity caused by cross-device deployment and provide highly consistent state input data for subsequent action prediction processes.

[0048] S20, Obtain an action from the initial action queue and execute the action;

[0049] In this embodiment, the process of obtaining actions relies on the organizational structure of the action queue, which is typically constructed in a first-in, first-out (FIFO) manner and stored in a local cache or temporary memory area. Each action instruction in the action queue contains fields such as target module identifier, parameter configuration set, expected duration, and execution priority. To extract the correct action to be executed, it is necessary to first locate the actual storage address of the queue in the cache. The storage structure can be implemented using a linear array, linked list, or circular buffer. By querying the position of the queue head pointer, the index of the currently pending action can be obtained.

[0050] After the action command is extracted, its internal structure fields need to be parsed to extract the action type and associated parameters. The action type may include movement, grasping, rotation, feedback monitoring, etc., while the parameters define specific control quantities such as execution amplitude, direction, speed, and constraint limits. These control parameters need to be converted into a sequence of control signals that can be recognized by the lower-level devices. The control signals can be represented using PWM (Pulse Width Modulation), serial port commands, CAN bus messages, or other industrial communication protocol formats, and timing and verification encapsulation are completed at the signal layer.

[0051] After the control signal sequence is generated, it is sent to the interface module of the hardware actuator. This interface module is typically embedded in the robot control board and is used to receive commands, drive execution, and collect feedback. The actuator may include servo motors, robotic arm joints, vision trackers, or environmental interaction components, etc. Each actuator performs a corresponding action based on the received signals. After the action is completed, the interface module returns status feedback information, including an execution completion flag, error code, timeout signal, or actual execution deviation. This feedback information is used to update the status of the action queue.

[0052] Based on the feedback, the queue management module moves the queue head pointer forward to the next unexecuted action instruction, maintaining the order and continuity of action execution. If the feedback contains exception information, it may trigger compensation logic or error handling mechanisms to prevent the execution deviation from continuing to spread. The entire acquisition and execution process is designed as a non-blocking structure to ensure the continuous schedulability of subsequent actions in the queue.

[0053] Action queues can be designed as circular cache structures to improve memory utilization in resource-constrained devices. Timestamps or version numbers can be appended to each action instruction to ensure correct action order when merging multi-source instruction streams. During parameter parsing, template matching mechanisms can be used for rapid instruction classification, such as directly mapping action type fields to control template indices, reducing runtime overhead. For signal generation, a hardware abstraction layer can encapsulate various communication protocols, making control instructions platform-portable.

[0054] Furthermore, a scoring system for action execution can be designed based on feedback information to adjust action priority or execution method in continuous tasks. For example, if there is high latency or execution deviation in the feedback, control parameters can be dynamically adjusted to improve action stability or fault tolerance. In addition, in high-reliability scenarios, a two-way handshake mechanism can be introduced before sending control signals to ensure that the actuator is ready to receive and execute the current instruction.

[0055] Example description: In the healthcare business, it can be used in automated injection devices to continuously perform operations such as dose delivery, angle adjustment, and feedback monitoring, ensuring safety and accuracy during drug delivery.

[0056] In fintech business scenarios, intelligent guidance robots can continuously perform tasks such as welcoming actions, path navigation, and customer interaction, and adjust their actions in real time according to changes in the environment. For example, when performing queue guidance actions, the voice broadcast content and path planning can be adjusted based on action feedback, improving service flexibility and response efficiency.

[0057] This embodiment achieves continuous task execution, controllable module-driven execution, and closed-loop anomaly response by retrieving and executing actions from an action queue. This mechanism reduces execution latency caused by waiting for instruction generation and improves scheduling efficiency through the queue structure. The linkage between the action execution process and the feedback mechanism ensures the dynamic updating capability of the action plan and lays the foundation for subsequent action prediction and fusion.

[0058] S30: When the remaining amount in the action queue is lower than a preset threshold, acquire new observation data;

[0059] In this embodiment, the remaining amount in the action queue refers to the number of currently unexecuted action instructions in the queue, used to dynamically assess whether action resources are about to be exhausted. This remaining amount is typically obtained by reading the difference between the head and tail pointers of the queue. If the queue uses a circular buffer structure, modulo operation is also required in conjunction with the capacity limit. The preset threshold is a fixed or adjustable parameter set by the scheduling strategy module before system operation, used to control the early triggering timing of action prediction. The setting of this threshold considers comprehensive factors such as execution latency, action block calculation time, and device response rate.

[0060] The remaining quantity is compared with a preset threshold in real time, typically done by a scheduling thread periodically calling a monitoring function. When the number of remaining actions in the current action queue is less than or equal to the threshold, the system immediately starts a new round of sensing tasks to ensure the queue is not interrupted. The first step of the sensing task is to activate a multimodal sensor array, which may include cameras, LiDAR, inertial measurement units, pressure sensors, etc., to capture multi-source information such as structural changes, target status, and interactive feedback in the environment.

[0061] After multimodal data acquisition, unified time synchronization and spatial registration are required to eliminate data alignment errors between sensors. This process relies on hardware clock synchronization protocols and attitude fusion algorithms, such as synchronization timestamp mechanisms and extended Kalman filters. After alignment, the original data from each modality can be reconstructed into observation data with a unified dimension and semantic structure through a format conversion module. Common data structures include tensors, multi-channel images, or structured key-value pairs.

[0062] The final observation data, serving as a snapshot of the environmental state, will be used as input for subsequent action predictions. This processing flow is designed for high responsiveness and low blocking to ensure that new prediction inputs are prepared in time before the action queue is nearly exhausted, avoiding interruptions or delays in action execution.

[0063] A dynamic threshold setting mechanism can be adopted, which adaptively adjusts the threshold based on the robot's current execution load, its own state, or the response time of the prediction model, thereby optimizing the lead time for action prediction. Alternatively, residual quantity change trend analysis can be added to the queue monitoring module to determine whether the threshold is about to be triggered through gradient analysis, so as to prepare sensing resources in advance.

[0064] In configuring multimodal sensors, different combinations can be selected based on the task scenario. For example, RGB-D cameras and microphone arrays can be used in indoor service environments, while LiDAR and torque sensors can be used in mechanical operation environments. Calibration templates, IMU compensation, or visual SLAM techniques can also be introduced during the spatial registration stage to further improve the stability and structural integrity of the observation data.

[0065] For the formatting module, data specifications required by different task models can be pre-defined, such as compressing pose information for static scenes and retaining voice channels and dynamic action labels for interactive scenes. The final output observation data should have interface consistency and data integrity to facilitate efficient parsing and processing by the strategy module.

[0066] This embodiment proactively acquires new observation data before the action queue is exhausted, thus overcoming the bottleneck between action execution and perception prediction, reducing system waiting time and prediction response latency. By utilizing real-time remaining data monitoring and dynamic perception activation mechanisms, it achieves efficient task continuity assurance while improving the system's responsiveness to sudden environmental changes.

[0067] S40, based on the similarity between the new observation data and the historical observation data, determine whether to trigger action prediction;

[0068] In this embodiment, new observation data refers to the current environmental state information collected and processed by the perception system after the remaining amount in the action queue falls below a threshold. Historical observation data refers to the observation data used by the system to generate the current action queue at a previous moment, and is usually stored in the cache or the state recording unit of the policy module. The core purpose of similarity judgment is to determine whether the current environmental state has changed significantly, thereby determining whether a new action prediction process needs to be triggered.

[0069] To make accurate judgments, comparable feature parameters need to be extracted from new and historical observation data. Common features include joint angles, pose coordinates, velocity vectors, semantic label distribution, and depth map information. Joint space feature parameters are one of the data representations that best reflect the state changes of the robot when interacting with its environment. They are usually collected by various sensor channels and parsed by the state estimation module.

[0070] The similarity determination process includes two parts: first, establishing a data alignment basis by projecting the feature parameters of two time points onto a consistent structure through interpolation, normalization, or dimensionality mapping; second, performing difference measurement calculations by using mathematical tools such as Euclidean distance, cosine similarity, structural similarity index SSIM, and KL divergence to form a quantitative difference measurement value.

[0071] The difference metric is then compared with a pre-set similarity threshold. This threshold is an empirically or learned decision boundary used to control the system's sensitivity to environmental changes. When the difference metric is greater than the threshold, it indicates a significant difference between the current environmental state and the previous prediction baseline, requiring a new action prediction to be triggered. If the difference metric is less than or equal to the threshold, and there are still unexecuted actions in the action queue, the prediction process can be skipped, and the current queue of actions can continue to be executed, avoiding redundant inference overhead.

[0072] Feature extractors based on neural networks can be used to extract high-level semantic features from raw observation data. For example, convolutional neural networks can be used to process image data, and Transformer models can be used to extract behavioral intentions from language instructions. For difference metrics, an end-to-end similarity determination subnetwork can be constructed and trained to output the probability of environmental changes, replacing traditional rule-based metric calculations.

[0073] A sliding window mechanism can also be introduced to maintain the sequence structure of historical observation data. This not only compares the current observation data with the previous frame but also assesses its changing trend in consecutive historical frames, thereby achieving dynamic judgment. When the system detects accelerated or significant changes in features, it can trigger early prediction even if the current difference value does not exceed a fixed threshold, thus providing stronger early warning capabilities.

[0074] For multimodal data input scenarios, different comparison weights can be assigned to different types of features through modal attention mechanisms. For example, the proportion of visual change features can be increased in dynamic environments, and the sensitivity of voice or action label change response can be enhanced in interactive scenarios.

[0075] Example: In the healthcare field, when a rehabilitation assistive robot monitors a patient's limb movements, if the current joint posture is highly similar to the state in the previous prediction, it can continue to execute the predetermined rehabilitation action sequence; when the patient exhibits a new posture change, such as unexpected extension or shaking, a new round of prediction is quickly triggered, adjusting the robot's assistive torque and movement rhythm.

[0076] In the fintech business, when service robots handle queuing customers, if the scene layout and personnel positions have not changed, they continue to execute the preset reception path; when they detect significant changes in the counter window status or customer interaction content, they automatically trigger action update strategies to adjust service behavior and improve response flexibility and service experience.

[0077] This embodiment introduces a similarity judgment between new and old observation data as a trigger mechanism for action prediction. This avoids unnecessary recalculation while maintaining inference response speed, thus saving computing resources and improving response efficiency. Simultaneously, this mechanism makes action updates more environmentally adaptive, maintaining stable action execution when environmental changes are insignificant and quickly entering a re-prediction process when the environment changes rapidly, significantly improving the system's robustness and sensitivity in dynamic environments.

[0078] S50, when it is determined that the trigger action prediction is to be triggered, the new observation data is sent to the policy server in a non-blocking manner, and a new action block is generated by the policy server;

[0079] In this embodiment, when the system determines that a new round of action prediction needs to be triggered, the observation data acquired at the current moment must be sent to the policy server as soon as possible to complete a new round of inference processing. Since there may still be some unexecuted actions in the action queue at this time, in order to avoid blocking the current action execution process during data transmission and policy inference, the communication mechanism must use a non-blocking method to complete the data transmission. Non-blocking transmission means that the execution thread is not actively paused during data transmission, allowing the system to continue processing other tasks, including the execution of the current action and sensor sampling, thereby realizing the parallelization of computation and communication.

[0080] To achieve non-blocking transmission, an asynchronous communication handle is first created. This handle can be a channel reference structure based on a thread pool, event loop, or asynchronous I / O library. After the communication handle is established, new observation data is encapsulated into structured data packets. These packets typically contain encoded observation information, timestamp identifiers, request sequence numbers, and other fields to ensure that the data can be correctly distributed and processed in the asynchronous channel.

[0081] After being encapsulated, the observation data is pushed to the receiving module of the policy server via an asynchronous handle. This module can register tasks in the concurrent request pool and start the model inference process in a background thread. The policy server contains a preloaded policy model, which performs forward computation based on the latest incoming observation data to generate a new set of action candidates. After inference is complete, the newly generated action blocks are written to a shared memory buffer for subsequent reading by the queue integration module.

[0082] A shared memory buffer is used to store policy output results, avoiding the repeated creation of temporary data structures or frequent calls to system-level memory allocation interfaces under high-frequency asynchronous calls. This structure enables rapid data exchange between the main control process and the policy prediction process, reducing latency jitter.

[0083] Non-blocking data transmission channels can be built using asynchronous communication protocols based on ZeroMQ or gRPC, supporting multi-threaded delivery and callback processing mechanisms. Asynchronous handles can be created using C++'s std::future and std::async mechanisms, or Python's asyncio event loop structure, to achieve concurrent request control over the policy server.

[0084] The encapsulation process of observation data can be compressed and type-checked using serialization formats such as JSON or Protocol Buffers. At the same time, checksums and unique identifiers can be attached to ensure that the policy server has context integrity when receiving and parsing data packets.

[0085] The model inference thread on the policy server can be scheduled using a thread pool to achieve multi-task load balancing. In resource-constrained scenarios, an inference priority management mechanism can be used to determine whether to postpone or merge some prediction requests. Action block generation is based on the output structure of the policy model; it can be a discrete action encoding sequence or an action trajectory prediction result with time-step control. To ensure that the generated results are readily available to the main control system, the action block is directly written to a shared memory buffer after inference is completed, and a semaphore or event trigger is set to notify the main control thread to read it.

[0086] Example: In the healthcare business, when a rehabilitation robot detects a significant change in the patient's joint state while executing the previous joint movement sequence, it can send a new motion prediction request to the rehabilitation strategy module in a non-blocking manner. This allows for the early completion of a new round of adjustment plans before the rehabilitation action is finished, reducing the idle time of the device between actions.

[0087] In the fintech business, when the welcoming and guiding robot executes a customer guidance action sequence, it immediately sends data to the service strategy server through an asynchronous channel when it receives observation information that the density of people in the queuing area is increasing. The server background generates a new service scheduling action block, which can seamlessly continue the current path and avoid path failure or service interruption due to prediction lag.

[0088] This embodiment transmits new observation data to the policy server in a non-blocking manner, decoupling the perception-computation-execution chain. This allows prediction calculations to be completed in advance while the remaining actions are being executed, significantly reducing waiting latency during action queue connection. Simultaneously, this mechanism supports parallel expansion of policy computation, providing time redundancy and resource foundation for subsequent multi-path prediction, priority ranking, and candidate scheme switching. This helps the system quickly respond to sudden changes in dynamic environments, improving the overall system control response speed and task processing stability.

[0089] S60, Receive a new action block generated by the policy server, and integrate the new action block with the action queue to generate an updated action queue;

[0090] In this embodiment, after receiving a new action block generated by the policy server through inference, it is necessary to effectively integrate this action block with the unexecuted parts of the current action queue. This integration process involves not only appending action instructions but also judging and processing overlapping parts in time sequence. To ensure the temporal continuity and semantic coherence of the integrated action queue, a timestamp mechanism needs to be introduced, and difference analysis needs to be performed on the time windows where there is overlap between the new and old action blocks.

[0091] First, the system receives structured action block data from the policy server. This data typically contains multiple action instruction units, each with an independent action type identifier, execution parameters, execution time window, and priority label. The receiving module writes this data into a temporary buffer and then begins comparison processing with the remaining action sequences in the current action queue.

[0092] The comparison process first identifies timestamp overlap intervals, i.e., the set of instructions in the new action block that partially or completely overlaps with the action at the tail of the current queue on the timeline. To avoid execution conflicts or logical redundancy caused by action overlap, these overlapping parts need to be fused. During the fusion process, alternative action instructions are generated based on parameters such as action time window, estimated target state, and policy confidence, using weighted averaging or multi-model ensemble methods. The fusion result retains the execution intent of the current queue while incorporating the adjustment direction of the newly predicted action.

[0093] For action instructions in the new action block that do not overlap with the old queue, they are appended to the end of the action queue to form a connected sequence of updated actions. To avoid timing jumps or repeated execution, the system regenerates the execution pointer of the update queue and updates the queue metadata structure, including length, sequence identifier, and valid range.

[0094] The fusion operation can achieve interval matching through the timestamp field attached to the action instruction structure. A time overlap threshold is set, for example, 100 milliseconds. If the start and end times of the new and old action instructions overlap beyond the threshold, it is marked as a conflict segment and enters the fusion processing logic. The fusion algorithm can use a linear weighting method, calculating the parameter values ​​of the fusion action based on the current system state and policy confidence, or selecting the action with higher priority as the retained instruction.

[0095] When appending action instructions, a double-ended queue structure can be constructed to retain the old action sequence after the current execution pointer and directly append the non-overlapping instructions from the new action block to the end. The entire integration operation is executed in the action queue scheduling module, and the action execution module is notified to switch to the new execution path through a queue update signal.

[0096] In a multi-threaded environment, a read-write lock mechanism can be introduced to prevent the queue structure from being overwritten or interrupted during execution. Simultaneously, to ensure the effectiveness of the merged queue, a validity check will be performed after each queue update to ensure that the time order is monotonically increasing, the instruction structure is complete, and the parameter values ​​are valid.

[0097] Example Explanation: In a healthcare scenario, suppose a robot is assisting a patient with rehabilitation exercise training. The current sequence of actions being performed is an upper limb extension series within a preset trajectory. When the system receives the patient's latest posture change, the policy server generates new upper limb adjustment action blocks. By identifying time overlap segments with the current actions, the system merges and updates the execution angles of some actions, while simultaneously adding the new, non-overlapping training actions to the end of the queue, allowing the rehabilitation exercises to adapt more smoothly to changes in the patient's position.

[0098] In fintech business scenarios, when a chatbot guides a customer, a path adjustment requirement arises due to a customer's location shift. At this point, a new action block includes partial modifications to the guidance path. The system integrates the path adjustment action with the current guidance path and appends newly added stations at the end of the guidance queue, achieving uninterrupted path correction and service flow continuity.

[0099] This embodiment achieves dynamic expansion and content optimization of the queue without interrupting the execution of the current task by aligning the new action block with the current action queue along the timeline, merging overlapping areas, and appending to the tail. This reduces redundant behavior in action execution and improves the naturalness and stability of action connections. Simultaneously, by introducing a fusion mechanism, the system has stronger adjustment capabilities when facing high-frequency environmental changes, avoiding control deviations caused by prediction mismatches or instruction conflicts.

[0100] S70, the updated action queue is used as a new action queue, and the process of action acquisition and execution, queue monitoring, similarity judgment, prediction triggering, queue integration and queue update is repeated until all actions are completed.

[0101] In this embodiment, after the system completes the update and integration of the action block, it needs to incorporate the action block into the current action control flow, replace the original action queue, establish a new execution context, and restart the action control loop accordingly. The action queue update is a synchronous reload process, marking an explicit transition in the system control state. This means that all previous action execution and scheduling records will be included in the execution history, and a new action scheduling baseline is established. This action is not only a structural queue update but also a transition operation of the system's internal scheduling state machine.

[0102] Marking the updated action queue as the current action queue requires resetting the execution pointer to the beginning of the new queue and clearing the action state cache bound to the old queue, such as interrupt flags, feedback signals, and execution window parameters. To prevent state confusion, the system internally establishes an action queue lifecycle tracking mechanism using unique identifiers (such as queue IDs or timestamps) to ensure that the currently scheduled action sequence is strictly consistent with the content generated by the policy server.

[0103] The system repeatedly executes actions, including action acquisition and execution, queue monitoring, similarity judgment, prediction triggering, queue integration, and queue update. It is a highly modular, multi-level feedback system with dynamic loop control capabilities. After each action is executed, the action execution module returns execution feedback (such as whether it was successful or whether the environment state matches expectations), triggering the scheduling module to check the remaining amount of unexecuted actions in the queue.

[0104] When the remaining amount falls below a set threshold, the environmental status monitoring module will be activated to actively collect current observation data. The collected new data will be matched with historical data using feature vectors or aligned with time series, and a similarity index will be calculated to determine whether it is necessary to trigger a new action prediction request. This similarity calculation can be based on vector distance (such as cosine similarity, Euclidean distance) or fuzzy matching using a trained clustering model, thereby addressing state changes in dynamic environments that are not entirely consistent but have similar trends.

[0105] If the system determines that action prediction needs to be triggered, it will send the new observation data to the policy server using non-blocking communication. While the data is being sent, the execution of the current action queue will not be paused, thus enabling parallel operation of control execution and policy updates, improving the system's robustness and efficiency in responding to real-time environmental changes.

[0106] After receiving the data, the policy server executes the action prediction model to generate a new action block. Upon arrival, this action block undergoes time-series analysis, conflict detection, and semantic fusion to integrate with the current queue, forming a new updated action queue. This updated action queue is then reactivated as the current queue, entering the next round of action execution and monitoring loop until all action queues have been executed.

[0107] The condition "all actions completed" is determined by the execution module based on the execution pointer position, the execution status return code, and the action execution feedback results. Specifically, when the execution pointer reaches the end of the action queue and all actions return a success status, the system will trigger the action flow termination logic, release the resource locks and caches bound to the current queue, and enter an idle waiting state or the next task scheduling cycle.

[0108] In practical deployments, updating the action queue settings can be achieved by replacing the reference pointer, rather than copying the queue contents to save memory resources. The execution control module automatically clears the old queue context and switches to the new queue pointer by listening for update signals. Furthermore, to ensure thread safety, the action queue switching process can employ atomic operations or introduce a mutex lock mechanism to prevent interruption and overwriting during queue execution.

[0109] Action acquisition and execution can be achieved by the scheduler controller periodically calling action nodes in the queue. Each node contains specific execution logic functions and parameter tables. Queue monitoring can trigger the next judgment logic by counting the number of unexecuted nodes or the time span.

[0110] The similarity assessment module can perform feature clustering locally using a lightweight graph model or a pre-trained vector space structure and match it with features in the historical observation state pool. Non-blocking communication adopts an event-driven mechanism, automatically triggering processing logic upon data arrival, without relying on the execution thread to wait for return values.

[0111] Action queue consolidation can be accomplished using a doubly linked list or a circular buffer structure, while retaining pointers to the previous version's queue to support rollback or difference analysis. Termination determination is achieved through a full-state feedback convergence mechanism, including action completion flags, timeout markers, and action failure count statistics.

[0112] Example Description: In a healthcare scenario, when performing gait assistance tasks, robots need to continuously guide rehabilitation patients through multiple sets of movement commands. When the robot detects a patient's posture shift and the action queue is not yet exhausted, it can predict new balance adjustment actions based on the posture change. The prediction results are immediately integrated into the current queue after generating new action blocks via a policy server. The robot can complete the queue update without interrupting the already started walking action and continue the assistance task with the new gait strategy, ensuring a stable, safe, and continuous rehabilitation process.

[0113] In smart service terminals within the fintech sector, service robots guide customers through processes such as form filling, identity verification, and product recommendations. When a change in customer behavior is detected (e.g., interruption of input, reselection of service type), the terminal system can collect real-time environmental data and predict new action paths (e.g., redirection to other service guidance processes). Once the new action block is generated, it is immediately integrated into the existing process control sequence. Robot behavior switching is natural and smooth, ensuring consistent service experience and uninterrupted task continuity, significantly improving interaction efficiency and service adaptability.

[0114] This embodiment, by repeatedly using the updated action queue and maintaining a continuous loop in the control flow, enables the system to continuously sense and respond to external changes in complex and dynamic environments, achieving seamless action updates and intelligent action orchestration. This process eliminates the idle delay problem caused by synchronous prediction in traditional control models, while improving the consistency and real-time performance of control behavior. Especially in scenarios with continuous action planning requirements and where the environmental state evolves over time, it can significantly enhance the robustness and adaptability of the system control chain.

[0115] This invention relates to the field of artificial intelligence technology and can be applied to business scenarios such as fintech and healthcare. It discloses an action queue control method, apparatus, device, and medium, comprising: acquiring initial observation data and sending it to a policy server; receiving an initial action queue generated by the policy server; acquiring and executing actions from the initial action queue; acquiring new observation data when the remaining amount in the action queue is lower than a preset threshold; determining whether action prediction is triggered based on the similarity between the new observation data and historical observation data; if action prediction is triggered, sending the new observation data to the policy server in a non-blocking manner and generating a new action block; receiving the new action block and integrating it with the action queue to generate an updated action queue; using the updated action queue as a new action queue and repeating the action acquisition, execution, and queue update process until the action is completed. This invention employs an asynchronous inference control mechanism, generating new action blocks and integrating queues in parallel before the action queue is exhausted, avoiding control blind spots and idle waiting during execution, effectively improving the real-time performance of action response and the continuity of system operation, and is suitable for intelligent control tasks in scenarios with limited computing resources.

[0116] In one embodiment, step S10 includes:

[0117] S101, start the multimodal sensor array, collect initial environmental state data to generate raw sensing data;

[0118] S102, Perform spatiotemporal alignment processing on the original sensing data to generate standard format observation data;

[0119] S103, Establish a secure communication link, and encrypt and transmit the standard format observation data to the policy server through the secure communication link;

[0120] S104, The standard format observation data is processed by calling the pre-trained policy model through the policy server;

[0121] S105, Receive the initial action block returned by the policy server, and construct the initial action block into an initial action queue.

[0122] In this embodiment, during system initialization, a multimodal sensor array is first activated to acquire raw environmental state data. The multimodal sensor array typically includes visual sensors (such as depth cameras or infrared imaging devices), auditory sensors (such as array microphones), tactile sensors (such as piezoelectric films or strain sensors), temperature and humidity sensors, and an inertial measurement unit (IMU). These sensors activate collaboratively in the temporal dimension and complement each other in spatial layout, comprehensively covering the physical environment information and dynamic changes of the current scene. The types of raw signals acquired by each sensor are not entirely the same, and the data formats and sampling frequencies also differ. Therefore, the directly acquired raw sensor data often exhibits characteristics of temporal misalignment, inconsistent scale, and dimensional heterogeneity.

[0123] To achieve the integration and unification of multi-source heterogeneous sensing information, spatiotemporal alignment processing of the raw sensor data is required. For temporal alignment, timestamp correction methods based on a global clock synchronization mechanism or dynamic alignment algorithms based on event-triggered backtracking interpolation can be used to ensure that data from different modalities are registered within the same time window. For spatial alignment, sensor calibration information and scene geometric constraints can be used to complete data coordinate transformation and fusion, ultimately generating observation data with a unified structure and standardized format. This standard format is typically organized in the form of tensors or multi-channel data blocks, possessing a unified spatial reference system and time base, and pre-set with necessary data integrity check codes or metadata tags to facilitate subsequent remote access and model input processing.

[0124] To ensure the secure transmission of observation data, the system establishes a secure communication link over which standard-format observation data is transmitted encrypted. The establishment of this secure communication link relies on a handshake using transport layer security protocols (such as TLS 1.3) and device authentication mechanisms (such as ECC-based device certificates) to negotiate encryption algorithms and keys. During encrypted transmission, observation data is encrypted using symmetric encryption algorithms (such as AES-GCM) supplemented with integrity check codes (such as HMAC-SHA256) to prevent man-in-the-middle attacks, data tampering, or replay attacks. A sequence number tracking mechanism and connection status monitoring strategy are implemented during communication to ensure data packet ordering, recoverability, and the stability of the communication link.

[0125] After the standard-format observation data is transmitted encrypted, it is decrypted on the policy server side and processed by the pre-trained policy model. The pre-trained policy model is a generalizable deep decision-making model, typically trained based on reinforcement learning, behavior cloning, imitation learning, or a hybrid expert system. It has been trained on a large amount of data in a multi-task environment and has the ability to make reasonable action inferences for different environmental states. The model receives standard-format observation data as input, extracts key state features, performs policy evaluation and action selection operations in its internal state space, and finally outputs a new action sequence that conforms to the semantics of the current state.

[0126] The output action information is assembled into initial action blocks within the policy server. Each action block can contain multiple atomic actions with an execution order. Each atomic action includes fields such as execution type, parameter vector, expected effect description, and effective time interval. Before being sent out, action blocks can be appended with a context checksum for validity verification and scheduling instruction identification at the execution end. Upon receiving the initial action blocks, the system constructs them into an initial action queue. During this construction process, the system converts each atomic action in the action block into an action queue node, inserts it into the queue structure according to the action order, and binds it to the current environment context identifier for consistency verification and state tracking during subsequent action execution.

[0127] Once the action queue is formed, the system will transfer the action scheduling state to the execution state and officially enter the action-driven control process. This marks the completion of the initial closed loop of environmental observation and action generation, laying the foundation for subsequent action execution, queue monitoring and dynamic prediction.

[0128] This embodiment significantly improves the modeling capability for complex dynamic scenes by activating a multimodal sensor array to acquire raw environmental state data and performing spatiotemporal alignment processing. This ensures coordination and consistency between sensors in both time and space, avoiding misjudgments caused by modal mismatch. Organizing observation data in a standard format before transmitting it to the policy server via an encrypted secure link effectively prevents sensitive information from being leaked or tampered with during transmission, enhancing the system's data security in real-world deployment scenarios. The policy server calls a pre-trained policy model to process the observation data and generate initial action blocks, enabling rapid reasoning from state to action sequence and avoiding performance bottlenecks caused by high-frequency, synchronous reasoning. Finally, the action blocks are constructed into an initial action queue, bound to environmental context information, and the action control process is initiated. This ensures that the system possesses a clear structure, coherent control, and strong dynamic adaptability in its initial state, laying the foundation for the entire closed-loop control system and achieving a safe, efficient, and intelligent initial action acquisition and scheduling mechanism.

[0129] In one embodiment, step S20 above includes:

[0130] S201, Locate the storage address of the initial action queue, and read the queue head pointer position from the storage address;

[0131] S202, extract the first action instruction from the initial action queue according to the position of the queue head pointer;

[0132] S203, parse the operation parameters of the first action command and generate a control signal sequence;

[0133] S204, send the control signal sequence to the actuator interface to drive the actuator to complete the action execution;

[0134] S205, Receive the action completion feedback returned by the actuator interface;

[0135] S206, based on the action completion feedback, update the position of the queue head pointer to the position of the next action instruction to be executed in the initial action queue.

[0136] In this embodiment, to achieve ordered scheduling and precise execution of actions in the initial action queue, the system first needs to locate the physical storage address of the action queue in memory or persistent storage. Action queues are generally constructed in the form of a linear structure or a circular buffer. Their storage addresses are explicitly recorded by the allocator during system initialization or action queue construction, or embedded in the global action scheduling table as reference identifiers. In actual implementation, the system indexes the action queue identifier through the scheduling control unit, retrieves the corresponding address mapping entry, and obtains the absolute position of the current action queue in the address space for subsequent pointer positioning and data retrieval operations.

[0137] After obtaining the storage address, the system reads the queue head pointer position from that address. This pointer identifies the position of the currently pending action instruction within the queue structure. The queue head pointer is typically represented by an offset address or node index. During dynamic scheduling, its atomic updates must be guaranteed to prevent duplicate execution of actions or instruction loss due to concurrent access. To address this, lightweight mutexes or a circular pointer spin strategy can be introduced to ensure the sequential consistency and memory visibility of read and write operations.

[0138] Extracting the header action instruction is a core step in the action execution process. The system accesses the corresponding entry in the queue based on the header pointer value and parses the entry into an action instruction object. Action instructions include fields such as operation type, control parameters, expected execution duration, priority flag, and fault tolerance policy flag. The extraction process requires not only reading the instruction body but also simultaneously extracting its context identifier and historical execution state to facilitate subsequent path consistency checks or anomaly recovery by the action scheduler.

[0139] For the extracted initial action command, the system further analyzes the operational parameters and generates a corresponding control signal sequence based on the control model. Operational parameters may include target position coordinates, velocity vector, torque limits, execution thresholds, and sensor trigger conditions. The system normalizes and converts the parameters according to the actuator model and the actual execution environment. The generation of the control signal sequence depends on the actuator interface specification and command protocol, such as pulse width modulation (PWM) control, serial port instruction sets, and CAN bus frame structures, ensuring that the generated control sequence matches the hardware interface and drive protocol of the actuator control module.

[0140] The generated control signal sequence is sent to the actuator interface module, which translates the high-level control signals into execution-level electrical or communication commands and transmits them to the specific actuator via a physical bus. The actuator interface module may be implemented based on an FPGA, microcontroller, or embedded operating system, and internally includes a state machine or task scheduling queue to buffer control signals, allocate channel resources, and drive the parallel response of multi-channel actuators. The actuator may include a servo motor, hydraulic actuator, robotic arm end effector, laser focusing mechanism, etc., which performs specific physical actions based on the received control signals.

[0141] Upon completion, the actuator interface module returns action completion feedback to the upper-layer module via its built-in sensor feedback channel or an action completion interrupt signal. This feedback data includes the execution result status code, execution latency statistics, error metrics, and execution interruption records, used to determine whether the action was successfully completed or if compensation logic needs to be triggered. Upon receiving this feedback, the system updates the queue head pointer to the position of the next pending action instruction. This update process must be performed within an atomic context to ensure strong consistency between action execution and pointer movement, preventing action sequence corruption or state drift.

[0142] Through the above operation process, the system realizes a mechanism for accurately extracting, parsing, executing, and dynamically advancing action sequences from the initial action queue, providing basic support for achieving uninterrupted continuous control processes.

[0143] This embodiment ensures efficient access to and management of action sequences by clearly defining the storage address and pointer position of the initial action queue, avoiding scheduling anomalies caused by memory reference invalidation or concurrency conflicts. It generates control signal sequences based on the operation parameters of action instructions, flexibly adapting to different types of actuator control protocols and improving the system's compatibility with various control devices. Through the actuator interface module, it performs protocol conversion and physical output of control signals, achieving accurate mapping from high-level action descriptions to low-level execution commands, ensuring consistency and efficiency in action execution. Utilizing an action completion feedback mechanism, the system can dynamically advance the queue pointer based on feedback data, achieving closed-loop execution and state tracking of action control. This constructs a control flow with action memory, adaptive advancement, and closed-loop execution, significantly improving execution stability and system responsiveness.

[0144] In one embodiment, step S30 above includes:

[0145] S301, monitors the number of remaining action instructions in the action queue in real time and obtains the remaining value;

[0146] S302, compare the remaining value with a preset threshold;

[0147] S303, when the remaining value is lower than a preset threshold, the multimodal sensor array is activated;

[0148] S304, real-time environmental data is collected through the multimodal sensor array to generate raw sensor data;

[0149] S305, Perform spatiotemporal alignment processing on the raw sensing data to generate standard format observation data;

[0150] S306, the standard format observation data is used as new observation data.

[0151] In this embodiment, the system continuously monitors the remaining number of action commands in the action queue during the execution of the action control process to determine whether it is necessary to reacquire environmental awareness information. This monitoring process is periodically triggered by the action scheduling management module or performed synchronously after each action execution feedback. The action queue is typically implemented in the form of a structured array, linked nodes, or message buffer. Each action command is accompanied by a unique identifier and status flag when enqueued. The scheduling module can obtain the remaining value by calculating the difference between the current total queue length and the number of executed actions, or by maintaining the offset distance between the tail pointer and the head execution pointer of the dynamic queue for real-time estimation.

[0152] After obtaining the current remaining value, the system compares it with a preset threshold. The preset threshold can be dynamically set based on task complexity, average action block length, system processing latency, and perception response latency, or it can be statically configured in the parameter management module according to the operating strategy. When the remaining value is less than this threshold, it means that the current action sequence is about to be exhausted, and the system needs to perform a perception update in advance to prepare for new action decisions. To prevent misjudgments or high-frequency triggering, a hysteresis threshold window mechanism can be introduced, that is, a sensor update is triggered only after the remaining value has been continuously below the threshold for more than a specific time window or after a certain number of executions.

[0153] When the triggering conditions are met, the control module immediately activates the multimodal sensor array. This sensor array may include an RGB camera, depth sensor, LiDAR, inertial measurement unit (IMU), haptic array, microphone array, ambient temperature and humidity sensor, etc., with the activation subset dynamically configured according to the specific task scenario. The activation process typically includes power-on wake-up, driver initialization, and loading of acquisition parameters, ensuring that all participating modalities synchronously enter the data preparation state.

[0154] After the sensor array is activated, the system immediately performs a one-time or short-term environmental data acquisition operation to obtain raw sensor data. This raw data includes multi-source heterogeneous data such as image frame sequences, point cloud data, voice signals, acceleration curves, and current feedback, providing basic input for subsequent strategy judgments. During the acquisition process, it is necessary to ensure the consistency of sampling time points for each modality. This can be achieved by broadcasting the acquisition trigger signal to all sensor nodes through a central clock synchronization module, or by setting a hardware-level synchronization line to perform lockstep sampling.

[0155] After acquisition, the raw sensor data enters the spatiotemporal alignment stage, which uniformly corrects the deviations of different modes in the time dimension and spatial coordinate system. Temporal alignment can be achieved through interpolation resampling, frame-level synchronization marking, delay compensation, etc., while spatial alignment is performed by coordinate transformation, projection fusion, or feature matching correction based on device calibration parameters (such as extrinsic parameter matrix, installation position deviation). The spatiotemporal alignment results constitute a unified data structure, typically an observation data container containing timestamps, coordinate system information, and features of each mode, with a standard format definition to facilitate subsequent strategy model parsing.

[0156] Finally, the processed standard-format observation data is defined as new observation data and used as an input source for subsequent determination of whether to trigger action prediction and policy model updates. This data structure may include additional attributes such as observation frame number, scene identification identifier, data confidence assessment, and modality coverage, supporting the system to flexibly select parsing strategies and paths according to the current task status.

[0157] For example, This inequality gives a safe lower bound on the action queue depth: the number of remaining action instructions g must at least cover the server's average inference latency E[l] s The number of actions consumed within the cycle. If Δt is the underlying control cycle and n is the number of instructions consumed per cycle, then the right-hand side represents the number of instructions required within the average computation latency. This is used to dynamically set a threshold to prevent the robot from idling due to queue exhaustion.

[0158] This embodiment continuously monitors the remaining number of action commands in the action queue, enabling the system to proactively trigger environmental perception before the action sequence is exhausted, thus avoiding idle behavior caused by control flow interruption and perception lag. By using a multimodal sensor array to collect real-time data and perform spatiotemporal alignment processing, a unified spatiotemporal perception representation can be constructed across different physical modalities, providing high-precision input for policy judgment. Through the definition of a standard-formatted observation data structure, the system can stably interface with the policy module and adapt to the perception heterogeneity in complex task environments, thereby achieving dynamic coordination and smooth transition between the action control process and the perception update process, significantly improving the continuous response capability and environmental adaptability of the robot system.

[0159] In one embodiment, step S40 above includes:

[0160] S401, Extract the joint space feature parameters of the new observation data;

[0161] S402, Extract the joint space feature parameters of the most recent observation data from the historical observation data;

[0162] S403, determine the difference measure between the joint space feature parameters of the new observation data and the joint space feature parameters of the most recent observation data;

[0163] S404, compare the difference measure with the similarity threshold;

[0164] S405, when the difference metric value is greater than the similarity threshold or the action queue is empty, action prediction is triggered;

[0165] S406, when the difference metric is not greater than the similarity threshold and the action queue is not empty, skip action prediction.

[0166] In this embodiment, to achieve dynamic filtering and predictive decision-making of observation data, the system needs to accurately compare the current environmental perception data with the past state to ensure that the strategy module is triggered only when necessary, thus conserving inference resources. Upon receiving new standard-format observation data, the system first extracts joint space feature parameters. These parameters typically include the spatial pose (position and attitude) of the end effector, the rotation angle or linear displacement values ​​of each major motion joint, velocity state, acceleration state, and motion direction vector. These parameters can be obtained directly from the robot's motion controller or derived backward from multimodal observation data, such as calculating the end effector's position trajectory from image sequences or estimating the changes in joint angles using depth data.

[0167] After completing feature extraction of the current data, the system retrieves the most recently acquired and processed standard-format observation data from the observation cache and performs the same joint space feature extraction operation on it. The "most recently" here is not necessarily the previous time slice, but can also be the keyframe data that was actually used to generate the action, based on the policy trigger point, to ensure temporal continuity.

[0168] The difference between current and historical features can be assessed using various difference metrics, including Euclidean distance, weighted Manhattan distance, cosine similarity inversion, Mahalanobis distance, or a custom combination of metrics. If the feature dimension is a pose quaternion or rotation matrix, a measurement mechanism in Lie group space is also required for nonlinear mapping. The system outputs the calculation result as a difference metric, reflecting the degree of pose or motion state deviation between the current state and the previous predicted state. The larger the difference value, the more severe the deviation of the current state from the predicted path, and the more necessary it is to update the policy model to generate a new action sequence.

[0169] The system then compares the difference metric with a preset similarity threshold. The similarity threshold can be set statically or dynamically adjusted based on scene changes, operational stability, and device load. The comparison result is used to decide whether to trigger a new round of action prediction. If the difference metric is greater than the similarity threshold, it indicates that the current environment state significantly deviates from the historical state, possibly due to task node switching, external intervention, or a sudden anomaly. In this case, the system should immediately activate the strategy module and regenerate the action block. Simultaneously, if the action queue is empty, prediction must be forcibly triggered even if the difference value is insufficient to ensure action continuity.

[0170] Conversely, when the difference metric is lower than or equal to the threshold and the current action queue has not been fully executed, it indicates that the current state is still within the policy coverage area. The system can directly skip the prediction process and continue to execute the existing action block, reducing redundant inference overhead and ensuring real-time performance and resource utilization efficiency.

[0171] For example, ||qt+1 -q t The inequality || < ε limits the Euclidean distance between the joint space state vectors of two consecutive frames. When the distance is less than the threshold ε, the system determines that the observed changes are insufficient, skips the prediction, and directly uses the existing action queue. t With q t+1 These are the joint state vectors at times t and t+1, respectively; ε is the similarity threshold, obtained from task accuracy requirements and sensor noise statistics.

[0172] This embodiment compares the differences in joint space features between new and historical observation data, enabling the system to accurately determine whether the environmental state has undergone substantial changes. This avoids redundant inference by repeatedly calling the strategy model during stable periods, thereby significantly reducing the model call frequency and computational load. In cases of drastic state changes or action exhaustion, timely action prediction is triggered to maintain the continuity of control flow, improving the system's ability to adapt quickly to dynamic environments and the robustness of behavioral responses, and realizing conditional scheduling and resource-aware control for inference triggering.

[0173] In one embodiment, step S50 above includes:

[0174] S501, Create an asynchronous communication handle and establish a non-blocking connection channel between the asynchronous communication handle and the policy server;

[0175] S502, the new observation data is encapsulated into an asynchronous transmission data packet;

[0176] S503, the asynchronous transmission data packet is sent to the policy server through the asynchronous communication handle;

[0177] S504, Start the background prediction thread of the policy server and call the pre-trained policy model to process the new observation data;

[0178] S505 generates new action block data by performing policy inference processing through a background prediction thread;

[0179] S506, the new action block data is written to the shared memory buffer.

[0180] In this embodiment, to avoid interruptions or delayed responses to control commands in subsequent stages of policy execution within a complex task control flow, the system must possess the capability for continuous execution and action prediction loops. Therefore, after the system receives a new action block generated by the policy server and completes its writing to the shared memory buffer, it must ensure that the current control flow can promptly and effectively integrate the new action block into the current action queue and build and update the action queue to support subsequent continuous control.

[0181] The integration process begins with reading action blocks. This reading operation does not rely on polling; instead, it triggers the execution thread to access new data via a shared memory monitoring mechanism or a signal wake-up mechanism. Before accessing the data, the system must determine its validity, including but not limited to verifying whether it is the latest version, whether it has a complete data frame structure, and whether its integrity has been confirmed through hash verification or CRC checksums. Only when the integrity requirements are met is the system allowed to parse the new action block data into a standard action instruction set.

[0182] The standard action instruction set is the result of transcoding and structuring the action vectors output by the policy. It typically includes metadata fields such as action category, location target, execution time window, and priority label. The structure of these fields must be consistent with the format of the existing action queue to support seamless integration.

[0183] During the integration phase, the system performs a data structure reconstruction operation on the current action queue. The current action queue is typically constructed using a deque or a priority queue to support high-frequency insertion and dynamic priority scheduling. Integration methods can be based on timestamp order, action importance tags, or dynamic reordering strategies. Reordering strategies can rely on heuristic algorithms, such as insertion sort based on path smoothness, action execution pre-time prediction, or even a semantic weight calculation model incorporating the current task context. The integrated action queue needs to regenerate its head and tail pointers and update the cache index to match the new action instruction sequence.

[0184] Updating the action queue not only reflects updates to the structured data but also triggers state transitions in the execution flow. After the original execution thread completes its current action, it no longer accesses the old queue but automatically switches to the updated action queue. To avoid instruction skipping or disordered execution, the system needs to set a version stamp or queue update flag after integration to ensure that the executor always references the latest queue data. The queue switching process can ensure consistency through atomic operations or transactional data structures, while also supporting the maintenance of queue state stability in high-concurrency control environments.

[0185] Furthermore, to support the system's adaptive expansion, this integration logic can be integrated into a state machine control framework. By introducing an action queue state transition table, the queue is clearly defined as being in three states: "pending integration," "integrating," and "updated." The state transition logic drives the calling rights of different modules. In multi-threaded scenarios, this mechanism effectively avoids resource contention and data access conflicts, achieving a dynamic closed loop in the action generation and execution process.

[0186] For example, l = t C→s +l S +t S→cThe formula breaks down the round-trip time of a single long-range inference operation into uplink network latency t. C→s Server inference time l S Downlink network latency t S→c This random variable is used to estimate the total potential delay of the system in the communication-computation closed loop.

[0187] This embodiment integrates newly generated action blocks with existing action queues to form an updated action queue, enabling continuous connection between policy prediction output and execution instruction sequence, effectively avoiding action interruptions caused by policy prediction delays. Through consistent data structure construction and update tag management, it ensures that new action instructions can be scheduled and executed correctly in real time. Simultaneously, the dynamic integration mechanism of action instructions supports adaptive insertion based on task priority, time window, or semantic tags, improving the system's responsiveness to task switching and scene changes.

[0188] In one embodiment, step S60 above includes:

[0189] S601, Receive a new action block generated by the policy server;

[0190] S602, Read the remaining action sequence from the action queue;

[0191] S603, identify the timestamp overlap interval between the new action block and the remaining action sequence;

[0192] S604, perform time-weighted fusion of the action instructions within the timestamp overlap interval to generate a fused action instruction;

[0193] S605, append the non-overlapping action instructions in the new action block to the end of the remaining action sequence to generate append action instructions;

[0194] S606, merge the fused action command and the append action command to generate an update action queue.

[0195] In this embodiment, after receiving a new action block generated by the policy server, the control system needs to precisely integrate the new action block with the remaining action sequences in the current action queue that have not yet been executed, in order to ensure the continuity of action execution and response efficiency. This integration process involves not only data splicing operations, but also time-based action fusion and semantic alignment of action sequences. The system first receives the new action block output by the policy server through the communication interface. This action block is usually composed of a set of structured action instructions with time tags. Each instruction includes elements such as execution start time, execution duration, joint control parameters, speed limit, and priority index. The data structure is consistent with the existing action queue to support standardized processing.

[0196] The system then reads the sequence of actions that have not yet been executed from the current action queue. This sequence is typically obtained based on the queue head pointer and the execution progress index. Following this, the system performs a timestamp alignment process, calculating the time stamp overlap between each instruction in the new action block and the instructions in the current remaining action sequence. Overlap detection can be achieved using an interval overlap detection algorithm, such as determining whether two time intervals [t1_start, t1_end] and [t2_start, t2_end] have a non-empty intersection.

[0197] Upon detecting a time overlap interval, the system performs fusion processing on action command pairs located within the overlap interval. The fusion processing uses a time-weighted approach to calculate the parameter output of the fused commands, where the control parameters of each overlapping command pair are weighted according to their time window and the overall degree of overlap. For example, for two action commands with partial time overlap, their joint control vectors can have fusion weights constructed based on the overlap interval length and the total duration, and then the fusion value for each control dimension can be calculated based on these weights. The weighting method can be linear weighting, exponential decay weighting, or optimization weighting based on minimizing prediction error, depending on the system task characteristics and control model strategy.

[0198] In addition to handling overlapping time intervals, the system also needs to extract action instructions from the new action block that do not overlap with the current action queue. These instructions are appended directly to the end of the remaining action sequence in chronological order, forming an appended action instruction set. During the appending process, it is necessary to verify whether the start time of the appended instructions is consecutive to the end of the existing queue. If a gap occurs, the system can generate transitional actions or state maintenance instructions based on task characteristics to fill the gap, thereby avoiding missing control instructions or idle actuators.

[0199] Ultimately, the system will merge the combined motion commands and additional motion commands to construct a new motion queue. This process requires maintaining the order and integrity of the queue structure, rebuilding index information, updating the timeline graph, and triggering a queue version flag update. This updated queue will be reattached to the motion execution module, replacing the old queue as the source for subsequent motion command acquisition and scheduling, ensuring the continuity and environmental adaptability of the robot control system.

[0200] For example, This weighted average is used to merge the new action block with the current remaining action sequence within the overlapping time period, smoothly transitioning instructions. It is the k-th action vector to be executed in the old queue. It is the corresponding action in the new block after time alignment; λ is adaptively adjusted according to the alignment error of the two time axes to ensure that the fusion result is both continuous and not excessively lagging.

[0201] Example Description: In the healthcare field, intelligent surgical robots need to continuously respond to dynamic interference factors such as changes in patient vital signs, tissue displacement, environmental occlusion, and intraoperative emergencies when performing image-guided minimally invasive procedures. This system is applied to perform dynamic neuronavigation puncture tasks, aiming to achieve high-precision puncture control of designated nerve targets under the background of patient breathing fluctuations and intraoperative instrument interference, while maintaining continuous observation, motion adjustment, and execution stability during the process.

[0202] The system first activates a multimodal sensor array deployed at the end effector of the surgical robotic arm, including an infrared depth camera, an intraoperative ultrasound probe, and an inertial measurement unit, to collect data such as structural images of the patient's surgical area, tissue deformation trajectories, and the posture of the surgical arm. This raw sensor data undergoes time synchronization and spatial alignment processing, is standardized into observation vectors usable for policy inference, and is transmitted to a policy server deployed within the hospital's local area network via an established TLS encrypted communication link.

[0203] The policy server invokes a deep reinforcement learning policy model trained on preoperative images to generate initial action blocks based on the input standard-format observation data. These blocks include continuous control commands such as robotic arm pose adjustment, fine-tuning of puncture direction, and needle advance distance control. The initial action blocks are constructed into an action queue and uploaded to the execution module to begin the actual operation.

[0204] The execution module reads the first action instruction from the action queue, parses the puncture speed, angle adjustment parameters and target position vector, generates continuous control signals, drives the surgical arm to advance precisely through the actuator interface, and receives action completion feedback from the joint drive system in real time to update the execution position pointer of the next action.

[0205] During operation, the system continuously monitors the remaining number of instructions in the action queue. When the number of instructions in the queue falls below a safe threshold (e.g., 3 action windows), the sensor array is reactivated to acquire real-time surgical area images and pose information, and new spatiotemporal alignment is performed to generate updated observation data.

[0206] At this point, the system further analyzes the differences between the new observation data and the previous observation, extracts the joint space feature parameters (such as angles, velocities, arm span, etc.) at the two time points, and calculates the difference metric. If the difference exceeds the dynamic change threshold, for example, due to a sudden increase in the patient's breathing or a slight displacement of the target tissue, or the current action queue is about to be empty, the system determines that a new round of strategy prediction needs to be triggered.

[0207] This prediction process is initiated asynchronously in a non-blocking manner. Observational data is encapsulated into transport packets and sent to the policy server via asynchronous communication handles, without blocking the current action execution flow. The background thread starts the policy inference model and generates new action blocks, writing them to the shared memory buffer. After the current action is completed, the execution module immediately reads the new action block from the shared buffer and merges it with the remaining part of the original action queue.

[0208] During the fusion process, the system identifies instruction portions with overlapping timestamps in the new and old actions, and applies a time-weighted mechanism to generate fused actions, ensuring that the currently advancing needle does not experience position jumps or speed fluctuations due to action block replacements. Furthermore, for new action portions without overlap, the system sequentially appends them to the end of the action queue. The final generated updated action queue contains multiple consecutive action instructions after fusion and appending processing, and the system uses this queue as the sole reference trajectory source for the current execution phase. Each action instruction includes the change in the robotic arm joint angle within the corresponding time window, the position vector of the end effector, the puncture depth increment, attitude fine-tuning parameters, and the velocity component under the attitude control strategy. The scheduling mechanism uses a fixed-time-cycle triggering method, reading the head action instruction of the action queue periodically through a high-precision scheduling clock.

[0209] After the scheduling thread starts, the system automatically parses the corresponding control target based on the parameters of the action at the head of the queue and generates a continuous control signal stream, which is transmitted in real time to the end effector (such as a six-degree-of-freedom robotic arm actuator, a puncture needle propulsion module, and an attitude stabilizer). In each control cycle, the system also collects execution feedback data in real time (including current pose, execution deviation, operating resistance, etc.) to confirm the stability and accuracy of the current action. If a significant feedback deviation occurs, the remaining control signal parameters will be dynamically adjusted for fine-tuning compensation.

[0210] This scheduling-feedback-adjustment mechanism will continuously operate to ensure that the spatial motion path of the robotic arm and puncture needle stably converges to the trajectory predicted by the strategy model. As the instructions in the updated action queue are gradually executed, the puncture path approaches the coordinates of the neural target point segment by segment in three-dimensional space. Finally, when the last action instruction is executed and the target arrival conditions are met (such as the puncture depth reaching the expected level, the posture error being below the safety threshold, and the tissue media impedance feedback conforming to the characteristics of entering the neural region), the system determines that the puncture task is completed and automatically enters the control hold or withdrawal phase.

[0211] In the fintech field, a high-frequency automated risk control and review system is being built. This system analyzes user transaction behavior in real time during the loan application process to identify abnormal patterns and drive automated processing actions. Throughout the entire process, the system continuously schedules review actions using asynchronous reasoning, thereby replacing the traditional serial review mechanism.

[0212] During the initialization phase, the system first activates the account behavior monitoring module. This module collects the user's initial interaction status by calling the financial transaction gateway, login behavior interface, and risk scoring module, including transaction details, device fingerprints, access frequency, and historical overdue status within the past 30 minutes, forming multi-source heterogeneous raw sensor data. Subsequently, this data stream undergoes format standardization processing through a unified behavior modeling interface, including time normalization, field mapping transformation, and strategy feature annotation, generating structured standard format observation data. To ensure secure data transmission between inference services, this standard data is encrypted using the TLS protocol and encapsulated into message data packets before being sent to the policy server deployed in the financial private cloud.

[0213] After receiving standard-formatted observation data, the strategy server performs rapid inference using its built-in high-frequency trading behavior prediction model (e.g., a state prediction model with a Transformer architecture). Based on the user's historical tag samples and the current context, the model predicts the behavioral evolution path and generates an initial action block containing a set of operational decisions, such as "freeze login channel," "mark high-risk behavior," or "request supplementary authentication materials." This initial action block is encoded and constructed into an initial action queue, which is cached in the instruction queue of the action execution scheduler.

[0214] Each round, the scheduler extracts instructions from the current action queue and parses their corresponding parameters, such as target account, control strategy, and response level, to generate an execution signal sequence. This sequence is sent to the corresponding service components in the financial system (such as the risk control engine, notification module, and risk control tagging engine) via API calls, driving the completion of the actual action. The executor interface returns feedback in real time, such as execution status, interface response code, or behavior response offset. The scheduler updates the action queue head pointer based on the feedback to ensure that instructions for the next round can be accurately obtained.

[0215] During execution, the system continuously monitors the number of remaining actions in the action queue. When the number of remaining instructions falls below a set threshold (e.g., less than 3), a new observation data acquisition process is immediately initiated. At this time, the system re-collects the user's recent behavior records, such as browsing behavior, click patterns, API call frequency, etc., and performs time-series standardization and feature extraction again to generate new observation data.

[0216] The system compares the new observation data with the observation data collected before the last execution, extracts behavioral label vectors and risk situation maps, and calculates similarity indicators using methods such as Euclidean distance or KL divergence. If the difference metric is greater than the similarity threshold (e.g., 0.4), it indicates a significant change in the user's state, and the system will trigger a new round of action prediction. During this process, the observation data is encapsulated into asynchronous message packets and sent to the policy server using non-blocking communication handles.

[0217] The policy server's background thread processes data in real time, performs rapid inference, and generates new action blocks. These new action blocks are written to the policy service's shared memory area, read by the action scheduler, and fused with the remaining action sequences. During fusion, the system identifies whether the timestamps or behavioral targets in the action instructions overlap. If there is overlap, a behavior-weighted fusion strategy is used; otherwise, the instructions are appended directly.

[0218] The updated action queue will again serve as the control basis, completing dynamic response tasks for high-frequency risk control strategies in the next cycle. The entire process supports continuous behavior monitoring and strategy adjustment, enabling dynamic perception and response scheduling of complex risk situations. For example, if the system identifies abnormal login behavior three times consecutively and predicts an upward trend in risk indicators, the automatically generated action block will include operations such as session cancellation for all terminals, SMS reminders to customers, and delayed loan approval, ensuring that risks are controlled in the early stages of data evolution.

[0219] This embodiment effectively solves the problems of timing misalignment and action conflict that may occur between predicted actions and the current execution state by performing timestamp overlap detection and fusion between the new action block generated by the policy server and the current remaining action sequence, and constructing an update action queue by orderly appending non-overlapping actions. The time-weighted fusion mechanism ensures a smooth transition of control instructions in the overlapping area, avoiding execution discontinuity or abnormal mechanical load caused by action switching. At the same time, the queue appending mechanism enables efficient connection of new actions, improves the overall system execution smoothness and the real-time performance of predictive control response, and significantly optimizes the control system's adaptability to dynamic environmental changes and the stability of instruction execution.

[0220] In one embodiment, an action queue control device is provided, which corresponds one-to-one with the action queue control method described in the above embodiments. (Refer to...) Figure 3 , Figure 3 This is a schematic diagram of the functional modules of a preferred embodiment of the action queue control device of the present invention. The modules include: observation data acquisition module 10, action execution module 20, queue monitoring module 30, similarity judgment module 40, asynchronous prediction triggering module 50, action queue integration module 60, and action flow control module 70. Detailed descriptions of each functional module are as follows:

[0221] The observation data acquisition module 10 is used to acquire initial observation data and send the initial observation data to the policy server, and to receive the initial action queue generated by the policy server.

[0222] Action execution module 20 is used to obtain actions from the initial action queue and execute the actions;

[0223] The queue monitoring module 30 is used to acquire new observation data when the remaining amount in the action queue is lower than a preset threshold.

[0224] The similarity judgment module 40 is used to determine whether to trigger action prediction based on the similarity between the new observation data and the historical observation data;

[0225] The asynchronous prediction triggering module 50 is used to send the new observation data to the policy server in a non-blocking manner when it is determined that a trigger action prediction is needed, and generate a new action block through the policy server.

[0226] Action queue integration module 60 is used to receive new action blocks generated by the policy server and integrate the new action blocks with the action queue to generate an updated action queue.

[0227] The action flow control module 70 is used to treat the updated action queue as a new action queue and repeatedly execute the action acquisition and execution, queue monitoring, similarity judgment, prediction triggering, queue integration and queue update process until all actions are completed.

[0228] In one embodiment, the observation data acquisition module 10 is specifically used for:

[0229] The multimodal sensor array is activated to collect initial environmental state data and generate raw sensing data.

[0230] The original sensor data is subjected to spatiotemporal alignment processing to generate standard format observation data;

[0231] Establish a secure communication link, and encrypt and transmit the standard format observation data to the policy server through the secure communication link;

[0232] The standard format observation data is processed by calling a pre-trained policy model through the policy server.

[0233] Receive the initial action block returned by the policy server and construct the initial action block into an initial action queue.

[0234] In one embodiment, the action execution module 20 is specifically used for:

[0235] Locate the storage address of the initial action queue, and read the queue head pointer position from the storage address;

[0236] Based on the position of the queue head pointer, extract the first action instruction from the initial action queue;

[0237] The operation parameters of the first action command are analyzed to generate a control signal sequence;

[0238] The control signal sequence is sent to the actuator interface to drive the actuator to complete the action.

[0239] Receive the action completion feedback returned by the actuator interface;

[0240] Based on the feedback of action completion, update the position of the queue head pointer to the position of the next action instruction to be executed in the initial action queue.

[0241] In one embodiment, the queue monitoring module 30 is specifically used for:

[0242] Monitor the number of remaining action instructions in the action queue in real time and obtain the remaining value;

[0243] Compare the remaining value with a preset threshold;

[0244] When the remaining value is lower than a preset threshold, the multimodal sensor array is activated;

[0245] The multimodal sensor array collects real-time environmental data to generate raw sensor data.

[0246] The original sensor data is subjected to spatiotemporal alignment processing to generate standard format observation data;

[0247] The standard format observation data is used as the new observation data.

[0248] In one embodiment, the similarity determination module 40 is specifically used for:

[0249] Extract the joint space feature parameters from the new observation data;

[0250] Extract the joint spatial feature parameters from the most recent observation data in the historical observation data;

[0251] Determine the difference measure between the joint space feature parameters of the new observation data and the joint space feature parameters of the most recent observation data;

[0252] Compare the difference metric with the similarity threshold;

[0253] Action prediction is triggered when the difference metric is greater than the similarity threshold or the action queue is empty.

[0254] When the difference metric is not greater than the similarity threshold and the action queue is not empty, skip action prediction.

[0255] In one embodiment, the asynchronous prediction triggering module 50 is specifically used for:

[0256] Create an asynchronous communication handle and establish a non-blocking connection channel between the asynchronous communication handle and the policy server;

[0257] The new observation data is encapsulated into asynchronous transmission data packets;

[0258] The asynchronous transmission data packet is sent to the policy server through the asynchronous communication handle;

[0259] Start the background prediction thread of the policy server and call the pre-trained policy model to process the new observation data;

[0260] The background prediction thread performs policy inference processing to generate new action block data.

[0261] Write the new action block data into the shared memory buffer.

[0262] In one embodiment, the action queue integration module 60 is specifically used for:

[0263] Receive a new action block generated by the policy server;

[0264] Read the remaining action sequence from the action queue;

[0265] Identify the timestamp overlap interval between the new action block and the remaining action sequence;

[0266] The action instructions within the timestamp overlap interval are time-weighted and fused to generate fused action instructions;

[0267] The non-overlapping action instructions in the new action block are appended to the end of the remaining action sequence to generate append action instructions;

[0268] The fused action command and the appended action command are merged to generate an update action queue.

[0269] In one embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 4 As shown, the computer device includes a processor, memory, network interface, and database connected via a system bus. The processor provides determination and control capabilities. The memory includes non-volatile and / or volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and database. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage media. The network interface is used for communication with external user terminals via a network connection. When the computer program is executed by the processor, it implements the functions or steps of an action queue control method on the server side.

[0270] In one embodiment, a computer device is provided, which may be a user terminal, and its internal structure diagram may be as follows: Figure 5As shown, the computer device includes a processor, memory, network interface, display screen, and input devices connected via a system bus. The processor provides determination and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage media. The network interface is used to communicate with an external server via a network connection. When the computer program is executed by the processor, it implements the functions or steps of an action queue control method on the user side.

[0271] In one embodiment, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to perform the following steps:

[0272] Acquire initial observation data and send the initial observation data to the policy server, and receive the initial action queue generated by the policy server;

[0273] Obtain an action from the initial action queue and execute the action;

[0274] When the remaining amount in the action queue is lower than a preset threshold, new observation data is acquired;

[0275] Based on the similarity between the new observation data and the historical observation data, determine whether to trigger action prediction;

[0276] When it is determined that an action prediction is triggered, the new observation data is sent to the policy server in a non-blocking manner, and a new action block is generated by the policy server.

[0277] Receive a new action block generated by the policy server, and integrate the new action block with the action queue to generate an updated action queue;

[0278] The updated action queue is used as a new action queue, and the process of action acquisition and execution, queue monitoring, similarity judgment, prediction triggering, queue integration and queue update is repeated until all actions are completed.

[0279] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, the computer program performing the following steps when executed by a processor:

[0280] Acquire initial observation data and send the initial observation data to the policy server, and receive the initial action queue generated by the policy server;

[0281] Obtain an action from the initial action queue and execute the action;

[0282] When the remaining amount in the action queue is lower than a preset threshold, new observation data is acquired;

[0283] Based on the similarity between the new observation data and the historical observation data, determine whether to trigger action prediction;

[0284] When it is determined that an action prediction is triggered, the new observation data is sent to the policy server in a non-blocking manner, and a new action block is generated by the policy server.

[0285] Receive a new action block generated by the policy server, and integrate the new action block with the action queue to generate an updated action queue;

[0286] The updated action queue is used as a new action queue, and the process of action acquisition and execution, queue monitoring, similarity judgment, prediction triggering, queue integration and queue update is repeated until all actions are completed.

[0287] It should be noted that the functions or steps that can be implemented by the computer-readable storage medium or computer device described above can be referred to the relevant descriptions on the server side and user side in the foregoing method embodiments. To avoid repetition, they will not be described one by one here.

[0288] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), Rambus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0289] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is used as an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above.

[0290] It should be noted that if any software tools or components not belonging to this company appear in the embodiments of this application, they are merely illustrative examples and do not represent actual use. The embodiments described above are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included within the protection scope of the present invention.

Claims

1. An action queue control method characterized by, The method comprises the following steps: acquiring initial observation data and sending the initial observation data to a policy server, receiving an initial action queue generated by the policy server; acquiring an action from the initial action queue and executing the action; when the remaining amount of the action queue is lower than a preset threshold, acquiring new observation data; determining whether to trigger action prediction based on the similarity between the new observation data and historical observation data; when it is determined to trigger action prediction, sending the new observation data to the policy server in a non-blocking manner, and generating a new action block by the policy server; receiving the new action block generated by the policy server, integrating the new action block with the action queue to generate an updated action queue; taking the updated action queue as a new action queue, and repeatedly executing the action acquisition and execution, queue monitoring, similarity determination, prediction triggering, queue integration and queue updating processes until all actions are completed.

2. The action queue control method of claim 1, wherein, The method comprises the following steps: starting a multi-modal sensor array to collect initial state data of an environment to generate original sensing data; performing spatio-temporal alignment processing on the original sensing data to generate standard format observation data; establishing a secure communication link and encrypting and transmitting the standard format observation data to a policy server through the secure communication link; processing the standard format observation data by calling a pre-trained policy model through the policy server; receiving an initial action block returned by the policy server and constructing the initial action block into an initial action queue.

3. The action queue control method of claim 1, wherein, The method comprises the following steps: locating a storage address of the initial action queue and reading a queue head pointer position from the storage address; extracting a first action instruction from the initial action queue according to the queue head pointer position; parsing operation parameters of the first action instruction to generate a control signal sequence; sending the control signal sequence to an actuator interface to drive an execution device to complete action execution; receiving action completion feedback returned by the actuator interface; updating the queue head pointer position to a position of a next to-be-executed action instruction in the initial action queue according to the action completion feedback.

4. The action queue control method of claim 1, wherein, The method comprises the following steps: monitoring the number of remaining action instructions of the action queue in real time to acquire a remaining amount value; comparing the remaining amount value with a preset threshold; when the remaining amount value is lower than the preset threshold, activating a multi-modal sensor array; collecting real-time data of an environment through the multi-modal sensor array to generate original sensing data; performing spatio-temporal alignment processing on the original sensing data to generate standard format observation data; taking the standard format observation data as new observation data.

5. The action queue control method of claim 1, wherein, The method comprises the following steps: extracting joint space feature parameters of the new observation data; extracting joint space feature parameters of the most recent observation data in historical observation data; determining a difference measure between the joint space feature parameters of the new observation data and the joint space feature parameters of the last observation data; comparing the difference measure with a similarity threshold; triggering action prediction when the difference measure is greater than the similarity threshold or the action queue is empty; skipping action prediction when the difference measure is not greater than the similarity threshold and the action queue is not empty.

6. The action queue control method of claim 1, wherein, when it is determined to trigger action prediction, sending the new observation data to the policy server in a non-blocking manner, generating a new action block through the policy server, including: creating an asynchronous communication handle and establishing a non-blocking connection channel between the asynchronous communication handle and the policy server; packaging the new observation data into an asynchronous transmission data packet; sending the asynchronous transmission data packet to the policy server through the asynchronous communication handle; starting a background prediction thread of the policy server, and calling a pre-trained policy model to process the new observation data; performing policy inference processing through the background prediction thread to generate new action block data; writing the new action block data into a shared memory buffer.

7. The action queue control method of claim 1, wherein, receiving the new action block generated by the policy server, integrating the new action block with the action queue to generate an updated action queue, including: receiving the new action block generated by the policy server; reading the remaining action sequence of the action queue; identifying the timestamp overlap interval between the new action block and the remaining action sequence; performing time-weighted fusion on the action instructions in the timestamp overlap interval to generate fused action instructions; appending the non-overlapping action instructions in the new action block to the tail of the remaining action sequence to generate appended action instructions; merging the fused action instructions and the appended action instructions to generate the updated action queue.

8. An action queue control apparatus characterized by comprising: The action queue control device includes: an observation data acquisition module configured to obtain initial observation data and send the initial observation data to a policy server, and receive an initial action queue generated by the policy server; an action execution module configured to obtain an action from the initial action queue and execute the action; a queue monitoring module configured to obtain new observation data when the remaining amount of the action queue is lower than a preset threshold; a similarity judgment module configured to determine whether to trigger action prediction based on the similarity between the new observation data and historical observation data; an asynchronous prediction triggering module configured to send the new observation data to the policy server in a non-blocking manner when it is determined to trigger action prediction, and generate a new action block through the policy server; an action queue integration module configured to receive the new action block generated by the policy server, and integrate the new action block with the action queue to generate an updated action queue; an action flow control module configured to use the updated action queue as a new action queue, and repeatedly perform the action obtaining and executing, queue monitoring, similarity judgment, prediction triggering, queue integration, and queue updating processes until all actions are completed.

9. A computer device, comprising: The computer device comprises a memory, a processor, and an action queue control program stored in the memory and executable on the processor, and the action queue control program, when executed by the processor, implements the steps of the action queue control method according to any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The storage medium stores an action queue control program, and the action queue control program, when executed by a processor, implements the steps of the action queue control method according to any one of claims 1-7.

Images