Robot motion state and acousto-optic warning time domain synchronous coupling control system and method

Abstract

The invention discloses a control system and method for synchronous coupling of a robot motion state and an acousto-optic warning time domain, relates to a man-machine interaction technology, and provides a scheme for solving the problems of motion state and acousto-optic separation and the like in the prior art. When the host receives a moving or grabbing task, the multi-mode synchronous planner generates a mechanical arm joint space trajectory, and meanwhile real-time kinematics feature vectors of the end effector in a Cartesian space are calculated in parallel; converting the physical motion parameters into target warning efficiency parameters according to a preset nonlinear model, and sending the target warning efficiency parameters to the intelligent warning terminal; the intelligent warning terminal directly sends real-time lighting effect data to the PID and aging compensation controller; sending the state switching instruction to a stack type finite-state machine for logic processing; and a hardware feedback signal is combined to form closed-loop control, and finally the LED array and the buzzer are driven to send out an acousto-optic signal which is strictly synchronous with the action of the robot. The device has the advantages that the warning lamp changes along with the speed of the mechanical arm, and the frequency of the buzzer changes suddenly along with acceleration.

Description

Technical Field

[0001] This invention relates to human-computer interaction technology for robots, and more particularly to a control system and method for time-domain synchronous coupling of robot motion state and audio-visual warning. Background Technology

[0002] With the increasing prevalence of collaborative robots in logistics, security inspection, and production line scenarios, safety during human-robot coexistence is of paramount importance. Existing robot warning systems suffer from a significant "perception-representation disconnect," primarily manifested as follows: 1. Discrete and delayed control logic: Existing systems mostly use threshold triggering mechanisms. This switch-like control method cannot reflect continuous changes in the degree of danger. For example, when a robot moves slowly at 0.1 m / s and moves at a high speed of 2 m / s, the warning lights often flash at the same frequency, making it impossible for personnel to intuitively perceive the level of potential collision risk.

[0003] 2. Insufficient versatility and linkage: Existing solutions mostly rely on dedicated buses, and the action execution code and lighting control code often run independently, only aligning with simple timestamps, lacking deep integration at the underlying algorithm level.

[0004] 3. Lack of state memory: Traditional state machines are mostly single-level transitions, and after the warning ends, they are often abruptly reset, unable to restore to the specific workflow state before the warning.

[0005] Therefore, there is an urgent need for a general control system that can map the robot's kinematic parameters to audible and visual warning parameters in real time and has highly reliable communication and state memory capabilities. Summary of the Invention

[0006] The purpose of this invention is to provide a control system and method for time-domain synchronous coupling of robot motion state and audio-visual warning, so as to solve the problems existing in the prior art.

[0007] The control system for synchronously coupling robot motion state with audible and visual warnings in the time domain, as described in this invention, includes a robot host and an intelligent warning terminal; the robot host and the intelligent warning terminal are connected via a USB to serial adapter. The robot host is equipped with a multimodal synchronous planner. When the host receives a movement or grasping task, the multimodal synchronous planner generates the joint space trajectory of the robotic arm while simultaneously calculating the real-time kinematic feature vector of the end effector in Cartesian space. The real-time kinematic feature vector is sent to the kinematic-optical effect mapping unit, which converts the physical motion parameters into target warning performance parameters according to a preset nonlinear model. Finally, the USB communication driver layer packages the real-time kinematic feature vector into a high-speed data stream and sends it to the intelligent warning terminal via the USB bus. The protocol parser inside the intelligent warning terminal receives high-speed data streams from the host and distributes them according to the protocol type: real-time light effect data is directly sent to the PID and aging compensation controller to drive the PWM generator; while high-priority state switching instructions are sent to a stacked finite state machine for logic processing; the intelligent warning terminal also integrates a current / temperature sensor, which sends hardware feedback signals back to the PID and aging compensation controller to form a closed-loop control, ultimately driving the LED array and buzzer to emit sound and light signals that are strictly synchronized with the robot's movements.

[0008] The control method described in this invention, which synchronously couples robot motion state with sound and light warning in the time domain, utilizes the system to control an intelligent warning terminal.

[0009] The control system and method for time-domain synchronous coupling of robot motion state and audio-visual warning, as described in this invention, have the advantage of constructing an accompanying light effect function while planning the robotic arm trajectory using interpolation algorithms such as Bezier curves. This means that the brightness of the warning light is no longer constant, but increases with the increase of the robotic arm speed, and the frequency of the buzzer increases with the abrupt change in acceleration.

[0010] Unlike traditional one-dimensional state transitions, this invention introduces a state stack with a depth of N. When a "red light flashing" interrupt is triggered, the system automatically pushes the current state; after the interrupt ends, it automatically pops back to its original state, ensuring the continuity of the task flow. On the other hand, the system incorporates current fingerprint monitoring and aging compensation. By collecting the drive current in real time, the MCU can identify the aging level of the LED and automatically increase the PWM duty cycle, ensuring constant warning brightness throughout its lifespan, while also providing self-checking protection against overcurrent and overtemperature. Attached Figure Description

[0011] Figure 1 This is a schematic diagram of the control system described in this invention.

[0012] Figure 2 This is a time-domain synchronization mapping curve of the coupling algorithm described in this invention.

[0013] Figure 3 This is a schematic diagram of the structure of the protocol frame described in this invention.

[0014] Figure 4 This is a flowchart illustrating the stacked finite state machine described in this invention.

[0015] Figure 5 This is a schematic diagram of the closed-loop compensation hardware circuit described in this invention. Detailed Implementation

[0016] like Figures 1 to 5As shown, the control system for synchronously coupling robot motion state and audible and visual warning in the time domain described in this invention adopts an asymmetric design in physical architecture. It consists of a robot host with strong computing power and an intelligent warning terminal with high real-time requirements connected through a USB to serial port adapter.

[0017] On the robot's host side, the core module is the multimodal synchronous planner. When the host receives a movement or grasping task, the multimodal synchronous planner generates the joint space trajectory of the robotic arm while simultaneously calculating the real-time kinematic feature vectors of the end effector in Cartesian space, including the linear velocity modulus. With acceleration modulus This set of kinematic data is then fed into a kinematic-optical effect mapping unit, which converts the physical motion parameters into target warning performance parameters, such as PWM duty cycle and flashing frequency, based on a preset nonlinear model. Finally, the USB communication driver layer packages these parameters into a high-speed data stream and sends it via the USB bus.

[0018] The intelligent warning terminal is based on a high-reliability MCU design. Its internal protocol parser receives high-speed data streams from the host and distributes them according to the protocol type: real-time light effect data is directly sent to the PID and aging compensation controller to drive the PWM generator; while high-priority state switching instructions (such as emergency stop) are sent to a stacked finite state machine for logic processing. The intelligent warning terminal also integrates current / temperature sensors, sending hardware feedback signals back to the PID and aging compensation controller to form a closed-loop control, ultimately driving the LED array and buzzer to emit sound and light signals strictly synchronized with the robot's movements.

[0019] To address the problem of perception-representation disconnect, the kinematic-optical effect mapping unit is equipped with a coupling algorithm based on temporal synchronization.

[0020] First, the system defines a comprehensive warning index. This is used to quantify the current level of potential risk. The comprehensive warning index... It is the linear velocity of the robotic arm's end effector. and acceleration Weighting function: ; in, and The maximum design speed and acceleration of the robotic arm; The speed weighting coefficient is... It should be noted that, in a simplified embodiment, a weighting coefficient can be set. At this time, the comprehensive warning index It can be simplified to only related to speed Related, but in this embodiment, the flicker frequency The control logic remains independent, meaning the system continues to monitor the acceleration modulus in real time. This allows for the determination of whether an emergency stop flash is triggered, thereby achieving a decoupled control effect where brightness changes smoothly with speed and frequency changes abruptly with an emergency stop.

[0021] Next, based on the Weber-Fechner logarithmic perception characteristics of visual light intensity changes, a brightness mapping function is defined to make the warning light more sensitive to brightness changes at low speeds, while tending to be more gradual and saturated at high speeds. The brightness mapping function is expressed as: ; in, For the target PWM duty cycle, Based on visible brightness, The brightness is at its maximum. This is a sensitivity adjustment factor used to adjust the curvature of the response curve as brightness changes with the risk index.

[0022] flicker frequency The control function is expressed as: ; in, The system's preset standby frequency; This is a high-frequency strobe frequency; The preset safety acceleration threshold is set, and its value does not exceed the maximum design acceleration of the robotic arm. It is usually set to 80% to 90% of the maximum speed is used to ensure that the highest level warning is triggered before the robotic arm reaches its physical limits. When acceleration modulus is detected... Exceeding the threshold At that moment, the flashing frequency instantly jumps to a high-frequency strobe state.

[0023] like Figure 2 As shown in the figure, the graph is divided into upper and lower coordinate systems, which intuitively demonstrate the time-domain response effect of the coupling algorithm. The upper coordinate system displays the input kinematic parameters: the solid curve represents the real-time linear velocity modulus of the robotic arm's end effector. The dashed curve represents the corresponding acceleration modulus. A pulse appears during the start and end phases. The coordinate system in the figure below shows the output warning effectiveness parameters: the solid curve represents the high-power LED brightness control signal after mapping calculation. The dashed curve represents the frequency control signal for the buzzer and strobe. During the acceleration phase, as the velocity modulus increases... As the speed increases, the PWM duty cycle exhibits a significant upward trend. During the constant speed phase, the speed remains constant, and the PWM duty cycle maintains a corresponding high brightness level, providing stable operational warnings. During the deceleration phase, as the speed gradually decreases, the PWM duty cycle decreases synchronously, providing a clear feedback of the decay process. The flicker frequency, on the other hand, makes a step response when there is a sudden pulse in acceleration.

[0024] The system also defines an enhanced communication protocol, such as Figure 3 As shown, the protocol frame is fixed at 8 bytes in length, introducing a streaming mode. When the function code FUNC in Byte 1 is set to 0x03, the MCU enters the streaming processing state. At this time, the data fields of Bytes 3 to 6 are no longer passively configured parameters, but directly carry real-time 16-bit control data. The specific combination method is as follows: DATA0 (Byte 3) is used as the high eight bits, combined with DATA1 (Byte 4) as the low eight bits, to form a complete 16-bit PWM duty cycle value; similarly, DATA2 (Byte 5) is used as the high eight bits, combined with DATA3 (Byte 6) as the low eight bits, to form a complete 16-bit frequency cycle value.

[0025] The host continuously sends this frame at a period of approximately 20ms. After receiving it, the MCU does not need complex instruction parsing and directly loads the data into the timer's shadow register, achieving millisecond-level light effect refresh. Furthermore, Byte 2 is defined as the real-time control word in streaming mode. Bit 7 is the Urgent bit. When the host detects an emergency stop signal, it sets the value of Bit 7 to 1 and sends the signal. Upon detecting this bit, the MCU's underlying serial port interrupt service routine immediately suspends the current DMA transfer and regular processing flow, prioritizing emergency actions (such as cutting off the motor power indicator), ensuring the highest priority for safety signals. See Table 1 below for specific bit definitions.

[0026] Table 1. Enhanced Communication Protocol Frame Structure Definition

[0027] The intelligent warning terminal is equipped with a stacked finite state machine. To address the need for nested multi-task operations in industrial scenarios, the stacked finite state machine is equipped with a memory function. For example... Figure 4 As shown, the defined states include Idle, Working, Warning, and Alert.

[0028] When the system is currently in the first state (low priority), if a higher priority event is received, the system first performs a PUSH operation, pushing the first state onto the state stack for storage, and then switches to the second state to execute the alert action. When the high-priority event ends, the system performs a POP operation, popping the previously saved first state from the top of the stack and automatically restoring to the first state to continue execution. The specific transition logic is shown in Table 2: Table 2. Stack-based state machine transition logic and operation table

[0029] This mechanism ensures that after the warning action ends, the robot can accurately return to the process instruction state before the interruption, thus avoiding process disruption.

[0030] On the other hand, in order to solve the problem of brightness decay caused by LED aging, the intelligent warning terminal is also equipped with, for example... Figure 5 The closed-loop compensation hardware circuit is shown. The MCU acquires the actual current flowing through the high-power LED in real time through a sampling resistor and an operational amplifier, and reads it through an ADC.

[0031] The system records the cumulative operating time of the high-power LED in the non-volatile memory (NVM). And calculate the aging compensation coefficient based on the preset attenuation model. : ; in: It is the maximum compensation gain for aging. It is the half-life time constant of a high-power LED, and the final output The value is: ; in, This is the temperature compensation coefficient.

[0032] The PID controller inside the MCU compares the target current with the actual current and dynamically adjusts the PWM duty cycle output to the MOSFET driver, thereby ensuring that the actual luminous brightness remains constant throughout the entire lifespan of the high-power LED. Furthermore, if the sampled actual current remains abnormal, the system will determine a hardware fault and enter a safe mode.

[0033] The control method described in this invention, which synchronously couples robot motion state with sound and light warning in the time domain, utilizes the system to control an intelligent warning terminal.

[0034] This invention upgrades traditional state machine-based warnings to analog warnings by strictly aligning robot actions and warning signals in the time domain and introducing a continuously changing mapping function. Combined with stacked state management and aging compensation mechanisms, it addresses the pain points of existing general-purpose warning light systems, such as rigid interaction, lack of feedback, and inadequate lifespan management. Through an asymmetric architecture design that integrates hardware and software, the following core technological breakthroughs are achieved: This system solves the problem of perception-representation disconnect, achieving a mimicry of warning information: it breaks the limitation of traditional warning systems that can only respond to discrete on / off signals. Utilizing a multimodal synchronous planning mechanism, a real-time continuous mapping channel is established from the robot arm's kinematic vectors (velocity, acceleration) to acoustic-optical physical parameters (PWM duty cycle, frequency). This strict temporal coupling allows the warning lights to "breathe" or "burst" like biological signs, fluctuating with the robot's kinetic energy, giving operators the ability to intuitively predict potential kinetic risks and significantly improving inherent safety in human-robot coexistence environments.

[0035] The stack mechanism endows the warning system with context memory capabilities, ensuring the continuity of task flow: Addressing the need for multi-level task nesting in industrial collaboration scenarios, a stack-based finite state machine (FSM) is innovatively introduced into the embedded terminal. Through a mechanism of pushing and saving onto the stack, interrupting execution, and popping from the stack for recovery, the problem of state loss after a high-priority warning (such as an emergency stop) interrupts normal process instructions is perfectly solved. This ensures that the robot can accurately recover to its pre-interruption operating state, improving the interaction efficiency and robustness of automated production lines.

[0036] A deterministic assurance system covering the entire lifecycle was constructed, enabling predictive maintenance of the equipment: At the transmission layer, a custom enhanced Stream Mode protocol and hardware preemption mechanism were used to overcome the communication latency bottleneck of the general USB serial port, ensuring millisecond-level response to control commands. At the execution layer, combined with current fingerprint monitoring and PID aging compensation algorithms, closed-loop correction of LED light decay was achieved, ensuring that the warning device maintains constant warning effectiveness throughout its entire lifecycle and resolving safety hazards caused by hardware aging.

[0037] For those skilled in the art, various other corresponding changes and modifications can be made based on the technical solutions and concepts described above, and all such changes and modifications should fall within the protection scope of the claims of this invention.

Claims

1. A control system for time-domain synchronous coupling of robot motion state and audio-visual warning, characterized in that, It includes a robot host and an intelligent warning terminal; the robot host and the intelligent warning terminal are connected via a USB to serial adapter; The robot host is equipped with a multimodal synchronous planner. When the host receives a movement or grasping task, the multimodal synchronous planner generates the joint space trajectory of the robotic arm while simultaneously calculating the real-time kinematic feature vector of the end effector in Cartesian space. The real-time kinematic feature vector is sent to the kinematic-optical effect mapping unit, which converts the physical motion parameters into target warning performance parameters according to a preset nonlinear model. Finally, the USB communication driver layer packages the real-time kinematic feature vector into a high-speed data stream and sends it to the intelligent warning terminal via the USB bus. The protocol parser inside the intelligent warning terminal receives high-speed data streams from the host and distributes them according to the protocol type: real-time light effect data is directly sent to the PID and aging compensation controller to drive the PWM generator; while high-priority state switching instructions are sent to a stacked finite state machine for logic processing; the intelligent warning terminal also integrates a current / temperature sensor, which sends hardware feedback signals back to the PID and aging compensation controller to form a closed-loop control, ultimately driving the LED array and buzzer to emit sound and light signals that are strictly synchronized with the robot's movements.

2. The control system for time-domain synchronous coupling of robot motion state and audio-visual warning as described in claim 1, characterized in that, The target warning performance parameters include PWM duty cycle and flashing frequency.

3. The control system for time-domain synchronous coupling of robot motion state and audio-visual warning as described in claim 2, characterized in that, The real-time kinematic feature vector includes the linear velocity magnitude. With acceleration modulus .

4. The control system for time-domain synchronous coupling of robot motion state and audio-visual warning as described in claim 3, characterized in that, The kinematic-optical mapping unit is equipped with a coupling algorithm based on temporal synchronization; First, the system defines a comprehensive warning index. Used to quantify the current level of potential risk; The comprehensive warning index It is the linear velocity of the robotic arm's end effector. and acceleration Weighting function: ； in, and The maximum design speed and acceleration of the robotic arm; The speed weighting coefficient is... ; Next, based on the Weber-Fechner logarithmic perception characteristics of visual light intensity changes, a luminance mapping function is defined; the luminance mapping function is expressed as: ； in, For the target PWM duty cycle, Based on visible brightness, The brightness is at its maximum. This is a sensitivity adjustment factor used to adjust the curvature of the response curve as brightness changes with the risk index; flicker frequency The control function is expressed as: ； in, The system's preset standby frequency; This is a high-frequency strobe frequency; The preset safety acceleration threshold is set, and its value does not exceed the maximum design acceleration of the robotic arm. When acceleration modulus is detected Exceeding the threshold At that moment, the flashing frequency instantly jumps to a high-frequency strobe state.

5. The control system for time-domain synchronous coupling of robot motion state and audio-visual warning as described in claim 4, characterized in that, An enhanced communication protocol is defined, with a fixed protocol frame length of 8 bytes, introducing a streaming mode. When the function code FUNC in Byte 1 is set to a specific value, the MCU enters streaming processing mode. At this time, the data fields of Bytes 3 to 6 directly carry real-time 16-bit control data, which is then combined as follows: DATA0 corresponding to Byte 3 is used as the high eight bits, and DATA1 corresponding to Byte 4 is used as the low eight bits to form a complete 16-bit PWM duty cycle value; DATA2 corresponding to Byte 5 is used as the high eight bits, and DATA3 corresponding to Byte 6 is used as the low eight bits to form a complete 16-bit frequency cycle value. The host continuously sends protocol frames at a period of approximately 20ms. After receiving the data, the MCU directly loads the data into the shadow register of the timer.

6. The control system for time-domain synchronous coupling of robot motion state and audio-visual warning as described in claim 5, characterized in that, In the enhanced communication protocol, Byte 2 is defined as a real-time control word in stream mode; Bit 7 is an emergency preemption bit; when the host detects an emergency stop signal, it sets the value of Bit 7 to 1 and sends it; after the MCU's underlying serial port interrupt service routine detects this emergency preemption bit, it immediately suspends the current DMA transfer and regular processing flow and prioritizes the execution of the emergency action.

7. The control system for time-domain synchronous coupling of robot motion state and audio-visual warning as described in claim 6, characterized in that, The stacked finite state machine has a memory function; the defined states include idle standby, normal operation, warning, and emergency interruption. When the system is currently in the first state, which has a lower priority, if a higher priority event is received, the system first performs a PUSH operation to push the first state onto the state stack for storage, and then switches to the second state to perform an alert action. When the high-priority event ends, the system performs a POP operation to pop the previously saved first state from the top of the stack and automatically restores to the first state to continue execution.

8. The control system for time-domain synchronous coupling of robot motion state and audio-visual warning as described in claim 7, characterized in that, The intelligent warning terminal is also equipped with a closed-loop compensation hardware circuit; the MCU collects the actual current flowing through the high-power LED in real time through a sampling resistor and an operational amplifier, and reads it through an ADC; The system records the cumulative operating time of the high-power LED in non-volatile memory. And calculate the aging compensation coefficient based on the preset attenuation model. : ； in: It is the maximum compensation gain for aging. It is the half-life time constant of a high-power LED, and the final output The value is: ； in, This is the temperature compensation coefficient; The PID controller inside the MCU compares the target current with the actual current and dynamically adjusts the PWM duty cycle output to the MOSFET driver, thereby ensuring that the actual luminous brightness remains constant throughout the entire life cycle of the high-power LED.

9. The control system for time-domain synchronous coupling of robot motion state and audio-visual warning as described in claim 8, characterized in that, If the actual current sampled by the closed-loop compensation hardware circuit remains abnormal, the system will determine that there is a hardware fault and enter a safe mode.

10. A control method for time-domain synchronous coupling of robot motion state and audible / visual warning, characterized in that, The intelligent warning terminal is controlled using the system described in any one of claims 1-9.

Images