A Collaborative Control Method and Device for an Electric Blowpipe Robot Based on Vision Sliding Mode Control
By decoupling the button and airflow control loops of the electric wind instrument robot through visual sliding mode control, and utilizing visual detection and single-loop sliding mode control, high-precision blow-press coordination control of the electric wind instrument robot is achieved. This solves the problem of inaccurate button timing and blowing intensity control in existing technologies, and improves the artistic expression of the performance and the stability of the system.
Patent Information
- Application Number
- CN202611627513.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-10-21
- Publication Date
- 2026-11-17
AI Technical Summary
Existing control systems for electric wind instruments cannot precisely control the timing of key presses and the intensity of air blowing, resulting in inaccurate performances and an inability to express dynamic performance expressions. Furthermore, the existing control architecture is complex, and parameter tuning is difficult, which can easily lead to control failures.
The visual sliding mode control method is adopted to decouple the robot's blow-press coordination system into a parallel button control loop and an airflow control loop. The actual button trigger time and airflow pressure are obtained by using a visual inspection device, and single-loop sliding mode control is used for compensation. Combined with the iterative learning feedforward of the forgetting factor, high-precision collaborative control is achieved.
It achieves high-precision synchronization between button presses and airflow, provides rapid response, avoids overall control system failure, enhances the artistic expression of the performance, and reduces system complexity and parameter tuning difficulty.
Smart Images

Figure CN123103716A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of robot control and automated performance technology, specifically to a collaborative control method and device for an electric wind instrument robot based on visual sliding mode control. Background Technology
[0002] With the continuous development of robot control and automated performance technology, various automated musical instrument playing robots are gradually appearing in people's field of vision. As an electronic musical instrument that generates MIDI signals through the coordination of airflow and keys, the automated performance of the electronic wind instrument places extremely high demands on the timing accuracy of key movements and the intensity control of the blown airflow.
[0003] Most existing electronic wind instrument playing robots employ open-loop control strategies. However, due to the objective existence of physical factors such as pneumatic delay, mechanical friction, and signal transmission lag, traditional open-loop control cannot effectively compensate for the key triggering time deviation caused by these factors, resulting in insufficient precision in the actual key timing during performance. Furthermore, existing solutions cannot precisely control the intensity of the breath, making it difficult to delicately express dynamic performance nuances such as crescendo and diminuendo, significantly limiting the artistic expressiveness of the robot's performance.
[0004] In other areas of automated performance, such as existing patents (application number CN202511892670.3, PID-based Piano Robot Performance Feedforward Compensation Dual-Loop Control System and Method), a nested inner and outer loop PID (Proportional-Integral-Derivative) control architecture is used. In this architecture, the two controllers operate nested and coupled, and to adapt to the piano's sound production mechanism, the control logic often includes complex hand movement compensation modes, resulting in a relatively cumbersome structure. The playing mechanism of an electronic wind instrument differs fundamentally from that of a piano. Its sound production relies on the highly coordinated operation of two variables: key timing and airflow intensity, without requiring complex finger / hand position movement compensation. Directly copying existing serial dual-loop nested or mode-switching control logic not only leads to difficulties in system parameter tuning and a tendency to accumulate drift errors, but also fails to efficiently regulate nonlinear air pressure changes, easily causing overall control degradation or failure. Summary of the Invention
[0005] To address the technical problems existing in the prior art, this invention provides a collaborative control method and device for an electric blowpipe robot based on vision sliding mode control. It breaks through the limitations of the existing serial coupling control architecture, has a simpler and clearer structure, faster response, and can achieve high-precision blowpipe collaborative control.
[0006] To achieve the above objectives, the present invention provides the following technical solution: This invention discloses a cooperative control method for an electric blowpipe robot based on vision sliding mode control, comprising: Analyze the musical score information to obtain the desired key timing information and the desired airflow intensity information; The robot's blow-press coordination system is decoupled into a parallel button control loop and an airflow control loop; In the key control loop, the actual key trigger time is obtained by a visual inspection device, and the key time error is calculated by combining the expected key timing information. Then, a combination of single-loop sliding mode control and iterative learning feedforward with a forgetting factor is used to calculate the total advance trigger time based on the key time error, so as to drive the key actuator to perform press time compensation. In the airflow control loop, the actual airflow pressure is acquired, and the air pressure error is calculated in combination with the desired airflow intensity information. Then, an independent single-loop sliding mode control is used to calculate the control quantity based on the air pressure error, so as to adjust the output of the air pressure actuator and realize the tracking of the target blowing intensity.
[0007] As a further improvement to the above scheme, in the key control loop, the total advance trigger time is synthesized by the real-time compensation amount of the sliding mode controller and the iterative learning feedforward amount of the current performance cycle, and the calculation formula is as follows: ; In the formula, For the first The total advance trigger time of each note; For the first Iterative learning feedforward for each performance cycle; The first output of the sliding mode controller Real-time compensation amount for each note ; This is the tuning gain for the button control circuit. For the button control circuit in the first Discrete sliding surface function of a note.
[0008] As a further improvement to the above scheme, the calculation formula for the discrete sliding surface function is as follows: ; The formula for calculating the iterative learning feedforward is: ; In the formula, For the first The timing error of pressing each note; The integral coefficient; The sampling period; For the first The first performance cycle The timing error of pressing each note; The forgetting factor is set, and ; The learning gain is set proportionally, and .
[0009] As a further improvement to the above scheme, the single-loop sliding mode control used in the airflow control loop is an incremental proportional sliding mode control, and the control quantity is the opening command of the airflow proportional valve. The calculation method includes: Calculate the discrete sliding surface of the pneumatic control loop: ; Calculate the opening increment based on the current sliding surface: ; Calculate the current control value based on the opening command and opening increment of the previous note, and then perform physical range limiting: ; In the formula, For the air pressure control loop in the first Discrete sliding surface function of each note; For the first The air pressure error of each note; The integral coefficient; The sampling period; For the first The increment of the opening of each note; Gain for the airflow control loop; and The first The note and the first The opening command for each note; This indicates that the aperture is limited to the set physical limit range. Inside.
[0010] As a further improvement to the above scheme, when calculating the discrete sliding surface of the air pressure control loop at the current note, it is also determined whether the opening command of the previous note has reached the maximum value of the physical amplitude limit range, i.e. or If the maximum value is reached, and the sign of the air pressure error of the current note causes the control quantity to continue to exceed the limit range, stop accumulating the integral term in the discrete sliding surface and keep the integral term at the value of the previous note; if the maximum value is not reached, the accumulation calculation of the integral term is performed normally.
[0011] As a further improvement to the above solution, the process of obtaining the actual key trigger time includes: The visual detection device is activated within a preset time period before the first note is played, the preset time period being set based on the delay between camera activation and communication detection; The recording timeline of the visual inspection device is aligned and synchronized with the timeline containing the desired key timing information. Under the synchronized timeline, the visual detection device captures the frame of the button linkage indicator light being lit and records the corresponding timestamp. The timestamp is used as the actual button trigger time to obtain non-contact feedback on the actual performance time.
[0012] As a further improvement to the above solution, the image captured by the visual inspection device is binarized, and connected component analysis is performed on the binarized image to extract the frame of the linked indicator light above the button when it is lit. The binarization process is completed by extracting the color features of the captured image and performing threshold segmentation within a preset target color gamut. The target color gamut is set based on the specific light emission color of the linked indicator light and has a preset color tolerance.
[0013] This invention also discloses a collaborative control device for an electric blowpipe robot based on vision sliding mode control, used to implement the above-mentioned collaborative control method for an electric blowpipe robot based on vision sliding mode control. The device includes: The music score parsing module is used to parse music score information and obtain the desired key timing information and desired airflow intensity information; The perception layer includes a visual inspection device and a barometric pressure sensing module, which are used to collect feedback signals of actual button trigger time and closed-loop feedback signals of actual airflow pressure, respectively. The controller includes a key sliding mode controller and an airflow sliding mode controller. The key sliding mode controller is used to acquire the actual key trigger time and calculate the key timing error by combining it with the desired key timing information. Then, it uses a combination of single-loop sliding mode control and iterative learning feedforward with a forgetting factor to calculate the total advance trigger time based on the key timing error. The airflow sliding mode controller is used to acquire the actual airflow pressure and calculate the air pressure error by combining it with the desired airflow intensity information. Then, it uses independent single-loop sliding mode control to calculate the control quantity based on the air pressure error. The execution layer includes a button actuator that receives signals from the button sliding mode controller to perform press compensation, and a pneumatic actuator that receives instructions from the airflow sliding mode controller to adjust the blowing intensity.
[0014] As a further improvement to the above solution, the visual inspection device includes a visual camera and an LED light positioned above the button for coordinated output; the button actuator includes a miniature cylinder; the pneumatic actuator includes a pneumatic source and an airflow proportional valve, and the airflow sliding mode controller drives the opening of the airflow proportional valve in real time via a communication bus.
[0015] As a further improvement to the above scheme, the button sliding mode controller and the airflow sliding mode controller are calculated independently and there is no nesting of inner and outer loops or mode switching logic; when a target variable in the blow-press coordination system is disturbed or feedback is abnormal, the other control loop maintains independent sliding mode action.
[0016] Compared with the prior art, the beneficial effects of the present invention are: 1. This invention achieves a breakthrough in the limitations of existing serially coupled control architectures by decoupling the blow-press coordination system into parallel-operating button control loops and airflow control loops. The structure is simpler and clearer, the response is faster, and it can achieve high-precision blow-press coordination control. The button and airflow loops perform independent single-loop sliding mode calculations, without inner or outer loop nesting or mode switching. When a variable experiences feedback anomalies, the other loop maintains independent closed-loop operation, avoiding localized stalls that could lead to overall control failure, thus ensuring the physical synchronization rate between airflow and button actions.
[0017] 2. This invention achieves accelerated convergence of periodic mechanical errors and suppresses the accumulation of measurement noise by employing an iterative learning feedforward with a forgetting factor combined with sliding mode control in the button control loop. By using the forgetting factor to perform an exponentially weighted average of historical periodic errors, it accurately compensates for repeatability deviations such as cylinder action delays while weakening control offsets caused by early random disturbances, thus maintaining the stability of the feedforward output during long-term cyclical performance.
[0018] 3. This invention achieves accurate tracking of target air pressure in nonlinear pneumatic pipelines and suppresses integral saturation overshoot under extreme blowing intensities by employing incremental single-loop sliding mode control in the airflow control loop and introducing an integral term stopping accumulation mechanism based on physical amplitude limiting and error sign judgment. This mechanism cuts off the over-integral path when the valve opening command reaches its maximum value and shows a tendency to exceed the limit, effectively shortening the recovery time of the system from saturation and ensuring airflow response sensitivity under extreme dynamic conditions.
[0019] 4. This invention uses a visual inspection device to capture the image frames of the button indicator lights and extracts the timestamps through threshold segmentation within a preset target color gamut. This achieves non-contact acquisition of the actual button press time and directly filters out ambient white light and background interference at the pixel level. This mechanism avoids time misjudgment caused by acoustic noise, reduces image processing complexity through tolerance-based specific color threshold segmentation and connected component analysis, and provides the system with accurate feedback input resistant to light and sound interference. Attached Figure Description
[0020] Figure 1 This is a front view of the collaborative control device for an electric blowpipe robot based on visual sliding mode control in an embodiment of the present invention.
[0021] Figure 2 for Figure 1 A three-dimensional structural diagram of the robot body.
[0022] Figure 3 for Figure 1 A schematic diagram of the three-dimensional structure of a vision inspection device.
[0023] Figure 4 This is a flowchart of a collaborative control method for an electric blowpipe robot based on visual sliding mode control, as described in an embodiment of the present invention.
[0024] Figure 5 This is a control logic block diagram of the blow-press coordination system in an embodiment of the present invention.
[0025] Figure 6 This is a logic block diagram of the airflow control loop in an embodiment of the present invention.
[0026] Figure 7 This is a logic block diagram of the button control circuit in an embodiment of the present invention.
[0027] In the diagram: 100, robot body; 200, electric blowpipe body; 300, vision inspection equipment; 301, guide rod; 302, adjustment bracket; 303, base; 304, vision camera; 400, execution layer; 401, button execution mechanism; 402, pneumatic execution mechanism; 500, platform. Detailed Implementation
[0028] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0029] Example 1 This embodiment provides a collaborative control device for an electric blowpipe robot based on vision sliding mode control. Please refer to [link to relevant documentation]. Figures 1 to 3 The device is used to control the robot body 100 to play the electronic wind instrument body 200. The device includes: a music score analysis module, a perception layer, a controller and an execution layer 400 (some modules are not shown in the figure).
[0030] The music score parsing module is used to parse music score information and obtain the desired key timing information and desired airflow intensity information. In this embodiment, the music score parsing module can be integrated into the host computer, and the output results can be added to the CC controller (Continuous Controller). The perception layer includes a visual inspection device 300 and a barometric pressure sensing module, which are used to collect feedback signals of actual button trigger time and closed-loop feedback signals of actual airflow pressure, respectively.
[0031] The visual inspection device 300 includes a visual camera 304 and an LED light positioned above the buttons for coordinated output. In this embodiment, the visual inspection device 300 may further include a guide rod 301, an adjustment bracket 302, and a base 303. The visual camera 304 is mounted on the adjustment bracket 302 and can be adjusted horizontally along the adjustment bracket 302. The adjustment bracket 302 is mounted on the vertically positioned guide rod 301 and can be adjusted vertically along the guide rod 301. The guide rod 301 is fixed to the base 303, and a suction cup or other fixing device may be provided at the bottom of the base 303 to stabilize the posture of the visual camera 304. Before the robot body 100 performs, the structural positions in the visual inspection device 300 can be pre-arranged to ensure that the visual camera 304 can capture the button linkage indicator lights of the robot body 100.
[0032] The controller includes a key sliding mode controller and an airflow sliding mode controller. The key sliding mode controller is used to acquire the actual key trigger time and calculate the key timing error by combining it with the desired key timing information. Then, it uses a combination of single-loop sliding mode control and an iterative learning feedforward quantity with a forgetting factor to calculate the total advance trigger time based on the key timing error. The airflow sliding mode controller is used to acquire the actual airflow pressure and calculate the air pressure error by combining it with the desired airflow intensity information. Then, it uses independent single-loop sliding mode control to calculate the control quantity based on the air pressure error.
[0033] The button sliding mode controller and the airflow sliding mode controller are calculated independently and there is no nesting of inner and outer loops or mode switching logic; when a target variable in the blow-press coordination system is disturbed or feedback is abnormal, the other control loop maintains independent sliding mode action.
[0034] It should be noted that the blow-press coordination system proposed in this invention is not a single new piece of hardware independent of the aforementioned modules, but rather a general term for the complete software and hardware interaction system involved in the automated performance of the electronic wind instrument in this device. By decoupling the blow-press coordination system into parallel-running button control loops and airflow control loops, this invention clarifies the independence of the two core actions (blowing and pressing the buttons) in logical calculation and their physical synchronization in the final performance.
[0035] The execution layer 400 includes a button actuator 401 that receives a button sliding mode controller signal to perform press compensation, and a pneumatic actuator 402 that receives an airflow sliding mode controller command to adjust the blowing intensity.
[0036] The button actuator 401 includes multiple miniature cylinders for controlling the robot's fingers to press the corresponding note buttons on the electronic wind instrument body 200; the pneumatic actuator 402 includes a pressure source and an airflow proportional valve, which supports the RS485 communication protocol. The airflow sliding controller drives the opening of the airflow proportional valve in real time via a communication bus, thereby adjusting the airflow size at the blowing port located at the head of the robot body 100, which is directly opposite the air intake of the electronic wind instrument body 200.
[0037] The data flow and connection relationships between the various physical modules in this device are as follows: A high-frequency data transmission channel is established between the perception layer and the controller. Specifically, the vision camera in the vision inspection equipment communicates with the button sliding mode controller in the microprocessor via a high-speed data transmission interface (such as GigE or USB 3.0 interface) or I / O port to transmit image frames and timestamp signals in real time. The air pressure sensing module communicates with the airflow sliding mode controller in the microprocessor via an A / D conversion circuit or a serial digital bus (such as I2C or SPI interface) to send the acquired analog / digital air pressure feedback signals to the airflow sliding mode controller for processing. A drive control connection is established between the controller and the execution layer. Specifically, the button sliding mode controller is connected to and drives the solenoid valve group through the I / O communication port of the microcontroller, thereby controlling the action of the miniature cylinder in the button actuator. The airflow sliding mode controller is connected to the airflow proportional valve in the air pressure actuator via an RS485 communication bus. Taking advantage of the strong anti-interference capability and suitability for long industrial wiring of the RS485 protocol, the opening command is stably sent to the airflow proportional valve.
[0038] In some embodiments, the robot body 100 can be fixedly mounted on the platform 500 for easy transfer and stage setup.
[0039] Please see Figures 4 to 7 This embodiment also provides a collaborative control method for an electric blowpipe robot based on visual sliding mode control. This method can be implemented using the above-mentioned device and includes steps S1 to S4.
[0040] S1. Analyze the musical score information to obtain the desired key timing information and the desired airflow intensity information.
[0041] S2. Decouple the robot's blow-press coordination system into a parallel-operating button control loop and airflow control loop.
[0042] S3. In the key control loop, the actual key trigger time is obtained, and the key timing error is calculated in combination with the expected key timing information. Then, a combination of single-loop sliding mode control and iterative learning feedforward with forgetting factor is used to calculate the total advance trigger time based on the key timing error, so as to drive the key actuator to perform pressing time compensation.
[0043] S4. In the airflow control loop, the actual airflow pressure is obtained, and the air pressure error is calculated in combination with the desired airflow intensity information. Then, an independent single-loop sliding mode control is used to calculate the control quantity based on the air pressure error, so as to adjust the output of the air pressure actuator and realize the tracking of the target blowing intensity.
[0044] The process of obtaining the actual key trigger time includes: The visual detection device is activated within a preset time period before the first note is played, the preset time period being set based on the delay between camera activation and communication detection; The recording timeline of the visual inspection device is aligned and synchronized with the timeline containing the desired key timing information. Under the synchronized timeline, the visual detection device captures the frame of the button linkage indicator light being lit and records the corresponding timestamp. The timestamp is used as the actual button trigger time to obtain non-contact feedback on the actual performance time.
[0045] In this embodiment, the image captured by the visual inspection device is binarized, and connected component analysis is performed on the binarized image to extract the frame of the linked indicator light above the button when it is lit. The binarization process is completed by extracting the color features of the captured image and performing threshold segmentation within a preset target color gamut. The target color gamut is set based on the specific luminous color of the button's linked indicator light and has a preset color tolerance, used to directly filter out ambient white light and background interference of non-target colors at the image pixel level, thereby ensuring the absolute accuracy of the actual button trigger time extraction.
[0046] The working process of the airflow control loop is described below: A dynamic model of the airflow proportional valve control path is performed, which can be approximated as first-order inertia plus pure delay, as follows: ; In the formula, Actual air pressure with respect to time The function, for The first derivative, and subsequent derivatives are expressed similarly; This is a proportional valve opening command. ; It is a time constant; For gain; It is a pure delay. Because... Since the sliding mode control is relatively small and robust to matching uncertainties, this embodiment will ignore the pure delay and simplify the model to an uncertain first-order system, achieving tracking through sliding mode. Therefore, the gas flow proportional valve control pneumatic dynamics model is as follows: ; The transformation yields: ; The error in air pressure change over time is defined as... ,have: ; In the formula, This is a function of the given air pressure changing over time.
[0047] After differentiating with respect to time: ; To eliminate steady-state error, a suitable integral sliding surface is selected. : ; In the formula, For integration variables, For the air pressure error from 0 to The integral term at time (i.e., the cumulative amount of error over time). The integral coefficient determines the proportion of the error integral term in the sliding surface, and also determines the speed at which the error decays and converges when the system state slides on the sliding surface.
[0048] After differentiating with respect to time: ; In this embodiment, an exponential reaching law is selected for sliding mode control, resulting in: ; In the formula, , ; For toggle items, This is the exponential term.
[0049] Combining the two equations, we have: ; Substitution and have: ; Control quantity Solving this problem yields the following results: ; Based on the equivalent control of sliding mode control To solve the requirements, let ,get: ; In practical engineering, for A barometric pressure sensor can be used to collect the actual air pressure in real time with a sampling period T, thus forming a closed-loop feedback. And for... Precise results are difficult to obtain, especially since sliding mode control is highly robust to matching uncertainties. To ensure ease of implementation, simplified forms are often adopted, such as... Designed to work only with sliding surfaces Proportional, i.e., proportional sliding mode control law. This is because when the system state is far from the sliding surface, the proportional term can provide a sufficiently strong control effect to drive it to approach; when it is close to the sliding surface, chattering can be suppressed by adjusting the gain. Although theoretically it can be obtained through system identification... , And obtained through offline differentiation However, in actual pneumatic systems and playing scenarios, there is strong nonlinearity, time-varying nature, and operating point dependence, making it difficult to solve for fixed parameters and cover all operating conditions. The sliding mode control scheme of this invention does not rely on an accurate model and adopts a single-gain proportional form, which ensures robustness and reduces the complexity of engineering implementation.
[0050] It should be further noted that the dynamic modeling of the airflow proportional valve control path and the derivation of the equivalent sliding mode control law are based on the actual physical state in the continuous time domain, aiming to fully elucidate the underlying physical follow-up laws of the system. Therefore, the independent variables in the above formulas are continuous time variables. However, in practical engineering applications, the controller of this invention needs to be deployed and executed on digital platforms such as microprocessors. Microprocessors operate with a fixed sampling period and cannot directly process continuous integral and differential functions; therefore, the continuous-time control law must be discretized. Thus, discrete sequence variables are introduced in the microprocessor deployment implementation section described below. The variable Characterizing the time node after discretization (corresponding to the first) Each note.
[0051] Therefore, this embodiment adopts the following simple and effective form: ; This means that the current control variable is added to the previous value with an increment proportional to the current sliding surface. This is equivalent to discrete proportional control, which is easy to implement and the parameter tuning is intuitive.
[0052] Here, the same sampling period is used. Discretization is performed: ; Discretizing the exponential reaching law, we obtain: ; Right now: ; Combining the discrete dynamics of the system, it can be proven that taking And update the proportional valve opening. ,when When it is large enough, it can guarantee It enters the sliding mode zone within a finite number of steps. In practice, this can be calibrated experimentally. .
[0053] In summary, the output of the airflow sliding mode controller is: ; In the formula, For the air pressure control loop in the first Discrete sliding surface function of each note; For the first The air pressure error of each note; The integral coefficient; The sampling period; For the first The increment of the opening of each note; The gain of the airflow control loop can be a fixed constant determined through tuning experiments; and The first The note and the first The opening command for each note; This indicates that the aperture is limited to the set physical limit range. Inside.
[0054] In this embodiment, when calculating the discrete sliding surface of the air pressure control loop at the current note, it can also be determined whether the opening command of the previous note has reached the maximum value of the physical amplitude limit range, i.e. or If the maximum value is reached, and the sign of the air pressure error of the current note causes the control quantity to continue to exceed the limit range, stop accumulating the integral term in the discrete sliding surface and keep the integral term at the value of the previous note; if the maximum value is not reached, the integral term is accumulated normally to suppress the integral saturation overshoot phenomenon that occurs when the airflow control loop deals with the extreme blowing intensity.
[0055] In this embodiment, the integral term freezing logic and the aforementioned physical limiting of the control quantity constitute two complementary but independent safety mechanisms. Physical limiting only imposes hardware-level constraints on the output command of the current control cycle and cannot change the influence of historical accumulated errors on the sliding surface state. Without freezing, when the system recovers from the extreme limiting state, the over-amplified integral term will dominate the sign of the sliding surface, causing the control quantity to linger at the limiting boundary for an extended period, i.e., integral saturation. This invention, by predicting the error sign and control trend, cuts off the over-integral path at the source of sliding surface calculation, effectively shortening the recovery time from saturation and ensuring dynamic response sensitivity under extreme contrasts. Furthermore, its judgment criterion is not the traditional error threshold, but rather a combination of the physical limiting boundary and the error sign, fundamentally different from the traditional PID integral separation strategy, resulting in a more sensitive response and a simpler, more direct principle.
[0056] The working process of the button control circuit is described below: First, the MIDI score is parsed and output. The score content is then adjusted and modified, and a CC controller is added for parsing to obtain the desired performance timing and fingering. With playing dynamics score .
[0057] Construct the error function, we have: ; in For the first The timing error of each note's keystroke; a positive value indicates that the actual timing is later than the target timing. The actual performance time is provided by a visual detection device. This module uses a high-speed camera to capture LED lighting frames and record the corresponding timestamps and frame information, representing the actual performance time.
[0058] The control objective of this embodiment is to make... Approaching 0. By adjusting the cylinder's advance trigger time. To compensate for errors.
[0059] First, in the continuous time domain, the error is defined. Sliding surface function To eliminate steady-state error, we take the integral form, which is: ; After differentiating with respect to time: ; Therefore, when the system enters ideal sliding mode motion, that is... At that time, The error decays exponentially, with the decay rate increasing from... Decide.
[0060] Meanwhile, to ensure the system state quickly approaches the sliding surface and suppress chattering, an exponential reaching law is adopted, which is: ; In the formula, , ; This is a switching term that ensures the state reaches the sliding surface within a finite time. This is the convergence coefficient; the larger it is, the faster the convergence. For the exponential term, when When the value is large, the dominant trend converges, making Smoothness approaches zero, reducing chattering.
[0061] To deploy the algorithm on a microprocessor, here we will further... Discretization is performed, resulting in: ; In the formula, This is the sampling period; The first-order forward difference approximation derivative.
[0062] ; Therefore, the discrete form of the synovial surface is: ; The discrete form of the reaching law is: ; The results were: ; For button control, the direct control quantity is the advance compensation time. The dynamics of the systematic error can be approximated as: ; Pressing the button in advance will advance the actual trigger time, reducing errors.
[0063] Here, a proportional control law is used and the gain is tuned. The control law is designed as follows: ,in The gain to be tuned is determined by substituting the control law into the sliding surface dynamics and adjusting it. This allows the system to meet the sliding mode arrival condition, thereby ensuring closed-loop stability.
[0064] Therefore, the discrete control law actually used in the button control loop is: ; ; To distinguish the air pressure control loop, the subscript 't' is added for differentiation: ; ; For multiple performances or repetitive sections of the same piece of music, while the sliding mode controller can effectively suppress disturbances in a single performance, recurring systematic errors, such as cylinder action delays and pressure build-up lags, can only slowly decay with each performance cycle. To further accelerate error convergence and utilize historical performance information, this system introduces an iterative learning feedforward module into the button control loop.
[0065] The mainstream iterative learning rate in the market is based on the fundamental P-type iterative learning law. That is, let the... The complete performance ( ) The key press timing error for each note is The corresponding feedforward compensation amount is ,have: ; in For proportional learning gain, it is usually taken as This learning law can make the error converge exponentially with the performance cycle, but it has problems such as the direct accumulation of measurement noise into the feedforward quantity, and the permanent retention of large errors in the early stages, which affects the control accuracy of subsequent cycles.
[0066] To suppress noise accumulation and enhance the system's adaptability to new environments, this device employs an improved iterative learning law with a forgetting factor, as follows: ; In the formula, Forgetting factor, satisfying This formula is equivalent to an exponentially weighted average of historical errors, with the weights decreasing exponentially with the period. Forgetting factor. The larger the value, the faster the weight of earlier cycle errors decays, thus effectively suppressing the cumulative effects of measurement noise and system drift.
[0067] Here we perform a simple convergence analysis: Under repetitive perturbation conditions, the error propagation relationship satisfies: ; In the formula, This is a bounded term, representing the error change caused by repetitive perturbations. When and When the modulus of the eigenvalues is less than 1, the system converges exponentially with the performance cycle. Compared to the basic P-type ILC, the learning law with the forgetting factor significantly improves noise resistance and robustness to non-repetitive perturbations while ensuring convergence.
[0068] Ultimately, the total advance trigger time of the button control loop is: ; in This is the real-time compensation value output by the sliding mode controller. This is the feedforward for iterative learning during the current performance cycle.
[0069] It should be noted that the numbering of steps S3 and S4 in this embodiment is merely for convenience in textual description and logical expansion, and does not constitute a limitation on the actual physical order of their execution. According to the core control architecture of this invention, the traditional control mode has been completely decoupled into two independent and parallel single-loop sliding mode control loops. This means that in the actual automated performance of an electronic wind instrument, the button control loop and the airflow control loop operate independently and in complete parallel. Steps S3 and S4 are executed concurrently in the microprocessor, each performing independent sliding mode calculations for button timing errors and air pressure errors, without any data dependency or nested coupling between inner and outer loops. Those skilled in the art should understand that even if a target variable in the system experiences a disturbance or feedback anomaly, the other control loop will still maintain independent sliding mode action. The synchronous and parallel working mechanism of S3 and S4 is the key to this invention avoiding the system accumulation errors easily caused by existing serial control architectures, thereby achieving efficient and rapid blow-press coordination.
[0070] In summary, the collaborative control method and apparatus for the electric blowpipe robot of the present invention have the following advantages: Using visual feedback (high-speed camera + LED indicator), the camera captures the moment the LED lights up, and the trigger time can be extracted with only simple image binarization and connected component analysis. Visual inspection is independent of the acoustic environment, has strong anti-interference capabilities, and its image processing complexity is far lower than that of audio feature extraction. Therefore, it is immune to environmental noise and suitable for live performance scenarios with environmental interference; visual inspection does not require complex feature extraction algorithms, making it easier to implement on embedded systems.
[0071] Employing a dual independent parallel single-loop sliding mode structure, the button circuit and the airflow circuit are completely uncoupled, allowing the controller to calculate and execute entirely independently. From a control theory perspective, the two objectives (timing + air pressure) are achieved through parallel independent dual loops. The decoupled control system parameters are simple to tune (each loop only requires tuning one gain parameter γ), the debugging cycle is short, and the two loops do not affect each other; a failure in one will not cause the other to fail.
[0072] Dual-variable parallel control simultaneously manages the button timing and airflow intensity. The sound production mechanism of an electronic wind instrument is a coordinated "blowing + pressing" action; the amount of air blown determines the volume and timbre, while the number of buttons presses determines the pitch. This application is specifically designed for this physical characteristic of electronic wind instruments, solving the technical problem of existing solutions being unable to precisely control the airflow intensity, thereby supporting expressive techniques such as crescendo, diminuendo, and sudden forte.
[0073] Furthermore, a weighted average learning law with a forgetting factor is employed, adaptively updating the feedforward quantity based on the actual error after each performance, and suppressing noise accumulation through the forgetting factor. Therefore, this invention continuously optimizes compensation accuracy using repeated performance information, possesses learning ability for multiple performances of the same piece of music, and the forgetting factor mechanism balances convergence speed and noise resistance.
[0074] In practical applications, pneumatic systems controlled by proportional valves exhibit strong nonlinearity, time-varying characteristics, and operating point dependence. Air path inertia, friction, and fluctuations in air source pressure all affect the air pressure response. Sliding mode control is inherently robust to these uncertainties, does not rely on an exact model, and can guarantee convergence using a single-gain proportional valve.
[0075] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.
Claims
1. A cooperative control method for an electric blowpipe robot based on vision sliding mode control, characterized in that, include: Analyze the musical score information to obtain the desired key timing information and the desired airflow intensity information; The robot's blow-press coordination system is decoupled into a parallel button control loop and an airflow control loop; In the key control loop, the actual key trigger time is obtained by a visual inspection device, and the key time error is calculated by combining the expected key timing information. Then, a combination of single-loop sliding mode control and iterative learning feedforward with a forgetting factor is used to calculate the total advance trigger time based on the key time error, so as to drive the key actuator to perform press time compensation. In the airflow control loop, the actual airflow pressure is acquired, and the air pressure error is calculated in combination with the desired airflow intensity information. Then, an independent single-loop sliding mode control is used to calculate the control quantity based on the air pressure error, so as to adjust the output of the air pressure actuator and realize the tracking of the target blowing intensity.
2. The cooperative control method for an electric blowpipe robot based on vision sliding mode control according to claim 1, characterized in that, In the key control loop, the total advance trigger time is synthesized from the real-time compensation amount of the sliding mode controller and the iterative learning feedforward amount of the current performance cycle, and the calculation formula is as follows: In the formula, For the first The total advance trigger time of each note; For the first Iterative learning feedforward for each performance cycle; The first output of the sliding mode controller Real-time compensation amount for each note ; This is the tuning gain for the button control circuit. For the button control circuit in the first Discrete sliding surface function of a note.
3. The cooperative control method for an electric blowpipe robot based on vision sliding mode control according to claim 2, characterized in that, The formula for calculating the discrete sliding surface function is as follows: The formula for calculating the iterative learning feedforward is: In the formula, For the first The timing error of pressing each note; The integral coefficient; The sampling period; For the first The first performance cycle The timing error of pressing each note; The forgetting factor is set, and ; The learning gain is set proportionally, and .
4. The collaborative control method for an electric blowpipe robot based on vision sliding mode control according to claim 1, characterized in that, In the airflow control loop, the single-loop sliding mode control used is an incremental proportional sliding mode control. The control quantity is the opening command of the airflow proportional valve, and the calculation method includes: Calculate the discrete sliding surface of the pneumatic control loop: Calculate the opening increment based on the current sliding surface: Calculate the current control value based on the opening command and opening increment of the previous note, and then perform physical range limiting: In the formula, For the air pressure control loop in the first Discrete sliding surface function of each note; For the first The air pressure error of each note; The integral coefficient; The sampling period; For the first The increment of the opening of each note; Gain for the airflow control loop; and The first The note and the first The opening command for each note; This indicates that the aperture is limited to the set physical limit range. Inside.
5. The cooperative control method for an electric blowpipe robot based on vision sliding mode control according to claim 4, characterized in that, When calculating the discrete sliding surface of the air pressure control loop for the current note, it is also determined whether the opening command of the previous note has reached the maximum or minimum value of the physical amplitude limit range. or If the maximum value is reached, and the sign of the air pressure error of the current note causes the control quantity to continue to exceed the limit range, stop accumulating the integral term in the discrete sliding surface and keep the integral term at the value of the previous note; if the maximum value is not reached, the accumulation calculation of the integral term is performed normally.
6. The cooperative control method for an electric blowpipe robot based on vision sliding mode control according to claim 1, characterized in that, The process of obtaining the actual key trigger time includes: The visual detection device is activated within a preset time period before the first note is played, the preset time period being set based on the delay between camera activation and communication detection; The recording timeline of the visual inspection device is aligned and synchronized with the timeline containing the desired key timing information. Under the synchronized timeline, the visual detection device captures the frame of the button linkage indicator light being lit and records the corresponding timestamp. The timestamp is used as the actual button trigger time to obtain non-contact feedback on the actual performance time.
7. The cooperative control method for an electric blowpipe robot based on vision sliding mode control according to claim 6, characterized in that, By binarizing the image captured by the visual inspection device and performing connected component analysis on the binarized image, the frame of the linked indicator light above the button when it is lit can be extracted. The binarization process is completed by extracting the color features of the captured image and performing threshold segmentation within a preset target color gamut. The target color gamut is set based on the specific light emission color of the linked indicator light and has a preset color tolerance.
8. A collaborative control device for an electric blowpipe robot based on vision sliding mode control, used to implement the collaborative control method for an electric blowpipe robot based on vision sliding mode control as described in any one of claims 1 to 7, characterized in that, The device includes: The music score parsing module is used to parse music score information and obtain the desired key timing information and desired airflow intensity information; The perception layer includes a visual inspection device and a barometric pressure sensing module, which are used to collect feedback signals of actual button trigger time and closed-loop feedback signals of actual airflow pressure, respectively. The controller includes a key sliding mode controller and an airflow sliding mode controller. The key sliding mode controller is used to acquire the actual key trigger time and calculate the key timing error by combining it with the desired key timing information. Then, it uses a combination of single-loop sliding mode control and iterative learning feedforward with a forgetting factor to calculate the total advance trigger time based on the key timing error. The airflow sliding mode controller is used to acquire the actual airflow pressure and calculate the air pressure error by combining it with the desired airflow intensity information. Then, it uses independent single-loop sliding mode control to calculate the control quantity based on the air pressure error. The execution layer includes a button actuator that receives signals from the button sliding mode controller to perform press compensation, and a pneumatic actuator that receives instructions from the airflow sliding mode controller to adjust the blowing intensity.
9. The collaborative control device for an electric blowpipe robot based on vision sliding mode control according to claim 8, characterized in that, The visual inspection device includes a visual camera and an LED light positioned above the button for coordinated output; the button actuator includes a miniature cylinder; the pneumatic actuator includes a pneumatic source and an airflow proportional valve, and the airflow sliding mode controller drives the opening of the airflow proportional valve in real time via a communication bus.
10. The collaborative control device for an electric blowpipe robot based on vision sliding mode control according to claim 8, characterized in that, The button sliding mode controller and the airflow sliding mode controller are calculated independently and there is no nesting of inner and outer loops or mode switching logic; when a target variable in the blow-press coordination system is disturbed or feedback is abnormal, the other control loop maintains independent sliding mode action.
Citation Information
Patent Citations
Piano robot playing feed-forward compensation double-loop control system and method based on PID
CN121340296A
