Motion capture-based latte art robot trajectory reproduction control method and system

Abstract

The application discloses a motion capture-based coffee latte art robot trajectory reproduction control method and system, and relates to the technical field of robot control. The method comprises the following steps: acquiring and preprocessing master latte art trajectory data, calculating a synchronization deviation index in combination with current milk foam conditions, performing synchronization state determination and dividing synchronization sections and desynchronization sections, and generating an overall diagnostic conclusion; decomposing the trajectory into a pattern shaping section set and a pattern connection section set according to the diagnostic conclusion, and extracting milk foam edge connection constraints; calculating milk foam edge connection deviations for each pattern connection section and performing hierarchical determination, thereby redistributing execution time, generating compensated spout speed and position sequences; and finally fusing the pattern shaping section and the connection section trajectory after time redistribution, converting into a joint space trajectory, and issuing to a controller for execution. The application realizes adaptive reproduction of master latte art trajectories under different milk foam conditions, and ensures the clarity and coherence of pattern formation.

Description

Technical Field

[0001] This invention relates to the field of robot control technology, and more specifically, to a motion capture-based method and system for reproducing the trajectory of a coffee latte art robot. Background Technology

[0002] With the continuous improvement of robotics technology and automation, cooking robots have gradually entered the stage of commercial application. Among them, coffee latte art robots have become a research hotspot due to their combination of artistic expression and standardization potential. At present, one of the main technical paths to realize robot latte art is trajectory reproduction, that is, recording the latte art operation trajectory of professional baristas through motion capture systems, and then driving the robot to reproduce the trajectory.

[0003] However, existing motion capture-based trajectory reproduction methods face significant challenges in practical applications. The effect of latte art highly depends on the real-time diffusion behavior of milk foam on the coffee surface. However, the consistency and temperature of the milk foam change dynamically, causing its diffusion speed to vary accordingly. Current methods often assume that the execution environment is identical to the recording environment, directly and rigidly sending the captured master trajectory to the robot for execution, without adapting the trajectory online based on actual milk foam conditions. When the actual diffusion speed differs from the recording speed, the spout movement speed and the milk foam edge diffusion speed will mismatch, easily leading to problems such as blurred edges, breaks, or milk foam accumulation in the reproduced pattern, severely affecting the clarity, consistency, and stability of the reproduced pattern.

[0004] To address the above problems, this invention proposes a solution. Summary of the Invention

[0005] In order to overcome the above-mentioned defects of the prior art, embodiments of the present invention provide a method and system for reproducing the trajectory of a coffee latte art robot based on motion capture, so as to solve the problems mentioned in the background art.

[0006] To achieve the above objectives, the present invention provides the following technical solution:

[0007] The motion capture-based coffee latte art robot trajectory reproduction control method includes the following steps: acquiring master latte art trajectory data collected and preprocessed by the motion capture system, calculating the planned speed of the spout at each sampling moment; calculating the synchronization deviation index and determining the synchronization state by combining the baseline diffusion speed under the current milk foam conditions, dividing the master latte art trajectory into a set of synchronized segments and a set of desynchronized segments; and generating an overall diagnostic conclusion based on the overall synchronization rate obtained from the synchronization segment's share of the total trajectory duration.

[0008] Based on the overall diagnostic conclusion, the master latte art trajectory is decomposed into a set of pattern shaping segments and a set of pattern connecting segments; the milk foam edge connection constraint between each pattern connecting segment and the adjacent pattern shaping segment is extracted. The milk foam edge connection constraint is used to describe the physical constraint of the milk foam edge spreading from the edge position of the previous pattern shaping segment to the edge position of the subsequent pattern shaping segment.

[0009] For each pattern connection segment, the milk foam edge connection deviation is calculated and graded based on its milk foam edge connection constraint; according to the graded determination result of the milk foam edge connection deviation, the execution time of the pattern connection segment is redistributed, and a compensated spout speed and position sequence is generated.

[0010] The original trajectory sequence of the pattern shaping section and the trajectory sequence of the pattern connection section after time redistribution are fused in time order to generate a complete fused trajectory, which is then converted into a robot joint space trajectory and sent to the controller for execution.

[0011] In a preferred embodiment, the difference between the planned spout speed and the expected speed under the reference coupling degree at each sampling time is divided by the expected speed to obtain the synchronization deviation index, wherein the expected speed under the reference coupling degree is the product of the reference coupling degree and the baseline diffusion speed under the current milk foam conditions.

[0012] Based on the absolute value of the synchronization deviation index, the synchronization status at each sampling moment is determined as synchronized, weakly desynchronized, or strongly desynchronized by setting a first synchronization deviation threshold and a second synchronization deviation threshold. Specifically, when the absolute value of the synchronization deviation index is less than the first synchronization deviation threshold, it is determined to be in a synchronized state; when the absolute value of the synchronization deviation index is greater than or equal to the first synchronization deviation threshold and less than the second synchronization deviation threshold, it is determined to be in a weakly desynchronized state; and when the absolute value of the synchronization deviation index is greater than or equal to the second synchronization deviation threshold, it is determined to be in a strongly desynchronized state.

[0013] In a preferred embodiment, based on the overall diagnostic conclusion, the master latte art trajectory is decomposed into a set of pattern shaping segments and a set of pattern connecting segments, including: when the overall synchronization rate is greater than a preset first synchronization rate threshold, the diagnostic conclusion is that it can be directly reproduced; when the overall synchronization rate is greater than a preset second synchronization rate threshold but does not exceed the first synchronization rate threshold, the diagnostic conclusion is that local correction is required; when the overall synchronization rate does not exceed the second synchronization rate threshold, the diagnostic conclusion is that full compensation is required.

[0014] When the diagnosis indicates that local correction is required, all segments in the synchronous segment set will be directly used as pattern shaping segments.

[0015] When the diagnosis conclusion is that full compensation is required, the set of synchronization segments is screened, and the synchronization segments that meet the following conditions are selected as pattern shaping segments: the absolute value of the mean of the synchronization deviation index is less than the preset mean threshold of stability, the standard deviation of the synchronization deviation index is less than the preset standard deviation threshold of stability, and the segment duration is greater than the preset minimum segment duration threshold.

[0016] The trajectory portion of the non-pattern-defined section is marked as the pattern-connecting section.

[0017] In a preferred embodiment, the extraction process of the milk foam edge connection constraint includes: calculating the edge position of the preceding pattern shaping section of the pattern connection segment, using the following formula: Where 'c' represents the identifier for the pattern connection section, For the first The edge position of the front pattern shaping section of the pattern connection section. For the first The preceding pattern shaping section is marked with the logo of the pattern connection section. The spout position at the end of the previous pattern shaping section. To create a delay distance at the edge, The unit vector representing the direction of spout movement at the end of the previous pattern shaping section;

[0018] The formula for calculating the edge position of the subsequent pattern shaping section in the pattern connection zone is as follows: ;in, For the first The edge position of the pattern shaping section after the pattern connection section, The spout position at the beginning of the later pattern shaping section. The unit vector representing the direction of spout movement at the beginning of the pattern-setting section;

[0019] The formula for calculating the diffusion distance of milk foam at the edge of the pattern connection section is as follows: ;in, For the first The distance of the milk foam edge diffusion in the pattern connection section represents the straight-line distance from the edge of the milk foam edge to the edge of the pattern shaping section.

[0020] In a preferred embodiment, for each pattern connection segment, the milk foam edge connection deviation is calculated based on its milk foam edge connection constraint, including: based on the milk foam edge diffusion distance. The baseline diffusion rate under current milk foam conditions and the original duration of the pattern connection section Calculate the edge connection deviation of milk foam. The calculation formula is as follows: Among them, a positive value for the edge connection deviation of the milk foam indicates that the actual diffusion distance of the milk foam edge is less than the required diffusion distance, while a negative value for the edge connection deviation of the milk foam indicates that the actual diffusion distance of the milk foam edge is greater than the required diffusion distance.

[0021] In a preferred embodiment, the execution time of the pattern connection segment is redistributed according to the grading determination result of the milk foam edge connection deviation, including: setting a connection deviation allowable threshold and a connection deviation severe threshold, and comparing the absolute value of the milk foam edge connection deviation with the threshold.

[0022] When the absolute value of the edge connection deviation of the milk foam is less than the allowable threshold for connection deviation, it is judged to be in the allowable level, and the original duration of the pattern connection segment remains unchanged.

[0023] When the absolute value of the edge connection deviation of the milk foam is greater than or equal to the allowable threshold for connection deviation but less than the severe threshold for connection deviation, it is judged as the compensation level; when the absolute value of the edge connection deviation of the milk foam is greater than or equal to the severe threshold for connection deviation, it is judged as the over-limit level.

[0024] For pattern connection segments that are determined to be at the compensation level or the over-limit level, time reallocation is performed. The original duration of the pattern connection segment is added to the time compensation amount to obtain the compensated execution duration. The time compensation amount is the quotient obtained by dividing the milk foam edge connection deviation by the baseline diffusion speed under the current milk foam conditions.

[0025] In a preferred embodiment, generating the compensated spout speed and position sequence includes: using the straight-line distance between the starting position of the subsequent pattern shaping section and the ending position of the preceding pattern shaping section as the spout displacement distance, and dividing the spout displacement distance by the compensated execution time to obtain the compensated spout speed.

[0026] The compensated spout speed is limited. When the compensated spout speed is less than the preset minimum safe speed, it is set to the minimum safe speed; when the compensated spout speed is greater than the preset maximum safe speed, it is set to the maximum safe speed.

[0027] If the speed limit is triggered and the speed limit is exceeded, the actual execution time and the corresponding actual milk foam edge diffusion distance are recalculated based on the spout speed after the speed limit is exceeded. The difference between the actual milk foam edge diffusion distance and the milk foam edge diffusion distance is used as the residual connection deviation. When the absolute value of the residual connection deviation exceeds the preset residual deviation threshold, the connection limit is exceeded and no speed sequence that can be sent is generated.

[0028] For pattern connection sections that do not trigger the connection over-limit sign, linear interpolation is used to plan the velocity direction, and a velocity sequence is generated by combining the compensated spout velocity. The position sequence is then generated by integration.

[0029] In a preferred embodiment, the original trajectory sequence of the pattern shaping section and the trajectory sequence of the pattern connecting section after time redistribution are fused in chronological order, including: alternating the pattern shaping section and the pattern connecting section in chronological order to construct the time axis of the fused trajectory;

[0030] For pattern-defined segments, keep the original spatial positions of trajectory points unchanged, and update the corresponding time labels to match the time axis of the fused trajectory.

[0031] For pattern-connecting segments, their time labels and spatial positions in the fusion trajectory are determined based on the generated position sequence and the compensated execution time.

[0032] The trajectory point sequences of all pattern-defined sections and pattern-connecting sections are merged according to the new time axis to form a complete fused trajectory that includes time, position, and velocity sequences.

[0033] In a preferred embodiment, generating a complete fusion trajectory further includes: setting a smoothing window at the junction of the pattern shaping section and the pattern connecting section, performing linear transition processing on the spout speed within the smoothing window, and regenerating the position sequence within the smoothing window through integration to ensure the speed continuity at the trajectory fusion point.

[0034] The motion capture-based coffee latte art robot trajectory reproduction control system includes the following modules: a synchronization diagnosis module, which is used to calculate the planned speed of the spout at each sampling moment based on the master spout trajectory data acquired and preprocessed by the motion capture system, calculate the synchronization deviation index and determine the synchronization status in combination with the current milk foam baseline diffusion speed, divide the master latte art trajectory into a set of synchronized segments and a set of desynchronized segments, and generate an overall diagnostic conclusion based on the proportion of synchronized segments.

[0035] The pattern segment decomposition module is used to decompose the master latte art trajectory into a set of pattern shaping segments and a set of pattern connecting segments based on the overall diagnostic conclusion, and to extract the milk foam edge connection constraints between each pattern connecting segment and the adjacent pattern shaping segment.

[0036] The time redistribution module is used to calculate the milk foam edge connection deviation based on the milk foam edge connection constraint for each pattern connection segment, and to obtain the compensated duration by eliminating the milk foam edge connection deviation through time redistribution.

[0037] The trajectory fusion execution module is used to fuse the original trajectory sequence of the pattern shaping section with the trajectory sequence of the pattern connecting section after time redistribution in chronological order, generate a complete fused trajectory, convert it into a robot joint space trajectory, and send it to the controller for execution.

[0038] The technical effects and advantages of the motion capture-based coffee latte art robot trajectory reproduction control method and system of this invention are as follows:

[0039] This invention achieves a quantitative assessment of the reproducibility of master latte art trajectories under current milk foam conditions by introducing a synchronization deviation index and a pre-execution synchronization diagnosis mechanism, avoiding the risk of pattern failure caused by blind execution in existing technologies. Secondly, through trajectory decomposition, the trajectory is divided into pattern shaping segments and connecting segments, preserving the fine movement characteristics of the master on key shaping structures such as the leaf tip and heart tip, while identifying weak links that need adjustment for environmental changes. Based on the unique physical process of milk foam edge diffusion in coffee latte art, this invention performs milk foam edge connecting deviation analysis and calculation, and derives a time redistribution formula accordingly. By dynamically adjusting the execution time of the pattern connecting segments, the problem of edge breakage or accumulation caused by changes in milk foam diffusion speed is effectively eliminated, ensuring the consistency of the pattern under different milk foam conditions. Through trajectory fusion and smoothing, the continuity and stability of robot movement are guaranteed. This method improves the success rate and edge clarity of complex latte art patterns under non-standard milk foam conditions, and significantly enhances the environmental adaptability of the latte art robot. Attached Figure Description

[0040] Figure 1 This is a schematic diagram of the trajectory reproduction control method for a coffee latte art robot based on motion capture according to the present invention;

[0041] Figure 2 This is a schematic diagram of the pattern segment decomposition and milk foam edge connection constraint extraction of the present invention;

[0042] Figure 3 This is a schematic diagram of the trajectory reproduction control system for a coffee latte art robot based on motion capture, as described in this invention. Detailed Implementation

[0043] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0044] Example

[0045] Please see Figure 1 As shown, this invention discloses a motion capture-based trajectory reproduction control method for a coffee latte art robot, comprising the following steps:

[0046] Step 1: Obtain the master latte art trajectory dataset collected and preprocessed by the motion capture system; obtain the baseline diffusion speed under the current milk foam conditions through the prediction experiment; calculate the synchronization deviation index, determine the synchronization status based on the synchronization deviation index, divide the master latte art trajectory into a set of synchronized segments and a set of desynchronized segments, calculate the proportion of synchronized segments to the total trajectory time to obtain the overall synchronization rate, and generate an overall diagnostic conclusion.

[0047] The purpose of this step is to perform a synchronization diagnosis on the master latte art trajectory acquired through motion capture before the robot performs the latte art, calculate the degree of matching between the planned spout speed at each moment in the trajectory and the expected diffusion speed under the current milk foam conditions, and output the three-value judgment results and overall diagnostic conclusions for each moment; specifically, it includes:

[0048] The master latte art trajectory data used in this invention is acquired through a motion capture system. When the motion capture system adopts a real-time on-site capture method, the three-dimensional coordinates of the marked points on the spout of the latte art kettle are collected in real time by an optical motion capture system such as OptiTrack or Vicon while the professional latte art artist is performing the latte art operation. The sampling frequency of the motion capture system is set to no less than 120Hz to ensure that the trajectory details of the spout during rapid movement are captured. The raw data obtained is the position time series of the spout marked points in the world coordinate system.

[0049] All acquired raw trajectory data must undergo uniform preprocessing. The preprocessing process includes four steps: coordinate system standardization, timestamp normalization, resampling, and filtering / denoising. Coordinate system standardization transforms the spout position coordinates obtained from motion capture into a standard coordinate system with the center of the coffee cup as the origin, the rim of the cup as the horizontal plane, and the vertical upward direction as the positive direction. Timestamp normalization sets the starting time of the trajectory to zero, and calculates the time offset of all subsequent times relative to the starting time, with the unit being uniformly seconds. Resampling resamples the raw data from different sampling frequencies to a standard analysis frequency. Among them, standard analysis frequency The initial value is set to 200 Hz in this embodiment, and the missing sampling points are filled by linear interpolation. The filtering and noise reduction is to apply a low-pass Butterworth filter with a cutoff frequency of 30 Hz to the resampled position sequence to eliminate the measurement noise of the motion capture system, while retaining the effective high-frequency components of the spout motion.

[0050] After preprocessing, the master latte art trajectory dataset is obtained. The dataset contains Each sampling point includes a timestamp. and the corresponding three-dimensional position of the spout Where i represents the sampling point identifier and Sampling interval Second;

[0051] Based on the preprocessed master latte art trajectory dataset The planned spout velocity is calculated at each sampling time. The planned spout velocity is defined as the projection modulus of the first derivative of the spout position with respect to time onto the liquid surface plane XY. The planned spout velocity is calculated using the central difference method. Specifically, for the ... At this time, each sampling point The method for calculating the planned speed of the spout is as follows: First, calculate the position difference vector between adjacent sampling points, then divide it by twice the sampling interval to obtain the velocity vector, and finally take the projection magnitude of this velocity vector on the liquid surface plane XY as the scalar value of the planned speed of the spout. The formula for calculating the planned speed of the spout is expressed as:

[0052] ;in, Indicates the sampling time The corresponding planned speed of the spout, where x represents the x-axis of the liquid surface. Indicates the first The sampling points are located on the x-axis of the liquid surface. Indicates the first The sampling points are located on the x-axis of the liquid surface, and y represents the y-axis of the liquid surface. Indicates the first The sampling points are located on the y-axis of the liquid surface. Indicates the first The position of each sampling point on the y-axis of the liquid surface;

[0053] For boundary sampling points, i.e. and The forward and backward differencing methods are used for calculation; after the calculation is completed, the time series of the planned spout speed is obtained. The unit is millimeters per second;

[0054] Before the robot performs latte art, it is necessary to estimate the diffusion rate under the current milk foam conditions; this invention obtains the baseline diffusion rate through a predictive experimental method. The baseline diffusion rate is used to characterize the ability of milk foam to diffuse on the surface of coffee liquid under the current milk foam state. The basic principle of the prediction experiment is: before the formal latte art begins, a small amount of milk foam sample is released onto the surface of coffee liquid, and the diffusion process of milk foam on the surface of liquid is observed through a visual sensor. The movement speed characteristics of the diffusion front are extracted, and then a baseline speed value that can represent the current milk foam diffusion ability is obtained. The method of determining the baseline diffusion rate is to select the instantaneous value, average value, peak value or other statistical characteristic value of the diffusion rate according to actual needs.

[0055] As a specific implementation method, the experiment is expected to adopt the following operation procedure: control the robot to tilt the latte art pitcher to a preset angle, which is set to 35 degrees in this embodiment, and drip a small amount of milk foam (approximately 0.5 ml) onto the edge of the coffee cup; simultaneously, activate the high-speed camera installed directly above the cup rim to capture the process of milk foam spreading on the liquid surface at a frame rate of 200 frames per second for 500 milliseconds; process the captured liquid surface image sequence to extract the movement speed of the milk foam diffusion front; the liquid surface image sequence process includes: first, converting the RGB image to the Lab color space; then, performing adaptive threshold segmentation based on the a channel to separate the milk foam area from the coffee area; extracting the boundary of the milk foam area as the diffusion front; calculating the normal displacement of the diffusion front between adjacent frames and dividing it by the frame interval time to obtain the diffusion speed; in this embodiment, the arithmetic mean of the diffusion speed within the shooting time is taken as the baseline diffusion speed. , measured millimeters per second;

[0056] Since the diffusion speed of milk foam changes with position and time during actual latte art, this invention establishes a diffusion speed field estimation model. The diffusion speed field estimation model is expressed as the estimated diffusion speed at the current moment equal to the baseline diffusion speed multiplied by the product of the position correction factor and the time correction factor. The position correction factor ranges from 0.85 to 1.15, and the time correction factor ranges from 0.90 to 1.00. This model is based on the following principle: the area near the spout diffuses faster due to the continuous impact of new milk foam, while the area far from the spout diffuses slower due to the decay of milk foam kinetic energy. As latte art progresses, the accumulation of milk foam on the liquid surface increases, hindering the diffusion of subsequent milk foam.

[0057] The Synchronization Deviation Index (SDI) is used to quantify the degree of mismatch between the planned spout speed and the baseline diffusion speed under current milk foam conditions. In latte art, to create clear and sharp pattern edges, a specific proportional relationship needs to be maintained between the spout movement speed and the milk foam diffusion speed. This invention defines this proportional relationship as the coupling degree. Professional latte artisans maintain a nearly constant coupling degree under different milk foam conditions. This invention uses the average coupling degree during master operations as a reference coupling degree. It determines the reference coupling degree by statistically analyzing motion capture data from multiple professional latte art artists. The initial value is 1.10; the synchronization deviation index is defined as the difference between the planned spout speed and the expected speed under the reference coupling degree, divided by the expected speed for normalization; the sign of the synchronization deviation index SDI carries physical meaning: a positive value indicates that the spout speed is too fast relative to the diffusion speed, which easily leads to pattern breakage; a negative value indicates that the spout speed is too slow relative to the diffusion speed, which easily leads to pattern accumulation; the calculation formula for the synchronization deviation index is expressed as: ;

[0058] Data set of master latte art trajectories The synchronization deviation index (SDI) time series is obtained by applying the formula for calculating the synchronization deviation index to each sampling point. ;

[0059] Based on the absolute value of the synchronization deviation index SDI, for each sampling time... The synchronization status is classified and determined; the present invention sets a first synchronization deviation threshold. Second synchronization deviation threshold As the classification boundary, the first synchronization deviation threshold The initial value is set to 0.15, and the second synchronization deviation threshold is... The initial value is set to 0.40;

[0060] Synchronization state is determined using the following classification function: ;

[0061] According to the above classification function, the synchronization state classification criterion is: when the absolute value of the synchronization deviation index is less than the first synchronization deviation threshold... When the time is right, it is determined that the moment is in a synchronized state, indicating that the spout speed and the diffusion speed are well matched;

[0062] When the absolute value of the synchronization deviation index is greater than or equal to the first synchronization deviation threshold And less than the second synchronization deviation threshold At that moment, it was determined that the state was weakly out of sync, indicating that there was a slight deviation.

[0063] When the absolute value of the synchronization deviation index is greater than or equal to the second synchronization deviation threshold At that moment, it is determined that the state is severely out of sync, indicating a serious deviation.

[0064] Applying the above synchronization state classification criteria to all sampling points yields a synchronization state sequence. ;

[0065] It should be noted that the above two thresholds are based on pattern quality experiments: Master latte art trajectories were executed at different synchronization deviation index levels, and professional latte artists blindly evaluated the generated patterns and recorded the edge sharpness scores. The experimental results showed that when the absolute value of the synchronization deviation index was less than the first synchronization deviation threshold, the edge sharpness of the pattern decreased imperceptibly; when the absolute value of the synchronization deviation index was between the two thresholds, the edges became slightly blurred; and when the absolute value of the synchronization deviation index was greater than or equal to the second synchronization deviation threshold, the pattern showed obvious breaks or severe accumulation.

[0066] Furthermore, based on the synchronization status determination results at each sampling point, the trajectory is divided into several continuous segments; the segmentation rules are as follows:

[0067] Adjacent sampling points with the same synchronization state are grouped into the same segment; when the synchronization state of a sampling point changes, the segment boundary is identified; after the division is completed, all segments with the same synchronization state are merged into a set of synchronized segments. Merge all segments in weak and strong desynchronization states into a set of desynchronization segments. Each synchronization segment is represented by its start and end times, i.e., the set of synchronization segments. Set of out-of-synchronization sections Where j represents the number of the synchronization segment and j=1,2, , This represents the total number of synchronization segments. Indicates the start and end times of the synchronization segment. Let j be the start time of the j-th synchronization segment. Let be the end time of the j-th synchronization segment; k represents the number of the desynchronization segment and k=1,2, , This represents the total number of out-of-synchronization sections. This indicates the start and end times of the synchronization failure segment. Let k be the start time of the k-th desynchronization segment. This represents the end time of the k-th desynchronization segment;

[0068] The time proportion of the synchronized segments to the total trajectory is calculated based on the obtained set of synchronized segments, and this proportion is used as the overall synchronization rate. The formula for calculating the overall synchronization rate is as follows: Based on overall synchronization rate Generate overall diagnostic conclusions; this invention sets a first synchronization rate threshold. Second synchronization rate threshold As a diagnostic boundary, the first synchronization rate threshold The initial value is set at 95%, and the second synchronization rate threshold is... The initial value is set to 70%;

[0069] The rule for determining the diagnostic conclusion is: when the overall synchronization rate... Greater than the first synchronization rate threshold At that time, the diagnosis conclusion was that it could be directly reproduced, indicating that the master's latte art trajectory could be directly sent to the robot for execution without any compensation processing;

[0070] When the overall synchronization rate Greater than the second synchronization rate threshold And not exceeding the first synchronization rate threshold At that time, the diagnosis was that local correction was required, indicating that most of the trajectory could be preserved and only a few out-of-synchronization sections needed to be processed.

[0071] When the overall synchronization rate Not exceeding the second synchronization rate threshold At that time, the diagnosis concluded that full compensation was required, indicating that there were many asynchronous sections and a comprehensive trajectory reconstruction was needed;

[0072] Based on the overall diagnostic conclusion, the corresponding decision logic is executed: if the diagnostic conclusion is that it can be directly reproduced, the preprocessed master latte art trajectory is directly sent to the robot controller for execution, and the process ends; if the diagnostic conclusion is that local correction or full compensation is required, the process proceeds to the next step to perform pattern segment decomposition processing.

[0073] Step 2: Based on the overall diagnostic conclusion, the master latte art trajectory is decomposed into a set of pattern shaping segments and a set of pattern connecting segments; the milk foam edge connection constraints between each pattern connecting segment and adjacent shaping segments are extracted. The milk foam edge connection constraints are used to describe the physical constraints of the milk foam edge spreading from the edge position of the previous shaping segment to the edge position of the subsequent shaping segment; among them, the pattern shaping segments are trajectory segments selected from the synchronization segments to reproduce the key structure of the pattern; the pattern connecting segments are trajectory segments that connect adjacent pattern shaping segments;

[0074] The purpose of this step is to call the Synchronization Deviation Index (SDI) time series output from step 1. Based on the segmentation results and the overall diagnostic conclusion, the corresponding processing mode is selected to decompose the master latte art trajectory into two categories: pattern shaping segments and pattern connecting segments. The milk foam edge connection constraints between each pattern connecting segment and the adjacent shaping segment are extracted.

[0075] At the beginning of this step, all output data from step 1 is read; the synchronization deviation index time series is read for subsequent calculation of the average synchronization deviation index value of the pattern connection segment; the set of synchronized segments and the set of desynchronized segments are read as candidate sets for pattern shaping segment screening and pattern connection segment marking; the overall diagnostic conclusion is read for selecting the processing mode.

[0076] Read the overall diagnostic conclusion and select the corresponding processing mode, including: the present invention defines two processing modes, namely local correction mode and full reconstruction mode;

[0077] When the overall diagnostic conclusion is that local correction is needed, the local correction mode is adopted. In this mode, the synchronization of most trajectories is good, and only a few out-of-synchronization sections need to be processed. Therefore, a lenient screening condition is adopted to set up the synchronization sections. Each section Add it directly to the pattern shaping section set ;

[0078] When the overall diagnosis conclusion is that full compensation is required, the full reconstruction mode is adopted. In this mode, there are many desynchronization segments. It is necessary to select the most stable segment from the synchronization segments as the pattern shaping segment. Therefore, strict screening conditions are adopted.

[0079] In the full reconstruction mode, it is necessary to adjust the set of synchronization segments. Further screening is performed; the screening criteria include two items: synchronization stability condition and duration condition; the synchronization stability condition is defined as follows: calculate the set of synchronization segments. The mean and standard deviation of the synchronization deviation index (SDI) of all sampling points are considered when the absolute value of the mean SDI is less than the stability mean threshold. Furthermore, the standard deviation of the synchronization deviation index (SDI) is less than the stability standard deviation threshold. When this happens, the synchronization section is deemed to meet the synchronization stability conditions;

[0080] Among them, the stability mean threshold The initial value is set to 0.10, and the stability standard deviation threshold is... The initial value is set to 0.03; the mean of the synchronization deviation index (SDI) reflects the average level of synchronization deviation within the synchronization segment, and the standard deviation of the SDI reflects the degree of fluctuation of the synchronization deviation. This dual constraint ensures that the selected pattern shaping segment has stable and reliable synchronization characteristics; the duration condition is defined as follows: the duration of the synchronization segment must be greater than the minimum synchronization segment duration threshold. Among them, the minimum segment duration threshold The initial value is set to 300 milliseconds. Too short a synchronization segment may be a synchronization state that occurs accidentally and is not suitable as a reliable pattern shaping segment.

[0081] Among them, the mean of the synchronization deviation index SDI within the synchronization section The calculation formula is expressed as:

[0082] ;in Synchronization section The number of sampling points within;

[0083] The formula for calculating the standard deviation of the synchronization deviation index (SDI) within the synchronization section is as follows:

[0084] ;

[0085] The specific operation of the filtering process is as follows: traverse the set of synchronization segments. Each synchronization segment in Calculate the mean of its synchronization deviation index SDI Synchronization Deviation Index (SDI) Standard Deviation and synchronization segment duration If both conditions are met , and If three conditions are met, then the segment will be added to the pattern shaping segment set. ;

[0086] After completing the pattern-defined segment selection, the non-pattern-defined segments are marked as pattern-connecting segments; on the timeline, the segment between two adjacent pattern-defined segments is the pattern-connecting segment; let the set of pattern-defined segments be... Arranged in chronological order Where M represents the total number of pattern-fixing segments, The time range is Then the set of pattern connection segments Include Section 1, No. The pattern connects to the section. The time range is ,in For each pattern connection section Record the identification of the preceding pattern shaping section as The final section of the pattern is marked as follows: ;

[0087] For each pattern connection section Calculate its original duration and the average synchronization deviation index (SDI) value within the segment; original duration The calculation method is based on the back boundary time. Subtract the front boundary time Average synchronization deviation index The calculation method is to calculate the arithmetic mean of the synchronization deviation index values ​​of all sampling points in the section. This value carries a positive or negative sign and reflects the overall deviation direction and degree of the spout velocity relative to the diffusion velocity in the pattern connection section.

[0088] Please see Figure 2 As shown, further, the milk foam edge connection constraint is extracted, which is used to describe the physical constraint that the milk foam edge of the pattern connection section needs to spread from the edge position of the previous pattern shaping section to the edge position of the subsequent pattern shaping section.

[0089] In latte art, the formation of the milk foam edge lags behind the spout position; this is because it takes time for the milk foam to diffuse and form a stable edge after falling from the spout onto the liquid surface; therefore, the edge formation delay distance is defined as follows. This is the distance between the spout position and the corresponding edge position, measured in the opposite direction of the spout's movement; its initial value is 4 mm; the calculation method for the unit vector of the spout's movement direction is as follows: Let the trajectory sampling interval be... For example, 5 milliseconds, taking the values ​​before and after time t. The duration of the spout position forms a differential displacement vector. ;in The preferred value is 25 milliseconds; when Cannot be obtained simultaneously near the segment boundary and At that time, forward difference was used instead. or backward difference ; to divide the displacement vector Normalization yields the unit vector of the spout's motion direction. ;

[0090] When the difference displacement vector Less than the preset minimum displacement threshold When using the spout motion direction unit vector from the previous valid moment, it is preferable to use the one-way unit vector from the previous valid moment. To avoid numerical instability;

[0091] The process of extracting the edge connection constraints of the milk foam includes calculating three aspects: the edge position of the previous pattern shaping section, the edge position of the subsequent pattern shaping section, and the diffusion distance of the milk foam edge; the edge position of the previous pattern shaping section... It refers to the pre-pattern shaping section The position of the edge of the milk foam on the liquid surface at the end of the current time is calculated as follows: take the spout position at the end of the previous pattern shaping segment. Subtract the edge delay distance The unit vector of the spout's motion direction at that moment The product of and is expressed by the formula:

[0092] ;in For the first The edge position of the previous pattern shaping section of the pattern connection section is taken as the XY plane component of the liquid surface plane;

[0093] Edge position of the post-pattern shaping section This refers to the post-pattern shaping section. The expected edge position of the milk foam at the start moment represents the target position that the edge of the milk foam should reach at the end of the pattern transition segment. Its calculation method is the same as that for the edge position of the previous pattern shaping segment, and the formula is expressed as follows:

[0094] ;in For the first The edge position of the pattern shaping section after the pattern connection section is taken as the XY plane component of the liquid surface plane.

[0095] Milk foam edge diffusion distance This refers to the straight-line distance that the edge of the milk foam needs to spread from the edge of the previous pattern shaping section to the edge of the next pattern shaping section within the pattern connection zone. The formula is expressed as:

[0096] ;in For the first The distance of the milk foam's edge diffusion in the connecting section of each pattern;

[0097] The edge position of the pre-pattern shaping section obtained above , edge position of the pattern shaping section Distance between the edge of the milk foam Assembled into a data structure for constraining the edge connections of milk foam; The edge connection constraint of the milk foam in each pattern connection segment is represented as follows: ;

[0098] It should be noted that the formation of the pattern shaping segment set in this embodiment is illustrated using the leaf pattern trajectory as an example; this step adopts a full reconstruction mode; the synchronous segment set... The five segments were filtered separately; taking the first segment as an example: this segment corresponds to the initial build-up phase, with a time range of (0.0, 1.2) seconds and a duration of 1.2 seconds. The SDI mean of the 240 sampling points within this segment was calculated to be -0.03, and the SDI standard deviation was 0.02; due to , and This segment meets all the screening criteria and is included in the pattern finalization segment set; after screening one by one, all five synchronous segments meet the criteria, and the pattern finalization segment set is complete. These correspond to the initial stroke and energy-building stage, the first leaf tip shaping stage, the second leaf tip shaping stage, the third leaf tip shaping stage, and the finishing and sealing stage of the leaf pattern; based on the time distribution of the shaping sections, four pattern connection sections are determined: lie in and Between these two points, corresponding to the first segment of leaf vein extension brushstrokes, the time range is (1.2, 2.1) seconds; lie in and Between these two points, corresponding to the second segment of leaf vein extension brushstrokes, the time range is (2.8, 3.9) seconds; lie in and Between 4.5 and 5.8 seconds; lie in and Between 6.4 and 7.2 seconds.

[0099] Step 3: For each pattern connection segment, calculate the milk foam edge connection deviation and classify it based on its milk foam edge connection constraint, average synchronization deviation index and diffusion speed under the current milk foam conditions; according to the classification result of the milk foam edge connection deviation, redistribute the execution time of the pattern connection segment and generate the compensated spout speed and position sequence.

[0100] For each pattern connection segment obtained in step 2 Read the distance of the milk foam's edge diffusion. Average synchronization deviation index and original duration Three data points; including the distance of milk foam diffusion at the edge. This indicates the distance the edge of the milk foam needs to spread from the edge of the previous pattern shaping section to the edge of the next pattern shaping section; this value can only be obtained from the pattern segment decomposition structure in step 2; average synchronization deviation index. The synchronization deviation exponential time series from step 1 is calculated and passed to this step via step 2; the baseline diffusion rate under the current milk foam conditions. The results were obtained from the prediction experiment in step 1 under the current milk foam conditions;

[0101] First, calculate the edge connection deviation of the milk foam. When the master records the latte art trajectory, the original duration of the pattern connection segment exactly satisfies the balance condition of the milk foam edge diffusion, that is, the diffusion distance of the milk foam edge is equal to the product of the diffusion speed recorded by the master and the original duration. When the robot executes the trajectory under different milk foam conditions, if it still executes according to the original duration, the actual diffusion distance of the milk foam edge is equal to the product of the baseline diffusion speed under the current milk foam condition and the original duration. Based on this, the edge connection deviation of the milk foam is defined as the difference between the expected diffusion distance and the actual diffusion distance, and the calculation formula is expressed as:

[0102] Among them, the baseline diffusion rate under current milk foam conditions. The results were obtained from the prediction experiment in step 1 under the current milk foam conditions;

[0103] Since the diffusion speed during master recording cannot be directly measured in actual execution, it is necessary to establish a quantitative relationship between the synchronization deviation index and the baseline diffusion speed under the current milk foam conditions. Based on the definition of the synchronization deviation index in step 1, the coupling relationship during master recording is substituted and algebraically rearranged to obtain a calculable form of the milk foam edge connection deviation:

[0104] ;

[0105] This formula indicates the average synchronization deviation index. The symbol directly determines the deviation of the milk foam edge connection. The symbol; when This indicates insufficient diffusion, which will lead to the risk of leaf vein breakage; when This indicates excessive diffusion, which may lead to edge accumulation.

[0106] Then, the deviation in the edge connection of the milk foam is graded and judged; this invention sets an allowable threshold for the connection deviation. and severe connection deviation threshold As the grading boundary, the allowable threshold for connection deviation is... The initial value is set to 0.5 mm, and the threshold for severe connection deviation is set. The initial value is set to 3.0 mm. Based on the comparison between the absolute value of the milk foam edge connection deviation and the above two thresholds, a three-level judgment is made for each pattern connection segment. The judgment rules are as follows: When the absolute value of the milk foam edge connection deviation is less than the allowable threshold for connection deviation, it is judged as the allowable level, indicating that the deviation is within the tolerance range of the natural diffusion of milk foam and the original duration can be maintained; when the absolute value of the milk foam edge connection deviation is greater than or equal to the allowable threshold for connection deviation and less than the severe threshold for connection deviation, it is judged as the compensation level, indicating that the deviation exceeds the tolerance but can be eliminated by time redistribution; when the absolute value of the milk foam edge connection deviation is greater than or equal to the severe threshold for connection deviation, it is judged as the over-limit level, indicating that the deviation is too large, a warning needs to be marked and the feasibility of compensation needs to be evaluated.

[0107] Next, the corresponding processing strategy is executed based on the classification judgment result; for pattern connection segments judged to be permissible, the original duration is maintained. Unchanged, the execution time after compensation Equal to the original duration For pattern connection sections determined to be at the compensation level or the over-limit level, execution time reallocation is performed; since the milk foam edge connection deviation... The distance by which diffusion is less or more within the original duration is represented. The amount of time required for compensation should be equal to that distance divided by the baseline diffusion rate under the current milk foam conditions. The compensated execution time is defined as the original duration plus the time compensation amount, expressed by the formula:

[0108] ;

[0109] when When the time compensation is positive, the execution time increases after compensation, and the spout delays waiting for the milk foam to spread at the edge and catch up; when When the time compensation amount is negative, the execution time is reduced after compensation, and the spout shortens the time to avoid premature accumulation of milk foam at the edge of the foam. This implementation method uses the method of first calculating the connection deviation and then converting the time compensation amount to highlight the time redistribution mechanism driven by the physical process of milk foam diffusion in coffee latte art.

[0110] Further calculations are needed to determine the compensated spout velocity within the pattern transition section; firstly, the spout displacement distance within the pattern transition section must be calculated. , defined as the straight-line distance between the starting position of the subsequent pattern shaping section and the ending position of the preceding pattern shaping section; the compensated spout speed is calculated by dividing the spout displacement distance by the compensated execution time, expressed by the formula:

[0111] ;

[0112] To ensure robot safety, the speed of the spout after compensation is limited; a minimum safe speed is defined. and maximum safe speed As a speed boundary, the minimum safe speed The initial value is set at 5 millimeters per second, and the maximum safe speed is... The initial value is set to 150 millimeters per second; when Set it to ,when Set it to When the limit is triggered, the actual execution time should be re-determined based on the ratio of the spout displacement distance to the speed after the limit is triggered.

[0113] Finally, the compensation effect is verified and a velocity sequence is generated. For segments determined to be at the compensation level, it is verified whether the milk foam edge connection deviation is eliminated to zero after time redistribution: substituting the compensated execution time, the actual diffusion distance of the milk foam edge is equal to the actual diffusion distance of the milk foam edge. The connection deviation is eliminated to zero; for sections determined to be beyond the limit and triggering speed limiting, the residual connection deviation is calculated and compared with the residual deviation threshold. Comparison, where the residual bias threshold The initial value is set to 1 mm; when the absolute value of the residual deviation exceeds A connection failure flag is output, and no downloadable velocity sequence is generated for this section to avoid introducing uncontrollable breakage or accumulation risks during fusion in step 4. For verified pattern connection sections, velocity and position sequences are generated. Velocity direction planning uses a linear interpolation method; the starting direction is determined by the spout movement direction at the end of the previous pattern shaping section, and the ending direction is determined by the spout movement direction at the beginning of the subsequent pattern shaping section. Velocity vector... It equals the scalar velocity of the spout after compensation multiplied by the unit vector of the direction of motion; taking the end position of the previous pattern shaping section as the starting point, the Euler integral is performed according to the velocity vector to generate the position sequence of the pattern connecting section, and assembled with the timestamp to form the output data structure of the section, which is called for the trajectory fusion in step 4.

[0114] Step 4: The original trajectory sequence of the pattern shaping section and the trajectory sequence of the pattern connecting section after time redistribution are alternately arranged in time order to fuse the trajectory and generate a complete fused trajectory; the fused trajectory is converted into a joint space trajectory that can be executed by the robot and sent to the controller for execution;

[0115] This step is used to reconstruct the time axis, reconstruct the trajectory, and merge the pattern shaping segment set output in step 2 and the pattern connecting segment set output in step 3. The merged trajectory is then sent to the robot controller to achieve the preservation of the master latte art trajectory and the adaptive adjustment of the connecting segments.

[0116] First, this step reads the pattern shaping segment set from the output of step 2 and obtains the original trajectory point sequence of each shaping segment. The trajectory of the pattern shaping segment is directly retained without any modification because this type of segment corresponds to key structures such as the leaf tip and the heart tip, and has a high-fidelity reproduction requirement. Further, this step reads the pattern connecting segment set from the output of step 3 and obtains the judgment level label, the execution time after compensation, the spout speed after compensation, whether the speed limit is triggered, the actual execution time, and the speed sequence and position sequence for each connecting segment. Among them, when step 3 outputs a connecting over-limit flag and the connecting segment has not generated a speed sequence that can be sent, this step does not perform fusion reconstruction and trajectory sending for the connecting segment to avoid introducing uncontrollable breakage or accumulation risks in subsequent execution, and outputs the over-limit alarm information of the corresponding segment.

[0117] The pattern-defined sections and pattern-connecting sections are arranged alternately in chronological order to form a complete trajectory sequence; the fusion order is as follows: ,in Indicates the first Each pattern-defined section, Indicates the first A pattern connection segment that has undergone time redistribution. ;

[0118] Because the duration of the pattern transition section changes, the overall timeline needs to be reconstructed; the duration of the pattern finalization section remains unchanged, while the duration of the pattern transition section is updated to the compensated execution duration calculated in step 3. The total duration of the fusion trajectory equals the sum of the durations of all defined segments plus the sum of the compensated execution durations of all connecting segments; to locate each segment on the new timeline, calculate the cumulative time offset of each segment relative to the original timeline, where the original timeline is the total duration of the master latte art trajectory; the first defined segment The offset of the first segment is zero, and the offset of the subsequent segment is equal to the sum of the duration changes of all the preceding connecting segments.

[0119] Regarding trajectory reconstruction: the trajectory points in the pattern shaping section retain their spatial positions, and only the time labels are updated to match the new time axis; the trajectory of the pattern connecting section is executed according to the output of step 3: when it is determined to be at the permissible level, the original duration is maintained, and the connecting section can directly use the original connecting trajectory point sequence and update the time labels according to the new time axis; when it is determined to be at the compensation level, the connecting section uses the velocity sequence and position sequence generated in step 3, with its trajectory starting from the end position of the previous pattern shaping section and using the position sequence generated by the Euler integral of the velocity vector as the standard; when it is determined to be at the over-limit level and the velocity limit is triggered, if the residual connecting deviation in step 3 meets the residual threshold requirement, the actual execution time and corresponding sequence output in step 3 are still used for fusion; if step 3 outputs a connecting over-limit flag and no velocity sequence that can be distributed is generated, then this step will not perform fusion execution and distribution for this section;

[0120] Regarding trajectory reconstruction: the trajectory points of the pattern shaping section retain their spatial positions, only updating the time labels to match the new time axis; the trajectory of the pattern connecting section is executed according to the output of step 3: when determined to be at the permissible level, the original duration is maintained, and the connecting section can directly use the original connecting trajectory point sequence and update the time labels according to the new time axis; when determined to be at the compensation level, the connecting section uses the velocity sequence and position sequence generated in step 3, with its trajectory starting from the end position of the previous pattern shaping section and using the position sequence generated by the Euler integral of the velocity vector as the reference; when determined to be at the over-limit level and triggering the velocity limit, if the residual connecting deviation in step 3 meets the residual threshold requirement, the actual execution time and corresponding sequence output in step 3 are still used for fusion; if step 3 outputs a connecting over-limit flag and no velocity sequence can be generated, this step will not perform fusion execution and distribution for this section; all trajectory point sequences of the pattern shaping section and the pattern connecting section are merged in the order of the new time axis to form a complete fused trajectory; the data structure of the fused trajectory is a sequence of time, position, and velocity;

[0121] Subsequently, the spout position sequence of the fused trajectory is converted into a robot joint angle sequence, and the inverse kinematics solver is called for each sampling point to calculate the corresponding joint angle to obtain the joint angle sequence. The spout posture is determined based on the direction of movement and the preset tilting angle; its inverse kinematics model is as follows: ,in These are the angle vectors of the six joints. This refers to the position of the spout. The spout posture is determined based on the direction of motion and the preset tilting angle. Before the trajectory is issued, an executability check is performed to check whether the joint angles are within the limit range and whether the joint angular velocity and joint angular acceleration exceed the upper limit. If there is a violation of the constraints, it is determined that the fused trajectory cannot be directly executed under the current constraints, and an execution alarm message or an unexecutable flag is output to avoid the risk of exceeding the limit during robot execution.

[0122] After the test is passed, the fused trajectory is sent to the robot controller. The robot controller performs interpolation motion according to the received trajectory and tracks the target trajectory at a preset control frequency to complete the latte art operation.

[0123] This step ensures the correct connection of the milk foam edge at each shaping segment through the following mechanisms: the pattern shaping segment maintains its original trajectory, and its milk foam edge formation process is consistent with that recorded by the master; the pattern connection segment is executed according to the graded judgment and time redistribution results in step 3, where the allowable level ensures that the connection deviation is within the allowable threshold range, the compensation level eliminates the connection deviation to zero without triggering the amplitude limit, and the over-limit level allows fusion execution when the residual threshold condition is met, and prohibits the generation and distribution of the speed sequence of the segment when it is not met; in addition, the connection between the shaping segment and the connection segment is processed by a smooth window, and there is no edge breakage or accumulation caused by speed abrupt change.

[0124] Please see Figure 3 As shown, the present invention discloses a motion capture-based coffee latte art robot trajectory reproduction control system, including the following modules: a synchronization diagnosis module, used to calculate the planned speed of the spout at each sampling moment on the master spout trajectory data acquired and preprocessed by the motion capture system, calculate the synchronization deviation index and determine the synchronization status in combination with the current milk foam baseline diffusion speed, divide the master latte art trajectory into a set of synchronized segments and a set of desynchronized segments, and generate an overall diagnostic conclusion based on the proportion of synchronized segments;

[0125] The pattern segment decomposition module is used to decompose the master latte art trajectory into a set of pattern shaping segments and a set of pattern connecting segments based on the overall diagnostic conclusion, and to extract the milk foam edge connection constraints between each pattern connecting segment and the adjacent pattern shaping segment.

[0126] The time redistribution module is used to calculate the milk foam edge connection deviation based on the milk foam edge connection constraint for each pattern connection segment, and to obtain the compensated duration by eliminating the milk foam edge connection deviation through time redistribution.

[0127] The trajectory fusion execution module is used to fuse the original trajectory sequence of the pattern shaping section with the trajectory sequence of the pattern connecting section after time redistribution in chronological order, generate a complete fused trajectory, convert it into a robot joint space trajectory, and send it to the controller for execution.

[0128] The above formulas are all dimensionless calculations. The formulas are derived from software simulations based on a large amount of collected data to obtain the most recent real-world results. The preset parameters in the formulas are set by those skilled in the art according to the actual situation.

[0129] The above embodiments can be implemented, in whole or in part, by software, hardware, firmware, or any other combination thereof. When implemented using software, the above embodiments can be implemented, in whole or in part, in the form of a computer program product.

[0130] Those skilled in the art will recognize that the modules and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and inventive constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0131] In addition, the functional modules in the various embodiments of this application can be integrated into one processing module, or each module can exist physically separately, or two or more modules can be integrated into one module.

[0132] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

[0133] In conclusion, the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A motion capture-based trajectory reproduction control method for coffee latte art robots, characterized in that, Includes the following steps: The master latte art trajectory data collected and preprocessed by the motion capture system is acquired, and the planned speed of the spout at each sampling moment is calculated. Combined with the baseline diffusion speed under the current milk foam conditions, the synchronization deviation index is calculated and the synchronization status is determined. The master latte art trajectory is divided into a set of synchronized segments and a set of desynchronized segments. Based on the overall synchronization rate obtained from the synchronization segment's share of the total trajectory duration, an overall diagnostic conclusion is generated. Based on the overall diagnostic conclusion, the master latte art trajectory is decomposed into a set of pattern shaping segments and a set of pattern connecting segments; the milk foam edge connection constraint between each pattern connecting segment and the adjacent pattern shaping segment is extracted. The milk foam edge connection constraint is used to describe the physical constraint of the milk foam edge spreading from the edge position of the previous pattern shaping segment to the edge position of the subsequent pattern shaping segment. For each pattern connection segment, the milk foam edge connection deviation is calculated and graded based on its milk foam edge connection constraint; according to the graded determination result of the milk foam edge connection deviation, the execution time of the pattern connection segment is redistributed, and a compensated spout speed and position sequence is generated. The original trajectory sequence of the pattern shaping section and the trajectory sequence of the pattern connection section after time redistribution are fused in time order to generate a complete fused trajectory, which is then converted into a robot joint space trajectory and sent to the controller for execution.

2. The motion capture-based trajectory reproduction control method for coffee latte art robots according to claim 1, characterized in that, The synchronization deviation index is calculated and the synchronization state is determined, including: dividing the difference between the planned speed of the spout and the expected speed under the reference coupling degree at each sampling time by the expected speed to obtain the synchronization deviation index, wherein the expected speed under the reference coupling degree is the product of the reference coupling degree and the baseline diffusion speed under the current milk foam conditions. Based on the absolute value of the synchronization deviation index, the synchronization status at each sampling moment is determined as synchronized, weakly desynchronized, or strongly desynchronized by setting a first synchronization deviation threshold and a second synchronization deviation threshold. Specifically, when the absolute value of the synchronization deviation index is less than the first synchronization deviation threshold, it is determined to be in a synchronized state; when the absolute value of the synchronization deviation index is greater than or equal to the first synchronization deviation threshold and less than the second synchronization deviation threshold, it is determined to be in a weakly desynchronized state; and when the absolute value of the synchronization deviation index is greater than or equal to the second synchronization deviation threshold, it is determined to be in a strongly desynchronized state.

3. The motion capture-based trajectory reproduction control method for coffee latte art robots according to claim 1, characterized in that, Based on the overall diagnostic conclusion, the master latte art trajectory is decomposed into a set of pattern shaping segments and a set of pattern connecting segments, including: when the overall synchronization rate is greater than the preset first synchronization rate threshold, the diagnostic conclusion is that it can be directly reproduced; when the overall synchronization rate is greater than the preset second synchronization rate threshold but does not exceed the first synchronization rate threshold, the diagnostic conclusion is that local correction is required; when the overall synchronization rate does not exceed the second synchronization rate threshold, the diagnostic conclusion is that full compensation is required. When the diagnosis indicates that local correction is required, all segments in the synchronous segment set will be directly used as pattern shaping segments. When the diagnosis conclusion is that full compensation is required, the set of synchronization segments is screened, and the synchronization segments that meet the following conditions are selected as pattern shaping segments: the absolute value of the mean of the synchronization deviation index is less than the preset mean threshold of stability, the standard deviation of the synchronization deviation index is less than the preset standard deviation threshold of stability, and the segment duration is greater than the preset minimum segment duration threshold. The trajectory portion of the non-pattern-defined section is marked as a pattern-connecting section, forming a set of pattern-connecting sections.

4. The motion capture-based trajectory reproduction control method for coffee latte art robots according to claim 1, characterized in that, The process of extracting the edge connection constraints of the milk foam includes: calculating the edge position of the preceding pattern shaping segment of the pattern connection section, using the following formula: Where 'c' represents the identifier for the pattern connection section, For the first The edge position of the front pattern shaping section of the pattern connection section. For the first The front pattern shaping section mark of each pattern connection section The spout position at the end of the previous pattern shaping section. To create a delay distance at the edge, The unit vector representing the direction of spout movement at the end of the previous pattern shaping section; The formula for calculating the edge position of the subsequent pattern shaping section in the pattern connection zone is as follows: ;in, For the first The edge position of the pattern shaping section after the pattern connection section, The spout position at the beginning of the later pattern shaping section. The unit vector representing the direction of spout movement at the beginning of the pattern-setting section; The formula for calculating the diffusion distance of milk foam at the edge of the pattern connection section is as follows: ;in, For the first The distance of the milk foam edge diffusion in the pattern connection section represents the straight-line distance from the edge of the milk foam edge to the edge of the pattern shaping section.

5. The motion capture-based trajectory reproduction control method for coffee latte art robots according to claim 4, characterized in that, For each pattern connection segment, the milk foam edge connection deviation is calculated based on its milk foam edge connection constraint, including: based on the milk foam edge diffusion distance. The baseline diffusion rate under current milk foam conditions and the original duration of the pattern connection section Calculate the edge connection deviation of milk foam. The calculation formula is as follows: Among them, a positive value for the edge connection deviation of the milk foam indicates that the actual diffusion distance of the milk foam edge is less than the required diffusion distance, while a negative value for the edge connection deviation of the milk foam indicates that the actual diffusion distance of the milk foam edge is greater than the required diffusion distance.

6. The motion capture-based trajectory reproduction control method for coffee latte art robots according to claim 1, characterized in that, Based on the grading results of the milk foam edge connection deviation, the execution time of the pattern connection segment is redistributed, including: setting a connection deviation allowable threshold and a connection deviation severe threshold, and comparing the absolute value of the milk foam edge connection deviation with the threshold. When the absolute value of the edge connection deviation of the milk foam is less than the allowable threshold for connection deviation, it is judged to be in the allowable level, and the original duration of the pattern connection segment remains unchanged. When the absolute value of the edge connection deviation of the milk foam is greater than or equal to the allowable threshold for connection deviation but less than the severe threshold for connection deviation, it is judged as the compensation level; when the absolute value of the edge connection deviation of the milk foam is greater than or equal to the severe threshold for connection deviation, it is judged as the over-limit level. For pattern connection segments that are determined to be at the compensation level or the over-limit level, time reallocation is performed. The original duration of the pattern connection segment is added to the time compensation amount to obtain the compensated execution duration. The time compensation amount is the quotient obtained by dividing the milk foam edge connection deviation by the baseline diffusion speed under the current milk foam conditions.

7. The motion capture-based trajectory reproduction control method for coffee latte art robots according to claim 6, characterized in that, The process of generating the compensated spout speed and position sequence includes: using the straight-line distance between the starting position of the subsequent pattern shaping section and the ending position of the preceding pattern shaping section as the spout displacement distance, and dividing the spout displacement distance by the compensated execution time to obtain the compensated spout speed. The compensated spout speed is limited. When the compensated spout speed is less than the preset minimum safe speed, it is set to the minimum safe speed; when the compensated spout speed is greater than the preset maximum safe speed, it is set to the maximum safe speed. If the speed limit is triggered and the speed limit is exceeded, the actual execution time and the corresponding actual milk foam edge diffusion distance are recalculated based on the spout speed after the speed limit is exceeded. The difference between the actual milk foam edge diffusion distance and the milk foam edge diffusion distance is used as the residual connection deviation. When the absolute value of the residual connection deviation exceeds the preset residual deviation threshold, the connection limit is exceeded and no speed sequence that can be sent is generated. For pattern connection sections that do not trigger the connection over-limit sign, linear interpolation is used to plan the velocity direction, and a velocity sequence is generated by combining the compensated spout velocity. The position sequence is then generated by integration.

8. The motion capture-based trajectory reproduction control method for coffee latte art robots according to claim 1, characterized in that, The original trajectory sequence of the pattern shaping section and the trajectory sequence of the pattern connecting section after time redistribution are fused in chronological order, including: alternating the pattern shaping section and the pattern connecting section in chronological order to construct the time axis of the fused trajectory; For pattern-defined segments, keep the original spatial positions of trajectory points unchanged, and update the corresponding time labels to match the time axis of the fused trajectory. For pattern-connecting segments, their time labels and spatial positions in the fusion trajectory are determined based on the generated position sequence and the compensated execution time. The trajectory point sequences of all pattern-defined sections and pattern-connecting sections are merged according to the new time axis to form a complete fused trajectory that includes time, position, and velocity sequences.

9. The motion capture-based trajectory reproduction control method for coffee latte art robots according to claim 8, characterized in that, Generating a complete fusion trajectory also includes: setting a smoothing window at the junction of the pattern shaping section and the pattern connecting section, performing linear transition processing on the spout speed within the smoothing window, and regenerating the position sequence within the smoothing window through integration to ensure the speed continuity at the trajectory fusion point.

10. A motion capture-based trajectory reproduction control system for a coffee latte art robot, used to implement the motion capture-based trajectory reproduction control method for a coffee latte art robot as described in any one of claims 1-9, characterized in that, It includes the following modules: Synchronization Diagnosis Module, which is used to calculate the planned speed of the spout at each sampling moment based on the master spout trajectory data acquired and preprocessed through the motion capture system, calculate the synchronization deviation index and determine the synchronization status in combination with the current milk foam baseline diffusion speed, divide the master latte art trajectory into a set of synchronized segments and a set of desynchronized segments, and generate an overall diagnostic conclusion based on the proportion of synchronized segments. The pattern segment decomposition module is used to decompose the master latte art trajectory into a set of pattern shaping segments and a set of pattern connecting segments based on the overall diagnostic conclusion, and to extract the milk foam edge connection constraints between each pattern connecting segment and the adjacent pattern shaping segment. The time redistribution module is used to calculate the milk foam edge connection deviation based on the milk foam edge connection constraint for each pattern connection segment, and to obtain the compensated duration by eliminating the milk foam edge connection deviation through time redistribution. The trajectory fusion execution module is used to fuse the original trajectory sequence of the pattern shaping section with the trajectory sequence of the pattern connecting section after time redistribution in chronological order, generate a complete fused trajectory, convert it into a robot joint space trajectory, and send it to the controller for execution.

Images