Automatic matching method of coffee latte art pattern based on user order

Abstract

The application discloses a coffee latte art pattern automatic matching method based on a user order, relates to the technical field of data processing, and is used for solving the problem of how to determine whether a candidate template has sufficient state switching space between adjacent patterns in combination with actual milk output response of equipment before latte art is executed and obtaining an executable template without destroying the integrity of a saved executable track; the method acquires an order pattern and determines a candidate latte art template, establishes a corresponding relationship between the order pattern object and a template pattern part, and distinguishes a main pattern and a decorative pattern; a mechanical arm moving path through a coffee base color area between adjacent template pattern parts is determined, a stop milk extension distance and a re-milk recovery distance are obtained according to front and back pattern execution parameter calling calibration records, corresponding positions are determined along the same path, and it is judged whether adjacent patterns can be kept separated; when the adjacent patterns cannot be kept separated, the corresponding decorative pattern is adjusted, the candidate template is re-matched, and re-judgment is performed until a final latte art template is determined.

Description

Technical Field

[0001] This invention relates to the field of data processing technology, and more specifically, to a method for automatically matching coffee latte art patterns based on user orders. Background Technology

[0002] With the development of automated coffee-making equipment, latte art is increasingly being created automatically using robotic arms in conjunction with milk dispensing mechanisms. Existing automated latte art equipment typically has a pre-built library of latte art patterns or templates, and saves corresponding robotic arm motion data and milk dispensing control data for different patterns. Upon receiving a user order, based on the user's selected basic latte art type, cup type, or order pattern created through scaling, rotation, movement, and adding new patterns, the system selects the appropriate latte art template from the pre-built library and then completes the latte art creation according to the corresponding robotic arm motion data and milk dispensing control data. For equipment that supports personalized patterns, a closer latte art template can be selected based on the image matching results between the order pattern and the template pattern.

[0003] However, when an order contains multiple separate milk patterns, a latte art template determined solely by the shape or degree of overlap of the patterns cannot accurately reflect the separation conditions of the patterns during the actual milk dispensing process. After completing one pattern, the robotic arm typically needs to stop dispensing milk and move to the next pattern position. Even after stopping dispensing, milk may continue to be discharged, and the milk lines require a certain amount of movement to stabilize after dispensing resumes. When there is insufficient space between adjacent patterns for this milk dispensing transition, the resulting milk traces may encroach on the coffee background area originally used to separate adjacent patterns, causing the originally separate patterns in the order to appear connected or have unclear boundaries. Therefore, determining an executable latte art template that maintains the separation of adjacent patterns while preserving the main pattern content of the order, based on the actual milk dispensing status of the current equipment, before the actual production process, is a technical problem that needs further resolution in automated coffee latte art. Summary of the Invention

[0004] In order to overcome the above-mentioned defects of the prior art, embodiments of the present invention provide an automatic coffee latte art pattern matching method based on user orders to solve the problems mentioned in the background art.

[0005] To achieve the above objectives, the present invention provides the following technical solution: The method for automatically matching coffee latte art patterns based on user orders includes the following steps: Obtain the order pattern, determine the candidate latte art template, establish the correspondence between the order pattern object and the template pattern part, and distinguish between the main pattern and the decorative pattern; The path length of the robotic arm through the coffee background area between adjacent template pattern parts is determined according to the production sequence. The calibration record is called based on the robotic arm movement speed and milk output per unit time before the previous template pattern part stops milking and when the next template pattern part starts milking, to obtain the milking stop extension distance and the milking restart recovery distance. The first position is determined by the milk stop extension distance along the forward direction of the robotic arm movement path, and the second position is determined by the milk recovery distance along the reverse direction of the same robotic arm movement path. When the second position is after the first position, and the length of the retainable path between the first and second positions within the coffee background area is not less than the discrete length of the cup surface space, it is determined that the adjacent template pattern parts can remain separated; otherwise, it is determined that they cannot remain separated. When separation cannot be maintained and the corresponding order pattern object contains decorative patterns, the corresponding decorative patterns are removed to form a copy of the order pattern. Candidate latte art templates are then re-matched and separation is re-judged until the final latte art template is determined.

[0006] In a preferred embodiment, an order pattern is obtained, candidate latte art templates are determined, a correspondence is established between the order pattern object and the template pattern portion, and the main pattern and decorative pattern are distinguished. The path length of the robotic arm through the coffee background area between adjacent template pattern parts is determined according to the production sequence. The calibration record is called based on the robotic arm movement speed and milk output per unit time before the previous template pattern part stops milking and when the next template pattern part starts milking, to obtain the milking stop extension distance and the milking restart recovery distance. The first position is determined by the milk stop extension distance along the forward direction of the robotic arm movement path, and the second position is determined by the milk recovery distance along the reverse direction of the same robotic arm movement path. When the second position is after the first position, and the length of the retainable path between the first and second positions within the coffee background area is not less than the discrete length of the cup surface space, it is determined that the adjacent template pattern parts can remain separated; otherwise, it is determined that they cannot remain separated. When separation cannot be maintained and the corresponding order pattern object contains decorative patterns, the corresponding decorative patterns are removed to form a copy of the order pattern. Candidate latte art templates are then re-matched and separation is re-judged until the final latte art template is determined.

[0007] In a preferred embodiment, the milk area, coffee background area and robotic arm position sequence in the candidate template are unified to the actual coordinate system of the cup surface according to the calibration relationship between the pixel coordinates of the cup surface image and the actual coordinates of the cup surface. The cup surface image is resampled at equal intervals according to the actual coordinates of the cup surface, and the actual distance between the centers of adjacent resampled points is determined as the discrete length of the cup surface space. The path length is obtained by taking the position where the robotic arm first leaves the previous template pattern as the starting point and the position where it first enters the next template pattern as the ending point. The length of the trajectory within the coffee-colored area in the robotic arm position sequence is accumulated.

[0008] In a preferred embodiment, determining the distance for extending the milk cessation period includes: Under the same robotic arm movement speed and milk output per unit time, multiple milk stop distance samples were obtained according to a preset calibration number of times. The position of the robotic arm when the stop milk dispensing command is issued is taken as the stop milk dispensing position, and the furthest position that still maintains continuity with the milk area formed before the stop milk dispensing along the direction of robotic arm movement is taken as the last continuous milk area position. The actual distance moved between the milk stop position and the last continuous milk position is used as the milk stop distance sample. The maximum distance among multiple milk stoppage distance samples is obtained, and then rounded upwards using the discrete length of the cup surface space as the unit to obtain the milk stoppage extension distance.

[0009] In a preferred embodiment, determining the re-milk production recovery distance includes: Under the same robotic arm moving speed and milk output per unit time, obtain multiple milk line widths for a stable milk output reference segment, and determine the stable width range based on the multiple milk line widths; Starting from the point where milk is re-discharged, continuously obtain the width of the milk line along the re-discharge trajectory, and determine the interval where the width of the milk line is continuously within a stable width range from the end of the trajectory in reverse. The position within this interval closest to the point where milk production resumes is determined as the stable recovery position. The actual distance traveled between the re-discharge position and the restored stable position is used as the re-discharge distance sample. The maximum distance among multiple re-discharge distance samples is obtained and normalized to obtain the re-discharge recovery distance.

[0010] In a preferred embodiment, the end position of the previous template pattern portion is used as the starting point of the cumulative distance of the robotic arm's movement path, and the cumulative distance corresponding to the milk stop extension distance is used as the first cumulative distance corresponding to the first position. The second position is determined by accumulating the distance of milk discharge recovery along the robotic arm's movement path from the starting position of the next template pattern portion, and then converting it into a second accumulated distance based on the starting point. When the second cumulative distance is greater than the first cumulative distance, the difference between the two is determined as the length of the path that can be retained. Otherwise, the length of the path that can be retained will be set to zero.

[0011] In a preferred embodiment, the pattern object derived from the user-defined basic lace template and formed through order editing is determined as the main pattern; After determining the basic latte art template, add new ones, and define the decorative pattern as an independent pattern object that is separated from the main pattern by a coffee background area with a width not less than the discrete length of the cup surface space. Determine the overlapping area between the template pattern and the order pattern object, and identify the order pattern object with the largest overlapping area as the corresponding object.

[0012] In a preferred embodiment, when the template pattern portion that cannot be kept separate does not have a corresponding order pattern object, the order pattern object removal is not performed, and the next candidate lace template is checked. When adjacent template pattern parts cannot remain separated, the corresponding order pattern object is determined according to the correspondence relationship; When both order pattern objects are primary patterns, keep the order pattern copies unchanged; When only one order pattern object is a decorative pattern, remove the corresponding decorative pattern to create a copy of the order pattern. When both order pattern objects are decorative patterns, two order pattern copies are created, and the decorative patterns with different production orders are removed from each copy.

[0013] In a preferred embodiment, the candidate latte art templates are rematched to the generated order pattern copies, the production order and robotic arm position sequence saved by the rematched candidate latte art templates are read, and the first position, second position and retainable path length corresponding to the adjacent template pattern parts are re-determined. When multiple candidate latte art templates are obtained by rematching the order pattern copy, the candidate latte art template that appears first in the order and has a complete production sequence, robotic arm position sequence and corresponding calibration record will be used as the rematch template. Obtain the retainable path lengths for each adjacent template pattern portion in the rematch template, and use the minimum value among them as the comparison path length for the order pattern copy; Determine the comparison path lengths for the two order pattern copies respectively, and retain the order pattern copy with the larger comparison path length.

[0014] In a preferred embodiment, when the comparison path lengths corresponding to two order pattern copies are the same, the actual area of ​​the decorative pattern removed from the two order pattern copies is determined according to the actual coordinates of the cup surface, and the order pattern copy with the smaller actual area of ​​the decorative pattern removed is retained. When the order pattern copy cannot be rematched with a candidate latte art template that has a complete production sequence, robotic arm position sequence and corresponding calibration record, the corresponding order pattern copy adjustment process ends. When all adjacent template patterns in the rematched candidate latte art templates can remain separated, the corresponding candidate latte art template is determined as the final latte art template.

[0015] The technical effects and advantages of this invention's automatic coffee latte art pattern matching method based on user orders are as follows: This invention associates the actual robotic arm movement path between adjacent patterns in candidate latte art templates with the milk stop extension distance and re-dispensing recovery distance corresponding to the current execution parameters. It determines the positional relationship between the previous and subsequent milk dispensing states along the same robotic arm movement path and judges whether adjacent patterns can remain separated during actual production based on the coffee background path that can be retained between the two positions. This ensures that template selection considers not only the visual matching degree of the order patterns but also the space occupied by the actual milk dispensing process of the current equipment. When adjacent patterns cannot remain separated, only the decorative patterns that can be removed are adjusted according to the correspondence between the order pattern object and the template pattern part. Then, the adjusted order pattern is used to rematch a template with a pre-stored production order and robotic arm position sequence, avoiding direct splicing or reconstruction of the original template trajectory. This improves the adaptability and execution reliability of the final latte art template to the actual milk dispensing process while maintaining the main pattern content. Attached Figure Description

[0016] Figure 1 This is a flowchart illustrating the automatic coffee latte art pattern matching method based on user orders according to the present invention. Figure 2 This is a schematic diagram showing the correspondence between the order pattern and the template pattern, as well as the division of the main pattern and decorative pattern in this invention; Figure 3 This is a schematic diagram showing the movement path of the robotic arm between adjacent template pattern portions of the present invention and the path length of the robotic arm passing through the coffee-colored background area. Figure 4 This is a schematic diagram illustrating the spatial formation of the milk stoppage extension distance and the milk re-production recovery distance according to the present invention; Figure 5 This is a schematic diagram illustrating the adjacent pattern separation judgment and order pattern copy re-matching of the present invention. Detailed Implementation

[0017] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0018] Example Please see Figure 1 This embodiment provides an automatic coffee latte art pattern matching method based on user orders, which is applied to an automatic coffee latte art device including an order processing terminal, a latte art template library, a robotic arm, a milk dispensing mechanism, an image acquisition device, and a processor.

[0019] The order processing terminal receives the user's selected cup type, basic latte art type, and pattern editing results. The latte art template library stores the latte art templates that the current automatic coffee latte art equipment can actually execute. Each latte art template is associated with at least the template image, template pattern parts, the production order of each template pattern part, the robotic arm position sequence, the robotic arm movement speed, and the milk output per unit time. In some embodiments, the cup type, the working height of the milk outlet relative to the cup surface, the milk dispensing mechanism status, and the calibration record number can also be associated with the template to retrieve the calibration record corresponding to the template execution conditions during formal order processing.

[0020] In this embodiment, the template pattern represents the continuous milk area formed during a single continuous milk dispensing process. When milk dispensing stops during the process, and the robotic arm moves to another position and resumes dispensing, the continuous milk area formed before dispensing stopped and the continuous milk area formed after dispensing resumes are respectively used as different template pattern portions. The production order refers to the order in which the robotic arm actually forms each template pattern portion; the complete production order in this application means that all template pattern portions in the candidate latte art template have a corresponding formation order record, and the sequential relationship between any adjacent template pattern portions can be determined based on this order; when a template pattern portion lacks a corresponding production order record, the template is not considered to have a complete production order.

[0021] The robotic arm position sequence represents the sequence of positions projected from the robotic arm's milk outlet onto the actual coordinate system of the cup surface, arranged according to execution time. During the template creation phase, multiple non-collinear calibration points with known actual positions can be set on the coffee cup's working plane. The robotic arm's milk outlet is controlled to sequentially reach the corresponding calibration points, obtaining paired position data between the robotic arm's control coordinates and the actual coordinates of the cup surface, and establishing a coordinate transformation relationship from the robotic arm's control coordinates to the actual coordinates of the cup surface. Before saving, the robotic arm position sequence corresponding to the template is transformed to the actual coordinate system of the cup surface according to the above coordinate transformation relationship, ensuring that the robotic arm position sequence and the pattern area in the template image use the same spatial coordinate reference.

[0022] It should be noted that the robotic arm movement path in this application refers to the complete motion trajectory formed by connecting consecutive position points in the order of execution time during the robotic arm's production process according to the candidate latte art template. This includes the entire movement trajectory of the robotic arm through the milk area and the coffee background area. The coffee background path length L represents the cumulative length of the trajectory segments within the coffee background area of ​​the aforementioned robotic arm movement path, used to characterize the actual available space length between adjacent template pattern portions for milk dispensing state transitions.

[0023] The robotic arm's movement speed can be directly read from the speed control data saved during template execution. When the template only saves discrete robotic arm positions and corresponding times, the robotic arm's movement speed can be determined based on the actual distance between two adjacent robotic arm positions and the corresponding time interval.

[0024] Milk output per unit time represents the mass of milk discharged by the milking mechanism per unit time. During the equipment parameter establishment phase, the milk discharged by the milking mechanism can be collected in an electronic weighing device. The increase in milk mass over a known duration is obtained, and the milk output per unit time is determined based on the increase in milk mass and the corresponding duration. Records of the milk output per unit time are established for different milking control states.

[0025] Before the equipment is put into use, the cup surface coordinates of the image acquisition device are calibrated. Multiple calibration points with known actual positions and non-collinearity are set on a plane corresponding to the working height of the coffee liquid surface. A planar mapping relationship is established based on the image pixel coordinates of the calibration points and the actual coordinates of the cup surface, and perspective correction is performed on the cup surface image using the planar mapping relationship.

[0026] For the perspective-corrected cup surface image, an equally spaced resampling grid is established according to the actual coordinates of the cup surface, so that the centers of adjacent resampling points correspond to the same actual distance. This actual distance is denoted as the discrete length δ of the cup surface space. The discrete length δ of the cup surface space is directly determined by the sampling step size of the resampling grid, and is used to unify the image region boundary position, the robotic arm trajectory distance, the milk stop extension distance, and the milk re-dispensing recovery distance to the same discrete unit of actual length.

[0027] When dealing with distance normalization, the actual distance to be normalized is divided by the discrete length δ of the cup surface space, rounded up, and then multiplied by δ. When dealing with whether to retain the coffee background path between two positions, the criterion is whether the actual path length between the two positions in the coffee background area is not less than δ.

[0028] Therefore, the discrete length δ of the cup surface space represents the unified discrete calculation step size in the actual coordinate system of the cup surface. Its value is obtained directly from the resampling process and is not determined by the theoretical limit resolution of the image acquisition device.

[0029] For example, in some embodiments, after perspective correction and resampling of the actual coordinates of the cup surface, the center of adjacent pixels corresponds to an actual cup surface length of 0.10 mm, and the discrete length δ of the cup surface space is 0.10 mm. 0.10 mm is only used to illustrate the method of parameter acquisition and does not limit other devices to using the same value.

[0030] Step S1: Obtain the order pattern, determine the candidate latte art template, and establish the correspondence between the order pattern object and the template pattern part; After a user submits an order, the processor reads the final confirmed order pattern. For orders created by the user based on the system's provided basic latte art template, the processor also reads the basic latte art template number and the pattern editing record. The pattern editing record can include new pattern entries, scaling ratio, rotation angle, and translation position.

[0031] Before performing template matching on the order pattern, first check whether the latte art template has complete actual execution data and corresponding milk output calibration records.

[0032] Specifically, the system reads the robotic arm movement speed and milk output per unit time used before milk dispensing stopped in each latte art template, and the robotic arm movement speed and milk output per unit time used when the corresponding template pattern begins to form after milk dispensing resumes. It then checks whether the corresponding parameter combinations have calibration records for the milk dispensing extension distance and the milk dispensing recovery distance. If all corresponding parameter combinations have calibration records, and the template has a complete production sequence and robotic arm position sequence, the latte art template is retained as a template that the current equipment can execute and participate in order pattern matching. If there are parameter combinations that have not been calibrated or a complete robotic arm position sequence is missing, the template is not considered as a candidate latte art template for the current order until the corresponding data is supplemented.

[0033] Therefore, subsequent candidate latte art templates are not selected from all templates based solely on pattern similarity, but rather from latte art templates for which the current equipment already has actual execution data and corresponding milk output calibration records.

[0034] Subsequently, the order pattern and the latte art template image were aligned to the actual coordinate system of the cup surface. The milk pattern in the order pattern and the milk pattern in the template image were used as the foreground area, and the coffee background was used as the background area.

[0035] In some embodiments, the overall pattern overlap between the order pattern and the template image is determined using the intersection-over-union ratio (IoU). Let the foreground region of the order pattern be A, and the foreground region of the template image be B, then the pattern overlap ratio R can be expressed as: R = |A∩B| / |A∪B|; where |A∩B| represents the combined area of ​​the foreground region of the order pattern and the foreground region of the template image, and |A∪B| represents the total area after merging the two foreground regions.

[0036] For all executable latte art templates of the same basic latte art type, determine the degree of pattern overlap and arrange them from highest to lowest overlap. When the degree of pattern overlap is the same, they can be arranged according to the fixed template numbers pre-saved in the template library, so that the same order patterns can repeatedly obtain the same candidate template arrangement results.

[0037] For example, a user selects a basic heart-shaped latte art template and adds two dots to the right of the heart. The template library contains multiple heart-shaped latte art templates, and the dot positions, production order, or robotic arm movement paths can differ between templates. After unifying the order pattern and the corresponding heart-shaped latte art template to the actual coordinate system of the cup surface, the degree of pattern overlap is determined, and a candidate latte art template arrangement order is formed.

[0038] After determining the candidate latte art templates, establish the correspondence between the order pattern object and the template pattern part in the candidate latte art template.

[0039] For orders created through editing operations, the order processing terminal saves the object number of the corresponding order pattern object when the user adds, moves, or changes the pattern. Therefore, it can obtain the order pattern object formed by the basic template and each newly added order pattern object.

[0040] After unifying the order pattern objects and template pattern parts to the actual coordinate system of the cup surface, the overlapping area between each order pattern object and each template pattern part is determined. For a given template pattern part, the order pattern object with the largest overlapping area is taken as the corresponding order pattern object.

[0041] When two order pattern objects have the same overlapping area with the same template pattern part, the actual distance from the center of the template pattern part to the center of the two order pattern objects can be further determined, and the order pattern object with the smaller center distance can be taken as the corresponding object.

[0042] The correspondence records both the template pattern part number and the order pattern object number.

[0043] An order pattern object can correspond to one or more template pattern parts. For example, if a dot pattern added by the user requires two independent milking processes during the actual production of the candidate template, then it can be associated with the same order pattern object by two template pattern parts. When the order pattern object is subsequently removed from the order pattern copy, it will no longer be included as part of the order pattern in the template matching process during the next round of establishing the correspondence.

[0044] like Figure 2 As shown, the main pattern P0 and decorative patterns P1 and P2 in the order pattern can correspond to one or more template pattern parts in the candidate lace template.

[0045] When a template pattern portion has no overlapping area with any of the order pattern objects, it can be recorded as having no corresponding order pattern object. The template pattern portion without a corresponding order pattern object still participates in the adjacent pattern separation judgment of the candidate templates; however, when the inability to maintain the adjacent relationship only involves the template pattern portion without a corresponding order pattern object, the order pattern object is not deleted based on this, and the next candidate lace template is checked.

[0046] Step S1 ultimately obtains the order pattern, order pattern object, candidate latte art template arrangement result, template pattern part, correspondence between order pattern object and template pattern part, production order, robotic arm position sequence, robotic arm moving speed, milk output per unit time, corresponding calibration record, and cup surface space discrete length δ, and inputs the above results into step S2.

[0047] Step S2: Distinguish between the main pattern and the decorative pattern, and determine the robotic arm movement path between adjacent template pattern parts; Step S2 calls the order pattern, basic latte art template number, pattern editing record, correspondence between order pattern object and template pattern part, and current candidate latte art template obtained in step S1.

[0048] In this embodiment, the main pattern represents the pattern content that originates from the basic latte art template selected by the user and is continuously retained during the subsequent template rematching process; the decorative pattern represents the pattern content that is added by the user after the basic latte art template is determined, and that can form an independent separation from the main pattern through the coffee background area.

[0049] The main pattern and decorative pattern are determined according to the source of the pattern object in the order and the spatial relationship between the patterns, not according to the size of the pattern area or the visual prominence. For new pattern objects and main patterns that do not overlap, the shortest actual distance between the boundary of the new pattern object and the boundary of the main pattern is determined as the coffee background color separation width between the two. When the coffee background color separation width is not less than the discrete length δ of the cup surface space, it is determined that there is a coffee background color area between the two that meets the independent separation condition.

[0050] Specifically, the original pattern of the basic pattern template is read according to the basic pattern template number, and the basic pattern template is transformed to the final position in the order according to the scaling ratio, rotation angle, and translation position in the order editing record. The pattern area that still originates from the basic pattern template after transformation is determined as the main pattern.

[0051] For new order pattern objects added after the basic lace template is determined, determine whether the new pattern object overlaps with the main pattern.

[0052] When a newly added pattern object does not overlap with the main pattern, and there is a coffee-colored background area between them with a width not less than the discrete length δ of the cup surface space, the newly added pattern object is determined as a decorative pattern.

[0053] When a newly added pattern object overlaps with the main pattern, connected component analysis is performed on the area of ​​the newly added pattern object outside the range of the main pattern. Parts located outside the range of the main pattern but not separated from it by a coffee-colored background area with a width not less than the discrete length δ of the cup surface space are incorporated into the main pattern and are not treated as independent deletion objects; only independent areas located outside the range of the main pattern and separated from it by a coffee-colored background area with a width not less than the discrete length δ of the cup surface space are identified as decorative patterns.

[0054] In some embodiments, users directly upload a complete latte art pattern instead of starting from a basic latte art template. In this case, the basic latte art template with the highest degree of overlap with the uploaded pattern can be determined based on the pattern overlap obtained in step S1, and the basic latte art template and the uploaded pattern are unified to the actual coordinate system of the cup surface. For each continuous pattern area in the basic latte art template, the uploaded pattern area with the largest overlapping area is identified to establish a correspondence. The area corresponding to the basic latte art template is used as the main pattern, and the remaining independent areas separated from the main pattern by a coffee background area with a width not less than the discrete length δ of the cup surface space are used as decorative patterns.

[0055] If the uploaded pattern cannot establish a sufficient correspondence with any basic lace template to determine the main pattern, then only the candidate lace template matching and spatial separation judgment of the adjacent template pattern parts are performed, and the adjustment process that requires the deletion of decorative patterns is not performed.

[0056] After determining the main pattern and decorative pattern, obtain the adjacent template pattern parts in the order that the candidate latte art templates are saved. Adjacent means that there are no other template pattern parts between the two template pattern parts in the production order.

[0057] The image used to determine the milk and coffee background areas is a template image associated with the current candidate latte art template, not the result image collected after the order is actually made.

[0058] If the template library already contains binary masks for the milk region and the coffee background region corresponding to the template image, the corresponding regions can be read directly. If the template library does not contain binary masks, the corresponding regions can be obtained using a fixed image processing procedure.

[0059] For example, the perspective-corrected template image can be converted to the CIELAB color space, the L* brightness channel can be extracted, the Otsu method can be used to determine the binary segmentation threshold of the current template image, the region with higher brightness can be used as the milk candidate region, and isolated connected regions with an actual area smaller than the actual area corresponding to one resampling grid cell can be removed as noise, thereby obtaining the milk region and the coffee background region.

[0060] Then, the robot arm position sequence saved in the current candidate latte art template is read. The position where the robot arm's movement path first leaves the boundary of the previous template pattern is determined as the end position of the previous template pattern. The position where the robot arm's movement path first enters the boundary of the next template pattern is determined as the start position of the next template pattern.

[0061] like Figure 3 As shown, the candidate latte art templates form a heart P0 first, and then a dot P1, according to the production order. The robotic arm moves from the end position of P0 to the beginning position of P1 along the robotic arm position sequence saved in the template.

[0062] All positions in the robotic arm's trajectory are located in the actual coordinate system of the cup surface. For two adjacent trajectory points Pi = (xi, yi) and Pi+1 = (xi+1, yi+1), the actual length li of the corresponding trajectory segment is determined according to the planar Euclidean distance: li = √[(xi+1 - xi)² + (yi+1 - yi)²]; The robotic arm sequentially checks the trajectory segments between the end position of the previous template pattern and the beginning position of the next template pattern, accumulating only the trajectory length within the coffee-colored area. When a trajectory segment crosses the boundary between the milk area and the coffee-colored area, the trajectory segment is split based on the intersection point with the area boundary, accumulating only the actual length on the coffee-colored side.

[0063] The accumulated actual length is recorded as the coffee background path length L between the current adjacent template pattern parts.

[0064] For example, the shortest straight-line distance between the heart P0 and the dot P1 outlines in the template image can be 6.0 mm, but according to the actual robotic arm position sequence in the template, the path length from the actual end position of the heart to the actual start position of the dot through the coffee-colored area can be 4.4 mm. Subsequent adjacent pattern separation judgments use the 4.4 mm coffee-colored path instead of the 6.0 mm shortest geometric distance between the patterns.

[0065] Step S2 finally obtains the main pattern, decorative pattern, adjacent template pattern parts in each candidate latte art template, the end position of the previous template pattern part, the start position of the next template pattern part, the movement trajectory of the robotic arm through the coffee background area between the two, and the corresponding coffee background path length L, and inputs the above results into step S3.

[0066] Step S3: Determine whether adjacent template pattern portions can remain separated based on the distance of milk stoppage extension and the distance of milk resumption. It should be noted that, in this embodiment, maintaining separation means that, after considering the milk extension after the previous template pattern stops dispensing milk and the recovery process before the next template pattern starts dispensing milk, there is still a coffee background area with a space discrete length δ of not less than the cup surface, thereby preventing the connection between two continuously formed milk areas.

[0067] like Figure 4 As shown, the milk stop extension distance Ds represents the actual distance the robotic arm moves from the milk stop position to the last continuous milk position after the milk stop command is issued; the milk restart recovery distance Dr represents the actual distance the robotic arm moves from the milk restart position to the position corresponding to the stable width of the milk line after the milk start command is issued again.

[0068] Both of the above distances are established in advance during the equipment deployment phase, after the milk dispensing mechanism is replaced, or during the equipment recalibration phase. When processing user orders, the corresponding calibration records are directly called.

[0069] Each set of calibration records for the milk dispensing extension distance and the milk dispensing recovery distance is associated with an execution parameter combination identifier and a calibration condition identifier. The execution parameter combination includes at least the robotic arm movement speed and the milk dispensing rate per unit time. The execution parameter combination identifier is used to uniquely correspond to a set of robotic arm movement speed and milk dispensing rate per unit time, and is not limited to using a numerical number, string number, or database index. The calibration condition identifier is used to distinguish the equipment execution conditions that affect the milk dispensing response. In some embodiments, the calibration condition identifier may include one or more of the following: cup type, working height of the milk outlet relative to the cup surface, and milk dispensing mechanism status.

[0070] When establishing calibration records, the execution parameter combinations, calibration conditions, and the corresponding distances for stopping milk production and resuming milk production are associated and saved. When processing user orders, the robot arm's movement speed and milk output per unit time before the previous template pattern stopped producing milk, and the robot arm's movement speed and milk output per unit time when the next template pattern began to form, are read from the candidate latte art templates. Calibration records matching the execution parameter combinations are then searched. If a calibration record is also associated with a calibration condition identifier, the execution conditions of the current template are further compared with the corresponding calibration conditions. If they match, the corresponding calibration record is invoked; otherwise, it is not invoked, and the current template is excluded from the candidate templates under the corresponding execution conditions.

[0071] When the template pattern uses a constant robotic arm movement speed and milk output per unit time, the calibration record is directly called according to the corresponding combination of execution parameters. When the robotic arm trajectory uses segmented speed or segmented milk output control, the milk stop extension distance is called according to the robotic arm movement speed and milk output per unit time corresponding to the last continuously executed parameter segment before the milk stop command is issued, and the milk restart recovery distance is called according to the robotic arm movement speed and milk output per unit time corresponding to the first continuously executed parameter segment used to form the next template pattern after milk restart.

[0072] The logic for determining the extended distance of the milk cessation is as follows: Under the calibrated robotic arm movement speed and milk output per unit time, the robotic arm first forms a continuous and stable straight milk area. A stop milk dispensing command is sent to the milk dispensing mechanism at the pre-recorded cup position, while the robotic arm continues to move in the current direction and speed.

[0073] The position where milk dispensing stops is determined by projecting the robotic arm's milk outlet onto the actual coordinate system of the cup surface when the stop milk dispensing command is issued.

[0074] The image acquisition device continuously acquires images of the cup surface during the calibration process and obtains the milk area. Starting from the position where milk dispensing stops, along the direction of the robotic arm's movement, the farthest position that remains connected to the continuous milk area formed before milk dispensing stopped is found, and this position is determined as the last continuous milk area. In some embodiments, by performing connectivity analysis on the milk areas in the continuously acquired cup surface images, areas with continuous pixel connections to the continuous milk area formed before milk dispensing stopped are determined as continuous milk areas. Isolated milk droplets that are only spatially adjacent to the aforementioned areas but have no connection relationship are not used as the basis for determining the last continuous milk area. Isolated milk droplets that appear after the last continuous milk droplet position but are not connected to the previous milk droplet area are not included in the distance of milk stoppage extension.

[0075] The distance traveled from the point where milk dispensing stops to the last continuous milk position along the actual position sequence of the robotic arm is accumulated to form a milk stopping distance sample.

[0076] Under the same combination of execution parameters and the same calibration conditions, the milk-stopping calibration process is repeated N times according to a preset calibration number. N is an integer greater than 1 set by the control program before the calibration task begins and remains unchanged during one calibration process with the current combination of execution parameters. In some embodiments, N can be determined based on the current equipment calibration accuracy requirements. When it is necessary to improve the stability of the calibration results, the number of repeated calibrations can be increased; when the equipment state is stable and the repetition error is small, the number of repeated calibrations can be reduced. In this embodiment, N is only used to determine the maximum effective sample range and is not limited to a specific value.

[0077] Each time the milk dispensing is calibrated, the position of the robotic arm's milk outlet projected onto the cup surface in the actual coordinate system when the milk dispensing stop command is issued is taken as the milk dispensing stop position. Starting from the milk dispensing stop position, the farthest position that is still connected to the continuous milk area before the milk dispensing stops is found along the robotic arm's movement direction, and this position is determined as the last continuous milk area. The actual movement distance between the milk dispensing stop position and the last continuous milk area is accumulated along the actual position sequence of the robotic arm to form the milk dispensing stop distance sample di.

[0078] When there are missing frames, missing corresponding robotic arm position sequences, or inability to determine continuous milk areas during the image acquisition process, the current record is marked as invalid, and the calibration is re-executed for the corresponding number of times, so that the current calibration task finally obtains N milk stop distance samples; the milk stop distance sample represents the original measurement distance obtained in a single calibration, and the milk stop extension distance Ds represents the distance parameter determined based on multiple milk stop distance samples and used for template judgment; The maximum milk stop distance dmax is obtained from N milk stop distance samples and rounded upwards in units of discrete cup surface length δ: Ds=ceil(dmax / δ)×δ; where ceil() represents the rounding function; the obtained Ds is used as the milk stop extension distance corresponding to the current execution parameter combination and the current calibration conditions, and a corresponding calibration record is established.

[0079] In this embodiment, the stable milk dispensing reference segment is the continuous trajectory segment between the start and stop milk dispensing transition segments when the robotic arm continuously forms milk lines according to the current combination of execution parameters. Within this trajectory segment, the robotic arm's moving speed and milk dispensing volume per unit time remain constant, and the line width sampling section is obtained according to the discrete length δ of the cup surface space. The stable milk dispensing reference segment is only used to obtain the stable width range corresponding to the current combination of execution parameters and does not participate in the actual distance accumulation for the milk dispensing recovery distance.

[0080] The logic for determining the distance to resume milk production is as follows: Under the current combination of execution parameters, the milk dispensing mechanism is made to dispense milk stably. Multiple sampling sections perpendicular to the direction of the robotic arm movement are obtained along the direction of the robotic arm movement according to the discrete length δ of the cup surface space. The length of the part where each sampling section intersects with the milk area in the actual coordinate system of the cup surface is taken as the width of the milk line.

[0081] The linewidth samples obtained from the stable milk discharge reference segment are arranged in ascending order to determine the first quartile Q1 and the third quartile Q3. Abnormal linewidths less than Q1 - 1.5 × IQR or greater than Q3 + 1.5 × IQR are identified according to IQR = Q3 - Q1. The minimum and maximum values ​​of the remaining linewidths, Wmin and Wmax, are combined to form a stable width range [Wmin, Wmax]. This stable width range represents the range of linewidth changes corresponding to the normal continuous milk discharge state under the current combination of execution parameters, and is used to determine whether the milk lines have returned to a stable state after re-discharge.

[0082] The robotic arm is then moved from the stopped milk dispensing state to the re-dispensing position, a milk dispensing start command is issued, and it continues to move along the preset re-dispensing calibration trajectory to the end of the calibration trajectory; the width of the milk line is continuously obtained along the corresponding trajectory according to the discrete length δ of the cup surface space.

[0083] Starting from the endpoint of the calibration trajectory, the width of the milk line at each sampling section is checked in reverse along the direction of the robotic arm's movement. A continuous interval is determined forward from the trajectory endpoint where the line width falls within the stable width range [Wmin, Wmax]. The sampling position closest to the re-dispensing position within this continuous interval is then identified as the restored stable position. If, during the re-dispensing calibration process, there are missing image acquisition frames, missing robotic arm position sequences, the milk line width corresponding to the calibration trajectory endpoint not falling within the stable width range, or a continuous interval within the stable width range cannot be obtained from the calibration trajectory endpoint, this record is marked as invalid, and the re-dispensing calibration is re-executed the corresponding number of times until N re-dispensing distance samples are obtained. Each re-dispensing distance sample represents the measured distance between the re-dispensing position and the restored stable position during a single calibration process. The re-dispensing recovery distance Dr represents the execution judgment parameter determined based on multiple re-dispensing distance samples.

[0084] A sample milk discharge distance ri is formed based on the actual movement distance of the robotic arm between the re-discharge position and the stable recovery position. N samples of milk discharge distance are obtained by repeating the calibration N times as the same as the milk stop calibration, and the maximum value rmax is obtained. The milk discharge recovery distance corresponding to the current combination of execution parameters and the current calibration conditions is obtained according to: Dr=ceil(rmax / δ)×δ.

[0085] Therefore, for each combination of robotic arm movement speed and milk output per unit time that the template can use, the corresponding Ds and Dr are saved in advance.

[0086] The separation logic for adjacent template pattern portions is as follows: like Figure 5 As shown, for each pair of adjacent template pattern parts in the candidate latte art templates, the coffee background path length L determined in step S2 is obtained; wherein, the coffee background path length L represents the cumulative length of the trajectory segment within the coffee background area in the robotic arm's movement path between the end position of the previous template pattern part and the start position of the next template pattern part.

[0087] The starting point of the cumulative distance of the robotic arm's movement path is taken as the end position of the previous template pattern. The first position is determined along the movement path according to the milk stop extension distance Ds. The cumulative distance C1 corresponding to the first position is: C1 = Ds; Starting from the position of the next template pattern part, accumulate the distance Dr along the same robotic arm movement path in reverse to restore milk production, and determine the second position; convert the second position into the cumulative distance starting from the end position of the previous template pattern part, then: C2 = L - Dr; When the second position is after the first position, the path length G that the robotic arm can retain across the coffee-colored area is: G = C2 - C1; Since the first position is determined by the distance Ds extended by the robotic arm moving forward along the path from the end position of the previous template pattern, the first cumulative distance C1 = Ds; Since the second position is determined by the distance Dr that resumes milk dispensing by moving backward along the same path from the beginning position of the next template pattern, the second cumulative distance C2 = L - Dr; Therefore, the retainable path length G can be further expressed as: G = L - Ds - Dr; When the second position is after the first position, and the actual path length G between the first and second positions in the coffee background area is not less than the discrete length δ of the cup surface space, it indicates that at least one coffee background path corresponding to the discrete length of the cup surface space is retained between the two positions, and it is determined that the current two template pattern parts can remain separated; the length of the robotic arm movement path between the first and second positions in the coffee background area is determined as the retainable path length G, where the retainable path length G represents the actual space length that can be retained between the current adjacent template pattern parts for pattern separation after deducting the milk stop extension distance Ds and the milk refill recovery distance Dr.

[0088] When the second position is after the first position, but the path length G of the robotic arm passing through the coffee background area between the first and second positions is less than the discrete length δ of the cup surface space, it indicates that the coffee background path retained between the two positions is less than the discrete length of the cup surface space, and it is determined that the current two template pattern parts cannot remain separated.

[0089] When the second position coincides with or is located before the first position, it indicates that the path occupied by the milk flow after the previous template pattern stopped flowing milk has been connected to or overlapped with the path occupied by the next template pattern to resume milk flow in order to restore a stable milk line width. It is determined that the two template pattern parts cannot remain separated, and the corresponding first position, second position, Ds, and Dr are recorded.

[0090] When Ds is greater than or equal to L, or Dr is greater than or equal to L, it indicates that the milk delivery response distance on one side has occupied the entire coffee background path, directly determining that the two template pattern parts cannot remain separated.

[0091] For example, in one embodiment, L is 5.2 mm, Ds is 2.0 mm, and Dr is 1.8 mm. The cumulative distance corresponding to the first position is 2.0 mm, and the cumulative distance corresponding to the second position is 3.4 mm, with a 1.4 mm coffee-colored path maintained between them. When δ is 0.10 mm, 1.4 mm is not less than δ, thus ensuring that the two template pattern portions remain separated.

[0092] In another embodiment, L is 4.45 mm, Ds is 2.20 mm, and Dr is 2.20 mm. Therefore, the cumulative distance corresponding to the first position is 2.20 mm, and the cumulative distance corresponding to the second position is 2.25 mm. There is only a 0.05 mm coffee-colored path between them. When δ is 0.10 mm, 0.05 mm is less than δ. Therefore, although the second position is still after the first position, it is still determined that the two template pattern parts cannot remain separated.

[0093] In another embodiment, L is 4.4 mm, Ds is 2.5 mm, and Dr is 2.1 mm. The cumulative distance corresponding to the first position is 2.5 mm, and the cumulative distance corresponding to the second position is 2.3 mm. The second position is located before the first position, so it is determined that the paths occupied by the two milk dispensing responses overlap.

[0094] The candidate latte art templates are evaluated one by one according to the production sequence. If all adjacent template patterns can remain separated, the current candidate latte art template is determined as the one that meets the actual milk dispensing space requirements of the current equipment.

[0095] Step S3 performs separation judgment one by one according to the arrangement order of the candidate latte art templates obtained in step S1. When a candidate latte art template appears for the first time in which all adjacent template patterns can remain separated, the current candidate latte art template is determined as the final latte art template.

[0096] When all candidate latte art templates corresponding to the current order pattern have adjacent template pattern parts that cannot be kept separate, the candidate latte art template with the highest order is determined as the current adjustment template; the adjacent template pattern parts in the current adjustment template are checked in turn according to the production order, and the first adjacent relationship that cannot be kept separate is determined as the current adjustment relationship. The template pattern part, first position, second position, retainable path length, milk stop extension distance and milk re-discharge recovery distance corresponding to the current adjustment relationship are entered into step S4.

[0097] Step S4: Adjust the decorative pattern according to the adjacent template pattern parts that cannot be kept separate, and rematch the candidate lace template; like Figure 5As shown, step S4 reads the adjacent template pattern parts that cannot be kept separate obtained in step S3, and determines the order pattern objects corresponding to the two template pattern parts that constitute the current adjacent relationship according to the correspondence established in step S1.

[0098] This step always adjusts the copy of the order pattern used for template matching, while the user's original order pattern and original order editing records remain unchanged.

[0099] First, determine whether the two corresponding order pattern objects belong to the main pattern or the decorative pattern.

[0100] When both order pattern objects are primary patterns, neither object is removed from the order pattern copy, and the order pattern copy adjustment process corresponding to the current candidate template is terminated.

[0101] When only one of the two order pattern objects is a decorative pattern, the decorative pattern is removed from the order pattern copy, and the corresponding area is restored to the coffee background color to form an adjusted order pattern copy. Then, the candidate latte art template matching in step S1 is re-executed based on the order pattern copy. In this embodiment, the candidate template arrangement order follows the candidate template sorting result formed in step S1 based on the degree of pattern overlap. When the degree of pattern overlap is the same, a fixed arrangement order is formed according to the template number.

[0102] For example, the original order pattern includes a main pattern P0, a decorative pattern P1, and a decorative pattern P2. P0 and P1 cannot be kept separate. Since P0 is the main pattern and P1 is the decorative pattern, P1 is removed from the order pattern copy to form an order pattern copy of P0 + P2.

[0103] After removing P1, the P0→P1 and P1→P2 robotic arm trajectories in the original candidate template are not spliced, extended, or regenerated. Instead, the P0+P2 template is used to rematch the latte art template. If template B is obtained through rematching, and the actual production order saved in template B is P0→P2, then the P0 to P2 robotic arm position sequence already saved in template B is directly read, and steps S2 and S3 are executed again.

[0104] The processing logic when both order pattern objects are decorative patterns is as follows: When two order pattern objects that cannot maintain a separation relationship are both decorative patterns, the decorative pattern to be removed is not predetermined based on the milk stoppage extension distance or milk re-production recovery distance corresponding to the two decorative patterns, but rather an adjustment process is formed based on the current order pattern copy to create two independent order pattern copies.

[0105] Copy the current order pattern to create a first order pattern copy and a second order pattern copy. In the first order pattern copy, remove the decorative pattern that was made earlier, restoring the corresponding area of ​​the removed decorative pattern to the coffee background color; in the second order pattern copy, remove the decorative pattern that was made later, restoring the corresponding area of ​​the removed decorative pattern to the coffee background color.

[0106] The first and second order pattern copies are used as new template matching inputs. Subsequent candidate latte art templates are determined by rematching the adjusted order pattern copies. The robotic arm trajectories before and after the removed decorative patterns in the original candidate latte art templates are not directly spliced, extended, or regenerated.

[0107] Therefore, in cases where both decorative patterns are involved and cannot maintain a separation relationship, the two possible decorative pattern removal methods each form an independent order pattern copy, and subsequent judgments are made based on the actual execution data saved by the candidate latte art templates obtained through rematching.

[0108] The logic for re-matching the adjusted order pattern copy is as follows: If only one copy of the order pattern is formed, steps S1 to S3 are re-executed using the corresponding copy of the order pattern.

[0109] After rematching, the production order and robotic arm position sequence of the new candidate latte art template are read, the correspondence between the order pattern object and the template pattern part is re-established, and the main pattern and remaining decorative pattern, adjacent template pattern parts, coffee background path lengths L, Ds, Dr, first position and second position are re-determined.

[0110] The actual execution data of the robotic arm always comes from the executable robotic arm position sequence that has been saved by the rematched latte art template, and does not directly generate a new trajectory based on the original candidate template.

[0111] In the case where two order pattern copies are formed, the candidate latte art templates are rematched using the first and second order pattern copies respectively. The rematched candidate latte art templates must have their own complete production sequence, robotic arm position sequence, and calibration records corresponding to the current execution parameter combination and calibration conditions; if the above conditions are met, steps S2 and S3 are re-executed using the two rematched candidate latte art templates respectively.

[0112] When multiple candidate latte art templates are obtained by rematching a copy of an order pattern, the candidate latte art template that is first in the list and has a complete production sequence, robotic arm position sequence, and corresponding calibration record is determined as the rematch template. The first position, second position, and retainable path length of the order pattern copy are subsequently determined based on this rematch template. The rematch template refers to the candidate latte art template that has a complete production sequence, robotic arm position sequence, and corresponding calibration record among the candidate latte art templates obtained by rematching the order pattern copy, and is used to determine the retainable path length between adjacent template pattern parts.

[0113] For a rematch template corresponding to a copy of an order pattern, determine the first and second positions corresponding to all adjacent template pattern portions respectively.

[0114] When the second position is after the first position, the actual path length between the first and second positions within the coffee-colored area is taken as the retainable path length corresponding to the current adjacency relationship; when the second position coincides with the first position or is before the first position, the corresponding retainable path length is recorded as zero.

[0115] When a rematching template has multiple adjacent template pattern partial relationships, the retainable path length corresponding to each adjacent relationship is determined, and the minimum value among them is used as the comparison path length corresponding to the current order pattern copy.

[0116] Compare the comparison path lengths of the candidate templates obtained by rematching the first order pattern copy and the second order pattern copy respectively. Keep the order pattern copy with the larger comparison path length to enter the next round of template matching and separation judgment. When the two comparison path lengths are the same, determine the actual area of ​​the two removed decorative patterns according to the actual coordinate calibration relationship of the cup surface, and keep the order pattern copy with the smaller actual area of ​​the removed decorative pattern.

[0117] The comparison path length represents the minimum coffee background separation space length that can be retained in the candidate template corresponding to the adjusted order pattern copy. The larger the value, the greater the space margin in the separation of adjacent patterns in the rematched candidate template. The comparison path length is determined by the retainable path length of each adjacent template pattern part in the rematched candidate latte art template, and is used to characterize the adjacent relationship with the smallest space margin in the current candidate latte art template.

[0118] For example, when the comparison path length of the candidate templates corresponding to the first order pattern copy and the second order pattern copy is 1.3 mm, the actual areas of the two removed decorative patterns are further compared; when the actual area of ​​the removed decorative pattern of the first order pattern copy is smaller, the first order pattern copy is retained and enters the next round of template matching.

[0119] If one of the order pattern copies cannot be rematched with a candidate template that has a complete production sequence, robotic arm position sequence, and corresponding calibration record, the order pattern copy adjustment process corresponding to that order pattern copy is terminated, and only the other order pattern copy that can obtain a candidate template is processed.

[0120] Each time a decorative pattern is removed, at least one decorative pattern object is reduced in the current order's pattern copy; therefore, the number of adjustments is limited by the number of decorative patterns in the current order.

[0121] When all adjacent template patterns in the candidate latte art templates can remain separated after rematching, the candidate latte art template that first meets the condition is determined as the final latte art template according to the order of the candidate latte art templates.

[0122] If all decorative patterns in the current order pattern copy adjustment process have been processed, and further adjustment would involve the main pattern, then the current order pattern copy adjustment process will be terminated. If there are other unprocessed order pattern copies, continue checking those other order pattern copies; if the adjustment process for all order pattern copies has been terminated and there are still no candidate latte art templates that meet the conditions, output the information that the current order does not have a latte art template that meets the milk dispensing space conditions of the current equipment, and keep the user's original order pattern unchanged.

[0123] In a complete example, the user's order pattern includes a large heart P0, dots P1 and dots P2, where P0 comes from the basic latte art template and is the main pattern; P1 and P2 are decorative patterns added by the user later and separated from P0 by a coffee-colored area with a width not less than the discrete length δ of the cup surface space.

[0124] In the initial candidate latte art templates, the template pattern parts corresponding to P0 and P1 cannot be kept separate; since P0 is the main pattern and P1 is the decorative pattern, P1 is removed from the order pattern copy to obtain P0 + P2; Template B is obtained by rematching P0+P2. The actual production order saved in template B is P0→P2. The robot arm position sequence from P0 to P2 already saved in template B is read, the robot arm coffee-colored movement path between the two is obtained again, and Ds and Dr corresponding to the actual execution parameters of template B are called.

[0125] If a 1.2 mm coffee-colored path is maintained between the first and second positions, and the current δ is 0.10 mm, then P0 and P2 can remain separated; when other adjacent template pattern parts in template B also meet the same judgment conditions, template B is determined as the final latte art template.

[0126] Throughout the process, the robotic arm execution trajectory of template B is not obtained by splicing or reconstructing the original candidate template trajectory, but directly comes from the robotic arm position sequence that has been saved in template B and can be executed by the current device.

[0127] Through steps S1 to S4, the order pattern is first used to determine candidate templates from latte art templates with complete actual execution data and corresponding milk discharge calibration records; then, the actual moving distance corresponding to the milk flow after the previous template pattern stops milking and the actual path required for the next template pattern to resume milk flow and restore stable line width are unified into the same coffee background path, and the positional relationship between the first position and the second position and the path length of the robotic arm through the coffee background area are used to determine whether adjacent patterns can remain separated during actual production.

[0128] When separation cannot be maintained, the adjustment method is determined based on the attributes of the order pattern objects involved in the current adjacent relationship. If both order pattern objects are primary patterns, the adjustment process for the current order pattern copy is terminated; if only one order pattern object is a decorative pattern, an order pattern copy with the corresponding decorative pattern removed is created; if both order pattern objects are decorative patterns, two order pattern copies are created, one with the decorative pattern removed first and the other with the decorative pattern removed later. The adjusted order pattern copies are then re-matched with candidate latte art templates, and the adjacent pattern separation judgment is re-executed based on the production order and robotic arm position sequence saved in the re-matched candidate latte art templates.

[0129] In the above processing, the image coordinate distance, robotic arm trajectory distance, milk stop extension distance, milk refill recovery distance, and the path length of the robotic arm through the coffee background area are all uniformly converted to the actual length in the actual coordinate system of the cup surface. The milk stop extension distance and milk refill recovery distance are derived from the calibration records pre-established under the corresponding execution parameter combinations and calibration conditions; the discrete length of the cup surface space is directly determined by the sampling step size of the cup surface actual coordinate resampling grid, and is used as a unified discrete length unit for distance normalization and coffee background path judgment.

[0130] The specific values ​​of coffee background path length, milk stop extension distance, milk refill recovery distance, and cup surface space discrete length listed in this embodiment are only used to illustrate the calculation process; different devices obtain the corresponding values ​​according to the same data acquisition, normalization, and judgment methods, and it is not required to use the same specific values.

[0131] Those skilled in the art will recognize that the modules and algorithm modules of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and inventive constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0132] In addition, the functional modules in the various embodiments of this application can be integrated into one processing module, or each module can exist physically separately, or two or more modules can be integrated into one module.

[0133] The above are merely specific embodiments of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

[0134] In conclusion, the above are merely preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for automatically matching coffee latte art patterns based on user orders, characterized in that, Includes the following steps: Obtain the order pattern, determine the candidate latte art template, establish the correspondence between the order pattern object and the template pattern part, and distinguish between the main pattern and the decorative pattern; The path length of the robotic arm through the coffee background area between adjacent template pattern parts is determined according to the production sequence. The calibration record is called based on the robotic arm movement speed and milk output per unit time before the previous template pattern part stops milking and when the next template pattern part starts milking, to obtain the milking stop extension distance and the milking restart recovery distance. The first position is determined by the milk stop extension distance along the forward direction of the robotic arm movement path, and the second position is determined by the milk recovery distance along the reverse direction of the same robotic arm movement path. When the second position is after the first position, and the retainable path length between the first position and the second position is not less than the discrete length of the cup surface space, it is determined that there is still a coffee background area between the adjacent template pattern parts that meets the discrete length requirement of the cup surface space, thus determining that the adjacent template pattern parts can remain separated. When separation cannot be maintained and the corresponding order pattern object contains decorative patterns, the corresponding decorative patterns are removed to form a copy of the order pattern. Candidate latte art templates are then re-matched and separation is re-judged until the final latte art template is determined.

2. The automatic coffee latte art pattern matching method based on user orders according to claim 1, characterized in that, Obtain the order pattern, determine the candidate latte art template, establish the correspondence between the order pattern object and the template pattern part, and distinguish between the main pattern and the decorative pattern; The path length of the robotic arm through the coffee background area between adjacent template pattern parts is determined according to the production sequence. The calibration record is called based on the robotic arm movement speed and milk output per unit time before the previous template pattern part stops milking and when the next template pattern part starts milking, to obtain the milking stop extension distance and the milking restart recovery distance. The first position is determined by the milk stop extension distance along the forward direction of the robotic arm movement path, and the second position is determined by the milk recovery distance along the reverse direction of the same robotic arm movement path. When the second position is after the first position, and the length of the retainable path between the first and second positions within the coffee background area is not less than the discrete length of the cup surface space, it is determined that the adjacent template pattern parts can remain separated; otherwise, it is determined that they cannot remain separated. When separation cannot be maintained and the corresponding order pattern object contains decorative patterns, the corresponding decorative patterns are removed to form a copy of the order pattern. Candidate latte art templates are then re-matched and separation is re-judged until the final latte art template is determined.

3. The automatic coffee latte art pattern matching method based on user orders according to claim 2, characterized in that, Based on the calibration relationship between the pixel coordinates of the cup surface image and the actual coordinates of the cup surface, the milk area, coffee background area and robotic arm position sequence in the candidate template are unified to the actual coordinate system of the cup surface. The cup surface image is resampled at equal intervals according to the actual coordinates of the cup surface, and the actual distance between the centers of adjacent resampled points is determined as the discrete length of the cup surface space. The path length is obtained by taking the position where the robotic arm first leaves the previous template pattern as the starting point and the position where it first enters the next template pattern as the ending point. The length of the trajectory within the coffee-colored area in the robotic arm position sequence is accumulated.

4. The automatic coffee latte art pattern matching method based on user orders according to claim 3, characterized in that, Determining the extension distance for weaning includes: Under the same robotic arm movement speed and milk output per unit time, multiple milk stop distance samples were obtained according to a preset calibration number of times. The position of the robotic arm when the stop milk dispensing command is issued is taken as the stop milk dispensing position, and the furthest position that still maintains continuity with the milk area formed before the stop milk dispensing along the direction of robotic arm movement is taken as the last continuous milk area position. The actual distance moved between the milk stop position and the last continuous milk position is used as the milk stop distance sample. The maximum distance among multiple milk stoppage distance samples is obtained, and then rounded upwards using the discrete length of the cup surface space as the unit to obtain the milk stoppage extension distance.

5. The automatic coffee latte art pattern matching method based on user orders according to claim 4, characterized in that, Determining the distance for resuming milk production includes: Under the same robotic arm moving speed and milk output per unit time, obtain multiple milk line widths for a stable milk output reference segment, and determine the stable width range based on the multiple milk line widths; Starting from the point where milk is re-discharged, continuously obtain the width of the milk line along the re-discharge trajectory, and determine the interval where the width of the milk line is continuously within a stable width range from the end of the trajectory in reverse. The position within this interval closest to the point where milk production resumes is determined as the stable recovery position. The actual distance traveled between the re-discharge position and the restored stable position is used as the re-discharge distance sample. The maximum distance among multiple re-discharge distance samples is obtained and normalized to obtain the re-discharge recovery distance.

6. The automatic coffee latte art pattern matching method based on user orders according to claim 5, characterized in that, The end position of the previous template pattern is used as the starting point of the cumulative distance of the robotic arm's movement path, and the cumulative distance corresponding to the milk stop extension distance is used as the first cumulative distance corresponding to the first position; The second position is determined by accumulating the distance of milk discharge recovery along the robotic arm's movement path from the starting position of the next template pattern portion, and then converting it into a second accumulated distance based on the starting point. When the second cumulative distance is greater than the first cumulative distance, the difference between the two is determined as the length of the path that can be retained. Otherwise, the length of the path that can be retained will be set to zero.

7. The automatic coffee latte art pattern matching method based on user orders according to claim 6, characterized in that, The pattern object derived from the user-defined basic latte art template and created through order editing is selected as the main pattern; After determining the basic latte art template, add new ones, and define the decorative pattern as an independent pattern object that is separated from the main pattern by a coffee background area with a width not less than the discrete length of the cup surface space. Determine the overlapping area between the template pattern and the order pattern object, and identify the order pattern object with the largest overlapping area as the corresponding object.

8. The automatic coffee latte art pattern matching method based on user orders according to claim 7, characterized in that, If the template pattern portion that cannot be separated does not have a corresponding order pattern object, the order pattern object removal is not performed, and the next candidate lattice template is checked. When adjacent template pattern parts cannot remain separated, the corresponding order pattern object is determined according to the correspondence relationship; When both order pattern objects are primary patterns, keep the order pattern copies unchanged; When only one order pattern object is a decorative pattern, remove the corresponding decorative pattern to create a copy of the order pattern. When both order pattern objects are decorative patterns, two order pattern copies are created, and the decorative patterns with different production orders are removed from each copy.

9. The automatic coffee latte art pattern matching method based on user orders according to claim 8, characterized in that, For each generated order pattern copy, rematch the candidate latte art templates, read the production order and robotic arm position sequence saved by the rematched candidate latte art templates, and redetermine the first position, second position, and retainable path length corresponding to the adjacent template pattern parts; When multiple candidate latte art templates are obtained by rematching the order pattern copy, the candidate latte art template that appears first in the order and has a complete production sequence, robotic arm position sequence and corresponding calibration record will be used as the rematch template. Obtain the retainable path lengths for each adjacent template pattern portion in the rematch template, and use the minimum value among them as the comparison path length for the order pattern copy; Determine the comparison path lengths for the two order pattern copies respectively, and retain the order pattern copy with the larger comparison path length.

10. The automatic coffee latte art pattern matching method based on user orders according to claim 9, characterized in that, When the comparison path lengths corresponding to two order pattern copies are the same, the actual area of ​​the decorative pattern removed from the two order pattern copies is determined based on the actual coordinates of the cup surface, and the order pattern copy with the smaller actual area of ​​the decorative pattern removed is retained; When the order pattern copy cannot be rematched with a candidate latte art template that has a complete production sequence, robotic arm position sequence and corresponding calibration record, the corresponding order pattern copy adjustment process ends. When all adjacent template patterns in the rematched candidate latte art templates can remain separated, the corresponding candidate latte art template is determined as the final latte art template.

Images