Lacquer trajectory determination method, device and electronic equipment

Abstract

This invention discloses a method, apparatus, and electronic device for determining latte art trajectories. The method includes: receiving a trajectory acquisition command; responding to the trajectory acquisition command, acquiring first movement trajectory data of a first handheld object, second movement trajectory data of a second handheld object, and a first image frame set including the first and second handheld objects; correcting the first and second movement trajectory data based on the first image frame set to obtain first corrected trajectory data of the first handheld object and second corrected trajectory data of the second handheld object; and obtaining the target latte art trajectory based on the first and second corrected trajectory data. This invention solves the technical problem in related technologies where the determined latte art trajectory is inaccurate.

Description

Technical Field

[0001] This invention relates to the field of trajectory determination, and more specifically, to a method, apparatus, and electronic device for determining the trajectory of latte art. Background Technology

[0002] As society develops, more and more consumers enjoy coffee, especially latte art. However, because latte artists create designs by hand, they cannot guarantee that the design will match the customer's requirements, affecting the final result. By using robots to simulate human hand movements, latte art can be created more efficiently, resulting in more consistent and aesthetically pleasing designs.

[0003] With the development of new business models, consumers have increasingly higher demands for latte art patterns. Enabling robots to create more designs requires replicating the movements of latte artists, and acquiring their motion data and trajectory is a crucial step. However, related technologies suffer from inaccuracies in determining the latte art trajectory.

[0004] There is currently no effective solution to the above problems. Summary of the Invention

[0005] This invention provides a method, apparatus, and electronic device for determining latte art trajectories, to at least solve the technical problem in the related art where the determined latte art trajectories are inaccurate.

[0006] According to one aspect of the present invention, a method for determining a latte art trajectory is provided, comprising: receiving a trajectory acquisition instruction, wherein the trajectory acquisition instruction carries a predetermined latte art pattern, the trajectory acquisition instruction being used to acquire a latte art trajectory of a predetermined object drawing the predetermined latte art pattern on a predetermined liquid surface, the predetermined liquid surface being the liquid surface of a liquid in a first handheld object of the predetermined object; responding to the trajectory acquisition instruction, acquiring first movement trajectory data of the first handheld object of the predetermined object, second movement trajectory data of a second handheld object of the predetermined object, and a first image frame set including the first handheld object and the second handheld object; correcting the first movement trajectory data and the second movement trajectory data according to the first image frame set to obtain first corrected trajectory data of the first handheld object and second corrected trajectory data of the second handheld object; and obtaining a target latte art trajectory according to the first corrected trajectory data and the second corrected trajectory data.

[0007] Optionally, the step of acquiring first movement trajectory data of the first handheld object of the predetermined object, second movement trajectory data of the second handheld object of the predetermined object, and a first image frame set including the first handheld object and the second handheld object in response to the trajectory acquisition command includes: acquiring a first initial pose of the first handheld object and a second initial pose of the second handheld object, and an initial pose image including the first handheld object and the second handheld object in response to the trajectory acquisition command; and acquiring the first movement trajectory data, the second movement trajectory data, and the first image frame set when the pose corresponding to the first handheld object in the initial pose image is the first predetermined pose and the pose corresponding to the second handheld object is the second predetermined pose.

[0008] Optionally, after obtaining the target latte art trajectory based on the first correction trajectory data and the second correction trajectory data, the method further includes: acquiring target flow data corresponding to each trajectory point in the second correction trajectory data, wherein the target flow data is the flow data of liquid flowing out of the second handheld object of the predetermined object; and obtaining the first latte art pattern trajectory information based on the target latte art trajectory and the target flow data corresponding to each trajectory point in the second correction trajectory data.

[0009] Optionally, after obtaining the target latte art trajectory based on the first correction trajectory data and the second correction trajectory data, the method further includes: acquiring a latte art drawing image corresponding to each trajectory point in the first correction trajectory data, wherein the latte art drawing image is an image of the liquid surface of the liquid in the first handheld object; and obtaining second latte art pattern trajectory information based on the target latte art trajectory and the latte art drawing image corresponding to each trajectory point in the first correction trajectory data.

[0010] Optionally, after obtaining the target latte art trajectory based on the first correction trajectory data and the second correction trajectory data, the method further includes: obtaining the capacity specification parameters of the first handheld object; and obtaining the third latte art pattern trajectory information based on the target latte art trajectory and the capacity specification parameters.

[0011] Optionally, after receiving the trajectory acquisition instruction, the method further includes: in response to the trajectory acquisition instruction, acquiring the first movement trajectory data, the second movement trajectory data, the third movement trajectory data of the predetermined part of the predetermined object, and a second image frame set including the first handheld object, the second handheld object, and the predetermined part.

[0012] Optionally, after obtaining the target latte art trajectory based on the first correction trajectory data and the second correction trajectory data, the method further includes: obtaining a target latte art pattern; determining the latte art approximation index between the target latte art pattern and the predetermined latte art pattern; if the latte art approximation index is less than a predetermined threshold, determining a difference pattern between the target latte art pattern and the predetermined latte art pattern; and adjusting the target latte art trajectory corresponding to the predetermined latte art pattern based on the difference pattern to obtain an updated latte art trajectory corresponding to the target latte art pattern.

[0013] According to one aspect of the present invention, a latte art trajectory determination device is provided, comprising: a receiving module for receiving a trajectory acquisition command, wherein the trajectory acquisition command carries a predetermined latte art pattern, the trajectory acquisition command being used to acquire a latte art trajectory of a predetermined object drawing the predetermined latte art pattern on a predetermined liquid surface, the predetermined liquid surface being the liquid surface of a first handheld object of the predetermined object; an acquisition module for acquiring, in response to the trajectory acquisition command, first movement trajectory data of the first handheld object of the predetermined object, second movement trajectory data of a second handheld object of the predetermined object, and a first image frame set including the first handheld object and the second handheld object; a correction module for correcting the first movement trajectory data and the second movement trajectory data according to the first image frame set, to obtain first corrected trajectory data of the first handheld object and second corrected trajectory data of the second handheld object; and a determination module for obtaining a target latte art trajectory according to the first corrected trajectory data and the second corrected trajectory data.

[0014] According to one aspect of the present invention, an electronic device is provided, comprising: a processor; and a memory for storing processor-executable instructions; wherein the processor is configured to execute the instructions to implement the latte art trajectory determination method described in any of the preceding claims.

[0015] According to one aspect of the present invention, a computer-readable storage medium is provided, wherein when the instructions in the computer-readable storage medium are executed by a processor of an electronic device, the electronic device is enabled to perform the latte art trajectory determination method described in any of the preceding claims.

[0016] In this embodiment of the invention, a trajectory acquisition command carrying a predetermined latte art pattern is received to acquire the latte art trajectory of a predetermined object drawing the predetermined latte art pattern on a predetermined liquid surface. In response to the trajectory acquisition command, first movement trajectory data of a first handheld object of the predetermined object, second movement trajectory data of a second handheld object of the predetermined object, and a first image frame set including the first and second handheld objects are acquired. Then, based on the first image frame set, the first and second movement trajectory data are corrected to obtain first corrected trajectory data of the first handheld object and second corrected trajectory data of the second handheld object. Finally, based on the first and second corrected trajectory data, the target latte art trajectory is obtained. Since the target latte art trajectory is obtained based on the first and second corrected trajectory data, the obtained target latte art trajectory is more accurate. Moreover, the correction is performed on the trajectory data using an image frame set, which can correct errors in three-dimensional space using two-dimensional images, making the corrected trajectory data more accurate, thereby solving the technical problem in related technologies where the determined latte art trajectory is inaccurate. Attached Figure Description

[0017] The accompanying drawings, which are included to provide a further understanding of the invention and form part of this application, illustrate exemplary embodiments of the invention and, together with their description, serve to explain the invention and do not constitute an undue limitation thereof. In the drawings:

[0018] Figure 1 This is a flowchart of a method for determining the latte art trajectory according to an embodiment of the present invention;

[0019] Figure 2 This is a schematic diagram of a latte art performer's latte art movements provided by an optional embodiment of the present invention;

[0020] Figure 3 This is a structural block diagram of a latte art trajectory determining device according to an embodiment of the present invention. Detailed Implementation

[0021] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0022] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0023] Example 1

[0024] According to an embodiment of the present invention, an embodiment of a method for determining a latte art trajectory is provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0025] Before introducing the process steps of the latte art trajectory determination method, let's first introduce the application scenario of this invention. The application scenario of this invention is that a predetermined object holds a first object and a second object. The predetermined object can be a latte artist. The first object is generally a liquid such as coffee, and the second object is a milk foam liquid or other liquid. The liquid in the second object can be poured onto the liquid in the first object, and latte art patterns can be drawn on the predetermined liquid surface of the liquid in the first object.

[0026] Figure 1 This is a flowchart of a method for determining the latte art trajectory according to an embodiment of the present invention, as shown below. Figure 1 As shown, the method includes the following steps:

[0027] Step S102: Receive trajectory acquisition instruction, wherein the trajectory acquisition instruction carries a predetermined latte art pattern, and the trajectory acquisition instruction is used to acquire the latte art trajectory of the predetermined object drawing the predetermined latte art pattern on the predetermined liquid surface, and the predetermined liquid surface is the liquid surface of the first handheld object of the predetermined object;

[0028] In step S102 provided in this application, a trajectory acquisition instruction carrying a predetermined latte art pattern is received, so that the predetermined object starts to draw the predetermined latte art pattern after receiving the trajectory acquisition instruction, so as to obtain the trajectory data corresponding to the predetermined latte art pattern.

[0029] Step S104: In response to the trajectory acquisition command, acquire the first movement trajectory data of the first handheld object of the predetermined object, the second movement trajectory data of the second handheld object of the predetermined object, and a first image frame set including the first handheld object and the second handheld object.

[0030] In step S104 provided in this application, since the actions performed by the hands are different when making latte art, it is necessary to acquire the trajectory data of the objects held in both hands separately. This trajectory data can be collected by a spatial sensor located on the object, such as a six-degree-of-freedom sensor, which can obtain the trajectory of the object's movement in space.

[0031] The first image frame set is a collection of images including both the first and second held objects. This first image frame set can be extracted from a recorded video or captured frame-by-frame; there is no limitation on this. Adaptive settings can be made according to the actual application and scenario.

[0032] It should be noted that the first set of image frames carries the frame time of the capture, and the trajectory data also carries the trajectory point time, so that the trajectory can be matched with the frame time and the trajectory point time, and the trajectory can be corrected better.

[0033] Step S106: Based on the first image frame set, correct the first motion trajectory data and the second motion trajectory data to obtain the first corrected trajectory data of the first handheld object and the second corrected trajectory data of the second handheld object;

[0034] In step S106 provided in this application, the three-dimensional spatial data can be corrected from the level of two-dimensional image and two-dimensional coordinates through the first image frame set. This ensures that the coordinates in the three-dimensional spatial data do not exceed the limitations of the two-dimensional coordinates, and that the trajectory points in the three-dimensional spatial data are constrained by the trajectory in the two-dimensional image, making the trajectory data more accurate.

[0035] Step S108: Based on the first correction trajectory data and the second correction trajectory data, the target latte art trajectory is obtained.

[0036] In step S108 provided in this application, the target latte art trajectory is obtained based on the first correction trajectory data and the second correction trajectory data.

[0037] Through the aforementioned steps S102-S108, a trajectory acquisition command carrying a predetermined latte art pattern is received to acquire the latte art trajectory of a predetermined object drawing the predetermined latte art pattern on a predetermined liquid surface. In response to the trajectory acquisition command, first movement trajectory data of the first handheld object of the predetermined object, second movement trajectory data of the second handheld object of the predetermined object, and a first image frame set including the first and second handheld objects are acquired. Then, based on the first image frame set, the first and second movement trajectory data are corrected to obtain first corrected trajectory data of the first handheld object and second corrected trajectory data of the second handheld object. Finally, based on the first and second corrected trajectory data, the target latte art trajectory is obtained. Since the target latte art trajectory is obtained based on the first and second corrected trajectory data, the obtained target latte art trajectory is more accurate. Moreover, the trajectory data is corrected using an image frame set, which can correct errors in three-dimensional space using two-dimensional images, making the corrected trajectory data more accurate, thereby solving the technical problem in related technologies where the determined latte art trajectory is inaccurate.

[0038] As an optional embodiment, in response to a trajectory acquisition command, acquiring first movement trajectory data of a first handheld object of a predetermined object, second movement trajectory data of a second handheld object of a predetermined object, and a first image frame set including the first handheld object and the second handheld object, includes: in response to a trajectory acquisition command, acquiring a first initial pose of the first handheld object, a second initial pose of the second handheld object, and an initial pose image including the first handheld object and the second handheld object; if the pose corresponding to the first handheld object in the initial pose image is the first predetermined pose and the pose corresponding to the second handheld object is the second predetermined pose, acquiring the first movement trajectory data, the second movement trajectory data, and the first image frame set.

[0039] In this embodiment, in response to a trajectory acquisition command, the first initial pose of the first held object, the second initial pose of the second held object, and an initial pose image are acquired. This is to determine whether the first and second initial poses are both located at their corresponding predetermined poses. If both the poses corresponding to the first and second held objects are located at their corresponding predetermined poses, it indicates that both hands holding the objects have reached the designated positions. At this point, it can be further determined whether the first initial pose is consistent with the pose of the first held object in the initial pose image, and whether the second initial pose is consistent with the pose of the second held object in the initial pose image. If they are consistent, the acquisition of first movement trajectory data, second movement trajectory data, and a first image frame set can begin.

[0040] This step ensures that the acquisition of the first movement trajectory data starts from the first initial pose, the second movement trajectory data starts from the second initial pose, and the first image frame set is based on the condition that the poses corresponding to the first and second handheld objects reach the corresponding predetermined poses before subsequent data acquisition. This is beneficial for the correction of the initial pose during subsequent robot latte art.

[0041] As an optional embodiment, after obtaining the target latte art trajectory based on the first correction trajectory data and the second correction trajectory data, the method further includes: acquiring target flow data corresponding to each trajectory point in the second correction trajectory data, wherein the target flow data is the flow data of liquid flowing out of the second handheld object of the predetermined object; and obtaining the first latte art pattern trajectory information based on the target latte art trajectory and the target flow data corresponding to each trajectory point in the second correction trajectory data.

[0042] In this embodiment, the second correction trajectory data is the trajectory data corresponding to the second handheld object, that is, the trajectory data corresponding to the handheld object to which the liquid is poured onto the predetermined liquid surface. The target flow rate data corresponding to each trajectory point is the flow rate data of the liquid flowing out of the handheld object at different trajectory points. For example, the target flow rate data at the first trajectory point is 5 ml / s, the target flow rate data at the second trajectory point is 10 ml / s, and so on. When the liquid in the second handheld object is milk foam, the target flow rate data is the flow rate data of the milk foam flowing out of the second handheld object of the predetermined object. By determining the flow rate data of the milk foam and the target latte art trajectory, not only can the movement trajectories of the first and second handheld objects be clearly known, but also the flow rate data of the liquid in the second handheld object at each trajectory point, which can also be understood as the flow rate data at each moment, which helps the robot to better draw the predetermined latte art pattern.

[0043] It should be noted that the target flow rate data can be calculated using the milk volume collected by the milk pitcher and the tilt angle in the trajectory, which yields data on the trajectory of the milk pitcher nozzle and the milk flow rate during the latte art process.

[0044] As an optional embodiment, after obtaining the target latte art trajectory based on the first correction trajectory data and the second correction trajectory data, the method further includes: acquiring a latte art drawing image corresponding to each trajectory point in the first correction trajectory data, wherein the latte art drawing image is an image of the liquid surface of the liquid in the first handheld object; and obtaining the second latte art pattern trajectory information based on the target latte art trajectory and the latte art drawing image corresponding to each trajectory point in the first correction trajectory data.

[0045] In this embodiment, latte art images corresponding to each trajectory point in the first correction trajectory data are acquired. When the liquid in the first handheld object is coffee, the latte art image is the image of the coffee surface in the first handheld object. By determining the image of the coffee surface and the target latte art trajectory, not only can the movement trajectories of the first and second handheld objects be clearly known, but also the patterns drawn by the liquid in the first handheld object at each trajectory point can be determined, which can also be understood as the patterns drawn at each moment. This helps the robot to better draw the predetermined latte art pattern based on the patterns on the surface of the liquid in the first handheld object at different trajectory points. When the latte art pattern drawn by the robot at the target trajectory point is inconsistent with the pattern in the latte art image corresponding to the target trajectory point, the degree of inconsistency can be used to determine whether the robot has correctly drawn the latte art. In the case of a latte art error, the robot can redraw the latte art, avoiding the phenomenon of the robot discovering the drawing failure after completion and having to redo it, which wastes time. It also avoids the phenomenon of directly serving the user the poor user experience caused by the failed drawing of liquid.

[0046] As an optional embodiment, after obtaining the target latte art trajectory based on the first correction trajectory data and the second correction trajectory data, the method further includes: obtaining the capacity specification parameters of the first handheld object; and obtaining the third latte art pattern trajectory information based on the target latte art trajectory and the capacity specification parameters.

[0047] In this embodiment, the volume specifications of the first handheld object are obtained. When the liquid in the first handheld object is coffee, the first handheld object can be a coffee cup. That is, when the volume specifications such as the diameter of the coffee cup are not the same, the third latte art pattern trajectory information can be obtained based on the target latte art trajectory and the volume specifications. Because when the volume specifications such as the diameter of the coffee cup are not the same, adjustments need to be made as needed to avoid the problem of milk foam overflowing due to the coffee cup diameter being too small, and also to avoid the problem of milk foam only being displayed in a small area in the middle due to the coffee cup diameter being too large, which is not aesthetically pleasing. By adjusting the third latte art pattern trajectory information as needed based on the target latte art trajectory and the volume specifications, it can be ensured that the predetermined latte art pattern drawn by the robot according to the third latte art pattern trajectory does not have the above-mentioned problems.

[0048] As an optional embodiment, after receiving the trajectory acquisition instruction, the method further includes: in response to the trajectory acquisition instruction, acquiring first movement trajectory data, second movement trajectory data, third movement trajectory data of a predetermined part of the predetermined object, and a second set of image frames including a first handheld object, a second handheld object, and the predetermined part.

[0049] In this embodiment, it is explained that in response to the trajectory acquisition command, in addition to acquiring the first and second movement trajectory data, a third movement trajectory data of a predetermined part of the predetermined object can also be acquired. At this time, the acquired image frame set is the second image frame set including the first handheld object, the second handheld object, and the predetermined part. The third movement trajectory data is the data corresponding to the predetermined part, such as data corresponding to the elbow, joint, etc. The settings can be adaptively configured according to the specific latte art pattern and the actual application and scenario.

[0050] As an optional embodiment, after obtaining the target latte art trajectory based on the first correction trajectory data and the second correction trajectory data, the method further includes: obtaining the target latte art pattern; determining the latte art approximation index between the target latte art pattern and the predetermined latte art pattern; if the latte art approximation index is less than a predetermined threshold, determining the difference pattern between the target latte art pattern and the predetermined latte art pattern; and adjusting the target latte art trajectory corresponding to the predetermined latte art pattern based on the difference pattern to obtain an updated latte art trajectory corresponding to the target latte art pattern.

[0051] In this embodiment, after obtaining the target latte art trajectory, the robot draws a predetermined latte art pattern based on the target trajectory. The target latte art pattern is then acquired, and a similarity index between the target and predetermined latte art patterns is determined. This similarity index is used to judge whether the target and predetermined latte art patterns drawn by the robot are similar. If the similarity index is less than a predetermined threshold, it indicates that the target and predetermined latte art patterns are not very similar and have a certain gap. In this case, a difference pattern between the target and predetermined latte art patterns can be determined. Based on the difference pattern, the target latte art trajectory corresponding to the predetermined latte art pattern is adjusted to obtain an updated latte art trajectory corresponding to the target latte art pattern, so that a latte art pattern more similar to the predetermined latte art pattern can be obtained in the next drawing.

[0052] Based on the above embodiments and optional embodiments, an optional implementation method is provided, which is described in detail below.

[0053] In related technologies, capturing the movements of latte art performers often involves optical motion capture devices or inertial motion capture devices. Optical motion capture devices obtain imaging information by reflecting the images from marker points to cameras at different locations, and then calculate the trajectory of the marker points in space. This requires a relatively high level of indoor space, is easily affected by reflective points, or can result in the marker points being lost due to the latte art performer's movements obscuring the marker points; it is also more expensive. Inertial motion capture devices use inertial sensors, which can measure the movements of the main skeletal parts of the human body in real time, are not limited by light, but have cumulative errors. Both methods of capturing latte art movements have a certain degree of error.

[0054] In view of this, an optional embodiment of the present invention provides a method for determining the latte art trajectory, which can accurately determine the latte art trajectory.

[0055] The principle is to install an inertial measurement unit (IMU), a six-degree-of-freedom sensor, and a camera on the latte art container. The sensors acquire spatial position and posture data of the latte art movements, while the camera periodically performs visual positioning, providing feedback to the sensor devices to correct accumulated errors, thereby obtaining accurate data on the latte art performer's movements. Figure 2 This is a schematic diagram of a latte art performer's latte art movements provided by an optional embodiment of the present invention, as shown below. Figure 2 As shown below, the process of determining the latte art trajectory will be introduced.

[0056] 1. A program used to visually locate a camera within the sensor's precision time scale while the sensor records motion data, compare the deviation between the two sets of data, and obtain accurate trajectory data through an algorithm.

[0057] 2. Fix two sets of sensors and camera devices to the bottom of the latte art pitcher and the bottom of the coffee cup, respectively. At the same time, install sensor devices at six points on the hands, upper arms, and forearms of the latte art performer's two arms. These locations can record the movement trajectory without affecting the latte art performer's latte art.

[0058] 3. Establish a spatial coordinate system and set the initial attitude of the sensor and camera devices;

[0059] 4. The coffee cup and milk pitcher are filled with measured amounts of coffee and milk foam, respectively. The barista performs latte art, and the equipment begins recording.

[0060] 5. During the above steps, the host computer connected to the 8 sets of equipment executes the program described in step 1 to record the trajectory of the cup, hand, and upper and lower arm of the left arm, as well as the trajectory of the cup, hand, and upper and lower arm of the right arm during the latte art performance.

[0061] 6. After the above steps are completed, the eight sets of data recorded by the equipment and corrected for cumulative errors in a timely manner are used to further calibrate the accurate trajectories of the latte art container and cup, including x, y, z, Rx, Ry, and Rz in the coordinate system, by using the relative positions between the equipment.

[0062] It should be noted that by measuring the milk volume in the milk container, the size parameters of the milk container, and the data in the trajectory of the latte art container, one can obtain the trajectory data of the milk container nozzle and the changes in milk flow during the latte art process, which can serve as important reference parameters for latte art data.

[0063] Through the above optional implementation methods, at least the following beneficial effects can be achieved: accurate recording of the latte art movement trajectory is realized, enabling the latte art produced by the robot executing this trajectory data to achieve the artistry of a latte artist. It also effectively solves the problem of inaccurate recording of latte artist movements, thus enabling rapid reproduction of latte art trajectories. This technical solution can quickly reproduce various patterns, with effects comparable to or even surpassing those of a latte artist, and the latte art patterns are stable, with no discrepancies in each production run. This frees up latte artists and allows for a more perfect integration of robots and latte art.

[0064] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that the present invention is not limited to the described order of actions, because according to the present invention, some steps can be performed in other orders or simultaneously. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions and modules involved are not necessarily essential to the present invention.

[0065] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods according to the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods of the various embodiments of the present invention.

[0066] Example 2

[0067] According to an embodiment of the present invention, an apparatus for implementing the above-described method for determining the latte art trajectory is also provided. Figure 3 This is a structural block diagram of the latte art trajectory determining device according to an embodiment of the present invention, as shown below. Figure 3 As shown, the device includes: a receiving module 302, an acquisition module 304, a correction module 306, and a determination module 308. The device will be described in detail below.

[0068] A receiving module 302 is used to receive a trajectory acquisition command, wherein the trajectory acquisition command carries a predetermined latte art pattern, and the trajectory acquisition command is used to acquire the latte art trajectory of a predetermined object drawing the predetermined latte art pattern on a predetermined liquid surface, the predetermined liquid surface being the liquid surface of the first handheld object of the predetermined object; an acquisition module 304 is connected to the receiving module 302, and is used to acquire, in response to the trajectory acquisition command, first movement trajectory data of the first handheld object of the predetermined object, second movement trajectory data of the second handheld object of the predetermined object, and a first image frame set including the first handheld object and the second handheld object; a correction module 306 is connected to the acquisition module 304, and is used to correct the first movement trajectory data and the second movement trajectory data according to the first image frame set to obtain first corrected trajectory data of the first handheld object and second corrected trajectory data of the second handheld object; a determining module 308 is connected to the correction module 306, and is used to obtain the target latte art trajectory according to the first corrected trajectory data and the second corrected trajectory data.

[0069] It should be noted that the receiving module 302, the acquiring module 304, the correction module 306 and the determining module 308 mentioned above correspond to steps S102 to S108 in the method for determining the latte art trajectory. The multiple modules and the corresponding steps are the same in terms of implementation examples and application scenarios, but are not limited to the content disclosed in the above embodiment 1.

[0070] Example 3

[0071] According to another aspect of the present invention, an electronic device is also provided, comprising: a processor; and a memory for storing processor-executable instructions, wherein the processor is configured to execute instructions to implement the latte art trajectory determination method of any of the above embodiments.

[0072] Example 4

[0073] According to another aspect of the present invention, a computer-readable storage medium is also provided, which, when the instructions in the computer-readable storage medium are executed by a processor of an electronic device, enables the electronic device to perform the latte art trajectory determination method described above.

[0074] The sequence numbers of the above embodiments of the present invention are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0075] In the above embodiments of the present invention, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0076] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. The device embodiments described above are merely illustrative; for example, the division of units can be a logical functional division, and in actual implementation, there may be other division methods. For instance, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual coupling, direct coupling, or communication connection may be through some interfaces; the indirect coupling or communication connection between units or modules may be electrical or other forms.

[0077] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0078] Furthermore, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0079] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, read-only memory (ROM), random access memory (RAM), portable hard drives, magnetic disks, or optical disks.

[0080] The above description is only a preferred embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A method of determining a pattern of a pull-up, characterized by, include: Receive trajectory acquisition instruction, wherein the trajectory acquisition instruction carries a predetermined latte art pattern, and the trajectory acquisition instruction is used to acquire the latte art trajectory of a predetermined object drawing the predetermined latte art pattern on a predetermined liquid surface, wherein the predetermined liquid surface is the liquid surface of the first handheld object of the predetermined object; In response to the trajectory acquisition command, the first movement trajectory data of the first handheld object of the predetermined object, the second movement trajectory data of the second handheld object of the predetermined object, and a first image frame set including the first handheld object and the second handheld object are acquired, wherein the trajectory data is acquired by a six-degree-of-freedom spatial sensor located on the corresponding handheld object, and the trajectory data includes the first movement trajectory data and the second movement trajectory data; Based on the first image frame set, the first movement trajectory data and the second movement trajectory data are corrected to obtain the first corrected trajectory data of the first handheld object and the second corrected trajectory data of the second handheld object. The first image frame set carries the frame time of the shooting, and the trajectory data carries the trajectory point time. The frame time and the trajectory point time are mapped, and the three-dimensional spatial data of the trajectory data is corrected based on the two-dimensional image and two-dimensional coordinates of the first image frame set. Based on the first correction trajectory data and the second correction trajectory data, the target latte art trajectory is obtained; The method further includes, after obtaining the target latte art trajectory based on the first and second correction trajectory data, acquiring target flow data corresponding to each trajectory point in the second correction trajectory data, wherein the target flow data is the flow rate of liquid flowing out of the second handheld object of the predetermined object; and obtaining first latte art pattern trajectory information based on the target latte art trajectory and the target flow data corresponding to each trajectory point in the second correction trajectory data, wherein the target flow data includes data on milk flow rate during the latte art process calculated using the milk volume in the latte art container and the tilt angle in the trajectory. The step of obtaining the target latte art trajectory based on the first correction trajectory data and the second correction trajectory data further includes: adjusting based on the capacity specification parameters of the first handheld object and the target latte art trajectory to obtain third latte art pattern trajectory information.

2. The method of claim 1, wherein, The process of acquiring first movement trajectory data of the first handheld object of the predetermined object, second movement trajectory data of the second handheld object of the predetermined object, and a first image frame set including the first handheld object and the second handheld object, in response to the trajectory acquisition command, includes: In response to the trajectory acquisition command, the first initial pose of the first handheld object and the second initial pose of the second handheld object are acquired, as well as an image including the initial poses of the first handheld object and the second handheld object; In the initial pose image, if the pose corresponding to the first handheld object is a first predetermined pose and the pose corresponding to the second handheld object is a second predetermined pose, the first movement trajectory data, the second movement trajectory data, and the first image frame set are acquired.

3. The method according to claim 1, characterized in that, After obtaining the target latte art trajectory based on the first correction trajectory data and the second correction trajectory data, the method further includes: Obtain the latte art drawing image corresponding to each trajectory point in the first correction trajectory data, wherein the latte art drawing image is an image of the liquid surface of the liquid in the first handheld object; Based on the target latte art trajectory and the latte art drawing image corresponding to each trajectory point in the first correction trajectory data, the second latte art pattern trajectory information is obtained.

4. The method according to claim 1, characterized in that, After receiving the trajectory acquisition command, the method further includes: In response to the trajectory acquisition command, the system acquires the first movement trajectory data, the second movement trajectory data, the third movement trajectory data of the predetermined part of the predetermined object, and a second set of image frames including the first handheld object, the second handheld object, and the predetermined part.

5. The method according to any one of claims 1 to 4, characterized in that, After obtaining the target latte art trajectory based on the first correction trajectory data and the second correction trajectory data, the method further includes: Obtain the target latte art pattern; Determine the latte art approximation index between the target latte art pattern and the predetermined latte art pattern; If the latte art approximation index is less than a predetermined threshold, the difference pattern between the target latte art pattern and the predetermined latte art pattern is determined. Based on the difference pattern, the target latte art trajectory corresponding to the predetermined latte art pattern is adjusted to obtain the updated latte art trajectory corresponding to the target latte art pattern.

6. A device for determining the trajectory of latte art, characterized in that, include: A receiving module is used to receive a trajectory acquisition instruction, wherein the trajectory acquisition instruction carries a predetermined latte art pattern, and the trajectory acquisition instruction is used to acquire the latte art trajectory of a predetermined object drawing the predetermined latte art pattern on a predetermined liquid surface, wherein the predetermined liquid surface is the liquid surface of the liquid in the first handheld object of the predetermined object; The acquisition module is configured to, in response to the trajectory acquisition command, acquire the first movement trajectory data of the first handheld object of the predetermined object, the second movement trajectory data of the second handheld object of the predetermined object, and a first image frame set including the first handheld object and the second handheld object, wherein the trajectory data is acquired by a six-degree-of-freedom spatial sensor located on the corresponding handheld object, and the trajectory data includes the first movement trajectory data and the second movement trajectory data; The correction module is used to correct the first motion trajectory data and the second motion trajectory data based on the first image frame set to obtain the first corrected trajectory data of the first handheld object and the second corrected trajectory data of the second handheld object. The first image frame set carries the frame time of the shooting, and the trajectory data carries the trajectory point time. The frame time is mapped to the trajectory point time, and the three-dimensional spatial data of the trajectory data is corrected based on the two-dimensional image and two-dimensional coordinates of the first image frame set. The determining module is used to obtain the target latte art trajectory based on the first correction trajectory data and the second correction trajectory data; The determining module is further configured to acquire target flow data corresponding to each trajectory point in the second correction trajectory data, wherein the target flow data is the flow rate of liquid flowing out of the second handheld object of the predetermined object; based on the target latte art trajectory and the target flow data corresponding to each trajectory point in the second correction trajectory data, the first latte art pattern trajectory information is obtained, wherein the target flow data includes data on milk flow rate during the latte art process calculated using the milk volume in the latte art container and the tilt angle in the trajectory; The device is further configured to, after obtaining the target latte art trajectory based on the first correction trajectory data and the second correction trajectory data, adjust the trajectory based on the capacity specification parameters of the first handheld object and the target latte art trajectory to obtain third latte art pattern trajectory information.

7. An electronic device, characterized in that, include: processor; Memory used to store the processor's executable instructions; The processor is configured to execute the instructions to implement the method for determining the latte art trajectory as described in any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that, When the instructions in the computer-readable storage medium are executed by the processor of the electronic device, the electronic device is able to perform the method for determining the latte art trajectory as described in any one of claims 1 to 5.

Images