Automatic beverage preparation method and system, medium and electronic equipment

Abstract

The invention provides an automatic beverage preparation method and system, a medium and electronic equipment. The automatic beverage preparation method comprises the following steps: obtaining an order instruction; performing real-time acquisition on a working area based on the order instruction to obtain a working area image; performing target detection processing on the working area image to obtain a unique identifier of a target ingredient container; performing online action matching on the unique identifier by using a lookup table mechanism to generate a motion mapping control strategy; driving a mechanical arm to execute a target batching operation action based on the motion mapping control strategy; and performing target batching operation according to the sequence identifier in the order instruction until the order task is completed. By means of the method, the robustness and flexibility of automatic beverage preparation can be improved.

Description

Technical Field

[0001] This application belongs to the field of artificial intelligence technology and relates to a robotic method for automatically preparing beverages, and particularly to an automatic beverage preparation method, system, medium and electronic equipment. Background Technology

[0002] With the development of artificial intelligence and robotics, intelligent robots are increasingly being used in the food industry, extending from industrial production to personalized catering services. The highly repetitive and customized nature of operations in scenarios such as beverage preparation provides an opportunity for fully automated processes. However, existing robot systems still face significant challenges in recognizing diverse ingredients, accurately grasping them, and adapting to dynamic environments. Insufficient visual perception capabilities and the complexity of motion planning have become key bottlenecks restricting the performance improvement of automated food preparation systems. Therefore, how to provide an automated beverage preparation method with low complexity and low computational cost while ensuring high precision and reliability has become one of the urgent problems to be solved in the industry.

[0003] Furthermore, with the increasing demand for personalized beverages, the automated production of freshly made drinks such as coffee, milk tea, and fruit tea has become an important direction for cost reduction and efficiency improvement in the catering industry. However, introducing robotic automation systems into dynamic, open store environments, rather than closed industrial production lines, faces unique technical challenges. Among them, the contradiction between the real-time performance and accuracy of visual perception systems and deployment costs is the core bottleneck restricting the large-scale commercialization of automated beverage preparation systems.

[0004] In beverage preparation scenarios, it is necessary to accurately identify and grasp dozens of different ingredient containers (such as jam, syrup, tapioca pearls, etc.) and add them quickly and precisely according to the order sequence. This requires the vision system to be able to identify the markings on the containers with extremely high accuracy in real time under complex lighting, slight occlusion, and various background interferences. However, existing high-precision deep learning vision models have high computational complexity and a large number of parameters, making them difficult to deploy directly on edge computing devices that are limited by store space, cost, and power consumption. If cloud computing is used, network latency and stability issues cannot meet the real-time requirement of "second-level" dispensing; if local devices with lower computing power are used, model accuracy or speed must be sacrificed, resulting in a high recognition error rate or slow response, which in turn leads to grasping errors, production line stoppages, and seriously affects dispensing efficiency and customer experience.

[0005] Therefore, how to implement a visual perception solution that simultaneously satisfies high-precision recognition, low-latency inference, and low resource consumption on resource-constrained edge computing devices, and how to closely coordinate it with the motion control of robots, has become a key technical problem that must be overcome to achieve a truly usable, reliable, and efficient fully automated system in the specific field of beverage preparation. Existing general solutions often fail to achieve a good balance in this triangular constraint, and a dedicated technical path deeply optimized for this scenario is urgently needed. Summary of the Invention

[0006] The purpose of this application is to provide an automated beverage preparation method, system, medium, and electronic device to improve the robustness and flexibility of automated beverage preparation.

[0007] In a first aspect, this application provides an automated beverage preparation method, comprising: acquiring an order instruction; acquiring a work area in real time based on the order instruction to obtain an image of the work area; performing target detection processing on the work area image to obtain a unique identifier of a target ingredient container, the unique identifier indicating the container corresponding to the target ingredient; performing online action matching on the unique identifier using a lookup table mechanism to generate a motion mapping control strategy, and constructing a lookup table based on the association between container position coordinates and actions; driving a robotic arm to perform a target ingredient operation based on the motion mapping control strategy; and performing the target ingredient operation according to the sequence identifier in the order instruction until the order task is completed.

[0008] In one implementation of the first aspect, the process of performing target detection processing on the work area image to obtain a unique identifier of the target ingredient container includes: using a target detection model to perform visual perception on the work area image to obtain container identifier detection information; and performing confidence filtering processing on the container identifier detection information to obtain the unique identifier.

[0009] In one implementation of the first aspect, the process of using a lookup table mechanism to perform online action matching on the unique identifier to generate a motion mapping control strategy includes: obtaining the container position coordinates based on the unique identifier; obtaining the action primitive sequence corresponding to the unique identifier based on the container position coordinates; and generating the motion mapping control strategy based on the container position coordinates and the action primitive sequence.

[0010] In one implementation of the first aspect, the automatic beverage preparation method further includes: offline calibration of the ingredient container to construct a lookup table, wherein the process of offline calibration of the ingredient container to construct the lookup table includes: offline calibration of the ingredient container to obtain the container position coordinates corresponding to the unique identifier of the ingredient container; offline planning based on the container position coordinates to obtain the action primitive sequence corresponding to the container position coordinates; and constructing the lookup table according to the unique identifier of the ingredient container, the container position coordinates, and the action primitive sequence corresponding to the container position coordinates.

[0011] In one implementation of the first aspect, the process of constructing the target detection model includes: constructing an image dataset based on identification card images in different scenarios; performing lightweight reconstruction of the backbone network of the YOLO model to obtain a lightweight network model; training the lightweight network model using the image dataset to obtain a converged model; and optimizing the converged model to obtain a target detection model.

[0012] In one implementation of the first aspect, the process of lightweighting and reconstructing the backbone network of the YOLO model to obtain a lightweight network model includes: replacing some standard convolutional layers in the backbone network with depthwise separable convolutional layers; combining the depthwise separable convolutional layers with an attention module in multiple levels to construct a feature enhancement module; and integrating the feature enhancement module with a pointwise convolutional layer to obtain the lightweight network model.

[0013] In one implementation of the first aspect, the process of optimizing the converged model to obtain the target detection model includes: performing channel pruning on the converged model to obtain an optimized model; and performing precision quantization on the optimized model to obtain the target detection model.

[0014] Secondly, this application provides an automated beverage preparation system, comprising: an order acquisition module for acquiring order instructions; an image acquisition module for real-time acquisition of a work area based on the order instructions to obtain an image of the work area; a target detection module for performing target detection processing on the work area image to obtain a unique identifier of a target ingredient container, the unique identifier indicating the container corresponding to the target ingredient; a motion strategy acquisition module for online action matching of the unique identifier using a lookup table mechanism to generate a motion mapping control strategy, and constructing a lookup table based on the association between container position coordinates and actions; an action execution module for driving a robotic arm to perform target ingredient operation actions based on the motion mapping control strategy; and performing target ingredient operations according to the sequence identifier in the order instructions until the order task is completed.

[0015] Thirdly, this application provides an electronic device, the electronic device comprising: a memory storing a computer program thereon; and a processor communicatively connected to the memory for executing the computer program to implement the above-described automatic beverage preparation method.

[0016] Fourthly, this application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by an electronic device, implements the above-described automatic beverage preparation method.

[0017] As described above, the automatic beverage preparation method, system, medium, and electronic device described in this application have the following beneficial effects:

[0018] By acquiring images of the work area in real time and performing target detection, the unique identifiers of ingredient containers can be accurately identified. A lookup table mechanism is constructed by directly mapping visual perception to actions, and the robotic arm control strategy is executed online based on action primitive sequences. This eliminates complex real-time motion planning algorithms, reducing system computational complexity, latency, and development costs while ensuring high response speed and high task completion rate. A lightweight network model is constructed by introducing depthwise separable convolutions, combining depthwise separable convolutional layers with attention mechanisms in multiple levels. This ensures the model is lightweight while improving its ability to represent complex features. Simultaneously, channel pruning and model quantization are performed on the model, enabling deployment on edge computing devices while maintaining ultra-high detection accuracy and achieving high-speed real-time inference.

[0019] The lightweight target detection model constructed in this application has achieved a series of key technical effects through synergistic optimization in the specific scenario of automated beverage preparation. While maintaining top-level accuracy, it significantly reduces computational complexity, memory usage, and power consumption, enabling high-precision vision guidance systems to be deployed in edge devices in real-world commercial scenarios at low cost and with low barriers to entry. This provides a practical and feasible technical solution for addressing the triangular bottleneck of "accuracy, speed, and cost" in automated beverage preparation. Attached Figure Description

[0020] Figure 1 The diagram shown is a schematic representation of the automatic beverage preparation method described in this application embodiment.

[0021] Figure 2 The diagram shows a process for generating a motion mapping control strategy as described in an embodiment of this application.

[0022] Figure 3 The diagram shown is a schematic representation of the process of constructing a target detection model as described in an embodiment of this application.

[0023] Figure 4 The diagram shows the process of obtaining a lightweight network model as described in an embodiment of this application.

[0024] Figure 5 The diagram shown is a structural schematic of the depth-separable convolution module described in an embodiment of this application.

[0025] Figure 6 The diagram shown is a structural schematic of the attention module described in an embodiment of this application.

[0026] Figure 7 The diagram shows the process of feature enhancement module processing as described in the embodiments of this application.

[0027] Figure 8 The diagram shown is a structural schematic of the automated beverage preparation system described in an embodiment of this application.

[0028] Figure 9 The diagram shown is a structural schematic of the electronic device described in an embodiment of this application. Detailed Implementation

[0029] The following specific examples illustrate the implementation of this application. Those skilled in the art can easily understand other advantages and effects of this application from the content disclosed in this specification. This application can also be implemented or applied through other different specific embodiments, and various details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of this application. It should be noted that, unless otherwise specified, the following embodiments and features in the embodiments can be combined with each other.

[0030] It should be noted that the illustrations provided in the following embodiments are only schematic representations of the basic concept of this application. Therefore, the drawings only show the components related to this application and are not drawn according to the actual number, shape and size of the components in the actual implementation. In the actual implementation, the form, quantity and proportion of each component can be arbitrarily changed, and the layout of the components may also be more complex.

[0031] With the development of artificial intelligence and robotics, intelligent robots are increasingly being used in the food industry, extending from industrial production to personalized catering services. The highly repetitive and customized operations in scenarios such as beverage preparation provide an opportunity for fully automated processes. However, existing robot systems still face significant challenges in recognizing diverse ingredients, accurately grasping them, and adapting to dynamic environments. Insufficient visual perception capabilities and the complexity of motion planning have become key bottlenecks restricting the performance improvement of automated food preparation systems.

[0032] Existing automated beverage machine systems lack real-time visual feedback, limiting their robustness and flexibility in dynamic environments. Regarding visual perception, while deep learning models, such as the YOLO series, have achieved a good balance in object detection, their standard models still suffer from unsatisfactory real-time performance and resource consumption on edge computing devices. In robot control, traditional methods rely on accurate environment modeling and complex inverse kinematics solutions, resulting in high computational costs and difficulty adapting to rapidly changing operational requirements.

[0033] Therefore, how to provide an automated beverage preparation method with low complexity and low computational cost while ensuring high precision and high reliability has become one of the urgent problems to be solved in the industry.

[0034] In response to at least the above-mentioned problems, the following embodiments of this application provide an automated beverage preparation method.

[0035] The technical solutions in the embodiments of this application will be described in detail below with reference to the accompanying drawings.

[0036] Figure 1 The diagram shown is a schematic representation of an automated beverage preparation method according to an embodiment of this application. Figure 1 As shown, the automatic beverage preparation method includes the following steps S11 to S16.

[0037] Step S11: Obtain order instructions. Each beverage includes multiple ingredients added according to a preset recipe order, each ingredient assigned a unique identifier. The order displays the ingredient identifiers in the order they are added, and the ingredient containers are labeled with corresponding identifiers, thus achieving precise matching between ingredient information and containers. The order instructions include the ingredients required to prepare the beverage and the order in which they are added. Beverages prepared include, but are not limited to, milk tea, coffee, tea drinks, fruit tea, dessert drinks, plant-based protein drinks, and functional beverages.

[0038] Step S12: Real-time image acquisition of the work area based on order instructions. The work area contains multiple ingredient containers; real-time image acquisition of this area allows for the acquisition of identification information for all ingredient containers.

[0039] Step S13: Perform target detection processing on the working area image to obtain a unique identifier for the target ingredient container. The unique identifier is used to indicate the container corresponding to the target ingredient. The unique identifier can be, for example, a numerical identifier, a letter identifier, a specific pattern, a QR code, or a color block. The specific identifier can be adjusted according to the actual scenario, and this application does not limit it.

[0040] In one embodiment of this application, the process of performing target detection processing on the work area image to obtain a unique identifier of the target ingredient container includes: using a target detection model to perform visual perception on the work area image to obtain container identifier detection information; and performing confidence filtering processing on the container identifier detection information to obtain the unique identifier.

[0041] Specifically, the working area image is input into the target detection model, and the identification detection information of all unique identifiers is output synchronously through a single forward propagation. The identification detection information of all unique identifiers is filtered and screened according to a preset confidence threshold, and the identification detection information with low confidence is filtered out to obtain the finally identified unique identifiers.

[0042] Step S14: Use a lookup table mechanism to perform online action matching on the unique identifier to generate a motion mapping control strategy, and construct a lookup table based on the association between container position coordinates and actions.

[0043] Figure 2 This diagram illustrates the process of generating a motion mapping control strategy in one embodiment of this application. Figure 2 As shown, the process of using a lookup table mechanism to perform online action matching on the unique identifier to generate a motion mapping control strategy includes the following steps S21 to S23.

[0044] Step S21: Obtain the container position coordinates based on the unique identifier. In the robot base coordinate system, each finally identified unique identifier corresponds to a predefined three-dimensional coordinate point. The three-dimensional coordinate point is the gripping position of the target ingredient container.

[0045] Step S22: Obtain the action primitive sequence corresponding to the unique identifier based on the container's position coordinates. Each coordinate point... The sequence of pre-programmed, collision-free action primitives is associated with the following actions: moving to a safe point above the container, descending to a gripping pose, activating the gripper to grasp, rising to a safe point, moving to a tilting position above the beverage cup, and releasing the gripper.

[0046] Step S23: Generate the motion mapping control strategy based on the container position coordinates and the action primitive sequence. When the finally identified unique identifier triggers the corresponding identifier's coordinate point... When that happens, the coordinates are called. The associated action primitive sequence is used to generate a motion map control strategy based on the robot's built-in inverse kinematics solver.

[0047] Step S15: Drive the robotic arm to perform the target batching operation based on the motion mapping control strategy.

[0048] For example, the robot is equipped with a multi-axis high-precision robotic arm and an end effector (such as an adaptive two-finger gripper) that can perform physical actions such as grasping, moving and pouring ingredients onto a target ingredient container.

[0049] Furthermore, the execution of the entire sequence of action primitives follows a closed-loop control mechanism, meaning that the triggering of each action is contingent upon the successful completion of the preceding action. Only when the "grab" action is confirmed to be successfully completed will the next "move to tilt position" action be triggered. This mechanism ensures the integrity and high fault tolerance of the entire operation process, fundamentally eliminating the risk of subsequent actions being invalidated due to the failure of preceding actions, thus guaranteeing the reliability and stability of the entire beverage preparation process.

[0050] Step S16: Perform the target ingredient dispensing operation according to the sequence identifier in the order instruction until the order task is completed.

[0051] For example, sequence identification can be implemented in various forms, including but not limited to list indexes and / or timestamp numbers. The list index represents the order of the target ingredients in the current order preparation list. The list index determines the execution order of the target ingredients throughout the entire ingredient preparation process, ensuring the logical coherence of multi-step operations. The timestamp number is generated based on the system clock or external clock signal at the time of order generation. It is a unique time sequence code that can not only distinguish ingredient preparation tasks from different orders but also provide accurate task identification in scenarios such as traceability, concurrent processing, or anomaly recovery.

[0052] Upon receiving an order instruction, the sequence identifier corresponding to the current beverage preparation task is first parsed to obtain the target ingredients and their execution order in the ingredient preparation process. Further, the unique identifier of the container corresponding to each target ingredient is obtained. A target detection model is used to collect images of the work area in real time and identify the unique identifier on the ingredient container. Based on this unique identifier, a lookup table mechanism is triggered, calling the container position coordinates and action primitive sequence corresponding to that unique identifier to drive the robotic arm to complete the grasping and pouring operations of the target ingredients. This achieves precise mapping and closed-loop control from the beverage preparation task to the ingredient preparation operation. Furthermore, the ingredient sequence identifiers specified in the order instruction drive the target ingredient operation one by one. Ingredient operation processing is triggered by continuous identifier triggering. Only after the action primitive sequence corresponding to the current identifier is fully executed and the ingredient addition operation is successfully completed will the ingredient operation of the next identifier in the subsequent instruction be triggered, thus starting the target ingredient process corresponding to the next identifier. This action triggering mechanism continues to execute until all ingredient identifiers in a beverage order have been executed in sequence, marking the completion of the order task.

[0053] In one embodiment of this application, the automatic beverage preparation method further includes: offline calibration of the ingredient container to construct a lookup table, wherein the process of offline calibration of the ingredient container to construct the lookup table includes steps S31 to S33.

[0054] Step S31: Perform offline calibration on the ingredient container to obtain the container position coordinates that uniquely identify the corresponding ingredient container.

[0055] Step S32: Perform offline planning based on the container position coordinates to obtain the action primitive sequence corresponding to the container position coordinates.

[0056] Step S33: Construct the lookup table based on the unique identifier of the ingredient container, the container position coordinates, and the action primitive sequence corresponding to the container position coordinates.

[0057] For example, the dispensing containers are calibrated offline once before system deployment. Offline calibration determines the precise three-dimensional coordinates of the target dispensing container corresponding to each unique identifier in the robot's base coordinate system. For each ingredient container, the three-dimensional coordinates are... Perform pre-planning and pre-programming of motion sequences, pre-test a set of collision-free, smooth motion primitive sequences and store them in a lookup table. An example motion primitive sequence is "move to the safe point above the container - descend to the gripping pose - activate the gripper to grip - rise to the safe point - move to the tilting position above the beverage cup - release the gripper".

[0058] After offline calibration, the system can enter the online automated beverage preparation mode. When the target detection model identifies the unique identifier "3" in the working area image, it retrieves the corresponding three-dimensional spatial coordinates P_3 and the pre-planned sequence of motion primitives strongly associated with the location of the ingredient from a pre-built lookup table based on the identified number "3". Subsequently, this sequence of motion is sent to the robotic arm controller. The inverse kinematics solver integrated into the controller accurately converts the target path in Cartesian space into the motion angle commands required by each joint, ultimately driving the robotic arm to smoothly and accurately execute the target operation of "adding target ingredient 3" at a preset speed and precision.

[0059] The online operation process is entirely based on a pre-defined sequence of motion primitives, eliminating the need for any real-time online trajectory planning calculations. This achieves millisecond-level rapid response, ensuring the continuity and efficiency of adding ingredients multiple times in the preparation of a beverage order. Simultaneously, the pre-planned sequence of motion primitives guarantees the smoothness and safety of each robotic arm movement, fundamentally avoiding path jitter or singularity issues that may arise from online planning, making each ingredient dispensing operation precise and stable.

[0060] Figure 3 This diagram illustrates the process of constructing a target detection model in one embodiment of this application. Figure 3 As shown, the process of constructing the target detection model includes the following steps S41 to S44.

[0061] Step S41: Construct an image dataset based on the identification card images in different scenarios.

[0062] For example, a custom dataset is constructed containing images of identification cards under different lighting conditions, angles, occlusions (less than 15%), and complex backgrounds, with a capacity of, for example, 10,000 images.

[0063] Step S42: Perform lightweight reconstruction of the backbone network of the YOLO model to obtain a lightweight network model.

[0064] Please see Figure 4 The process of lightweighting and reconstructing the backbone network of the YOLO model to obtain a lightweight network model includes the following steps S51 to S53.

[0065] Step S51: Replace some standard convolutional layers in the backbone network with depthwise separable convolutional layers. The network architecture of the depthwise separable convolutional module can be found in [reference needed]. Figure 5 The input image features are fed into a depthwise separable convolutional module, sequentially passing through depthwise separable convolutional layers, batch normalization, SILU activation function, pointwise convolutional layers, and secondary batch normalization to output features. Replacing some standard convolutional networks in the YOLOv5s backbone network with depthwise separable convolutional layers can significantly reduce the number of parameters and computational cost. Step S52: The depthwise separable convolutional layers are combined with an attention module in multiple stages to construct a feature enhancement module.

[0066] Please see Figure 6 The attention module consists of a global average pooling layer, a 1×1×C feature compression layer, a fully connected layer FC1, a ReLU activation function layer, a fully connected layer FC2, a Sigmoid activation function layer, and channel weight adjustment. The attention module, through multi-layer combinations, constitutes the CBAM (Convolutional Block Attention Module) attention mechanism, where the input features are sequentially processed by multiple SE attention modules to obtain the output features.

[0067] An attention module (such as the SE attention module) is introduced after the depthwise separable convolutional layer, and the two are stacked in multiple levels to construct a deeper feature extraction network. The attention module compresses the feature map through global average pooling and learns the weights of each channel through two fully connected layers to incentivize important feature channels and suppress secondary channels. In this process, the attention module can accurately capture the dependencies between channels, dynamically weight the feature map output by the depthwise separable convolution, effectively suppress background noise and highlight key digit region information, enhancing the model's ability to perceive key feature channels for unique identifiers. Through repeated feature refinement and enhancement via multi-level stacking, the model's sensitivity to and representation ability of digit details is significantly improved, enhancing the model's extraction effect on digit features. In particular, common beverage preparation environments often contain complex backgrounds (such as other containers on the workbench, equipment shadows, lighting changes, etc.), and these interfering factors can easily affect the accurate identification of unique identifiers. By using the attention module to weight and compress the feature map and learn the importance weights of each channel, it can dynamically focus on the key features of the digit region and effectively suppress interference from irrelevant background information. This means that in actual operation, even if the ingredient container is under uneven lighting or slightly obscured, the model can still accurately locate the unique identifier, ensuring the reliability of visual recognition.

[0068] Through multi-level stacking, attention modules and depthwise separable convolutional layers are alternately combined to form a hierarchical feature extraction mechanism. Depthwise separable convolutions are responsible for efficiently extracting basic features, while the attention modules weight and enhance features at each layer based on channel dimensions, enabling the model to continuously highlight number-related information flow as it propagates layer by layer. This iterative enhancement mechanism significantly improves the model's sensitivity to numerical details—for example, when number cards are slightly worn, tilted, or blurry, the model can still accurately distinguish easily confused numbers such as "6" and "8," or "1" and "7," avoiding errors in ingredient grabbing due to misidentification, improving the success rate of visual recognition, and ensuring low-latency coordination between robotic arm movements and visual feedback.

[0069] Step S53: Integrate the feature enhancement module with the pointwise convolutional layer to obtain the lightweight network model.

[0070] By utilizing pointwise convolutional layers to deeply fuse the input image data and the feature maps output by the feature enhancement module, the low-level information of the original input is effectively integrated with the enhanced deep features. Furthermore, adaptive weighting and information recombination between feature channels are performed through linear combinations of pointwise convolutions, ensuring both the lightweight nature of the network model and maximizing the retention of key feature information, thus enhancing the lightweight network model's ability to represent complex features. For example, the feature processing process using the lightweight network model includes the following stages: First, the working region image is input into the LightC3 module of the lightweight network model, and the first-stage features are output after processing. Then, the first-stage features are input into the feature enhancement module for further extraction, outputting the second-stage features. The second-stage features are input into the feature enhancement module again for repeated processing, sequentially generating the third-stage and fourth-stage features. The fourth-stage features undergo feature matrix transformation to obtain transformed features, which are then subjected to feature decoding, feature normalization, batch standardization, and filtering processes, finally outputting the enhanced features.

[0071] Please see Figure 7 The process of inputting the first-stage features into the feature enhancement module for feature extraction and outputting the second-stage features includes:

[0072] (1) Perform 1×1 convolution dimensionality reduction on the first stage features, and then split the resulting feature map to obtain the first feature.

[0073] (2) The features after dimensionality reduction by 1×1 convolution are input into the attention module. After further feature processing by 3×3 depthwise separable convolution and the attention module, the features are then increased in dimensionality by 1×1 convolution. The residuals of the increased features are added to the original input to obtain the second feature. (3) The first feature and the second feature are concatenated. The concatenated features are further processed by the attention module. Finally, the number of channels is adjusted by 1×1 convolution to output the second-stage feature.

[0074] Step S43: Train the lightweight network model using the image dataset to obtain a converged model.

[0075] For example, a custom dataset is divided into a training set, a validation set, and a test set according to a preset ratio (e.g., 8:1:1). The lightweight network model is iteratively trained using the training set. During training, the lightweight network model continuously adjusts its internal parameters through a single forward propagation to minimize the loss function. When the performance metrics (e.g., accuracy, loss value) of the lightweight network model on the validation set stabilize and no longer significantly improve or decrease, it indicates that the lightweight network model training has reached convergence, and a converged model is obtained.

[0076] Furthermore, the lightweight network model is implemented based on the PyTorch framework and trained for 300 epochs using the SGD (Stochastic Gradient Descent) optimizer. The input images in the training set directly output unique identifier detection results after a single forward propagation of the lightweight model. The converged model was tested on the test set, achieving 99.6% mAP@0.5 (mean Average Precision at 0.5), and reaching an inference speed of 35 FPS (35 Frames Per Second) on the Jetson Nano edge device, perfectly balancing accuracy and real-time performance. Performance comparisons of different vision models on the test set are shown in Table 1.

[0077] Table 1. Performance comparison results of different visual models on the test set.

[0078] In automated beverage preparation scenarios, robots need to identify unique identifiers on multiple ingredient containers in real time to accurately grasp and pour the corresponding ingredients. This process places extremely high demands on the model's inference speed and resource consumption, especially when deployed on edge computing devices (such as the NVIDIA Jetson Nano). By introducing depthwise separable convolutional layers into the YOLOv5s backbone network, the system can significantly reduce the number of model parameters and computational burden without sacrificing detection accuracy. Depthwise separable convolution decomposes standard convolution into depthwise convolution and pointwise convolution, greatly reducing the complexity of each layer's computation, allowing the model to run stably at a high frame rate of 35 FPS on the Jetson Nano edge device. This also means that in actual operation, whenever the robotic arm prepares to grasp an ingredient container, the target detection model can complete the identification and localization of the unique identifier with almost no delay, ensuring a rapid response to subsequent actions.

[0079] Step S44: Optimize the convergence model to obtain the target detection model.

[0080] In one embodiment of this application, the process of optimizing the convergence model to obtain the target detection model includes the following steps S61 to S62.

[0081] Step S61: Perform channel pruning on the converged model to obtain an optimized model. Channel pruning is used to prune channels according to their importance, removing unimportant channels from the converged model to obtain the optimized model.

[0082] For example, for a converged model that has been trained, the importance of 64 channels in a certain convolutional layer is quantitatively evaluated using the L1 norm. The evaluation results show that the absolute values ​​of the weights of 19 channels are low, and their contribution to the final output of the model is weak. Therefore, these 19 redundant channels and their related connections are pruned as a whole, retaining only the 45 key channels.

[0083] Channel pruning is performed on the converged model. By calculating the weight or gradient of each channel, the importance of each channel in feature extraction is evaluated. Redundant channels with low contribution to model performance are removed according to a preset pruning ratio or channel importance threshold, and the remaining channels are reconnected to ensure the continuity and effectiveness of the network structure. Removing redundant computational paths through channel pruning significantly reduces the model's computational complexity and number of parameters, achieving model lightweighting, reducing memory consumption, and making it easier to deploy on resource-constrained edge devices. It also provides a better foundation for subsequent quantization processing.

[0084] Step S62: Perform precision quantization on the optimized model to obtain the target detection model. Precision quantization is performed on the optimized model by quantizing the FP32 weights into INT8 format. This quantization reduces the number of parameters in the optimized model to approximately 2.5M, a 65% reduction compared to the traditional YOLOv5s (7.2M).

[0085] Specifically, precision quantization is performed on the optimized model, converting the weights and activation values, originally represented as 32-bit floating-point numbers (FP32), into 8-bit integer (INT8) format. This ensures that the quantized model maintains high precision while compressing the parameter format, significantly reducing the model's parameter storage requirements. Precision quantization reduces the model's parameter size to approximately 2.5M, a 65% reduction compared to traditional YOLOv5s (7.2M). INT8 operations offer acceleration advantages on specific hardware (such as NPUs and DSPs), significantly improving the model's inference speed and enabling it to meet the demands of real-time applications.

[0086] Channel pruning and precision quantization were performed on the convergence model, significantly reducing the number of parameters in the object detection model (from 7.2M to approximately 2.5M). This directly reduced the memory usage and computing power consumption of the edge computing device during operation. This means that in continuous, high-intensity beverage preparation tasks, the object detection model for beverage preparation can maintain stable operation for extended periods, avoiding inference delays or system lags caused by resource overload, and ensuring a rapid response in the preparation process of each beverage.

[0087] By using pruning to remove redundant computational paths, the forward propagation speed of the model is further improved (reaching 35 FPS on Jetson Nano), providing more time margin for the robotic arm's real-time grasping. When the target detection model identifies unique identifiers at a higher frame rate, the system can trigger the corresponding action sequence earlier, thereby compressing the average time of a single task to less than 4.2 seconds and improving the overall beverage dispensing efficiency.

[0088] The pruned model is quantized, making it easier to deploy in power-constrained commercial devices. This lowers the hardware threshold and operating costs, and provides a foundation for a more compact structure and less accuracy loss. As a result, the final deployed model can be adapted to embedded platforms with a smaller size while maintaining a high accuracy of 99.6%, enabling a smooth migration from laboratory to actual beverage preparation scenarios.

[0089] By employing lightweight deep convolutional layers, multi-level combinations of deep convolution and attention modules, channel pruning, and model quantization, the constructed object detection model can be efficiently deployed on resource-constrained edge devices. This provides highly reliable, low-latency visual guidance for beverage preparation robots in dynamic, fast-paced real-world scenarios. For example, the optimized object detection model underwent 100 consecutive end-to-end task tests. Each task randomly assigned a target number, requiring the completion of the entire process from recognition to successful pouring. The overall performance test results are shown in Table 2.

[0090] Evaluation Metrics Total number of tasks 100 Number of successful visual recognition attempts 100 Number of successful grabs by the robotic arm 98 Final number of task completions 97 Grasping success rate 98% Task completion rate 97% Average task time consumption 4.2 ± 0.3 seconds

[0091] Table 2 Overall performance test results

[0092] The following will provide a detailed description of the automated beverage preparation method provided in this application through a specific example. It should be noted that the content of this example is only for explaining and illustrating the automated beverage preparation method provided in this application, and is not intended to limit the scope of protection of this application in any way. In specific applications, corresponding steps can be added or deleted based on this example according to actual needs. The automated beverage preparation method in this example includes the following steps S100 to S400.

[0093] Step S100: Obtain beverage order instructions and trigger the acquisition of images in the work area based on the beverage order instructions.

[0094] Step S200: The working area image is processed using a target detection model to identify and output the unique identifier on the target ingredient container.

[0095] Step S300: Based on the identified unique identifier, the three-dimensional coordinate points of the predefined target ingredient container are matched using a lookup table to obtain the corresponding action primitive sequence.

[0096] Step S400: Based on the motion primitive sequence, a multi-axis high-precision robotic arm and end effector are used to grasp, move or tilt the target ingredients until all ingredients on the beverage order are completed.

[0097] It should be noted that the labels S100 to S400 are only used to indicate different steps, and not to restrict the execution order of these steps.

[0098] In summary, this application directly maps visual perception to actions through offline calibration and converts the identification results into a pre-programmed reliable action sequence by constructing a lookup table mechanism. When a beverage order needs to be executed, online execution is performed based on the lookup table, abandoning complex real-time motion planning algorithms. This reduces the system's computational complexity, latency, and development costs while ensuring high response speed and high task completion rate. Furthermore, this application performs deep lightweight optimization on the YOLOv5s model by introducing depthwise separable convolutions and combining depthwise separable convolutional layers with attention mechanisms in multiple levels. This ensures the model is lightweight while improving its ability to represent complex features. Simultaneously, channel pruning and model quantization are performed on the model, enabling deployment on edge computing devices while maintaining ultra-high detection accuracy and achieving high-speed real-time inference. Finally, this application validates the object detection model through over 100 consecutive end-to-end task tests, achieving a 98% capture success rate and a 97% task completion rate, with an average single task time of only 4.2 seconds, demonstrating the high reliability and efficiency of this method.

[0099] The scope of protection for the automatic beverage preparation method described in this application is not limited to the order of steps listed in this application. Any solution implemented by adding, subtracting, or replacing steps in the prior art based on the principles of this application is included within the scope of protection of this application.

[0100] This application also provides an automated beverage preparation device. The automated beverage preparation device includes a vision unit, a computing unit, and an execution unit. Ten ingredient containers labeled with different number cards (e.g., 0-9) are placed on a standardized simulated beverage preparation workbench. The vision unit is fixed directly above the work area to acquire real-time images of the work area. The vision unit uses a Hikvision MV-EB435i industrial camera, and the real-time image resolution of the work area can reach 1280x720. The computing unit is deployed on an edge device, and its core algorithm is deployed on an NVIDIA Jetson Nano development board. The training and development verification of the computing unit algorithm can be performed on a computer equipped with an Intel Core i7-10700K CPU and an NVIDIA GeForce RTX 3060 GPU. The execution unit uses a 6-axis high-precision desktop robotic arm with an adaptive two-finger gripper at its end.

[0101] This application also provides an automatic beverage preparation system, which can implement the automatic beverage preparation method described in this application. However, the implementation device of the automatic beverage preparation method described in this application includes, but is not limited to, the structure of the automatic beverage preparation system listed in this embodiment. All structural modifications and substitutions of the prior art made based on the principles of this application are included within the protection scope of this application.

[0102] Figure 8 The diagram shown is a structural schematic of an automated beverage preparation system according to an embodiment of this application. Figure 8 As shown, the automated beverage preparation system 1 includes: an order acquisition module 11, an image acquisition module 12, a target detection module 13, a motion strategy acquisition module 14, and an action execution module 15. The order acquisition module 11 acquires order instructions. The image acquisition module 12 acquires images of the work area in real time based on the order instructions. The target detection module 13 performs target detection processing on the work area images to obtain unique identifiers for target ingredient containers, which indicate the containers corresponding to the target ingredients. The motion strategy acquisition module 14 uses a lookup table mechanism to perform online action matching on the unique identifiers to generate a motion mapping control strategy, constructing a lookup table based on the association between container position coordinates and actions. The action execution module 15 drives a robotic arm to perform target ingredient operation actions based on the motion mapping control strategy; it performs target ingredient operations according to the sequence identifiers in the order instructions until the order task is completed.

[0103] It should be noted that, Figure 8 The modules in the automated beverage preparation system 1 shown are... Figure 1 The steps in the automated beverage preparation method correspond one-to-one, and will not be elaborated here.

[0104] In the embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, or methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative. For instance, the division of modules / units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple modules or units may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection of apparatuses or modules or units may be electrical, mechanical, or other forms.

[0105] The modules / units described as separate components may or may not be physically separate. The components shown as modules / units may or may not be physical modules; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules / units can be selected to achieve the objectives of the embodiments of this application, depending on actual needs. For example, the functional modules / units in the various embodiments of this application may be integrated into one processing module, or each module / unit may exist physically separately, or two or more modules / units may be integrated into one module / unit.

[0106] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0107] This application also provides a computer-readable storage medium storing a computer program thereon. When executed by a processor, the computer program implements the automatic beverage preparation method provided in this application. Those skilled in the art will understand that all or part of the steps in the methods of the above embodiments can be implemented by a program instructing a processor. The program can be stored in a computer-readable storage medium, which is a non-transitory medium, such as random access memory, read-only memory, flash memory, hard disk, solid-state hard disk, magnetic tape, floppy disk, optical disk, and any combination thereof. The above storage medium can be any available medium accessible to a computer or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., digital video disc (DVD)), or a semiconductor medium (e.g., solid-state disk (SSD)).

[0108] This application embodiment may also provide an electronic device. Figure 9 The diagram shown is a structural schematic of an electronic device 200 according to an embodiment of this application. Figure 9 As shown, in this embodiment, the electronic device 200 includes a memory 201 and a processor 202.

[0109] The memory 201 is used to store computer programs. In some possible implementations, the memory 201 may include various media capable of storing program code, such as ROM, RAM, magnetic disk, USB flash drive, memory card, or optical disk.

[0110] In this embodiment, memory 201 may include a computer system readable medium in the form of volatile memory, such as RAM and / or cache memory. Electronic device 200 may further include other removable / non-removable, volatile / non-volatile computer system storage media. Memory 201 may include at least one program product having a set (e.g., at least one) of program modules configured to perform the functions of the embodiments of this application.

[0111] The processor 202 is connected to the memory 201 and is used to execute the computer program stored in the memory 201 so that the electronic device 200 performs an automatic beverage preparation method.

[0112] For example, processor 202 may be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc. In other embodiments, processor 202 may also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0113] In some implementations, the electronic device 200 provided in this application embodiment may further include a display 203. The display 203 is communicatively connected to the memory 201 and the processor 202, and is used to display a graphical user interface (GUI) related to the automated beverage preparation method.

[0114] In this embodiment, the display 203 may include a display screen (display panel). In some implementations, the display panel may be configured using a liquid crystal display (LCD), an organic light-emitting diode (OLED), or other similar forms. Furthermore, the display 203 may also be a touch panel (touchscreen, touch screen), which may include a display screen and a touch-sensitive surface. When the touch-sensitive surface detects a touch operation on or near it, it transmits the information to the processor 202 to determine the type of touch event. Subsequently, the processor 202 provides corresponding visual output on the display device based on the type of touch event.

[0115] The descriptions of the processes or structures corresponding to the above figures each have their own emphasis. For parts of a process or structure that are not described in detail, please refer to the relevant descriptions of other processes or structures.

[0116] The above embodiments are merely illustrative of the principles and effects of this application and are not intended to limit this application. Any person skilled in the art can modify or alter the above embodiments without departing from the spirit and scope of this application. Therefore, all equivalent modifications or alterations made by those skilled in the art without departing from the spirit and technical concept disclosed in this application should still be covered by the claims of this application.

Claims

1. A method of automatic beverage preparation, characterized in that, The automatic beverage preparation method comprises: obtaining an order instruction; real-time acquisition of a work area based on the order instruction to obtain a work area image; target detection processing of the work area image to obtain a unique identifier of a target ingredient container, the unique identifier being used to indicate a container corresponding to a target ingredient; online action matching of the unique identifier using a lookup table mechanism to generate a motion mapping control strategy, a lookup table being constructed based on the association between container position coordinates and actions; driving a mechanical arm to perform a target ingredient operation action based on the motion mapping control strategy; target ingredient operation according to a sequence identifier in the order instruction until the order task is completed.

2. The automatic beverage preparation method according to claim 1, characterized in that, The process of target detection processing of the work area image to obtain a unique identifier of a target ingredient container comprises: visual perception of the work area image using a target detection model to obtain container identifier detection information; confidence screening processing of the container identifier detection information to obtain the unique identifier.

3. The automatic beverage preparation method according to claim 1, characterized in that, The process of online action matching of the unique identifier using a lookup table mechanism to generate a motion mapping control strategy comprises: obtaining container position coordinates based on the unique identifier; obtaining an action primitive sequence corresponding to the unique identifier based on the container position coordinates; generating the motion mapping control strategy according to the container position coordinates and the action primitive sequence.

4. The automatic beverage preparation method according to claim 1, characterized in that, The automatic beverage preparation method further comprises offline calibration of an ingredient container to construct a lookup table, wherein the process of offline calibration of an ingredient container to construct a lookup table comprises: offline calibration of the ingredient container to obtain container position coordinates of an ingredient container corresponding to a unique identifier; offline planning based on the container position coordinates to obtain an action primitive sequence corresponding to the container position coordinates; constructing the lookup table according to the unique identifier of the ingredient container, the container position coordinates and the action primitive sequence corresponding to the container position coordinates.

5. The automatic beverage preparation method according to claim 1, characterized in that, The process of constructing the target detection model comprises: constructing an image dataset according to identifier card images under different scenarios; lightweight reconstruction of a backbone network of a YOLO model to obtain a lightweight network model; training the lightweight network model using the image dataset to obtain a converged model; optimization processing of the converged model to obtain a target detection model.

6. The automatic beverage preparation method according to claim 5, characterized in that, The process of lightweight reconstruction of a backbone network of a YOLO model to obtain a lightweight network model comprises: replacing part of the standard convolution layers in the backbone network with depth separable convolution layers; multistage combination of the depth separable convolution layers and attention modules to construct a feature enhancement module; integration of the feature enhancement module and point-wise convolution layers to obtain the lightweight network model.

7. The automatic beverage preparation method according to claim 5, characterized in that, The process of optimization processing of the converged model to obtain a target detection model comprises: channel pruning processing of the converged model to obtain an optimized model; precision quantization processing of the optimized model to obtain the target detection model.

8. An automatic beverage preparation system, characterized in that, The automatic beverage preparation system comprises: an order obtaining module that obtains an order instruction; An image acquisition module acquires the working area in real time based on the order instruction to obtain a working area image; A target detection module performs target detection processing on the working area image to obtain a unique identifier of a target ingredient container, the unique identifier being used to indicate a container corresponding to a target ingredient; A motion strategy acquisition module performs online action matching on the unique identifier using a lookup table mechanism to generate a motion mapping control strategy, and constructs a lookup table based on the association between container position coordinates and actions; An action execution module drives a mechanical arm to perform a target ingredient operation action based on the motion mapping control strategy; and performs target ingredient operations according to the sequence identifier in the order instruction until the order task is completed.

9. An electronic device, comprising: The electronic device comprises: a memory having a computer program stored thereon; a processor in communication with the memory, configured to execute the computer program to implement the automatic beverage preparation method of any one of claims 1 to 7.

10. A computer-readable storage medium having stored thereon a computer program, characterized in that, The program is executed by the electronic device to implement the automatic beverage preparation method of any one of claims 1 to 7.

Images