Tea leaf recognition and packaging method and device applied to tea leaf vending machine

Abstract

Embodiments of the present disclosure disclose a tea leaf recognition and packaging method and device applied to a tea leaf vending machine. A specific embodiment of the method comprises: obtaining a replenishment information set according to tea leaf inventory information, tea leaf storage cost and tea leaf replenishment cost corresponding to each tea leaf category sold by a tea leaf vending machine to be replenished; releasing corresponding tea leaves to a turntable according to the replenishment information set; collecting hyperspectral images of the tea leaves on the turntable to obtain tea leaf hyperspectral image information; performing fine-grained tea leaf grade recognition on the tea leaf hyperspectral image information to obtain tea leaf grade information; collecting tea leaf weight information; performing volume recognition on the tea leaves to obtain tea leaf volume information; packaging the tea leaves according to the tea leaf grade information and the tea leaf volume information to obtain packaged tea leaves; and conveying the packaged tea leaves to the tea leaf vending machine. The embodiment can improve the overall packaging efficiency of the tea leaves.

Description

Technical Field

[0001] The embodiments disclosed herein relate to the field of vending machine technology, specifically to a tea identification and packaging method and apparatus applied to a tea vending machine. Background Technology

[0002] Tea vending machines are widely popular due to their convenience. The restocking process for tea vending machines typically includes two key steps: tea identification and packaging. Currently, the common method for identifying and packaging tea is to use chemical analysis to determine the type of tea, followed by packaging it in fixed sizes.

[0003] However, when using the above methods, a common technical problem is that replenishment decisions fail to comprehensively weigh the relationship between tea inventory costs, replenishment costs, and equipment capacity, which can easily lead to tea stockpiling or excessively frequent replenishment. At the same time, because tea can deteriorate in quality during storage, and the existing packaging process does not introduce a real-time grading mechanism, it is impossible to grade the tea according to its current actual quality, resulting in a discrepancy between the quality of the packaged tea and the labeled grade, thereby reducing overall packaging efficiency. Summary of the Invention

[0004] The summary portion of this disclosure is intended to provide a brief overview of the concepts, which will be described in detail in the detailed description portion. This summary portion is not intended to identify key or essential features of the claimed technical solutions, nor is it intended to limit the scope of the claimed technical solutions.

[0005] Some embodiments of this disclosure propose a method and apparatus for tea identification and packaging applied to tea vending machines to solve the technical problems mentioned in the background section above.

[0006] In a first aspect, some embodiments of this disclosure provide a tea identification and packaging method applied to a tea vending machine. The method includes: generating replenishment information based on tea inventory information, tea storage costs, and tea replenishment costs corresponding to each tea category in a tea category set sold by the tea vending machine to be replenished, thus obtaining a replenishment information set; releasing the corresponding tea onto a turntable via a feeder according to the replenishment information set; acquiring hyperspectral images of the tea on the turntable using a hyperspectral imager to obtain hyperspectral image information of the tea; performing fine-grained tea grade identification on the hyperspectral image information of the tea based on a preset tea grade classification model to obtain tea grade information; in response to determining that the tea grade information meets preset replenishment grade conditions, acquiring tea weight information via a weight sensor at the bottom of the turntable; in response to determining that the tea weight information meets preset replenishment weight conditions, performing volume identification on the tea using a camera configured on a robotic arm to obtain tea volume information; packaging the tea according to the tea grade information and the tea volume information to obtain packaged tea; and conveying the packaged tea to the tea vending machine to be replenished.

[0007] Secondly, some embodiments of this disclosure provide a tea identification and packaging device for a tea vending machine. The device includes: a generation unit configured to generate replenishment information based on tea inventory information, tea storage cost, and tea replenishment cost corresponding to each tea category in a tea category set sold by the tea vending machine to be replenished, thereby obtaining a replenishment information set; a release unit configured to release the corresponding tea onto a turntable via a feeder according to the replenishment information set; a hyperspectral image acquisition unit configured to acquire hyperspectral images of the tea on the turntable using a hyperspectral imager, thereby obtaining hyperspectral image information of the tea; and a fine-grained tea grade identification unit configured to identify tea grades based on preset tea grades. A classification model performs fine-grained tea grade identification on the hyperspectral image information of the tea leaves to obtain tea grade information; a data acquisition unit is configured to acquire tea weight information through a weight sensor at the bottom of the turntable in response to determining that the tea grade information meets a preset replenishment grade condition; a volume recognition unit is configured to perform volume recognition on the tea leaves through a camera mounted on a robotic arm in response to determining that the tea weight information meets a preset replenishment weight condition to obtain tea volume information; and a packaging unit is configured to package the tea leaves according to the tea grade information and the tea volume information to obtain packaged tea leaves, and to transport the packaged tea leaves to the tea vending machine awaiting replenishment.

[0008] Thirdly, some embodiments of this disclosure provide an electronic device, including: one or more processors; and a storage device having one or more programs stored thereon, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement the method described in any implementation of the first aspect above.

[0009] Fourthly, some embodiments of this disclosure provide a computer-readable medium having a computer program stored thereon, wherein the program, when executed by a processor, implements the method described in any of the implementations of the first aspect above.

[0010] The various embodiments of this disclosure have the following beneficial effects: the overall packaging efficiency is improved through the tea identification and packaging method applied to tea vending machines according to some embodiments of this disclosure. Specifically, the reason for the insufficient overall packaging efficiency is that the replenishment decision fails to comprehensively balance the relationship between tea inventory costs, replenishment costs, and equipment capacity, which easily leads to tea stockpiling or excessively frequent replenishment; at the same time, since tea quality deteriorates during storage, and the existing packaging process does not introduce a real-time grading mechanism, it is impossible to grade the tea according to its current actual quality, resulting in the quality of packaged tea not matching the labeled grade. Based on this, the tea identification and packaging method applied to tea vending machines according to some embodiments of this disclosure first generates replenishment information based on the tea inventory information, tea storage costs, and tea replenishment costs corresponding to each tea category in the tea category set sold by the tea vending machine to be replenished, thus obtaining a replenishment information set. Therefore, the overall storage and replenishment costs can be minimized without stockouts. Secondly, based on the above replenishment information set, the corresponding tea is released onto the turntable by a feeder. Next, a hyperspectral imager is used to acquire hyperspectral images of the tea leaves on the turntable, obtaining hyperspectral image information of the tea leaves. This non-contact, non-destructive hyperspectral image acquisition of the tea leaves yields hyperspectral image data containing rich spectral and spatial information. Then, based on a preset tea grading model, fine-grained tea grading is performed on the hyperspectral image information to obtain tea grading information. This allows for real-time output of accurate tea grading information, dynamically reflecting potential quality changes during storage. Following this, in response to determining that the tea grading information meets preset replenishment grading conditions, a weight sensor at the bottom of the turntable acquires the weight information of the tea leaves. Then, in response to determining that the tea weight information meets preset replenishment weight conditions, a camera mounted on the robotic arm performs volume recognition on the tea leaves, obtaining tea volume information. Since different teas have varying degrees of fluffiness, relying solely on weight for packaging can easily lead to an increased breakage rate; volume recognition improves the tea packaging effect. Finally, based on the aforementioned tea grade and volume information, the tea is packaged to obtain packaged tea, which is then transported to the aforementioned tea vending machine awaiting restocking. Packaging based on both tea grade and volume not only ensures consistency between product quality and label grade but also significantly improves packaging efficiency. Attached Figure Description

[0011] The above and other features, advantages, and aspects of the embodiments of this disclosure will become more apparent from the accompanying drawings and the following detailed description. Throughout the drawings, the same or similar reference numerals denote the same or similar elements. It should be understood that the drawings are schematic, and elements are not necessarily drawn to scale.

[0012] Figure 1 This is a schematic diagram illustrating an application scenario of the tea identification and packaging method for a tea vending machine, which is one of the embodiments of this disclosure.

[0013] Figure 2 This is a flowchart of some embodiments of a tea identification and packaging method applied to a tea vending machine according to the present disclosure;

[0014] Figure 3 This is a structural diagram of the initial tea grading classification model for a tea identification and packaging method applied to a tea vending machine, based on the present disclosure.

[0015] Figure 4 This is an image acquisition flowchart of a tea identification and packaging method for tea vending machines according to the present disclosure;

[0016] Figure 5 These are schematic diagrams illustrating the structure of some embodiments of a tea identification and packaging device applied to a tea vending machine according to the present disclosure;

[0017] Figure 6 This is a schematic diagram of the structure of an electronic device suitable for implementing some embodiments of the present disclosure. Detailed Implementation

[0018] Embodiments of this disclosure will now be described in more detail with reference to the accompanying drawings. While some embodiments of this disclosure are shown in the drawings, it should be understood that this disclosure can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of this disclosure. It should be understood that the accompanying drawings and embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of protection of this disclosure.

[0019] It should also be noted that, for ease of description, only the parts relevant to the invention are shown in the accompanying drawings. Unless otherwise specified, the embodiments and features described in this disclosure can be combined with each other.

[0020] It should be noted that the concepts of "first" and "second" mentioned in this disclosure are used only to distinguish different devices, modules or units, and are not used to limit the order of functions performed by these devices, modules or units or their interdependencies.

[0021] It should be noted that the terms "a" and "a plurality of" used in this disclosure are illustrative rather than restrictive, and those skilled in the art should understand that, unless otherwise expressly indicated in the context, they should be understood as "one or more".

[0022] The names of messages or information exchanged between multiple devices in the embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of such messages or information.

[0023] This disclosure will now be described in detail with reference to the accompanying drawings and embodiments.

[0024] Figure 1 This is a schematic diagram illustrating an application scenario of the tea identification and packaging method for a tea vending machine, which is one of the embodiments of this disclosure.

[0025] exist Figure 1 In the application scenario, firstly, the computing device 101 can acquire the tea inventory information, tea storage cost, tea replenishment cost, and replenishment information corresponding to each tea category in the tea category set sold by the tea vending machine 102 to be replenished, thus obtaining a replenishment information set. Then, the computing device 101 can control the feeder 103 to release the corresponding tea onto the turntable 104 based on the above replenishment information set. Afterwards, the computing device 101 can acquire hyperspectral images of the tea on the turntable using the hyperspectral imager 105, thus obtaining hyperspectral image information of the tea. Next, the computing device 101 can perform fine-grained tea grade identification on the above hyperspectral image information of the tea based on a preset tea grade classification model, thus obtaining tea grade information. In response to determining that the above tea grade information meets the preset replenishment grade conditions, the computing device 101 can acquire tea weight information through the weight sensor 106 at the bottom of the turntable. In response to determining that the weight information of the tea meets the preset replenishment weight conditions, the computing device 101 can use the camera 108 configured on the robotic arm 107 to perform volume recognition on the tea and obtain the tea volume information. Then, the computing device 101 can package the tea according to the tea grade information and the tea volume information to obtain packaged tea, and transport the packaged tea to the tea vending machine 102 awaiting replenishment.

[0026] It should be noted that the aforementioned computing device 101 can be either hardware or software. When the computing device is hardware, it can be implemented as a distributed cluster composed of multiple servers or terminal devices, or as a single server or a single terminal device. When the computing device is software, it can be installed in the hardware devices listed above. It can be implemented as, for example, multiple software programs or software modules used to provide distributed services, or as a single software program or software module. No specific limitations are made here. It should be understood that... Figure 1 The number of computing devices in the system can be arbitrary, depending on the implementation requirements.

[0027] Continue to refer to Figure 2The diagram illustrates a flow 200 of some embodiments of a tea identification and packaging method applied to a tea vending machine according to the present disclosure. This tea identification and packaging method applied to a tea vending machine includes the following steps:

[0028] Step 201: Based on the tea inventory information, tea storage cost, and tea replenishment cost corresponding to each tea category in the tea category set sold by the tea vending machine to be replenished, generate replenishment information to obtain a replenishment information set.

[0029] In some embodiments, the implementer of the tea identification and packaging method applied to a tea vending machine (e.g., Figure 1 The computing device 101 shown can generate replenishment information based on the tea inventory information, tea storage cost, and tea replenishment cost corresponding to each tea category in the tea category set sold by the tea vending machine to be replenished, thus obtaining a replenishment information set. The aforementioned executing entity can communicate with the aforementioned tea vending machine to be replenished to obtain the tea inventory information, tea storage cost, and tea replenishment cost corresponding to each tea category in the tea category set sold by the tea vending machine to be replenished. The aforementioned tea vending machine to be replenished can be a vending machine used to sell packaged tea. The aforementioned tea vending machine to be replenished can correspond to a tea vending machine storage capacity. The aforementioned tea vending machine storage capacity can represent the maximum quantity of packaged tea that the tea vending machine can store. The aforementioned tea category set can be a collection of tea categories sold by the aforementioned tea vending machine to be replenished. Each tea category in the aforementioned tea category set can correspond to a type, grade, and weight. Each tea category in the aforementioned tea category set can also correspond to tea inventory information, tea storage cost, and tea replenishment cost. The aforementioned tea inventory information represents the remaining quantity of the aforementioned tea category in the aforementioned tea vending machine awaiting replenishment. The aforementioned tea storage cost can be the preset cost per unit time required to store a single tea leaf in the aforementioned tea vending machine awaiting replenishment. The aforementioned unit time can be 1 day. The aforementioned tea replenishment cost can be the cost of replenishing the aforementioned tea category per transaction. The aforementioned tea replenishment cost may include: tea ordering costs, labor costs, replenishment electricity costs, etc.

[0030] In practice, the aforementioned implementing entity can evenly distribute the storage capacity of the tea vending machine to each tea category, obtaining an average tea capacity. For each tea category in the aforementioned tea category set, firstly, the difference between the average tea capacity and the corresponding tea inventory information for that tea category can be determined as the replenishment quantity. Then, the tea category, the replenishment quantity, and the preset replenishment time can be determined as the replenishment information. For example, the replenishment time could be two days later.

[0031] As an example, assume the tea vending machine mentioned above has a storage capacity of 40, meaning it can store 40 packaged teas. Let the tea categories sold include four types: "A: Green Tea - Grade 1 - 5g; B: Green Tea - Grade 2 - 5g; C: Green Tea - Grade 2 - 10g; D: Black Tea - Grade 1 - 10g". The corresponding tea inventory information is 10; 4; 2; and 6 respectively. Therefore, the average capacity for each tea category is 40 divided by 4, which equals 10. Thus, the replenishment quantity for tea category A is 0, for tea category B it is 6, for tea category C it is 8, and for tea category D it is 4. The replenishment information set could be: "Green Tea - Grade 2 - 5g, 6, 2 days; Green Tea - Grade 2 - 10g, 8, 2 days; Black Tea - Grade 1 - 10g, 4, 2 days".

[0032] Optionally, the aforementioned implementing entity generates replenishment information based on the tea inventory information, tea storage costs, and tea replenishment costs corresponding to each tea category in the tea category set sold by the tea vending machine to be replenished, thus obtaining a replenishment information set, which may include the following steps:

[0033] The first step, for each tea category in the above tea category groups, is to perform the following steps:

[0034] The first sub-step involves obtaining the tea demand rate corresponding to the aforementioned tea categories. This tea demand rate can be the sales volume per unit time for each tea category. In practice, the executing entity can obtain the tea demand rate corresponding to each tea category in the vending machine awaiting replenishment. For example, the demand rate for 5g of grade 2 green tea is 2 per day.

[0035] The second sub-step involves generating an economic order quantity (EOQ) based on a pre-defined algorithm, taking into account the tea demand rate, the tea storage cost corresponding to the tea category, and the tea replenishment cost. This EOQ algorithm can be the Economic Order Quantity (EOQ) algorithm. The EOQ minimizes the sum of replenishment and storage costs. In practice, firstly, the product of the tea demand rate and the tea replenishment cost can be multiplied by 2 and then divided by the tea storage cost to obtain the initial EOQ. Then, the square root of this initial EOQ, rounded down, is used to determine the final EOQ.

[0036] In practice, replenishing stock according to the above-mentioned economic order quantity can minimize the total storage and replenishment costs.

[0037] The third sub-step generates a replenishment threshold and replenishment time based on the aforementioned tea inventory information and tea demand rate. The replenishment threshold can be a reorder point (ROP). The replenishment threshold can also be the number of items. The replenishment time is the time point corresponding to the reorder point. In practice, the aforementioned tea demand rate can be used as the replenishment threshold. The replenishment time can be obtained by subtracting one from the ratio between the aforementioned tea inventory information and the aforementioned tea demand rate.

[0038] As an example, assuming the demand rate for 5g of Grade 2 Green Tea is 2 units per day, the replenishment threshold can be 2. Replenishment is required when the inventory of 5g of Grade 2 Green Tea is less than or equal to 2 units. Otherwise, a stockout may occur. Assuming the current inventory of 5g of Grade 2 Green Tea is 4 units, the replenishment time is 4 / 2 - 1 = 1 day. This means replenishment is required 1 day from the current time.

[0039] The fourth sub-step involves using the aforementioned tea inventory information and the aforementioned economic order quantity. and The aforementioned replenishment threshold generates a predicted inventory sequence. This predicted inventory sequence represents the daily inventory quantity of the aforementioned tea category in the aforementioned tea vending machine to be replenished over a future period. This future period can be one month. The predicted inventory sequence can contain 30 values. In practice, the tea inventory information can be used as the first value in the predicted inventory sequence. Then, for each value in the predicted inventory sequence other than the first value, the tea demand rate can be subtracted from the previous value to obtain the initial inventory quantity. When the initial inventory quantity is greater than the replenishment threshold, it can be used as the predicted inventory quantity. When the initial inventory quantity is less than or equal to the replenishment threshold, it can be added to the economic order quantity to obtain the predicted inventory quantity.

[0040] The second step involves summing the generated predicted inventory sequences element by element to obtain the total inventory sequence. This total inventory sequence represents the daily inventory quantity of all tea categories in the aforementioned tea vending machines awaiting replenishment over a future period.

[0041] The third step involves determining that each total inventory in the aforementioned total inventory sequence is less than or equal to the preset storage capacity of the tea vending machine, and then defining the generated economic order quantity and replenishment time as a replenishment information set. Each piece of replenishment information in this set may include the tea category, economic order quantity, and replenishment time.

[0042] Fourth, in response to determining that there is a total inventory in the above total inventory sequence that exceeds the preset storage capacity of the tea vending machine, the following steps are executed:

[0043] The first sub-step involves constructing the objective function for each tea category within the aforementioned tea category set. The dependent variable of this objective function can be the total cost across all tea categories. The independent variable can include the replenishment estimate for each tea category. This replenishment estimate can be the replenishment quantity to be determined. In practice, for each tea category in the aforementioned tea category set, the target replenishment cost can be obtained by multiplying the tea demand rate corresponding to that category by the tea replenishment cost and then dividing by the replenishment estimate. Next, the target storage cost can be obtained by multiplying the replenishment estimate by the tea storage cost and then dividing by 2. Then, the sum of the target replenishment cost and the target storage cost can be determined as the total cost per tea leaf. Finally, the sum of the determined total costs per tea leaf can be determined as the total cost. Therefore, the objective function can be "Total Cost = Σ[(Tea Demand Rate × Tea Replenishment Cost ÷ Replenishment Estimate) + (Replenishment Estimate × Tea Storage Cost ÷ 2)]". This objective function has corresponding constraints. The above constraint is that the sum of half of the estimated replenishment quantities for each of the above tea categories must be less than or equal to the storage capacity of the tea vending machine.

[0044] The second sub-step involves optimizing the objective function to generate the target replenishment quantity for each tea category. In practice, a pre-defined optimization algorithm can be used to determine the values ​​of each replenishment estimate that minimizes the total cost of the objective function, which will then be used as the replenishment quantity. This optimization algorithm can be either an enumeration method or an optimization algorithm.

[0045] The third sub-step involves defining the generated replenishment times and quantities as a replenishment information set. Each piece of replenishment information in this set may include the tea category, the optimal replenishment quantity, and the replenishment time.

[0046] Steps one through four of the above-mentioned optional solutions, and their related contents, constitute an inventive point of this disclosure, solving the technical problem of "tea stockpiling or excessively frequent replenishment." The cause of tea stockpiling or excessively frequent replenishment is often that replenishment decisions fail to comprehensively balance the relationship between tea inventory costs, replenishment costs, and equipment capacity. Solving these factors can avoid tea stockpiling or excessively frequent replenishment. To achieve this, firstly, the sales volume per unit time for each tea category in the vending machine to be replenished is acquired in real time. Then, combined with an economic order quantity (EOQ) algorithm, the economic order quantity for each tea category is determined to ensure that the sum of replenishment and storage costs for a single category is minimized. Simultaneously, replenishment thresholds and replenishment times are determined by combining inventory information and demand rates to avoid stockout risks. Secondly, by generating a predicted inventory quantity sequence for each category and superimposing it to obtain a total inventory quantity sequence, the inventory occupancy of the vending machine for a future period is predicted to identify whether the total inventory exceeds the total capacity of the vending machine. When the inventory does not exceed the vending machine capacity, the economic order quantity can be used as the final replenishment quantity to maximize cost-effectiveness. When inventory exceeds the vending machine's capacity, a total cost objective function for each tea category is constructed, and an optimization algorithm is used to solve for the optimal target replenishment quantity. This minimizes the overall cost of all teas within the vending machine's capacity constraint, reducing losses from tea stockpiling or excessively frequent replenishment.

[0047] Step 202: Based on the replenishment information set, release the corresponding tea leaves onto the turntable via the feeder.

[0048] In some embodiments, the executing entity can release the corresponding tea leaves onto the turntable via a feeder, based on the replenishment information set. The feeder can be a screw feeder. Each tea category in the tea category set can be stored in a storage bin, and each storage bin can be equipped with a feeder at its end. In practice, for any replenishment information in the replenishment information set, the executing entity can control the feeder corresponding to the tea category to release tea leaves of the corresponding weight onto the turntable.

[0049] As an example, suppose the replenishment information set contains the following information: Green Tea - Grade 2 - 5g, 6 pieces, 2 days. After 2 days, the aforementioned execution entity can locate the storage box for Green Tea - Grade 2. It then controls the screw feeder at its end to evenly push the Green Tea - Grade 2 tea leaves from the bottom of the storage box onto a turntable. A weight sensor can be installed at the bottom of the turntable. The screw feeder can initially release most of the tea leaves at a relatively high speed (e.g., 5g × 90% = 4.5g), then slowly release the tea leaves at a low speed until the weight sensor collects a weight of 5g. Once the feeding is complete, the replenishment information can be updated by decreasing the replenishment quantity by one. The updated replenishment information is: Green Tea - Grade 2 - 5g, 5 pieces, 0 days. Afterwards, when the replenishment quantity in the replenishment information set is not zero, the aforementioned execution entity can perform the replenishment step again.

[0050] Step 203: Use a hyperspectral imager to acquire hyperspectral images of the tea leaves on the turntable, and obtain hyperspectral image information of the tea leaves.

[0051] In some embodiments, the aforementioned executing entity can acquire hyperspectral images of the tea leaves on the turntable using a hyperspectral imager to obtain hyperspectral image information of the tea leaves. The hyperspectral imager can be used to acquire hyperspectral images of the tea leaves. The hyperspectral imager may include, but is not limited to, a hyperspectral analyzer, a camera lens, and a halogen lamp illumination unit. The hyperspectral image information of the tea leaves may include the type of tea and the hyperspectral image of the tea leaves. The hyperspectral image of the tea leaves can be represented as a multi-channel image, where each pixel of the image corresponds to a reflectance in the visible to near-infrared (e.g., 400nm-1000nm) wavelength range. In practice, the aforementioned executing entity can control the hyperspectral imager to photograph the tea leaves on the turntable to obtain hyperspectral image information of the tea leaves.

[0052] Optionally, the aforementioned executing entity acquires hyperspectral images of the tea leaves on the turntable using a hyperspectral imager to obtain hyperspectral image information of the tea leaves, which may include the following steps:

[0053] The first step involves controlling the camera to acquire a preliminary image of the area corresponding to the turntable. This camera can be communicatively connected to the execution entity. The camera can be directly facing the turntable. In practice, the execution entity can control the camera to capture images of the area corresponding to the turntable, thus obtaining a preliminary image.

[0054] The second step is to perform object detection on the preliminary image to obtain the object detection result. This result can indicate the presence or absence of an object. In practice, a pre-defined object detection model can be used to perform object detection on the preliminary image. This object detection model can be a pre-trained binary classification model. This model can be used to determine whether the preliminary image contains tea leaves. The object detection model can include a feature extraction layer, a global pooling layer, and a classification layer. The feature extraction layer can be MobileNet. The classification layer can include a fully connected layer and an activation function (e.g., sigmoid).

[0055] As an example, firstly, image data can be collected when the turntable is empty and when tea leaves are placed on it, resulting in a sample image dataset. Next, each sample image in this dataset can be labeled as "present" or "absent," resulting in a labeled image set. Then, using this labeled image set, an initial binary classification model is trained to obtain an object detection model. This initial binary classification model can include the feature extraction layer and the classification layer mentioned above. Finally, the trained object detection model is used to perform forward inference on the initial images to obtain the object detection result.

[0056] Thirdly, in response to the determination that the above-mentioned object detection result meets the preset object presence condition, based on a preset time interval, the camera is controlled to acquire images of the tea leaves on the turntable, obtaining a first tea leaf image and a second tea leaf image. The above-mentioned object presence condition can be that the above-mentioned object detection result indicates the presence of an object. The above-mentioned time interval can be 1 second. In practice, when the above-mentioned object detection result indicates the presence of an object, the camera can be controlled to acquire images of the tea leaves on the turntable twice based on the above-mentioned time interval, obtaining a first tea leaf image and a second tea leaf image.

[0057] The fourth step is to determine the overall grayscale difference between the first tea leaf image and the second tea leaf image. In practice, firstly, the first and second tea leaf images can be converted to grayscale to obtain a first grayscale image and a second grayscale image. Then, pixel-by-pixel grayscale differences can be calculated between the first and second grayscale images to obtain a grayscale difference matrix. Finally, the average value among the various grayscale differences in the grayscale difference matrix can be determined as the overall grayscale difference.

[0058] Fifth, in response to determining that the overall grayscale difference is less than or equal to a preset grayscale threshold, the preset static compliance information is determined as the static detection result. The grayscale threshold can be a preset value, without specific limitations. The static compliance information can be "static compliance" or represented by "1". The static detection result can be either static compliance or static non-compliance. In practice, when the overall grayscale difference is less than or equal to the grayscale threshold, the static compliance information can be determined as the static detection result.

[0059] Step 6: In response to determining that the above static detection result meets the preset static conditions, control the hyperspectral imager to acquire hyperspectral images of the tea leaves, obtaining hyperspectral image information of the tea leaves. The above static conditions can be that the static detection result meets the static standards. In practice, when the above static detection result meets the static standards, the executing entity can control the hyperspectral imager to acquire hyperspectral images of the tea leaves, obtaining hyperspectral image information of the tea leaves.

[0060] Optionally, when the static detection result is deemed unsatisfactory, the executing entity can, after the aforementioned time interval, control the camera to acquire an image of the tea leaves on the turntable, obtaining a third tea leaf image. Then, the second tea leaf image can be used as the first tea leaf image, and the third tea leaf image as the second tea leaf image, and steps four and five can be executed again to generate a static detection result. This process continues until the static detection result meets the aforementioned static conditions.

[0061] Step 204: Based on the preset tea grade classification model, perform fine-grained tea grade identification on the hyperspectral image information of tea to obtain tea grade information.

[0062] In some embodiments, the executing entity can perform fine-grained tea grade identification on the hyperspectral image information of the tea leaves based on a preset tea grade classification model to obtain tea grade information. The tea grade classification model can be a classification model used to identify tea grades. For example, the tea grade classification model can be a tea variety and grade identification model based on convolutional neural networks and near-infrared spectroscopy, or it can be a hyperspectral image tea identification algorithm based on machine learning. In practice, the executing entity can input the hyperspectral image information of the tea leaves into the tea grade classification model to output tea grade information.

[0063] In practice, due to differences in ambient temperature, humidity, and light conditions, tea may oxidize and become damp in storage boxes, causing changes in its color, aroma, and appearance. These changes may result in the actual grade of the tea not matching the required category. Therefore, real-time grade identification of the tea is necessary during replenishment to ensure that the tea meets the required grade.

[0064] Optionally, the aforementioned executing entity, based on a preset tea grade classification model, performs fine-grained tea grade identification on the aforementioned hyperspectral image information of tea to obtain tea grade information, which may include the following steps:

[0065] The first step involves acquiring a set of hyperspectral images of sample tea leaves and a pre-defined initial tea grading classification model. The set of sample tea leaf hyperspectral images can be a collection of pre-acquired and labeled hyperspectral images of tea leaves. Each sample tea leaf hyperspectral image in the set corresponds to a ground truth value for the tea grading. The initial tea grading classification model may include a one-heat coding layer, a principal component analysis (PCA) layer, a backbone network, a coarse classification head network, a texture enhancement branch network, and a fine classification head network. The structure diagram of the initial tea grading classification model can be shown as follows: Figure 3 As shown.

[0066] As an example, firstly, hyperspectral images of different types and grades of tea can be collected to obtain a raw hyperspectral image set of tea. Next, each raw hyperspectral image in the original tea image set can be labeled to indicate the corresponding tea type and grade, resulting in a sample tea hyperspectral image set. The tea grade can be "Grade 1", "Grade 2", "Grade 3", etc.

[0067] The second step involves training the initial tea grading classification model using the aforementioned set of sample tea hyperspectral images, resulting in a trained tea grading classification model. This trained model shares the same structure as the initial model, but differs in parameters. In practice, each sample tea hyperspectral image from the aforementioned set can be used as input to the initial model, and the corresponding true tea grade value can be used as the expected output. This process is then used to train the initial model, yielding the trained tea grading classification model.

[0068] The third step involves optimizing the trained tea grade classification model using a pre-defined particle swarm optimization (PSO) algorithm to obtain an optimized tea grade classification model, which serves as the final tea grade classification model. This trained tea grade classification model may include hyperparameters such as learning rate, weight decay, dropout rate, and batch size. In practice, each hyperparameter in the trained tea grade classification model can be optimized using a pre-defined PSO algorithm to generate optimized hyperparameters. The optimized hyperparameters are then used to update the trained tea grade classification model, resulting in the final optimized tea grade classification model.

[0069] The fourth step involves performing fine-grained tea grade identification on the hyperspectral image information of the tea leaves, based on the aforementioned tea grade classification model, to obtain the tea grade information. In practice, the hyperspectral image information of the tea leaves can be input into the aforementioned tea grade classification model to output the tea grade information.

[0070] In practice, a common technical challenge in tea grading is that different tea categories (e.g., green tea, black tea) have different grading standards. Training a separate grading model for each category would significantly increase storage and computational overhead. However, using a single model to grade all tea categories would reduce the accuracy of tea grading. Therefore, the following solution is proposed.

[0071] Optionally, the execution entity, based on the aforementioned tea grade classification model, performs fine-grained tea grade identification on the aforementioned hyperspectral image information of tea to obtain tea grade information, which may include the following steps:

[0072] The first step is to determine the tea category vectors for the tea varieties included in the aforementioned hyperspectral image information. In practice, this can be achieved by first performing one-hot encoding on each tea category in the tea category set to generate a tea category vector, thus obtaining a tea category vector set. Next, the tea category vectors corresponding to the tea varieties included in the aforementioned hyperspectral image information can be determined.

[0073] As an example, suppose the categories of each tea type in the tea category set are "black tea, green tea, and Pu'er tea", and the corresponding tea category vectors are "[0, 0, 1], [0, 1, 0], [1, 0, 0]". When the tea category corresponding to the above hyperspectral image information is black tea, its corresponding tea category vector is [0, 0, 1].

[0074] The second step involves performing principal component analysis (PCA) on the hyperspectral images of tea leaves included in the aforementioned tea leaf hyperspectral image information to obtain a three-channel principal component image. In practice, a preset principal component analysis (PCA) technique can be used to perform PCA on the hyperspectral images of tea leaves corresponding to the aforementioned tea leaf hyperspectral image information, and to determine the images corresponding to the first three principal components as the three-channel principal component image. The width and height of the three-channel principal component image can be the same as the width and height of the aforementioned hyperspectral image of tea leaves, and the number of channels is three.

[0075] As an example, assuming the dimensions of the aforementioned hyperspectral image of tea leaves are H×W×B, performing 3D principal component analysis on the image yields three principal component images as a three-channel principal component image, with dimensions of H×W×3. Here, H represents the height of the image, and W represents the width. For example, the dimensions of the three-channel principal component image could be 512×512×3.

[0076] The third step involves joint feature extraction from the tea category vector and the three-channel principal component image to obtain the hyperspectral features of the tea. In practice, the aforementioned backbone network can be used to perform joint feature extraction from the tea category vector and the three-channel principal component image to obtain the hyperspectral features of the tea. The aforementioned backbone network can include multiple residual blocks and multiple linear layers. The number of residual blocks is the same as the number of linear layers. The structure of the aforementioned backbone network can be as follows: Figure 3 As shown.

[0077] Specifically, firstly, the tea category vector can be linearly mapped using a first linear layer to obtain a first scaling feature and a first translation feature. Both the first scaling feature and the first translation feature can have a dimension of 1×1×C, where C represents the number of feature channels. No specific limitation is made here. Next, the three-channel principal component image can be input into the first residual block to obtain a first image feature. The dimension of the first image feature can be H×W×C. Then, the product of the first image feature and the first scaling feature can be determined as the scaling feature. The dimension of the scaling feature can be H×W×C. Then, the first translation feature can be expanded to the H×W×C dimension, and the sum of the scaling feature and the expanded first translation feature can be determined as the modulation image feature. Secondly, the tea category vector can be input into a second linear layer to obtain a second scaling feature and a second translation feature. The modulation image feature is then input into a second residual block to obtain a second image feature. Following the above operation of determining the modulation image feature, the second image feature is scaled and translated based on the second scaling feature and the second translation feature to obtain the second modulation image feature. Similarly, the features of the final output modulated image are used as the hyperspectral features of tea leaves.

[0078] In practice, using tea category vectors to modulate tea image features can enable a single model to adaptively learn the grade classification patterns of different tea categories, thereby effectively reducing model storage requirements and overall computing power consumption while maintaining classification accuracy.

[0079] The fourth step is to determine the initial classification result corresponding to the hyperspectral features of the tea leaves. This initial classification result includes the initial tea grade and the initial classification confidence level. In practice, the initial classification result corresponding to the hyperspectral features of the tea leaves can be determined using the coarse classification head network described above. This coarse classification head network can output the corresponding tea grade based on the input features. The coarse classification head network may include a fully connected layer and an activation function (e.g., softmax).

[0080] Fifth, in response to the determination that the initial classification confidence level is less than a preset confidence threshold, the following steps are performed:

[0081] The first sub-step involves extracting gray-level co-occurrence features from the three-channel principal component images to obtain the gray-level co-occurrence features of tea leaves. The confidence threshold can be a preset value and is not specifically limited here. In practice, when the initial classification confidence is less than the confidence threshold, the gray-level co-occurrence matrix (GLCM) technique included in the texture enhancement branch network can be used to extract gray-level co-occurrence features from the three-channel principal component images to obtain the gray-level co-occurrence features of tea leaves. The dimension of the gray-level co-occurrence matrix can be M×M×3, where M represents the width and height of the gray-level co-occurrence matrix. For example, the dimension of the gray-level co-occurrence matrix is ​​128×128×3.

[0082] The second sub-step involves fusing the aforementioned hyperspectral features and gray-level co-occurrence features of the tea leaves to obtain a fused feature. In practice, this can be achieved first by concatenating the hyperspectral features and gray-level co-occurrence features of the tea leaves to obtain a concatenated feature. Then, a pre-defined linear layer can be used to linearly map the concatenated feature to obtain the fused feature. The dimension of the fused feature can be the same as the dimension of the hyperspectral features of the tea leaves.

[0083] The third sub-step involves determining the fine-grained classification tea grade corresponding to the aforementioned fused features, which serves as the tea grade information. In practice, this fine-grained classification head network can be used to determine the fine-grained classification tea grade corresponding to the aforementioned fused features, serving as the tea grade information. The aforementioned fine-grained classification head network may include the aforementioned fully connected layers and the aforementioned activation function (Softmax). The aforementioned fine-grained classification head network can share parameters with the aforementioned coarse-grained classification head network.

[0084] Step 6: In response to determining that the confidence level of the initial classification is greater than or equal to the confidence threshold, the initial classification of the tea grade is determined as tea grade information. In practice, when the confidence level of the initial classification is less than or equal to the confidence threshold, the initial classification of the tea grade can be determined as tea grade information.

[0085] Steps one through six of the aforementioned optional solutions, along with their related content, constitute an inventive point of this disclosure, solving the aforementioned technical problem: "reduced accuracy in tea grade identification." The reason for reduced accuracy in tea grade identification is often that, due to the different grading standards for different tea categories (e.g., green tea, black tea), training a separate grading classification model for each tea category would significantly increase model storage and computational overhead. However, using a single model to classify all categories of tea would reduce the accuracy of tea grade identification. Solving these factors can improve the accuracy of tea grade identification. To achieve this effect, firstly, principal component analysis is performed on the hyperspectral images of tea leaves, extracting the first three principal component images. This effectively compresses the data dimensionality while retaining important detailed information, enhancing the model's ability to distinguish subtle grade differences. Next, in the feature extraction stage, the category vectors of known tea categories are introduced, and a conditional modulation mechanism guides the hyperspectral features with category priors, enabling the single model to adaptively learn the grade features of different tea categories. Subsequently, to avoid misclassification of easily confused samples with similar features, a confidence threshold was introduced. Only ambiguous samples with low confidence thresholds underwent fine-grained processing, avoiding unnecessary computational overhead. Next, gray-level co-occurrence features were extracted from low-confidence samples to obtain fine-grained texture features of the tea hyperspectral image. Finally, fine-grained grade recognition was performed by fusing the hyperspectral features and gray-level co-occurrence features of the tea, significantly improving the classification accuracy of error-prone samples. Thus, accurate grade recognition of multiple tea categories was achieved under a single model architecture. This avoids the storage and computing power consumption caused by multi-model deployment and reduces the accuracy loss caused by general models ignoring category differences, making it easy to deploy on edge devices.

[0086] Step 205: In response to determining that the tea grade information meets the preset replenishment grade conditions, the weight information of the tea is collected by the weight sensor at the bottom of the turntable.

[0087] In some embodiments, the executing entity may, in response to determining that the tea grade information meets preset replenishment grade conditions, collect tea weight information through a weight sensor at the bottom of the turntable. The replenishment grade conditions may be that the replenishment information set contains replenishment information corresponding to the tea category and tea grade information.

[0088] As an example, suppose the tea variety mentioned above is green tea, and the tea grade information after tea grading is grade 2. When the replenishment information contains the following information: "Green tea - grade 2 - 5g, 6 pieces, 2 days", then it is determined that the tea grade information meets the replenishment grade conditions.

[0089] Optionally, the implementing entity may also return the tea in response to determining that the tea grade information does not meet the preset replenishment grade conditions. In practice, when the tea grade information does not meet the replenishment grade conditions, the tea can be returned to the corresponding storage box.

[0090] As an example, suppose the tea mentioned above is classified as green tea, and its grade information after tea grading is level 1. If there is no replenishment information in the replenishment information set that reads "Green Tea - Level 1", then it is determined that the tea grade information does not meet the replenishment grade requirements.

[0091] Step 206: In response to determining that the weight information of the tea leaves meets the preset replenishment weight conditions, the volume of the tea leaves is identified by a camera configured on the robotic arm to obtain the volume information of the tea leaves.

[0092] In some embodiments, the execution entity may, in response to determining that the tea weight information meets a preset replenishment weight condition, perform volume recognition on the tea using a camera configured on a robotic arm to obtain tea volume information. The replenishment weight condition may be that the replenishment information set contains replenishment information corresponding to the tea category, tea grade, and tea weight information. The camera may be mounted on the end effector of the robotic arm. The robotic arm may be used to change the position of the camera. In practice, firstly, the execution entity may control the robotic arm to change its pose to control the camera to acquire multi-view images of the tea, obtaining a tea image set. Then, a preset 3D reconstruction algorithm may be used to perform 3D reconstruction on each tea image in the tea image set, obtaining a 3D point cloud of the tea. The 3D reconstruction algorithm may be a Structure from Motion (SFM) algorithm. Finally, the volume of the tea 3D point cloud can be determined as the tea volume information.

[0093] Optionally, in response to determining that the weight information of the tea leaves meets a preset replenishment weight condition, the executing entity performs volume recognition on the tea leaves using a camera configured on the robotic arm to obtain the volume information of the tea leaves. This may include the following steps:

[0094] The first step involves adjusting the robotic arm and controlling the turntable to rotate based on each latitude information in a pre-defined latitude information set. This allows the camera to capture multiple images of the tea leaves, resulting in a single-dimensional tea leaf image set. Each latitude information in the set can be a distinct value representing the latitude of the robotic arm's end effector, i.e., the camera's latitude. In practice, the executing entity can adjust the robotic arm to the latitude corresponding to the given latitude information. Next, the turntable is rotated. A marking sheet, such as an AprilTag label, can be placed on the turntable and fixed at its center. The executing entity can control the camera to photograph the marking sheet on the turntable to determine the relative pose between the robotic arm and the turntable in real time and to ascertain whether the turntable has completed a 360-degree rotation. During the turntable's rotation, the executing entity can control the camera to capture multiple images of the tea leaves, resulting in a single-dimensional tea leaf image set. The flowchart for this image acquisition process is as follows: Figure 4 As shown.

[0095] As an example, the latitude information set mentioned above could include: {10°, 40°, 70°}. At each latitude, the turntable can rotate one full circle to capture several images (e.g., one image every 30°, for a total of 12 images) as a single-latitude tea leaf image set.

[0096] The second step involves performing coarse volume recognition on each of the obtained single-dimensional tea image sets to obtain initial tea volume information. This initial tea volume information can include the initial tea volume center and the initial tea volume radius. The initial tea volume center can be represented by longitude, latitude, and radius. The initial tea volume radius can be a distance value. In practice, firstly, a preset 3D modeling algorithm can be used to perform 3D modeling on each of the obtained single-dimensional tea image sets to generate a coarse 3D model of the tea. This coarse 3D model can be a 3D model representing the tea. Then, the centroid of this coarse 3D model can be determined as the initial tea volume center. Secondly, the radius of the largest circumscribed sphere of the coarse 3D model can be multiplied by 1.2 to obtain the initial tea volume radius. The 3D modeling algorithm can be the VisualHull algorithm.

[0097] The third step is to generate a dense viewpoint pose information set based on the initial tea leaf volume information. Each dense viewpoint pose information set can be characterized by latitude and radius. In practice, firstly, an initial tea leaf volume sphere can be generated based on the initial tea leaf volume center and radius. Then, regardless of longitude, sampling points at different latitudes of the initial tea leaf volume sphere are uniformly sampled as dense viewpoint pose information.

[0098] The fourth step involves adjusting the robotic arm and controlling the turntable to rotate based on each dense viewpoint pose information in the aforementioned dense viewpoint pose information set, so as to acquire multiple images of the tea leaves through the camera and obtain a dense tea leaf image set.

[0099] As an example, after obtaining the dense viewpoint pose information set, the control terminal controls the robotic arm to reset to the initial pose (0 longitude, 10 latitude, 0.05 meters). Then, the turntable is started, and the control terminal begins acquiring images. During this process, the pose is judged in real time to calculate whether the turntable has completed a full rotation. If not, the rotation continues; if completed, the turntable is stopped, and the robotic arm is moved to the next pose (e.g., 0 longitude, 20 latitude, 0.04 meters). This cycle continues until all target poses have been acquired. Throughout the entire movement, the aforementioned execution entity continuously judges the poses of markers, etc., to determine the relative poses of the turntable and the robotic arm.

[0100] The fifth step involves performing volumetric precision recognition on each of the obtained dense tea leaf image groups to obtain the tea leaf volume information. This volume information can be a value representing the tea leaves themselves. In practice, a pre-defined 3D modeling algorithm can be used to perform volumetric precision recognition on each of the obtained dense tea leaf image groups to obtain a 3D model of the tea leaves. The volume of this 3D model is then determined as the tea leaf volume information. This 3D modeling algorithm can be a multi-view stereo matching (MVS) algorithm.

[0101] Specifically, firstly, foreground segmentation can be performed on dense tea leaf image groups. Combined with high-precision pose information, a multi-view stereo vision algorithm is used to generate a high-density point cloud. Then, surface reconstruction (such as Poisson reconstruction) is performed on the point cloud to obtain a detailed 3D model of the tea leaves. Finally, the closed volume of this model is calculated as the final volume information of the tea leaves.

[0102] Optionally, the aforementioned executing entity may also, in response to determining that the tea weight information does not meet the preset replenishment weight condition, adjust the tea on the turntable and collect the tea weight information again through the aforementioned weight sensor. In practice, when the tea weight information does not meet the preset replenishment weight condition, the following steps can be performed:

[0103] The first step involves returning the tea leaves from the turntable when their weight exceeds the target replenishment weight. The screw feeder then releases the tea leaves from the storage bin corresponding to the tea category. The weight sensor is then used to collect the tea weight again until it equals the target weight. For example, suppose the tea category is green tea, the grade is 2, and the weight is 10g. If the replenishment information contains the message: "Green tea - Grade 2 - 5g, 6 pieces, 2 days," then the tea weight exceeds the target replenishment weight.

[0104] The second step involves releasing tea leaves from the storage bin corresponding to the tea category using the screw feeder when the tea weight information is less than or equal to the target replenishment information. The weight information is then collected again by the weight sensor until it equals the target weight.

[0105] Step 207: Based on the tea grade information and tea volume information, package the tea to obtain packaged tea, and then transport the packaged tea to the tea vending machine waiting for replenishment.

[0106] In some embodiments, the executing entity can package the tea according to the tea grade information and the tea volume information to obtain packaged tea, and then transport the packaged tea to the tea vending machine awaiting replenishment. In practice, firstly, the executing entity can obtain packaging bags corresponding to the tea grade information and the tea volume information using a preset packaging device. For example, "Black Tea - Grade 1" can correspond to a Grade 1 packaging bag. The tea volume information of 200ml can correspond to a medium-sized packaging bag. Then, the packaging device is controlled to package the tea using the packaging bags to obtain packaged tea. The packaging device can be a vertical filling and sealing packaging machine. Afterwards, the packaged tea can be transported to the tea vending machine awaiting replenishment using a conveyor device. The conveyor device can be a device used to deliver packaged tea to the replenishment port of the tea vending machine awaiting replenishment. For example, the conveyor device can be a belt conveyor or a roller conveyor.

[0107] Further reference Figure 5 As an implementation of the methods shown in the above figures, this disclosure provides some embodiments of a tea identification and packaging device applied to a tea vending machine. These device embodiments are similar to... Figure 2 Corresponding to the method embodiments shown, the tea identification and packaging device applied to tea vending machines can be specifically applied to various electronic devices.

[0108] like Figure 5As shown, a tea identification and packaging device 500 applied to a tea vending machine in some embodiments includes: a generation unit 501, a release unit 502, a hyperspectral image acquisition unit 503, a fine-grained tea grade identification unit 504, an acquisition unit 505, a volume identification unit 506, and a packaging unit 507. The generation unit 501 is configured to generate replenishment information based on the tea inventory information, tea storage cost, and tea replenishment cost corresponding to each tea category in the tea category set sold by the tea vending machine to be replenished, thus obtaining a replenishment information set. The release unit 502 is configured to release the corresponding tea onto the turntable via a feeder according to the replenishment information set. The hyperspectral image acquisition unit 503 is configured to acquire hyperspectral images of the tea on the turntable using a hyperspectral imager, thus obtaining hyperspectral image information of the tea. The fine-grained tea grade identification unit 504 is configured to process the hyperspectral image information of the tea based on a preset tea grade classification model. Fine-grained tea grade identification obtains tea grade information; acquisition unit 505 is configured to acquire tea weight information through a weight sensor at the bottom of the turntable in response to determining that the tea grade information meets preset replenishment grade conditions; volume recognition unit 506 is configured to perform volume recognition on the tea through a camera configured on the robotic arm in response to determining that the tea weight information meets preset replenishment weight conditions, obtain tea volume information; packaging unit 507 is configured to package the tea according to the tea grade information and the tea volume information, obtain packaged tea, and transport the packaged tea to the tea vending machine to be replenished.

[0109] It is understandable that the units and references described in the tea identification and packaging device 500 applied to tea vending machines are related to... Figure 2 The steps in the described method correspond to each other. Therefore, the operations, features, and beneficial effects described above for the method are also applicable to the tea identification and packaging device 500 and the units contained therein applied to the tea vending machine, and will not be repeated here.

[0110] The following is for reference. Figure 6 It shows a schematic diagram of the structure of an electronic device 600 (e.g., a computing device) suitable for implementing some embodiments of the present disclosure. Figure 6 The electronic device shown is merely an example and should not be construed as limiting the functionality and scope of the embodiments of this disclosure.

[0111] like Figure 6As shown, electronic device 600 may include processing device 601 (e.g., central processing unit, graphics processor, etc.), which can perform various appropriate actions and processes according to a program stored in read-only memory (ROM) 602 or a program loaded from storage device 608 into random access memory (RAM) 603. RAM 603 also stores various programs and data required for the operation of electronic device 600. Processing device 601, ROM 602, and RAM 603 are interconnected via bus 604. Input / output (I / O) interface 605 is also connected to bus 604.

[0112] Typically, the following devices can be connected to I / O interface 605: input devices 606 including, for example, touchscreens, touchpads, keyboards, mice, cameras, microphones, accelerometers, gyroscopes, etc.; output devices 607 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 608 including, for example, magnetic tapes, hard disks, etc.; and communication devices 609. Communication device 609 allows electronic device 600 to communicate wirelessly or wiredly with other devices to exchange data. Although Figure 6 An electronic device 600 with various devices is shown; however, it should be understood that it is not required to implement or possess all of the devices shown. More or fewer devices may be implemented or possessed alternatively. Figure 6 Each box shown can represent a device or multiple devices as needed.

[0113] In particular, according to some embodiments of this disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, some embodiments of this disclosure include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device 609, or installed from a storage device 608, or installed from a ROM 602. When the computer program is executed by the processing device 601, it performs the functions defined above in the methods of some embodiments of this disclosure.

[0114] It should be noted that, in some embodiments of this disclosure, the computer-readable medium may be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium may be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In some embodiments of this disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In some embodiments of this disclosure, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium can be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wires, optical fibers, RF (radio frequency), etc., or any suitable combination thereof.

[0115] In some implementations, clients and servers can communicate using any currently known or future-developed network protocol such as HTTP (Hypertext Transfer Protocol) and can interconnect with digital data communication (e.g., communication networks) of any form or medium. Examples of communication networks include local area networks (“LANs”), wide area networks (“WANs”), the Internet (e.g., the Internet of Things), and peer-to-peer networks (e.g., ad hoc peer-to-peer networks), as well as any currently known or future-developed networks.

[0116] The aforementioned computer-readable medium may be included in the aforementioned electronic device; or it may exist independently and not assembled into the electronic device. The aforementioned computer-readable medium carries one or more programs that, when executed by the electronic device, cause the electronic device to: generate replenishment information based on the tea inventory information, tea storage cost, and tea replenishment cost corresponding to each tea category in the tea category set sold by the tea vending machine to be replenished, thus obtaining a replenishment information set; release the corresponding tea onto the turntable via a feeder based on the replenishment information set; acquire hyperspectral images of the tea on the turntable using a hyperspectral imager, thus obtaining hyperspectral image information of the tea; and classify the tea based on a preset tea grade classification model. The aforementioned hyperspectral image information of tea is used for fine-grained tea grade identification to obtain tea grade information; in response to determining that the aforementioned tea grade information meets the preset replenishment grade conditions, the weight information of the tea is collected by the weight sensor at the bottom of the aforementioned turntable; in response to determining that the aforementioned tea weight information meets the preset replenishment weight conditions, the volume of the aforementioned tea is identified by the camera configured on the robotic arm to obtain tea volume information; based on the aforementioned tea grade information and the aforementioned tea volume information, the aforementioned tea is packaged to obtain packaged tea, and the aforementioned packaged tea is transported to the aforementioned tea vending machine awaiting replenishment.

[0117] Computer program code for performing operations of some embodiments of this disclosure can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0118] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0119] The functions described above in this document can be performed at least in part by one or more hardware logic components. For example, exemplary types of hardware logic components that can be used, without limitation, include: field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), system-on-a-chip (SoCs), complex programmable logic devices (CPLDs), and so on.

[0120] The above description is merely a selection of preferred embodiments of this disclosure and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of the invention involved in the embodiments of this disclosure is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the above-described inventive concept. For example, technical solutions formed by substituting the above-described features with (but not limited to) technical features with similar functions disclosed in the embodiments of this disclosure.

Claims

1. A method for tea identification and packaging applied to a tea vending machine, characterized in that, include: Based on the tea inventory information, tea storage cost, and tea replenishment cost corresponding to each tea category in the tea category set sold by the tea vending machine to be replenished, replenishment information is generated to obtain a replenishment information set. Based on the replenishment information set, the corresponding tea leaves are released onto the turntable via the feeder; The tea leaves on the turntable are captured using a hyperspectral imager to obtain hyperspectral image information of the tea leaves. Based on a preset tea grade classification model, fine-grained tea grade identification is performed on the hyperspectral image information of the tea to obtain tea grade information; In response to determining that the tea grade information meets the preset replenishment grade conditions, the weight information of the tea is collected by the weight sensor at the bottom of the turntable; In response to determining that the weight information of the tea leaves meets the preset replenishment weight conditions, the volume of the tea leaves is identified by a camera configured on the robotic arm to obtain the volume information of the tea leaves; Based on the tea grade information and the tea volume information, the tea is packaged to obtain packaged tea, and the packaged tea is then transported to the tea vending machine that needs to be replenished. The generation of replenishment information includes: For each tea category in the aforementioned tea category set, perform the following steps: Obtain the tea demand rate corresponding to the tea category, wherein the tea demand rate can be the sales volume of the tea category per unit time; Based on a preset economic order quantity algorithm, an economic order quantity is generated according to the tea demand rate, the tea storage cost corresponding to the tea category, and the tea replenishment cost. Based on the tea inventory information and the tea demand rate, a replenishment threshold and replenishment time are generated; Based on the tea inventory information and the economic order quantity and The replenishment threshold is used to generate a predicted inventory sequence. The total inventory sequence is obtained by adding each of the generated predicted inventory sequences element by element. In response to determining that each total inventory in the total inventory sequence is less than or equal to the preset tea vending machine storage capacity, the generated economic order batches and replenishment times are determined as a replenishment information set; In response to determining that there exists a total inventory in the total inventory sequence that exceeds the preset storage capacity of the tea vending machine, the following steps are performed: Construct an objective function for each tea category in the tea category set, wherein the dependent variable of the objective function can be the total cost among the various tea categories; The objective function is optimized to generate the target replenishment quantity for each tea category; The generated replenishment times and replenishment quantities are defined as replenishment information sets.

2. The method according to claim 1, characterized in that, The method further includes: In response to the determination that the tea grade information does not meet the preset replenishment grade conditions, the tea is returned.

3. The method according to claim 1, characterized in that, The method further includes: In response to determining that the tea weight information does not meet the preset replenishment weight condition, the tea on the turntable is adjusted, and the tea weight information is collected again through the weight sensor.

4. The method according to claim 1, characterized in that, The process of acquiring hyperspectral images of the tea leaves on the turntable using a hyperspectral imager to obtain hyperspectral image information of the tea leaves includes: The camera is controlled to acquire a preliminary image of the area corresponding to the turntable. The preliminary image is subjected to object detection to obtain object detection results; In response to determining that the object detection result meets the preset object condition, based on a preset time interval, the camera is controlled to acquire images of the tea leaves on the turntable to obtain a first tea leaf image and a second tea leaf image. Determine the overall grayscale difference between the first tea leaf image and the second tea leaf image; In response to determining that the overall grayscale difference is less than or equal to a preset grayscale threshold, the preset static compliance information is determined as the static detection result; In response to determining that the static detection result meets the preset static conditions, the hyperspectral imager is controlled to acquire hyperspectral images of the tea leaves, thereby obtaining hyperspectral image information of the tea leaves.

5. The method according to claim 1, characterized in that, The tea grade classification model based on a preset method performs fine-grained tea grade identification on the hyperspectral image information of the tea leaves to obtain tea grade information, including: A set of hyperspectral images of sample tea leaves and a preset initial tea grade classification model are obtained, wherein each sample tea hyperspectral image in the set of sample tea hyperspectral images corresponds to a true value of tea grade; Based on the set of hyperspectral images of the sample tea leaves, the initial tea grade classification model is trained to obtain a trained tea grade classification model. Based on a preset particle swarm optimization algorithm, the trained tea grade classification model is optimized to obtain an optimized tea grade classification model, which is used as the tea grade classification model. Based on the tea grade classification model, fine-grained tea grade identification is performed on the hyperspectral image information of the tea to obtain tea grade information.

6. The method according to claim 1, characterized in that, The process of using a camera mounted on a robotic arm to perform volume recognition on the tea leaves and obtain volume information includes: Based on each latitude information in the preset latitude information set, the robotic arm is adjusted and the turntable is controlled to rotate so that the camera can capture multiple images of the tea leaves to obtain a single-latitude tea leaf image group. The volume of each single-dimensional tea image group is coarsely identified to obtain the initial tea volume information; Based on the initial tea leaf volume information, a dense viewpoint pose information set is generated; Based on each dense viewpoint pose information in the dense viewpoint pose information set, the robotic arm is adjusted and the turntable is controlled to rotate, so as to acquire multiple images of the tea leaves through the camera to obtain a dense tea leaf image group. Volumetric precision recognition is performed on each of the obtained dense tea leaf image groups to obtain the volumetric information of the tea leaves.

7. A tea identification and packaging device for use in tea vending machines, characterized in that, include: The generation unit is configured to generate replenishment information based on the tea inventory information, tea storage cost, and tea replenishment cost corresponding to each tea category in the tea category set sold by the tea vending machine to be replenished, thus obtaining a replenishment information set; wherein, generating replenishment information includes: For each tea category in the aforementioned tea category set, perform the following steps: Obtain the tea demand rate corresponding to the tea category, wherein the tea demand rate can be the sales volume of the tea category per unit time; Based on a preset economic order quantity algorithm, an economic order quantity is generated according to the tea demand rate, the tea storage cost corresponding to the tea category, and the tea replenishment cost. Based on the tea inventory information and the tea demand rate, a replenishment threshold and replenishment time are generated; Based on the tea inventory information and the economic order quantity and The replenishment threshold is used to generate a predicted inventory sequence. The total inventory sequence is obtained by adding each of the generated predicted inventory sequences element by element. In response to determining that each total inventory in the total inventory sequence is less than or equal to the preset tea vending machine storage capacity, the generated economic order batches and replenishment times are determined as a replenishment information set; In response to determining that there exists a total inventory in the total inventory sequence that exceeds the preset storage capacity of the tea vending machine, the following steps are performed: Construct an objective function for each tea category in the tea category set, wherein the dependent variable of the objective function can be the total cost among the various tea categories; The objective function is optimized to generate the target replenishment quantity for each tea category; Each replenishment time and each replenishment quantity generated is defined as a replenishment information set; The release unit is configured to release the corresponding tea leaves onto the turntable via a feeder according to the replenishment information set; The hyperspectral image acquisition unit is configured to acquire hyperspectral images of the tea leaves on the turntable using a hyperspectral imager, thereby obtaining hyperspectral image information of the tea leaves. The fine-grained tea grade recognition unit is configured to perform fine-grained tea grade recognition on the hyperspectral image information of the tea based on a preset tea grade classification model to obtain tea grade information; The data acquisition unit is configured to acquire tea weight information via a weight sensor at the bottom of the turntable in response to determining that the tea grade information meets a preset replenishment grade condition. The volume recognition unit is configured to, in response to determining that the weight information of the tea leaves meets a preset replenishment weight condition, perform volume recognition on the tea leaves using a camera configured on the robotic arm to obtain the volume information of the tea leaves; The packaging unit is configured to package the tea according to the tea grade information and the tea volume information to obtain packaged tea, and to transport the packaged tea to the tea vending machine to be replenished.

8. An electronic device, characterized in that, include: One or more processors; A storage device on which one or more programs are stored; When the one or more programs are executed by the one or more processors, the one or more processors implement the method as described in any one of claims 1 to 6.

9. A computer-readable medium, characterized in that, It stores a computer program thereon, wherein the computer program, when executed by a processor, implements the method as described in any one of claims 1 to 6.

Images