Automatic beverage machine with visual detection and detection method thereof

Abstract

The application discloses an automatic beverage machine with visual detection and a detection method thereof. The automatic beverage machine with visual detection comprises a cabinet, a storage module for storing beverage capsules, an extraction module for extracting beverages by using the beverage capsules, a conveying module for receiving the beverage capsules falling from the storage module and conveying the beverage capsules to the extraction module, a cup falling module for storing and falling beverage containers, a container moving module, and a visual detection assembly. The container moving module can move axially below the cup falling module and the extraction module, and align the beverage containers placed at the top of the container moving module with one of the cup falling outlet and the extraction outlet. The visual detection assembly is used for detecting whether the modules, containers or capsules inside the cabinet operate according to the preset situation. Through the visual detection assembly and the detection method, the automatic beverage machine is monitored visually in the whole process from raw material storage, material falling, container supply, conveying to packaging, and the frequency of manual intervention and operation and maintenance cost are reduced.

Description

Technical Field

[0001] This invention relates to the field of automatic beverage machine technology, and in particular to an automatic beverage machine with visual inspection and its inspection method. Background Technology

[0002] An automatic beverage machine is a device that responds to user orders, instantly storing and transporting beverage ingredients, as well as brewing and extracting them, ultimately providing the user with a ready-to-drink beverage. However, hardware devices are not immune to malfunctions, breakdowns, or stockouts.

[0003] Most existing automatic beverage machines only respond when users discover malfunctions or stock shortages during use, prompting them to report to maintenance personnel. Maintenance personnel then dispatch to the machine's location for repairs. However, depending on the machine's application scenario, not all users have the contact information for maintenance personnel or the awareness to report issues. For example, when an automatic beverage machine is placed in a public area (such as a subway station), a customer who discovers a malfunction might simply choose not to use it.

[0004] In another scenario, maintenance personnel will periodically visit the distribution points to inspect, repair, or restock the beverage machines.

[0005] Regardless of the situation, the malfunction information of the automatic beverage machine cannot be fed back in a timely manner, affecting the efficiency of repair or restocking, and thus affecting the user experience. Summary of the Invention

[0006] The problem solved by this invention is to provide an automatic beverage machine with visual detection and a detection method thereof.

[0007] In a first aspect, embodiments of the present invention provide an automatic beverage machine with visual detection capabilities, comprising: Cabinet; A storage module, located inside the cabinet, is used to store beverage capsules; An extraction module, located inside the cabinet, is used to extract beverages using the beverage capsules; the bottom of the extraction module has an extraction outlet for extracting beverages. A conveying module is movably positioned between the storage module and the extraction module, used to receive the beverage capsules falling from the storage module and transport them to the extraction module; A cup-dropping module is used to store and drop beverage containers; the bottom end of the cup-dropping module has a cup-dropping opening for dropping beverage containers. A container moving module is movably disposed below the cup-dropping module and the extraction module; the container moving module can move axially below the cup-dropping module and the extraction module, aligning the beverage container placed at the top of the container moving module with one of the cup-dropping outlet and the extraction outlet; and A visual inspection component is installed inside the cabinet to detect whether at least one of the modules, containers, and capsules inside the cabinet moves according to a preset scenario.

[0008] In some embodiments, the visual inspection component includes a first visual inspection device and a second visual inspection device. The first visual inspection device is disposed on one side of the storage module and is positioned opposite to the initial position of the conveying module. The initial position of the conveying module is a position for receiving the falling beverage capsule. The second visual inspection device is disposed on one side of the container moving module.

[0009] In some embodiments, the automatic beverage machine with visual detection further includes a lid-dropping module and a lid-pressing module; the lid-dropping module and the lid-pressing module are arranged side by side with the extraction module and the cup-dropping module; the lid-dropping module is used to store beverage lids, and the bottom end of the lid-dropping module has a lid-dropping opening for dropping beverage lids; the lid-pressing module is movably disposed inside the cabinet, and the lid-pressing module can move vertically; the lid-pressing module is used to press the beverage lid on the beverage container placed at the top of the container moving module; the lid-dropping opening and the bottom end of the lid-pressing module are directly opposite the top end of the container moving module, and the container moving module can align the beverage container placed at the top of the container moving module with one of the lid-dropping opening and the bottom end of the lid-pressing module.

[0010] Secondly, embodiments of the present invention provide a detection method for an automatic beverage machine, applied to the automatic beverage machine with visual detection described in the first aspect. The automatic beverage machine further includes a control unit disposed within the cabinet. The visual detection component is electrically connected to the control unit, the control unit is electrically connected to each module in the automatic beverage machine, and the control unit is also communicatively connected to a cloud server. The detection method includes: If a beverage preparation instruction is received, the image of the target cargo channel in the storage module collected by the first vision detection device at the first moment is used as the first image data; The image of the target material channel at the second moment after the material dropping command is triggered is obtained as the second image data; The first image data and the second image data are compared at the pixel level to calculate the capsule displacement vector. Based on the capsule displacement vector, it is determined whether the target capsule has fallen and the inventory record is updated.

[0011] In some embodiments, after acquiring the image of the target cargo channel in the storage module collected by the visual detection component at a first moment as the first image data, the method further includes: Based on the first image data, determine whether the beverage capsules on the target cargo channel are in a suspended state; If there is no suspended state, send a dropping command to the storage module.

[0012] In some embodiments, determining whether a beverage capsule on the target delivery channel is suspended in mid-air based on the first image data includes: The distance between the edges of any adjacent capsules is identified. When a gap larger than the diameter of a single capsule is detected between the edges of adjacent capsules, it is determined to be a suspended gap. Based on the location and quantity of the suspended space gaps, a replenishment quality score is generated and linked to the replenishment record of the corresponding target cargo channel.

[0013] In some embodiments, the detection method for the automatic beverage machine further includes: If a replenishment instruction is received, the real-time image of the target cargo channel in the storage module during the replenishment process is obtained as the third image data; The posture information of the capsule is identified based on the third image data, including the angle between the capsule axis and the cargo channel axis; when the angle exceeds a preset threshold, an abnormal posture prompt is generated.

[0014] In some embodiments, the detection method for the automatic beverage machine further includes: If a cup-dropping command is received, the dynamic image sequence of the cup-dropping module dropping the beverage container, collected by the second vision detection device, is used as the fourth image data. Based on the fourth image data, the cup's posture during the falling process is identified to detect whether there is cup overlap or abnormal cup opening orientation.

[0015] In some embodiments, the detection method for the automatic beverage machine further includes: If a container delivery instruction is received, real-time images of the container moving module during the delivery of the beverage container are obtained as the fifth image data. Based on the fifth image data, the positional offset of the cup on the container moving module is tracked. When the positional offset exceeds the tolerance range, the risk of cup slippage is determined and a delivery pause command is triggered.

[0016] In some embodiments, the detection method for the automatic beverage machine further includes: If a capping command is received, obtain a comparison image before and after the capping action is performed as the sixth image data; Based on the sixth image data, the concentricity deviation between the cup lid and the cup body is identified. When the concentricity deviation exceeds the sealing threshold, an alarm for abnormal lid sealing is generated.

[0017] Beneficial Effects: This invention provides an automatic beverage machine with visual detection and its detection method. The automatic beverage machine with visual detection includes: a cabinet; a storage module disposed inside the cabinet for storing beverage capsules; an extraction module disposed inside the cabinet for extracting beverages using the beverage capsules; the bottom end of the extraction module has an extraction outlet for extracting beverages; a conveying module movably disposed between the storage module and the extraction module for receiving beverage capsules falling from the storage module and transporting them to the extraction module; and a cup-dispensing module for... The system includes a beverage container storage and dispensing mechanism; a dispensing port at the bottom of the dispensing module for dispensing the beverage container; a container moving module movably positioned below the dispensing module and the extraction module; the container moving module can move axially below the dispensing module and the extraction module, aligning the beverage container placed at the top of the moving module with one of the dispensing port and the extraction port; and a visual inspection component, located inside the cabinet, for detecting whether at least one of the modules, containers, and capsules inside the cabinet moves according to a preset scenario. By setting up the visual inspection component and using a detection method, the automatic beverage machine achieves full-process visual monitoring from raw material storage, dispensing, container supply, transportation to packaging, reducing the frequency of manual intervention and maintenance costs. Attached Figure Description

[0018] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the following description of the embodiments will be briefly introduced. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1 A flowchart illustrating the detection method for an automatic beverage machine provided in an embodiment of the present invention; Figure 2 This is a sub-flowchart of the detection method for an automatic beverage machine provided in an embodiment of the present invention; Figure 3 This is a sub-flowchart of the detection method for an automatic beverage machine provided in an embodiment of the present invention; Figure 4 A flowchart illustrating the detection method for an automatic beverage machine provided in an embodiment of the present invention; Figure 5 An internal structural diagram of an automatic beverage machine with visual detection provided in an embodiment of the present invention; Figure 6An internal structural diagram of an automatic beverage machine with visual detection provided in an embodiment of the present invention, viewed from another perspective; Figure 7 An internal side view of an automatic beverage machine with visual detection provided in an embodiment of the present invention; Figure 8 An internal front view of an automatic beverage machine with visual inspection provided in an embodiment of the present invention; Figure 9 This is an internal top view of an automatic beverage machine with visual inspection provided in an embodiment of the present invention.

[0020] The specific reference numerals in the attached figures are as follows: 1. Storage module; 2. Extraction module; 3. Conveying module; 4. Cup dropping module; 5. Container moving module; 6. Cap dropping module; 7. Capping module; 8. First vision inspection device; 9. Second vision inspection device; 10. Picking port. Detailed Implementation

[0021] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0022] It should be understood that, when used in this specification and the appended claims, the terms "comprising" and "including" indicate the presence of the described features, integrals, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or collections thereof.

[0023] It should also be understood that the terminology used in this specification is for the purpose of describing particular embodiments only and is not intended to limit the invention. As used in this specification and the appended claims, the singular forms “a,” “an,” and “the” are intended to include the plural forms unless the context clearly indicates otherwise.

[0024] It should also be further understood that the term "and / or" as used in this specification and the appended claims refers to any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0025] Please see Figures 5-9 , Figure 5 An internal structural diagram of an automatic beverage machine with visual detection provided in an embodiment of the present invention; Figure 6An internal structural diagram of an automatic beverage machine with visual detection provided in an embodiment of the present invention, viewed from another perspective; Figure 7 An internal side view of an automatic beverage machine with visual detection provided in an embodiment of the present invention; Figure 8 An internal front view of an automatic beverage machine with visual inspection provided in an embodiment of the present invention; Figure 9 This is an internal top view of an automatic beverage machine with visual detection provided in an embodiment of the present invention. The embodiment of the present invention provides an automatic beverage machine with visual detection, comprising: a cabinet (not shown); a storage module 1, disposed inside the cabinet, for storing beverage capsules; an extraction module 2, disposed inside the cabinet, for extracting beverages using the beverage capsules; the bottom end of the extraction module 2 has an extraction outlet for extracting beverages; a conveying module 3, movably disposed between the storage module 1 and the extraction module 2, for receiving beverage capsules falling from the storage module 1 and transporting them to the extraction module 2; and an ice-making module (not shown), which is connected to the extraction module 2. The system comprises: an ice-making module 2 for producing, storing, and discharging ice; an ice-making module with an ice outlet for discharging ice; a cup-dropping module 4 for storing and discharging beverage containers; a cup-dropping opening at the bottom of the cup-dropping module 4 for discharging beverage containers; and a container moving module 5 movably positioned below the extraction module 2, ice-making module, and cup-dropping module 4. The container moving module 5 can move axially below the extraction module 2, ice-making module, and cup-dropping module 4, aligning the beverage container placed at the top of the container moving module 5 with one of the extraction outlet, ice outlet, or cup-dropping opening. A visual inspection component, located inside the cabinet, is used to detect whether at least one of the modules, containers, and capsules inside the cabinet moves according to a preset scenario.

[0026] In this embodiment, the cabinet is a hollow cavity structure housing a control unit, a storage module 1, an extraction module 2, a conveying module 3, an ice-making module, a cup-dropping module 4, and a container moving module 5. The extraction module 2 may additionally include a brewing component for heating drinking water, allowing the extraction module 2 to extract the beverage capsules using hot water. The control unit can be a micro-CPU or other control device with computing capabilities. The control unit is electrically connected to the storage module 1, extraction module 2, conveying module 3, brewing component, ice-making module, cup-dropping module 4, container moving module 5, and vision detection component. It can send control commands to each module or component or receive status information from each module or detection information from the vision detection component to control the operation of each module within the cabinet. The cabinet can be a rectangular prism structure, with one side of the prism serving as a door. A hinge is provided on one side of the door, which is hinged to the rest of the cabinet, allowing the door to open from the cabinet via the hinge. After the cabinet door is opened, the various modules inside are exposed, allowing maintenance personnel to perform repairs, maintenance, and restocking on the visually inspected automatic beverage machine. During daily operation, the cabinet door can be locked with a latch to prevent damage to the internal modules. A retrieval slot 10 is provided on the cabinet door, and this slot 10 can be used in conjunction with a movable retrieval door. The retrieval door can be a sliding door structure located on the inside of the cabinet door (i.e., the side closest to the interior of the cabinet). When the visually inspected automatic beverage machine is not in operation, or when a complete beverage production cycle is not completed, the retrieval slot 10 is closed by the retrieval door. When the visually inspected automatic beverage machine completes a full beverage production cycle and needs to provide the user with a prepared hot or cold beverage, the retrieval door will slide open to one side, releasing the closure of the retrieval slot 10, allowing the user to obtain the brewed beverage located inside the cabinet through the retrieval slot 10. A touchscreen can be installed on the side of the cabinet door facing the customer for ordering, and a camera can be further installed on the touchscreen for facial recognition payment and other operations. Ventilation vents can be added to the other sides of the cabinet without doors, and exhaust fans can be connected to them to ventilate the interior, ensuring heat dissipation during operation and maintaining the storage quality of the beverage capsules. In addition, lighting can be installed inside the cabinet to provide good visibility for maintenance personnel. The touchscreen can also be electrically connected to a control unit, which can receive order information from the touchscreen and control other modules to produce beverages accordingly.

[0027] Storage module 1 is a rotatable module containing a cylindrical rotating cylinder rotatably mounted inside the cabinet, with its axis of rotation perpendicular to the ground. Multiple channels are arranged around the side walls of the rotating cylinder, with their central axes parallel to each other and all perpendicular to the ground. Each channel can store multiple beverage capsules. The beverage capsules contain beverage ingredients for brewing, which can be in powder or concentrate form. Each beverage capsule has an opening at the top and a storage cavity at the bottom. The opening engages with the channel, allowing the capsule to move up and down within the corresponding channel. Multiple beverage capsules can be stacked within a single channel; when the bottom capsule falls, the other capsules above it naturally move downwards.

[0028] Specifically, each conveyor channel has two sets of push rods at the bottom of its sidewall. These push rods are located above and below the bottommost beverage capsule, with the upper push rod positioned between the bottommost capsule and the capsule adjacent to it. Under normal conditions, the lower push rod is extended and protrudes from the sidewall of the conveyor channel, supporting all the beverage capsules from below the bottommost capsule. When the bottommost capsule needs to be lowered, the upper push rod extends to support all the other capsules except the bottommost one. Then, the lower push rod retracts into the conveyor channel, allowing the bottommost capsule to fall. The lower push rod then extends again to push the falling capsule out of the conveyor channel. Before the bottommost capsule falls, the conveyor module 3 moves to the designated falling position. After the capsule falls from the storage module 1, it is received by the conveyor module 3 and transported to the extraction module 2. Subsequently, the top rod at the top retracts into the cargo channel, allowing the other beverage capsules located at the top to move naturally downwards and be supported by the bottom rod.

[0029] When the beverage capsule is transported to extraction module 2, it is moved by conveyor module 3 to a position below the extraction outlet of extraction module 2. Extraction module 2 then performs the extraction operation on the beverage capsule. During extraction, the extraction outlet in extraction module 2 punctures the sealing membrane at the opening of the beverage capsule and extends into the capsule. The brewing component pumps pre-heated water into the beverage capsule. Specifically, hot water sprays from the extraction outlet and enters the beverage capsule. The hot water then soaks and extracts the beverage ingredients within the capsule.

[0030] Before the extraction operation, the cup-dropping module 4 will perform a cup-dropping operation. Specifically, the cup-dropping module 4 stores a certain number of beverage containers, which can be beverage cups or beverage bowls, or other containers capable of holding beverages. During the cup-dropping operation, the cup-dropping module 4 separates a single beverage container from multiple overlapping stored beverage containers and drops that beverage container. At this time, the container moving module 5 has moved below the cup-dropping module 4, and the beverage container, after falling from the cup-dropping module 4, will land on the container moving module 5.

[0031] The visual inspection component may include a first visual inspection device 8 and a second visual inspection device 9. The first visual inspection device 8 may be a first camera, which is set on one side of the storage module 1 and opposite to the initial position of the conveying module 3. It is used to detect whether the storage module 1 operates according to a preset scenario and whether the beverage capsules in the storage channel are assembled in the channel in a preset state. The initial position of the conveying module 3 is the position for receiving the falling beverage capsules. The second visual inspection device 9 may be a second camera, which is set on one side of the container moving module 5. It is used to detect whether the beverage container moves along a preset trajectory, including whether the cup is correctly placed, whether it is stably transported by the container moving module 5, and whether there is leakage. The visual inspection component sends the collected information to the control unit, which analyzes the collected information according to the pre-stored detection strategy and sends corresponding control commands to other modules or sends information, such as information on restocking or maintenance, to the cloud server connected to the control unit.

[0032] In one embodiment, the automatic beverage machine with visual detection further includes a lid-dropping module 6 and a lid-pressing module 7; the lid-dropping module 6 and the lid-pressing module 7 are arranged side by side with the ice-making module, the extraction module 2, and the cup-dropping module 4; the lid-dropping module 6 is used to store beverage lids, and the bottom end of the lid-dropping module 6 has a lid-dropping opening for dropping beverage lids; the lid-pressing module 7 is movably disposed inside the cabinet, and the lid-pressing module 7 can move vertically; the lid-pressing module 7 is used to press the beverage lid placed on the beverage container at the top of the container moving module 5; the lid-dropping opening and the bottom end of the lid-pressing module 7 are directly opposite the top end of the container moving module 5, and the container moving module 5 can align the beverage container placed at the top of the container moving module 5 with one of the lid-dropping opening and the bottom end of the lid-pressing module 7.

[0033] In this embodiment, the cap-dropping module 6 can store a certain number of beverage caps and drop them onto the beverage container placed on top of the container moving module 5. Then, the cap-pressing module 7 moves downwards and presses the caps firmly onto the beverage container, thus forming a fully packaged beverage for delivery to the user. Specifically, the beverage caps in the cap-dropping module 6 are stacked and stored within it. When capping is required, the container moving module 5 moves the beverage container already containing the beverage to directly below the cap-dropping opening. Then, the bottommost cap falls from the opening and onto the top of the beverage container. Subsequently, the container moving module 5 moves the beverage container with the cap to below the cap-pressing module 7. The cap-pressing module 7 moves downwards and abuts against the top of the cap, applying pressure and pressing it down a certain distance, thus sealing the beverage container tightly. Since the cap-dropping module 6, cap-pressing module 7, ice-making module, extraction module 2, and cup-dropping module 4 are all arranged side-by-side, the movement track of the container moving module 5 only needs to be set as a straight track. The container moving module 5 only needs to move the beverage container in a straight direction, thereby effectively improving the efficiency of beverage production and packaging. Alternatively, the cup-dropping module 4, ice-making module, extraction module 2, cap-dropping module 6, and cap-pressing module 7 can be arranged sequentially along the movement axis of the container moving module 5. This allows the container moving module 5 to complete all beverage production and packaging steps without reciprocating motion, simply moving in one direction, further improving production efficiency. The second vision device can also be used to detect abnormalities in the cap-dropping and cap-pressing processes.

[0034] See also Figure 1 This invention provides a detection method for an automatic beverage machine, applied to the automatic beverage machine with visual detection described in the above embodiments. The automatic beverage machine also includes a control unit disposed in the cabinet. The visual detection component is electrically connected to the control unit, the control unit is electrically connected to each module in the automatic beverage machine, and the control unit is also communicatively connected to a cloud server. The detection method includes steps S10 to S30.

[0035] S10. If a beverage preparation instruction is received, the image of the target cargo channel in the storage module collected by the first vision detection device at the first moment is obtained as the first image data.

[0036] In this embodiment, if a beverage preparation instruction is received, the image of the target cargo channel in the storage module at the first moment, collected by the first visual detection device, is used as the first image data. The first visual detection device is located on one side of the storage module and is used to collect images of the cargo channels in the storage module.

[0037] The beverage preparation instruction can be an internal instruction generated by the control unit after the user selects a beverage via the touchscreen, or a start instruction triggered after verification by the payment system. The first visual detection device is a high-resolution industrial camera located on one side of the storage module, with its field of view covering the side of the storage module closest to it. The first moment specifically refers to the instant before the material dropping action is executed, such as 100-200 milliseconds before the push rod is activated. The control unit sends a trigger signal to the corresponding first visual detection device based on the channel number information carried in the beverage preparation instruction. Upon receiving the trigger signal, the first visual detection device acquires an optical image of the target channel using a CMOS or CCD sensor, generates digital image data after analog-to-digital conversion, and transmits it to the control unit. The first image data is a RAW or JPEG format color / grayscale image with a resolution of no less than 1280×720 pixels to ensure clear identification of the outline features of individual beverage capsules. The control unit temporarily stores the received first image data in a local cache as a reference image for subsequent comparison. Simultaneously, the control unit extracts the timestamp information of the first image data, associates it with the order ID of the current beverage preparation task, and establishes a data index for subsequent traceability. By establishing a visual baseline state before the production process begins, a reliable reference system is provided for judging whether the subsequent material cutting is successful, avoiding misjudgments caused by unknown initial state. At the same time, by binding timestamps with orders, data traceability throughout the entire process is achieved.

[0038] See also Figure 2 In one embodiment, after acquiring the image of the target cargo channel in the storage module collected by the visual detection component at a first moment as the first image data, the method further includes: S11. Based on the first image data, determine whether the beverage capsules on the target cargo channel are suspended in mid-air.

[0039] In this embodiment, the system determines whether the beverage capsules on the target delivery channel are suspended based on the first image data. This step is performed after S10 and before sending the dropping command, serving as a preliminary safety check. The control unit analyzes the first image data to identify the arrangement of the capsules within the delivery channel. The suspended state refers to capsules not being tightly arranged, with unexpected gaps existing within the delivery channel, which may lead to the risk of the top rod not being able to lift properly or multiple capsules falling at once.

[0040] S12. If there is no suspended state, send a dropping command to the storage module.

[0041] In this embodiment, if there is no suspended state, a dropping command is sent to the storage module. After the control unit confirms that the capsules in the delivery channel are tightly arranged and there is no suspended state, it generates a dropping command and sends it to the motor driver of the storage module to start the push rod action. If a suspended state is detected, a replenishment quality warning is first sent to the cloud server or sent to the maintenance personnel's terminal through the cloud server, prompting them to reorganize the delivery channel or replenish the goods, and suspending the current production process.

[0042] Please refer to this together. Figure 3 In one embodiment, determining whether the beverage capsules on the target delivery channel are suspended in mid-air based on the first image data includes: S111. Identify the distance between the edges of any adjacent capsules. When a gap larger than the diameter of a single capsule is detected between the edges of adjacent capsules, it is determined to be a suspended gap.

[0043] In this embodiment, the distance between the edges of any adjacent capsules is identified. When a gap larger than the diameter of a single capsule is detected between the edges of adjacent capsules, it is determined to be a suspended gap, i.e., a suspended state exists. Specifically, the control unit uses an edge detection algorithm (such as the Sobel operator or the Laplacian operator) to extract the end face edges of all capsules in the first image data and fits them as elliptical or circular contours. Subsequently, the minimum distance dgap between adjacent contours in the direction of the cargo channel axis (i.e., the direction perpendicular to the ground) is calculated. The diameter of a single capsule is a pre-calibrated pixel value Dcap. If dgap > k·Dcap (where k is a tolerance coefficient, which can be 0.8-1.2), a suspended gap is determined to exist. The system can further determine whether the gap is at the top (there is a gap at the top) or in the middle (there is a hole in the capsule stack) based on the gap position.

[0044] S112. Based on the location and quantity of the suspended space gaps, generate a replenishment quality score and associate it with the replenishment record of the corresponding target cargo channel.

[0045] In this embodiment, a replenishment quality score is generated based on the location and quantity of the suspended gaps, and associated with the replenishment record of the corresponding target channel. The replenishment quality score Q can be calculated using the formula Q=100-∑(wi·gi), where gi is the pixel area of ​​the i-th suspended gap, and wi is the position weight (the top gap has a higher weight). The control unit associates the replenishment quality score with the current channel number and timestamp, stores it in a local database or uploads it to a cloud server, and uses it as a basis for evaluating the quality of replenishment operations by maintenance personnel, and to predict the probability of the channel becoming congested. For example, the lower the replenishment quality score, the higher the probability of congestion. In this way, preventive detection before material drop avoids mechanical failures caused by improper replenishment (such as damage to the mechanism by the top rod or blockage caused by the falling of two beverage capsules), and at the same time, the quantitative scoring mechanism promotes the standardization of maintenance operations and reduces long-term maintenance costs.

[0046] S20. Obtain the image of the target material channel at the second moment after the material dropping command is triggered as the second image data.

[0047] In this embodiment, the image of the target channel at the second moment after the dropping command is triggered is acquired as the second image data. The dropping command is a control signal sent by the control unit to the drive motor of the storage module after confirming that the first image data acquisition is complete and determining that the beverage capsules on the target channel are not suspended. This signal drives the push rod to fall the bottommost beverage capsule. The second moment specifically refers to the time point at which the dropping action is theoretically completed, for example, 300-500 milliseconds after the push rod retracts, ensuring that the capsule has completed falling or is in a stable stuck state. The second image data is also acquired by the first vision detection device, which is automatically triggered by the control unit after sending the dropping command, based on preset delay parameters (considering the push rod action time and the capsule falling time). The format and resolution of the second image data are consistent with the first image data to ensure the accuracy of pixel-level comparison. By precisely controlling the delay of the secondary sampling, the transient results after dropping are captured, providing direct visual evidence for judging the actual execution effect of the mechanical action, and making up for the inability of relying solely on sensors (such as photoelectric switches) to identify the capsule posture.

[0048] S30. Compare the first image data and the second image data at the pixel level, calculate the capsule displacement vector, determine whether the target capsule has fallen based on the capsule displacement vector, and update the inventory record.

[0049] In this embodiment, the first image data and the second image data are compared pixel-level to calculate the capsule displacement vector. Based on the capsule displacement vector, it is determined whether the target capsule has fallen, and the inventory record is updated. This step is executed by the image processing module in the control unit. The image processing module can be a software algorithm integrated into the main control chip, or it can be a separate FPGA or NPU hardware acceleration unit. If the capsule falls, the inventory quantity is decremented by one. When the inventory quantity is less than the safety quantity, the controller sends a message of insufficient stock to the cloud server or the mobile terminal of the maintenance personnel to remind the maintenance personnel to replenish the stock.

[0050] In one embodiment, determining whether the target capsule has fallen based on the capsule displacement vector and updating the inventory record includes: Extract the first centroid coordinates of the target capsule from the first image data and the second centroid coordinates of the corresponding capsule from the second image data; calculate the displacement difference between the first centroid coordinates and the second centroid coordinates; when the displacement difference is less than the effective falling threshold, it is determined that the capsule has not fallen; when it is determined that the capsule has not fallen, send a pause motion command to the conveying module and drive the push rod of the storage module to perform a secondary ejection action, and obtain the image again to verify the falling status.

[0051] In this embodiment, the first centroid coordinates of the target capsule in the first image data and the second centroid coordinates of the corresponding capsule in the second image data are extracted. Specifically, the control unit first preprocesses the two frames of images, including Gaussian filtering for noise reduction and illumination normalization, to eliminate interference caused by changes in ambient light. Subsequently, an edge detection algorithm or a deep learning-based instance segmentation model is used to extract the contour information of the target capsule in the image. Based on the extracted set of contour pixels, the centroid coordinates of the capsule are calculated using the geometric moment calculation method: for pixels (xi, yi) within the contour, the centroid coordinates (Cx, Cy) are calculated using the formula Cx = N∑xi, Cy = N∑yi, where N is the total number of pixels within the contour. The first centroid coordinates correspond to the geometric center position of the target capsule in the first image data, and the second centroid coordinates correspond to the geometric center position of the same capsule (if displacement has occurred) or adjacent capsules (if the target capsule has fallen) in the second image data.

[0052] Calculate the displacement difference between the coordinates of the first centroid and the coordinates of the second centroid. If the displacement difference is less than the effective fall threshold, it is determined that the fall did not occur. The displacement difference ΔD is calculated using the distance formula: ΔD2 = (Cx2 - Cx1) 2 +(Cy2-Cy1) 2 We calculate the square root of ΔD2 to obtain ΔD.

[0053] Where (Cx1, Cy1) are the first centroid coordinates, and (Cx2, Cy2) are the second centroid coordinates. The effective drop threshold is an empirical value determined based on the pixel size of the capsule in the image and its actual physical size, for example, 80%-120% of the capsule diameter in pixels. If ΔD is greater than or equal to this threshold, it indicates that the capsule has undergone a significant positional change and is judged to have fallen normally; if ΔD is less than this threshold, it indicates that the capsule's position has remained basically unchanged and is judged to be stuck or suspended.

[0054] Based on the judgment result, the corresponding control command is executed and the inventory record is updated. If the target capsule is determined to have fallen, the control unit sends a start command to the conveyor module, driving it to transport the beverage capsule to the extraction module; simultaneously, the control unit sends an inventory decrement command to the cloud server, updating the remaining inventory quantity of the target channel. If it is determined that the capsule has not fallen, the control unit immediately sends a pause command to the conveyor module, blocking subsequent processes and preventing the production of empty cups; simultaneously, the control unit sends a secondary ejection command to the drive motor of the storage module, driving the ejector rod to extend again to attempt to release the stuck capsule. After the secondary ejection action is completed, the control unit triggers the first vision detection device again to collect the image at the third moment as new second image data, repeating the comparison process of S30. If the capsule is determined to have fallen successfully after the secondary ejection, the normal process is resumed; if it is still determined that the capsule has not fallen, a fault code is generated and sent to the cloud server to notify the maintenance personnel for on-site handling. In this way, through the automatic error correction mechanism triggered by visual feedback, common jamming problems are resolved without manual intervention, improving the first-pass yield of the equipment and reducing downtime caused by minor faults.

[0055] Furthermore, after determining that the target capsule has fallen, image data of the top of the conveying module is acquired at the third moment after the material dropping command is triggered, and the target capsule is judged to have fallen to the target position based on the image data.

[0056] In this embodiment, the initial position of the conveying module is the location for receiving the falling beverage capsules, that is, directly below the target channel. The third moment is specifically a point in time several seconds after the second moment, for example, calculated according to the formula for free fall, which determines the time required for the beverage capsule to fall to the top of the conveying module. After the storage module triggers the dropping command and determines that the target capsule has fallen from the target channel, the image data of the top of the conveying module at the third moment, collected by the first visual detection device, is acquired. The first visual detection device is located on one side of the storage module, and is close to the conveying module; therefore, the conveying module is also within the detection range of the first visual detection device. Specifically, the top of the conveying module is a circular area with a concave center to receive the capsule. By judging whether the center point (top view) of the target capsule in the image data of the top of the conveying module coincides with the center point of the top of the conveying module, it is determined whether the target capsule has fallen onto the conveying module and whether it is located at the target position, that is, the receiving area on the top of the conveying module. If the center points of the two coincide, it is determined that the target capsule has fallen to the target position, and a transport command is sent to the conveying module to control the conveying module to transport the target capsule to the extraction module. If the center points of the two do not coincide, or the target capsule is not even on the transport module, it is determined that the target capsule has not landed at the target position. A pause movement command is sent to the transport module to block the subsequent process and prevent the empty cup from being made or the target capsule from falling off the transport module during transportation.

[0057] Please refer to this together. Figure 4 In one embodiment, the detection method of the automatic beverage machine further includes: S40. If a replenishment instruction is received, obtain the real-time image of the target cargo channel in the storage module during the replenishment process as the third image data.

[0058] In this embodiment, if a replenishment command is received, the real-time image of the target cargo channel in the storage module during the replenishment process is acquired as third image data. The replenishment command is triggered by maintenance personnel via a touchscreen or a dedicated maintenance APP, indicating that the system has entered replenishment mode.

[0059] The third image data is acquired in real time by the first vision detection device in the form of a video stream (e.g., 10-15 frames per second) and transmitted to the control unit for processing, used to monitor the capsule posture during the replenishment process.

[0060] S50. Based on the third image data, identify the posture information of the capsule being added, the posture information including the angle between the capsule axis and the cargo channel axis; when the angle is detected to exceed a preset threshold, generate a posture abnormality prompt.

[0061] In this embodiment, the posture information of the replenished capsule is identified based on the third image data. The posture information includes the angle between the capsule axis and the channel axis. When the angle exceeds a preset threshold, a posture abnormality prompt is generated. The control unit estimates the posture of the capsule in the third image data: first, it obtains the ellipse equation of the capsule end face through an ellipse fitting algorithm, where the major axis direction represents the axial direction of the capsule; then, it calculates the angle θ between this major axis direction and the perpendicular direction of the channel (channel axis direction). The preset threshold is 90 degrees. If θ exceeds this threshold, it indicates that the capsule is severely tilted. The control unit directly or through a cloud server sends a posture abnormality prompt (vibration + voice) to the smart terminal (such as a smartwatch or mobile APP) worn by the maintenance personnel, prompting them to adjust the posture of the capsule to prevent it from getting stuck in the channel during material delivery. In this way, real-time visual guidance is achieved in the replenishment process, transforming post-event correction into in-process intervention, reducing subsequent jamming failures caused by improper replenishment posture, and improving maintenance efficiency.

[0062] S60. If a cup-dropping instruction is received, the dynamic image sequence of the cup-dropping module dropping the beverage container, collected by the second vision detection device, is used as the fourth image data.

[0063] In this embodiment, upon receiving a cup-dropping command, the dynamic image sequence of the cup-dropping module as the beverage container falls, captured by the second vision detection device, is used as the fourth image data. The cup-dropping command is generated by the control unit after confirming successful beverage capsule dispensing, triggering the cup-dropping module's action. The second vision detection device is an industrial camera located on the lower side of the cabinet, covering the free-fall area from the cup-dropping opening to the receiving position of the container moving module, as well as the moving area of ​​the container moving module. The fourth image data is a high-speed continuous image sequence or short video clip, used to capture the dynamic process of the cup falling.

[0064] S70. Based on the fourth image data, identify the cup's posture during the falling process and detect whether there is cup overlap or abnormal cup opening orientation.

[0065] In this embodiment, the cup's posture during the falling process is identified based on the fourth image data to detect whether there is cup overlap or abnormal cup rim orientation. The control unit performs frame-by-frame analysis of the fourth image data: first, the cup outline in motion is extracted using background subtraction; then, the number and geometric features of the cup outline in each frame are calculated. If two or more cup outlines are detected to maintain a fixed relative position in time (overlap rate consistently greater than 90%) and fall simultaneously, it is determined to be cup overlap (double cup falling). For the cup rim orientation, the opening features of the cup outline (the cup rim is usually circular or elliptical, and the cup bottom is a closed shape) are analyzed to determine whether the cup is inverted. If double cups or inversion are detected, the control unit immediately sends a stop delivery command to the container moving module or a stop extraction command to the extraction module, and generates a fault alarm. In this way, dynamic visual monitoring solves the common problems of double cup adhesion and abnormal posture during the cup falling process, avoids unqualified containers from entering subsequent processes, and prevents beverage spillage or equipment contamination, thereby improving the stability of the output quality.

[0066] S80. If a container delivery instruction is received, the real-time image of the beverage container being delivered by the container moving module is obtained as the fifth image data.

[0067] In this embodiment, if a container transport command is received, real-time images of the beverage container being transported by the container moving module are acquired as the fifth image data. The container transport command is generated after confirming that the cup is placed normally, driving the container moving module to move along the guide rail. The fifth image data is acquired in real time by the second vision detection device in the form of a video stream, with a field of view covering the entire transport path of the container moving module, focusing on the stability of the cup placed at the top of the container moving module.

[0068] S90. Based on the fifth image data, track the positional offset of the cup on the container moving module. When the positional offset exceeds the tolerance range, determine the risk of cup slippage and trigger a delivery pause command.

[0069] In this embodiment, the positional offset of the cup on the container moving module is tracked based on the fifth image data. When the positional offset exceeds the tolerance range, a risk of cup slippage is determined and a transport pause command is triggered. The control unit uses template matching or feature point tracking (such as KLT optical flow) algorithms to track the relative position of the bottom center point of the cup relative to the top platform edge of the container moving module in consecutive frames. The pixel displacement ΔP between the current position and the initial placement position is calculated and converted into the actual physical displacement ΔL according to the camera calibration parameters. The tolerance range is set according to the cup diameter and platform size, for example, preset to ±5 mm. If |ΔL|>5mm, it indicates that the cup has slipped or tilted during transport. The control unit immediately sends a transport pause command to the motor driver of the container moving module and generates a position abnormality alarm to prevent the cup from being misaligned in subsequent extraction or capping stations, which could lead to accidents. In this way, dynamic stability monitoring of the transportation process can be achieved, and cup slippage caused by uneven platform or excessive acceleration can be detected in time, avoiding mechanical collisions or sealing failures caused by positional deviations during high-pressure extraction or capping.

[0070] S100. If a capping instruction is received, obtain a comparison image before and after the capping action is performed as the sixth image data.

[0071] In this embodiment, if a capping command is received, a comparison image before and after the capping action is executed is acquired as the sixth image data. The capping command is generated after the beverage extraction is completed. Specifically, the container moving module transports the beverage container to below the cap dropping module. After the cap is dropped, the container moving module transports the beverage container to below the capping module, triggering the capping module's action. The sixth image data includes a "pre-capping image" acquired before the capping module descends and a "post-capping image" acquired after the capping module resets, acquired by the second vision inspection device at the capping station.

[0072] S110. Based on the sixth image data, identify the concentricity deviation between the cup lid and the cup body. When the concentricity deviation exceeds the sealing threshold, generate a lid-pressing abnormality alarm.

[0073] In this embodiment, the concentricity deviation between the cup lid and the cup body is identified based on the sixth image data. When the concentricity deviation exceeds the sealing threshold, a cap-pressing abnormality alarm is generated. The control unit processes the pressed image: first, it identifies the outer contour of the top of the cup body and the outer contour of the cup lid, and obtains the center coordinates (xc, yc) and (xl, yl) of both through ellipse fitting. The concentricity deviation is then calculated. .

[0074] The sealing threshold is set based on the width of the lid's sealing ring, for example, 2-3 mm. If the decc exceeds this threshold, it indicates that the lid misalignment may lead to a poor seal. The control unit sends a capping anomaly alarm to the cloud server, indicating a risk of leakage for the beverage and suggesting manual re-inspection or disposal. Simultaneously, this anomaly is recorded for statistical analysis of the mechanical wear of the capping module. In this way, visual inspection enables online full inspection of capping quality, ensuring the integrity of the seal for every beverage, preventing transportation leaks and food safety hazards caused by cap misalignment, and providing data support for preventative equipment maintenance.

[0075] In a further embodiment, after the beverage preparation is complete, i.e., all the aforementioned processes are normal, the controller sends an opening command to the dispensing door to stop it from closing the dispensing opening. The container moving module is located on the side of the cabinet closest to the dispensing door, allowing the user to easily reach and remove the beverage from it. Since the second vision detection device is located on one side of the container moving module, it can also detect when the user reaches into the cabinet through the dispensing opening. The detection method for the automatic beverage machine also includes: After the pickup door triggers the opening command, the system acquires the seventh image data collected by the second vision detection device. Based on the seventh image data, it determines whether the user's limbs and beverage have both exited the pickup slot. Only when the user's limbs and beverage have both exited the pickup slot will the system send a closing command to the pickup door.

[0076] In this embodiment, the pickup door is closed while the beverage is still being prepared to prevent users from being scalded by hot water when their limbs are inserted into the pickup opening. Once the beverage is prepared and the pickup door opens, the user inserts their hand into the opening. At this point, the system continuously acquires seventh image data collected by the second visual detection device and determines whether the user's limbs or the beverage are present in the seventh image data. If, at any given moment, the seventh image data does not contain either the user's limbs or the beverage, meaning both have left the detection range of the second visual detection device and exited the pickup opening, a closing command is sent to the pickup door. Otherwise, no closing command is sent to the pickup door, leaving it open to prevent the door from pinching the user when it closes or closing before the user has taken the beverage, which would negatively impact the user experience.

[0077] Through the above embodiments, the present invention realizes full-process visual monitoring of automatic beverage machines from raw material storage, feeding, container supply, transportation to packaging and retrieval. Through multi-camera collaboration, time-series comparison analysis and closed-loop control, the intelligence level and operational reliability of the equipment are improved, and the frequency of manual intervention and maintenance costs are reduced.

[0078] The above are merely specific embodiments of the present invention, but the scope of protection of the present invention is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in the present invention, and these modifications or substitutions should all be covered within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. An automatic beverage machine equipped with visual detection, characterized in that, include: Cabinet; A storage module, located inside the cabinet, is used to store beverage capsules; An extraction module, located inside the cabinet, is used to extract beverages using the beverage capsules; the bottom of the extraction module has an extraction outlet for extracting beverages. A conveying module is movably positioned between the storage module and the extraction module, used to receive the beverage capsules falling from the storage module and transport them to the extraction module; A cup-dropping module is used to store and drop beverage containers; the bottom end of the cup-dropping module has a cup-dropping opening for dropping beverage containers. A container moving module is movably disposed below the cup-dropping module and the extraction module; the container moving module can move axially below the cup-dropping module and the extraction module, and align the beverage container placed at the top of the container moving module with one of the cup-dropping outlet and the extraction port; as well as A visual inspection component is installed inside the cabinet to detect whether at least one of the modules, containers, and capsules inside the cabinet moves according to a preset scenario.

2. The automatic beverage machine with visual detection according to claim 1, characterized in that, The visual inspection component includes a first visual inspection device and a second visual inspection device. The first visual inspection device is disposed on one side of the storage module and is positioned opposite to the initial position of the conveying module. The initial position of the conveying module is a position for receiving the falling beverage capsules. The second visual inspection device is disposed on one side of the container moving module.

3. The automatic beverage machine with visual detection according to claim 1, characterized in that, It also includes a lid-dropping module and a lid-pressing module; the lid-dropping module and the lid-pressing module are arranged side by side with the extraction module and the cup-dropping module; the lid-dropping module is used to store beverage lids, and the bottom end of the lid-dropping module has a lid-dropping opening for dropping beverage lids; the lid-pressing module is movably disposed inside the cabinet, and the lid-pressing module can move vertically; the lid-pressing module is used to press the beverage lid on the beverage container placed at the top of the container moving module; the lid-dropping opening and the bottom end of the lid-pressing module are directly opposite the top end of the container moving module, and the container moving module can align the beverage container placed at the top of the container moving module with one of the lid-dropping opening and the bottom end of the lid-pressing module.

4. A method for detecting an automatic beverage machine, characterized in that, The automatic beverage machine with visual detection as described in any one of claims 1 to 3 further includes a control unit disposed within the cabinet, the visual detection component being electrically connected to the control unit, the control unit being electrically connected to each module in the automatic beverage machine, and the control unit also being communicatively connected to a cloud server; the detection method includes: If a beverage preparation instruction is received, the image of the target cargo channel in the storage module collected by the first vision detection device at the first moment is used as the first image data; The image of the target material channel at the second moment after the material dropping command is triggered is obtained as the second image data; The first image data and the second image data are compared at the pixel level to calculate the capsule displacement vector. Based on the capsule displacement vector, it is determined whether the target capsule has fallen and the inventory record is updated.

5. The detection method for an automatic beverage machine according to claim 4, characterized in that, After acquiring the image of the target cargo channel in the storage module collected by the vision detection component at the first moment as the first image data, the method further includes: Based on the first image data, determine whether the beverage capsules on the target cargo channel are in a suspended state; If there is no suspended state, send a dropping command to the storage module.

6. The detection method for an automatic beverage machine according to claim 5, characterized in that, The step of determining whether the beverage capsules on the target delivery channel are suspended in mid-air based on the first image data includes: The distance between the edges of any adjacent capsules is identified. When a gap larger than the diameter of a single capsule is detected between the edges of adjacent capsules, it is determined to be a suspended gap. Based on the location and quantity of the suspended space gaps, a replenishment quality score is generated and linked to the replenishment record of the corresponding target cargo channel.

7. The detection method for an automatic beverage machine according to claim 4, characterized in that, Also includes: If a replenishment instruction is received, the real-time image of the target cargo channel in the storage module during the replenishment process is obtained as the third image data; The posture information of the capsule is identified based on the third image data, including the angle between the capsule axis and the cargo channel axis; when the angle exceeds a preset threshold, an abnormal posture prompt is generated.

8. The detection method for an automatic beverage machine according to claim 4, characterized in that, Also includes: If a cup-dropping command is received, the dynamic image sequence of the cup-dropping module dropping the beverage container, collected by the second vision detection device, is used as the fourth image data. Based on the fourth image data, the cup's posture during the falling process is identified to detect whether there is cup overlap or abnormal cup opening orientation.

9. The detection method for an automatic beverage machine according to claim 4, characterized in that, Also includes: If a container delivery instruction is received, real-time images of the container moving module during the delivery of the beverage container are obtained as the fifth image data. Based on the fifth image data, the positional offset of the cup on the container moving module is tracked. When the positional offset exceeds the tolerance range, the risk of cup slippage is determined and a delivery pause command is triggered.

10. The detection method for an automatic beverage machine according to claim 4, characterized in that, Also includes: If a capping command is received, obtain a comparison image before and after the capping action is performed as the sixth image data; Based on the sixth image data, the concentricity deviation between the cup lid and the cup body is identified. When the concentricity deviation exceeds the sealing threshold, an alarm for abnormal lid sealing is generated.

Images