Intelligent goods cabinet, interactive control method and system of intelligent goods cabinet

Abstract

The application discloses an intelligent goods cabinet, an interactive control method and system of the intelligent goods cabinet, and relates to the field of unmanned retail.The intelligent goods cabinet comprises a main control device arranged at the upper right corner of a cabinet body of the intelligent goods cabinet, wherein the main control device is provided with a first image acquisition device and a second image acquisition device, the first image acquisition device is arranged in the main control device through a sliding groove, the first image acquisition device slides in the sliding groove so that the distance between the first image acquisition device and the second image acquisition device can be adjusted, the acquisition range of the first image acquisition device is inside and outside the cabinet body of the intelligent goods cabinet, and the acquisition range of the second image acquisition device is outside the cabinet body of the intelligent goods cabinet.Based on the above-mentioned intelligent goods cabinet, the application also correspondingly provides an interactive control method and system of the intelligent goods cabinet.The application has the advantages of accurately identifying the category and quantity of goods and generating order information, reducing manual intervention, and improving the shopping experience of users.

Description

[0001] This invention is a divisional application of the invention patent application filed on November 9, 2023, entitled "Intelligent Vending Machine Interactive Control Method, Device and System Based on Machine Vision", with application number 202311483613.0. Technical Field

[0002] This invention relates to the field of Internet of Things (IoT) technology, and in particular to a smart vending machine, a smart vending machine interactive control method and system. Background Technology

[0003] Smart vending machines are self-service sales systems that utilize IoT, machine vision, and other technologies to provide goods and services in a more efficient and convenient way. Typically equipped with cameras, sensors, displays, and internet connectivity, smart vending machines can automatically monitor inventory, transmit data in real time, interact with users, and process payments. Available 24 / 7, smart vending machines are not limited by store opening hours, allowing users to purchase goods at any time without waiting, reducing queue time. Furthermore, they can automatically monitor and replenish inventory, reducing product expiration and waste. By analyzing sales data, retailers can better understand market demand and user preferences, enabling more informed stocking decisions.

[0004] Smart vending machines bring countless conveniences to users, but some problems often exist during use: hardware components of smart vending machines, such as cameras and electronic locks, may malfunction, leading to interruptions or errors in interaction; defects in camera image acquisition may result in the inability to accurately identify the category or quantity of goods; large data transmission delays may lead to long interaction processes or failed purchases, etc. These problems seriously affect the user experience. Summary of the Invention

[0005] In view of this, embodiments of the present invention provide a smart vending machine, a smart vending machine interactive control method and system, to solve the technical problem in the prior art that inaccurate image acquisition by unmanned retail vending machines affects image recognition, thereby resulting in a poor user experience.

[0006] In a first aspect, embodiments of the present invention provide a smart vending machine, the smart vending machine comprising: a main control device disposed at the upper right corner of the smart vending machine body, the main control device being provided with a first image acquisition device and a second image acquisition device, wherein the first image acquisition device is disposed in the main control device via a sliding groove, and the first image acquisition device slides in the sliding groove to adjust the distance between itself and the second image acquisition device; the acquisition range of the first image acquisition device is inside and outside the smart vending machine body, and the acquisition range of the second image acquisition device is outside the smart vending machine body.

[0007] Preferably, the angle between the viewing angle of the first image acquisition device and the plane where the smart vending machine door is located in the vertical direction is between 55° and 65°.

[0008] Preferably, the angle between the viewing angle of the second image acquisition device and the plane of the smart vending machine door in the horizontal direction is 55° to 65°.

[0009] Preferably, the second image acquisition device is fixedly installed at the upper right corner of the smart vending machine door, and the position of the first image acquisition device is adjusted by sliding according to the position, acquisition angle and acquisition range of the second image acquisition device.

[0010] Secondly, the present invention provides an interactive control method for an intelligent vending machine, the interactive control method comprising:

[0011] The first and second image acquisition devices of the smart vending machine are controlled to acquire a first continuous image at a preset time.

[0012] Once the first continuous image is successfully acquired, the electronic lock of the smart vending machine is controlled to perform an unlocking operation.

[0013] The first and second image acquisition devices are controlled to start acquiring the second continuous image until a locking signal is received from the electronic lock;

[0014] The acquired second continuous image is sent to the server, which then drives the server to send order information to the smart terminal.

[0015] Preferably, the first image acquisition device and the second image acquisition device controlling the smart vending machine acquire the first continuous image at a preset time, including:

[0016] The first image acquisition device and the second image acquisition device are controlled to simultaneously acquire the first sub-continuous image and the second sub-continuous image at a preset time.

[0017] The first continuous image is obtained by merging the first sub-continuous image and the second sub-continuous image.

[0018] Preferably, controlling the first image acquisition device and the second image acquisition device to start acquiring the second continuous image until receiving the locking signal from the electronic lock includes:

[0019] Extract the frame image containing the smart vending machine door information from the second continuous image, and denote it as the door image;

[0020] Cabinet door status information is generated in real time based on the cabinet door image, wherein the cabinet door status information includes at least one of cabinet door open and cabinet door closed.

[0021] When the locking signal is received, the cabinet door status information is obtained;

[0022] If the cabinet door status information indicates that the cabinet door is closed, then the image acquisition device is controlled to stop acquiring the second continuous image.

[0023] Preferably, controlling the first image acquisition device and the second image acquisition device to start acquiring the second continuous image until receiving the locking signal from the electronic lock includes:

[0024] The first and second image acquisition devices are controlled to capture the user's movement trajectory from opening the cabinet door to taking out the goods;

[0025] The product area where the user retrieved the item is located is determined based on the movement trajectory;

[0026] By comparing the frame images of the product area before the user takes out the product with the frame images of the product area after the user takes out the product, the information of the product taken out by the user can be determined.

[0027] Preferably, controlling the image acquisition device to acquire the second consecutive images further includes:

[0028] The initial frame image is obtained by extracting several frames of images initially acquired by the first image acquisition device and the second image acquisition device.

[0029] Obtain the sharpness level of the initial frame image and determine whether the sharpness level meets the preset image analysis requirements;

[0030] If not, adjust the acquisition frame rate of the first image acquisition device and the second image acquisition device according to the clarity level and the preset image analysis requirements;

[0031] The first few frames of images acquired by the first and second image acquisition devices after the frame rate is adjusted are denoted as the adjusted frame images.

[0032] Obtain the sharpness level of the adjusted frame image and determine whether the sharpness level of the adjusted frame image meets the preset image analysis requirements;

[0033] Continue until the resolution level meets the preset image analysis requirements.

[0034] Thirdly, the present invention provides an interactive control system for an intelligent vending machine, the system comprising: an intelligent vending machine and a server communicatively connected to the intelligent vending machine, wherein the intelligent vending machine is any of the intelligent vending machines described in the preceding claims.

[0035] In summary, the beneficial effects of the present invention are as follows:

[0036] The smart vending machine, the smart vending machine interactive control method and system provided in this invention embodiment, by setting a first image acquisition device and a second image acquisition device at different positions on the smart vending machine, and by adjusting the position between the two, can be widely applied to smart vending machines of different sizes. Furthermore, it can achieve real-time and accurate image acquisition for users during shopping, so as to accurately identify the product category and quantity and generate order information, making the entire transaction process more convenient and efficient, and providing users with a good and convenient shopping experience. Attached Figure Description

[0037] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the embodiments of the present invention will be briefly introduced below. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort, and these are all within the protection scope of the present invention.

[0038] Figure 1 This is a schematic diagram of the intelligent vending machine system according to Embodiment 1 of the present invention.

[0039] Figure 2 This is a structural schematic diagram of the intelligent vending machine according to Embodiment 1 of the present invention.

[0040] Figure 3 This is a flowchart illustrating the intelligent vending machine interactive control method based on machine vision according to Embodiment 1 of the present invention.

[0041] Figure 4 yes Figure 3 A schematic diagram of the specific process for step S1.

[0042] Figure 5 yes Figure 3 A schematic diagram of the specific process for step S1.

[0043] Figure 6 yes Figure 3 A schematic diagram of the specific process for step S3.

[0044] Figure 7 yes Figure 3 A schematic diagram of the specific process for step S4.

[0045] Figure 8 yes Figure 3 A schematic diagram of the specific process for step S5.

[0046] Figure 9 This is a flowchart illustrating the intelligent vending machine interactive control method based on machine vision according to Embodiment 2 of the present invention.

[0047] Figure 10 This is a schematic diagram of the structure of the intelligent vending machine interactive control device based on machine vision according to Embodiment 3 of the present invention.

[0048] Figure 11 This is a schematic diagram of the structure of the intelligent vending machine interactive control device based on machine vision according to Embodiment 4 of the present invention.

[0049] Figure 12 This is a schematic diagram of the structure of the intelligent vending machine main control system according to an embodiment of the present invention. Detailed Implementation

[0050] The features and exemplary embodiments of various aspects of the present invention will now be described in detail. To make the objectives, technical solutions, and advantages of the present invention clearer, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only configured to explain the present invention and are not configured to limit the present invention. For those skilled in the art, the present invention can be practiced without some of these specific details. The following description of the embodiments is merely intended to provide a better understanding of the present invention by illustrating examples of the invention.

[0051] It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising..." does not exclude the presence of additional identical elements in the process, method, article, or apparatus that includes said element.

[0052] Example 1

[0053] This invention provides a machine vision-based intelligent vending machine interaction control method, applicable to the intelligent vending machine itself within an intelligent vending machine system. For example... Figure 1As shown, the smart vending machine system includes a smart vending machine 1 and a server 2. The smart vending machine 1 and the server 2 are connected via wired or wireless means. In practical applications, after receiving a purchase request from a user, the smart vending machine will open its door to allow the user to select goods. During this process, the smart vending machine uses image acquisition devices such as MIPI cameras, IPC cameras, or USB cameras set on its top to capture the user's selection of goods and sends the captured images or videos to the server. The server analyzes and processes this image data to determine the type and quantity of goods purchased by the user, and then sends the order information to the user's smartphone or the human-computer interaction device of the smart vending machine, such as the display screen. After the user confirms that the order information is correct, they pay the corresponding amount to complete the transaction.

[0054] In one embodiment, such as Figure 1 As shown, a main control device 10 for the smart vending machine is located in the upper right corner. This main control device 10 has two image acquisition devices: a first image acquisition device 11 and a second image acquisition device 12. The first image acquisition device 11 is mounted in the main control device via a sliding groove (not shown), allowing it to slide within the groove and adjust its distance from the second image acquisition device 12. Preferably, the first image acquisition device 11 captures images of the inside and outside of the smart vending machine, while the second image acquisition device 12 captures images of the outside of the smart vending machine. The two image acquisition devices capture images of the inside and outside of the smart vending machine and send them to a server for analysis of user behavior and operations, such as the type and quantity of goods taken by the user, to determine order information. Preferably, the angle between the viewing direction of the first image acquisition device 11 and the plane of the smart vending machine door in the vertical direction is between 55° and 65°, and the angle between the viewing direction of the second image acquisition device 12 and the plane of the smart vending machine door in the horizontal direction is between 55° and 65°. During installation, the second image acquisition device 12 is fixedly installed at the upper right corner of the smart vending machine door. The position of the first image acquisition device is adjusted by sliding according to the position, acquisition angle, and acquisition range of the second image acquisition device 12. By continuously adjusting the distance between the first image acquisition device 11 and the second image acquisition device 12, as well as the viewing angles of the first image acquisition device 11 and the second image acquisition device 12, the acquisition range of the first image acquisition device 11 and the acquisition range of the second image acquisition device 12 are optimized. The optimal effect can be determined according to the actual application. For example, the acquisition range of the first image acquisition device needs to cover a first preset area, the acquisition range of the second image acquisition device needs to cover a second preset area, and the overlap between the first preset area and the second preset area is greater than or equal to a certain size. The first preset area and the second preset area can be any area inside or outside the cabinet that can be acquired by the two image acquisition devices.

[0055] Please see Figure 3 The intelligent vending machine interaction control method based on machine vision specifically includes the following steps:

[0056] S1: In response to the start command from the server, control the hardware device of the smart vending machine to perform a hardware self-test and obtain the hardware self-test result, the hardware self-test result including normal or abnormal.

[0057] S2: When the hardware self-test result is normal, control the image acquisition device of the smart vending machine to acquire the first continuous image at a preset time.

[0058] S3: When the first continuous image is successfully acquired, control the electronic lock of the smart vending machine to perform the unlocking operation;

[0059] S4: Control the image acquisition device to start acquiring the second consecutive image until a locking signal is received from the electronic lock;

[0060] S5: Send the acquired second continuous image to the server and drive the server to send order information to the smart terminal.

[0061] Specifically, when a user wants to purchase goods from the smart vending machine, they can scan the QR code provided by the smart vending machine with their smartphone or click the purchase button on the smart vending machine's human-computer interaction device (such as a touch screen) to send a start request to the server. Upon receiving the start request from the smart terminal, the server will issue a start command to the smart vending machine. After receiving the start command, the smart vending machine's main control system will first perform a hardware self-test to ensure that the smart vending machine hardware can operate normally and stably throughout the entire transaction process. After the hardware self-test is completed, the system will obtain the result of the hardware self-test, i.e., normal or abnormal. If the hardware self-test result is abnormal, the system will push the abnormal information to the smart terminal to inform the user that the smart vending machine cannot be used and to the smart vending machine operator to inform the operator to perform timely maintenance.

[0062] When the hardware self-test result is normal, the image acquisition device acquires a first series of images for a preset time. This preset time can be determined based on actual conditions, preferably 2-3 seconds. Once the first series of images is successfully acquired, the smart vending machine sends an unlocking command to the electronic lock and unlocks it. The user can then open the door and select goods. Simultaneously, the image acquisition device continuously acquires images of the user selecting goods, obtaining a second series of images. This continues until the user finishes selecting goods and closes the door, triggering the electronic lock to send a locking signal. The acquired second series of images is then sent to the server. The server analyzes and processes these images, identifying the type and quantity of goods selected by the user. Based on this, it determines the order amount and then pushes the order information to the smart terminal to inform the user. The user then completes the payment based on the order information, thus completing the transaction. After the transaction is completed, preferably, the smart vending machine enters standby mode to save power.

[0063] Please see Figure 4 In one embodiment, in response to a start command from the server, the smart vending machine is controlled to perform a hardware self-test and obtain the hardware self-test result, wherein the hardware self-test result includes normal or abnormal, including:

[0064] S11: In response to the start command, control the main control system of the smart vending machine to perform system initialization;

[0065] S12: Detect and verify the working status of the hardware devices in the smart vending machine, wherein the hardware devices include one or more of an image acquisition device, an electronic lock, and a network communication module;

[0066] S13: When the system is successfully initialized and the hardware devices are all in normal working condition, the hardware self-test result is normal; otherwise, it is abnormal.

[0067] Specifically, after receiving the start command, the smart vending machine first performs basic system initialization, including allocating memory and resources. After successful system initialization, it begins to detect and verify the working status of the image acquisition device, the electronic lock, the network communication module, and the temperature and humidity sensor inside the vending machine. If the working status of the image acquisition device, electronic lock, network communication module, and other hardware devices is normal, the hardware self-test result is normal; otherwise, it is abnormal.

[0068] In one embodiment, see Figure 5 In response to a start command from the server, controlling the smart vending machine to perform a hardware self-test and obtain the hardware self-test results also includes:

[0069] S14: The process information generated during the hardware self-test is sent to the server in real time. The process information will be input into the server's preset hardware self-test prediction learning model and generate hardware self-test result prediction information.

[0070] S15: Obtain the hardware self-test result prediction information and determine necessary and unnecessary self-test items based on it;

[0071] S16: Perform the hardware self-test operation of the smart vending machine according to the necessary self-test items.

[0072] To improve hardware self-test efficiency, process and result information generated during the hardware self-test is sent to the server in real time. A machine learning model—a preset hardware self-test prediction learning model—is pre-programmed on the server. This model, trained and learned from historical data, can predict the hardware self-test result under current conditions based on the process information generated during the real-time hardware self-test and returns the predicted result information. The smart vending machine uses this predicted result information to determine necessary and unnecessary self-test items. For example, if the server-returned hardware self-test result prediction information indicates that the connection status of the image acquisition device may be abnormal, then the connection status detection and verification of the image acquisition device must be performed. If multiple hardware self-test results indicate that the temperature and humidity meter is stable, then the temperature and humidity meter detection can be considered an unnecessary self-test item. Adding a machine learning model to the hardware self-test process to predict the smart vending machine's self-test result under current hardware conditions can skip unnecessary self-test steps, shorten self-test time, optimize system resource usage, improve efficiency, and also help predict potentially problematic hardware devices to notify operators to take preventative maintenance measures, thus improving user experience.

[0073] In this embodiment, the image acquisition device is as follows: Figure 2 The diagram shows at least a first image acquisition device and a second image acquisition device. The acquisition areas of the first and second image acquisition devices do not completely overlap. Therefore, the first continuous image here includes a first sub-continuous image and a second sub-continuous image acquired by the first image acquisition device. Please refer to [link to relevant documentation]. Figure 6 When the hardware self-test result is normal, the image acquisition device of the smart vending machine is controlled to acquire a first continuous image at a preset time, and the acquired first continuous image includes:

[0074] S21: Control the first image acquisition device and the second image acquisition device to simultaneously acquire the first sub-continuous image and the second sub-continuous image at a preset time;

[0075] S22: Combine the first sub-continuous image and the second sub-continuous image to obtain the first continuous image.

[0076] Specifically, when the hardware self-test result is normal, the first and second image acquisition devices are controlled to simultaneously acquire continuous images within a preset time period, which are respectively denoted as the first sub-continuous image and the second sub-continuous image. Since the acquisition ranges of the first and second image acquisition devices do not completely overlap, the first and second sub-continuous images record different information, which will be used to obtain user operations and product category quantities, etc. The first and second sub-continuous images are then combined to obtain the first continuous image. Furthermore, analysis and processing can be performed to determine whether the first and second sub-continuous images meet preset requirements, such as resolution, frame rate, and image acquisition range. Only if the acquired images meet the preset requirements will the order information obtained from the subsequent analysis and processing of these images be accurate. If they do not meet the preset requirements, the smart vending machine's image acquisition device can be notified to correct the acquisition parameters and re-acquire images that meet the preset requirements.

[0077] After the first series of images is successfully acquired, the smart vending machine unlocks its electronic lock, allowing the user to open the door and select goods. Simultaneously, the image acquisition device continuously captures images of the user selecting goods, creating a second series of images, until the user finishes selecting and closes the door, triggering the electronic lock to send a locking signal. To ensure the integrity and accuracy of the second series of image acquisition, please refer to [link to relevant documentation]. Figure 7 The control of the image acquisition device to start acquiring a second consecutive image until a locking signal is received from the electronic lock includes:

[0078] S31: Extract the frame image that includes the smart vending machine door information from the second continuous image, and denot it as the door image;

[0079] S32: Generate cabinet door status information in real time based on the cabinet door image, wherein the cabinet door status information includes at least one of cabinet door open and cabinet door closed;

[0080] S33: Obtain the cabinet door status information when the locking signal is received;

[0081] S34: If the cabinet door status information indicates that the cabinet door is closed, then control the image acquisition device to stop acquiring the second continuous image.

[0082] Specifically, the second continuous image records information such as the user opening the cabinet door after unlocking, the door's open state, selecting goods and the type and quantity of selected goods, and closing the cabinet door after selection. When the cabinet door is closed, the electronic lock will lock accordingly and send a locking information to the main control system. At this time, the main control system considers that the user has completed the goods retrieval and ends the current acquisition of the second continuous image. To avoid abnormal states of the electronic lock during use that could cause the locking information to be sent prematurely, ending the acquisition of the second continuous image before the cabinet door is closed, thus affecting the accuracy of the order due to incomplete information acquisition, in this embodiment of the invention, the frame image containing the cabinet door information in the second continuous image is extracted and denoted as the cabinet door image. Real-time cabinet door status information is obtained based on the image. When the locking information of the electronic lock is received, it is further determined whether the cabinet door is closed based on the cabinet door status information. If so, the image acquisition device is notified to stop acquiring the second continuous image.

[0083] In practical applications, when a user opens a cabinet door to retrieve a product, they may unintentionally or intentionally obscure the product, preventing the image acquisition device from capturing an image of the product and thus making it impossible to obtain product information. To prevent the image acquisition device from failing to directly capture images of the product during the retrieval process, in one embodiment, controlling the image acquisition device to capture a second consecutive image includes:

[0084] The image acquisition device is controlled to capture the user's movement trajectory from opening the cabinet door to taking out the goods;

[0085] The product area where the user retrieved the item is located is determined based on the movement trajectory;

[0086] By comparing the frame images of the product area before the user takes out the product with the frame images of the product area after the user takes out the product, the information of the product taken out by the user can be determined.

[0087] In this embodiment, the acquisition range of the image acquisition device is adjusted so that it can completely capture the overall image of the user and the images of each product area within the cabinet. Since smart vending machines often hold a variety of products with different prices, it is preferable to place similar products with the same or similar prices in the same area. For example, beverages with the same or similar prices but lower prices are placed on the first shelf of the smart vending machine, which is the first product area. Higher-priced beverages are placed on the second shelf (the second product area), bread with the same or similar prices but lower prices is placed on the third shelf (the third product area), and higher-priced items are placed on the fourth shelf (the fourth product area), and so on. When acquiring the first continuous image after hardware self-testing, the acquired product placement information and the information of the product area where each product is located can be recorded. When a user opens the smart vending machine door, the image acquisition device begins to capture a second series of images. These images record the user's movement trajectory, from opening the door, reaching into the vending machine, placing their hand in a specific product area, and moving their hand out of the vending machine. If the user unintentionally or intentionally obscures a product during the process of retrieving it, preventing the image acquisition device from obtaining product image information, the device determines the product area where the retrieved product is located based on the user's movement trajectory. Simultaneously, it extracts frame images of the product area before the user retrieves the product and frame images of the product area after the user retrieves the product. By comparing these two types of frame images, the quantity and price of the retrieved product are determined.

[0088] Furthermore, in practical applications, the following situations may occur: the image acquisition device's frame rate is set too low, and the user's action of taking out the goods is too fast, resulting in blurry images and affecting the confirmation of product information. In one embodiment, controlling the image acquisition device to acquire a second continuous image further includes:

[0089] S301: Extract several frames of images initially acquired by the image acquisition device, which are the initial frame images;

[0090] S302: Obtain the sharpness level of the initial frame image and determine whether the sharpness level meets the preset image analysis requirements;

[0091] S303: If not, adjust the acquisition frame rate of the image acquisition device according to the resolution level and the preset image analysis requirements;

[0092] S304: Reacquire the previous few frames of images acquired by the image acquisition device after adjusting the acquisition frame rate, and record them as the adjusted frame images;

[0093] S305: Obtain the sharpness level of the adjusted frame image and determine whether the sharpness level of the adjusted frame image meets the preset image analysis requirements; repeat steps S303 to S305 until the sharpness level meets the preset image analysis requirements.

[0094] To ensure that the image clarity acquired by the image acquisition device meets the requirements of subsequent image analysis, in this embodiment, multiple clarity levels can be set before image acquisition to determine the preset image analysis requirements and corresponding clarity levels needed to obtain the required product information from the acquired images. At the start of image acquisition, the first few frames are extracted and their clarity is checked against the preset image analysis requirements. If not, the frame rate of the image acquisition device needs to be adjusted to meet these requirements. In other embodiments, while adjusting the frame rate, parameters affecting image clarity, such as exposure and light source, can also be adjusted. This ensures that the image acquisition device, even under conditions of rapid user movement, can acquire clear images for subsequent product information retrieval.

[0095] In practical applications, image acquisition devices may be unintentionally or maliciously obstructed. In one embodiment, the image acquisition device includes at least two image acquisition devices. When one image acquisition device is detected to be obstructed, the image acquisition range of the other image acquisition device is automatically adjusted so that the acquired image meets the requirements for subsequent image analysis to obtain product information. Furthermore, the frame rate of the unobstructed image acquisition device can be increased, and parameters such as light source brightness, exposure, sensitivity, and autofocus can be adjusted to ensure that even using only one image acquisition device, an image meeting the requirements for subsequent image analysis can be acquired. If both image acquisition devices are detected to be obstructed, the smart vending machine can be controlled to issue an alarm and initiate a door locking operation. Simultaneously, the information regarding the obstruction of the image acquisition devices can be reported to the operator via a server to notify the operator to promptly resolve the obstruction fault.

[0096] In one embodiment, an edge computing module is installed at the smart vending machine, and after controlling the image acquisition device to start acquiring a second continuous image until a locking signal is received from the electronic lock, the following is also included:

[0097] Determine whether the connection between the smart vending machine and the server is normal;

[0098] If not, the second continuous image is sent to the edge computing module for image analysis and processing to generate order information.

[0099] Under normal communication network conditions, the captured images are sent to the server for image analysis and processing to generate an order, which is then pushed to the smart terminal. When significant latency or anomalies occur in the communication network, an edge computing module is installed at the smart vending machine to ensure normal transaction processing. In the event of communication latency or anomalies, the captured images can be directly transmitted to the edge computing module, which then identifies the product information based on the captured images and generates an order. This avoids the inability to generate orders and complete transactions in a timely manner when there is a delay or anomaly in the connection with the server.

[0100] In one embodiment, the method of controlling the image acquisition device to acquire the second consecutive images further includes:

[0101] The acquired second continuous image is sent to the edge computing module in real time for analysis and processing.

[0102] When the edge computing module detects that the user has taken out an item and has not taken out any other items within a preset interval time based on the second continuous image, or when it detects that the user has left the smart vending machine, it controls the smart vending machine to send a settlement instruction to the server to drive the server to generate order information.

[0103] Control the smart vending machine door to perform a closing operation.

[0104] If a user forgets to close the locker door after taking out goods, or if the door remains open for some reason, in order to avoid affecting the price settlement and unnecessary energy consumption, the edge computing module can analyze the images captured by the image acquisition device to determine whether the user has taken out goods and whether there has been any other operation to take out goods within a preset interval, or if the user has left the smart vending machine. On the one hand, the smart vending machine can promptly send a settlement instruction to the server to notify the server to settle the transaction and generate an order. On the other hand, the locker door can be automatically closed to avoid energy consumption.

[0105] In one embodiment, the image acquisition device is as follows: Figure 2 The image shown includes at least a first image acquisition device and a second image acquisition device, wherein the acquisition areas of the first image acquisition device and the second image acquisition device do not completely overlap, and the second continuous image includes a third sub-continuous image acquired by the first image acquisition device and a fourth sub-continuous image acquired by the second image acquisition device. (See also...) Figure 8 The step of sending the acquired second continuous image to the server and driving the server to send order information to the smart terminal includes:

[0106] S41: Obtain the first key frame image in the third sub-continuous image that records preset key information, and perform data segmentation processing on the first key frame image to obtain the first preset key image therein.

[0107] S42: Obtain the second key frame image in the fourth sub-continuous image that records preset key information, and perform data segmentation processing on the second key frame image to obtain the second preset key image therein;

[0108] S43: Obtain the data volume of the first preset key image and the data volume of the second preset key image respectively, and denot them as the third data volume and the fourth data volume;

[0109] S44: Determine the third transmission channel corresponding to the first preset key image and the fourth transmission channel corresponding to the second preset key image based on the third data volume and the fourth data volume, respectively.

[0110] S45: The first preset key image and the second preset key image are sent to the server through the third transmission channel and the fourth transmission channel respectively, and the server is driven to perform data processing and analysis based on the first preset key image and the second preset key image and then send the order information to the smart terminal.

[0111] Specifically, to improve data transmission and image processing efficiency, in this embodiment of the invention, before transmitting image data to the server, the image data is first processed by intelligent data segmentation, extracting only keyframe images and image information of key areas within the keyframe images. For example, the first image acquisition device's acquisition range includes the inside and outside of the smart vending machine, while the second image acquisition device's acquisition range is the outside of the smart vending machine. The second continuous image includes overall information about the user standing outside the smart vending machine, information about the smart vending machine door, overall information about the inside of the smart vending machine, information about all goods, information about the user's product selection process, and information about the user taking out goods. Since order information is mainly determined through information about the user's product selection process and information about the user taking out goods (i.e., preset key information), the image recording this information is recorded as a keyframe image. The frame image recording this preset key information in the third sub-continuous image is recorded as the first keyframe image. Similarly, the frame image recording this preset key information in the fourth sub-continuous image is recorded as the first keyframe image. The frame image containing key information is designated as the second keyframe image. Further, information about key regions is extracted from these keyframe images. For example, during the user's shopping process, besides the actions of the arm reaching into the smart vending machine to select and grab goods, other body information such as posture and facial expressions do not contribute to the accuracy of order information generation. Therefore, intelligent data segmentation is performed on each first and second keyframe image, removing images of the user's body or face, and only acquiring images of regions that are effective in determining the type and quantity of goods selected by the user for order generation. These are designated as the first and second preset key images, respectively. The third and fourth data volumes corresponding to the first and second preset key images are then obtained. The third and fourth data volumes are reduced compared to the data volumes of the third and fourth sub-continuous images, thereby alleviating data transmission pressure and improving processing efficiency. Preferably, suitable third and fourth transmission channels are dynamically allocated to the first and second preset key images based on the size of the third and fourth data volumes. For example, when the third data volume is large, the bandwidth of the third transmission channel is adjusted to a larger bandwidth. After receiving the first and second preset key images, the server analyzes and processes them to obtain information such as the type of goods selected by the user and related data. Based on this information, it generates a corresponding order and sends it to the user's smartphone or the human-computer interaction device of the smart vending machine. The user then pays for the goods according to the order information, thus completing the transaction.

[0112] In one embodiment, when the image acquisition device malfunctions due to obstruction or improper parameter settings, or when communication between the smart vending machine and the server fails, the edge computing module installed in the smart vending machine can be used for automatic fault handling. The machine vision-based smart vending machine interactive control method includes:

[0113] In response to a start command from the server, the hardware device of the smart vending machine is controlled to perform a hardware self-test and obtain the hardware self-test result, wherein the hardware self-test result includes normal or abnormal.

[0114] When the hardware self-test result is normal, the image acquisition device of the smart vending machine is controlled to acquire the first continuous image at a preset time.

[0115] Once the first continuous image is successfully acquired, the electronic lock of the smart vending machine is controlled to perform an unlocking operation.

[0116] When the first continuous image acquisition fails, the reason for the acquisition failure is obtained and sent to the edge computing module, wherein the edge computing module formulates a corresponding image acquisition device processing strategy based on the reason for the acquisition failure.

[0117] The smart vending machine is controlled to adjust the image acquisition device according to the image acquisition device processing strategy and then perform the unlocking operation.

[0118] The image acquisition device is controlled to start acquiring a second continuous image and transmit the second continuous image to the edge computing module in real time, wherein the edge computing module is used to analyze and process the received second continuous image;

[0119] When the smart vending machine receives a locking signal from the electronic lock within the preset maximum retrieval time, it determines whether the connection between the smart vending machine and the server is normal. If yes, it sends the collected second continuous image to the server and drives the server to send order information to the smart terminal. If no, it controls the edge computing module to generate order information based on the received second continuous image and sends the order information to the smart terminal.

[0120] If the smart vending machine does not receive the locking signal within the preset maximum retrieval time, the edge computing module is controlled to determine whether the user has finished taking the goods based on the second continuous image. If so, it is determined whether the connection between the smart vending machine and the server is normal. If the connection is normal, the smart vending machine is controlled to send a settlement instruction to the server to drive the server to generate order information. If the connection is abnormal, the edge computing module is controlled to generate order information based on the received second continuous image and send the order information to the smart terminal.

[0121] Specifically, after the smart vending machine starts up in response to the start command from the server and performs a hardware self-test, the image acquisition device acquires the first continuous images at a preset time. If there are no abnormalities in the image acquisition device, the first continuous image acquisition is successful, and the electronic lock of the smart vending machine is controlled to perform an unlocking operation. If the image acquisition device fails to acquire images normally due to improper parameter settings or partial obstruction, the first continuous image acquisition is considered to have failed. The reason for the acquisition failure is obtained and sent to the edge computing module. The edge computing module formulates a corresponding image acquisition device processing strategy based on the reason for the acquisition failure: for example, if the image acquisition device is maliciously obstructed, another hidden image acquisition device is started to acquire images, or a voice alarm is issued to inform the user not to obstruct the image acquisition device; if the image acquisition device parameter settings are improper, the parameters of the image acquisition device are adjusted. After adjusting the image acquisition device to ensure it can acquire images normally, the unlocking operation is performed. The user takes the goods after the cabinet door opens, and the image acquisition device captures the process of the user taking the goods. To ensure a smooth transaction, the image acquisition device sends the images to the edge computing module for real-time image analysis while acquiring the second consecutive image. When the smart vending machine receives a locking signal from the electronic lock within a preset maximum retrieval time (the preset maximum retrieval time can be determined based on actual conditions; for example, based on historical data analysis, if users generally complete transactions within 10 minutes, then the preset maximum retrieval time can be set to 10 minutes), it indicates that the user has taken the goods and closed the cabinet door. Then, it is determined whether the connection between the smart vending machine and the server is normal. If the connection is normal, the acquired second consecutive image is sent to the server, and the server is driven to send order information to the smart terminal. If the connection is abnormal due to network latency or malicious signal blocking, the edge computing module is controlled to generate order information based on the received second consecutive image and send the order information to the smart terminal.

[0122] If the smart vending machine does not receive the locking signal within the preset maximum retrieval time, it indicates that the user may have forgotten to close the door or is intentionally leaving it open. In this case, the edge computing module determines whether the user has finished retrieving the goods based on the second continuous image. This determination can be made by the edge computing module analyzing the second continuous image to detect if the user has taken out the goods and not taken any other items, or if the user has left the smart vending machine. If either of these conditions is met, it is considered that the user has finished retrieving the goods. At this point, it is determined whether the connection between the smart vending machine and the server is normal. If the connection is normal, the smart vending machine sends a settlement instruction to the server to drive the server to generate order information. If the connection is abnormal, the edge computing module generates order information based on the received second continuous image and sends the order information to the smart terminal. In this embodiment, considering the possibility of abnormalities in the image acquisition device and / or communication connection with the server during the transaction process, the edge computing module installed at the smart vending machine is used to handle abnormal situations in a timely manner to ensure smooth transactions and improve user experience.

[0123] Example 2

[0124] This invention provides a machine vision-based intelligent vending machine interactive control method, applied to a server. The server connects to the intelligent vending machine via wired or wireless means, receives image data transmitted by the intelligent vending machine, and analyzes and processes the image data to assist the intelligent vending machine in completing product identification and transactions. Please refer to [link to relevant documentation]. Figure 9 The machine vision-based intelligent vending machine interactive control method includes:

[0125] S10: Receive a startup request from a smart terminal and send a startup command to the smart vending machine. The startup command will drive the smart vending machine to perform hardware self-test, acquire a first continuous image and a second continuous image. The smart terminal includes any one of a smartphone or a human-computer interaction device for the smart vending machine.

[0126] S20: Receive a second continuous image from the smart vending machine, wherein the second continuous image is an image continuously acquired by the image acquisition device of the smart vending machine from the time the electronic lock is unlocked until it is locked;

[0127] S30: Perform image recognition and analysis processing on the second continuous image and generate order information;

[0128] S40: Send the order information to the smart terminal.

[0129] Specifically, when a user wants to purchase goods from the smart vending machine, they can scan the QR code provided by the smart vending machine with their smartphone or click the purchase button on the smart vending machine's human-computer interaction device (such as a touch screen) to send a start request to the server. Upon receiving the start request from the smart terminal, the server will send a start command to the smart vending machine. After the smart vending machine receives the start command, starts up, and completes its hardware self-test, it will control the image acquisition device to capture the first continuous images over a preset time. Once the first continuous images are successfully acquired, the smart vending machine will unlock its electronic lock, allowing the user to open the door and select goods. Simultaneously, the image acquisition device will continuously capture images of the user's selection process to obtain the second continuous images. This process continues until the user finishes selecting goods and closes the door, triggering the electronic lock to send a locking signal. The acquired second continuous images will then be sent to the server. The server will analyze and process the acquired second continuous images to identify the type and quantity of goods selected by the user, determine the order amount based on the type and quantity, and then push the order information to the smart terminal to inform the user. The user can then complete the payment based on the order information, thus completing the transaction.

[0130] In summary, the machine vision-based smart vending machine interactive control method provided by this invention responds to a server start command, controls the smart vending machine to perform hardware self-tests and obtain the self-test results; when the hardware self-test results are normal, it controls the smart vending machine's image acquisition device to acquire a first series of images at a preset time; after the first series of images are successfully acquired, it controls the smart vending machine's electronic lock to perform an unlocking operation; it controls the image acquisition device to start acquiring a second series of images until it receives a locking signal from the electronic lock; it sends the acquired second series of images to the server and drives the server to send order information to the smart terminal, thus realizing the smart vending machine interactive control and commodity transaction process. This invention ensures the normal operation and stability of the device through hardware self-testing, and accurately identifies commodity categories and quantities and generates order information by real-time image acquisition and monitoring and sending images to the server for image analysis, making the entire transaction process more convenient and efficient, and providing users with a good and convenient purchasing experience.

[0131] Example 3

[0132] Please see Figure 10 This invention provides a machine vision-based intelligent vending machine interactive control device 200, applied to an intelligent vending machine. The device 200 includes:

[0133] The self-test module 201 is used to control the hardware device of the smart vending machine to perform a hardware self-test and obtain the hardware self-test result in response to a start command from the server, wherein the hardware self-test result includes normal or abnormal.

[0134] The first acquisition module 202 is used to control the image acquisition device of the smart vending machine to acquire a first continuous image at a preset time when the hardware self-test result is normal.

[0135] The unlocking module 203 is used to control the electronic lock of the smart vending machine to perform an unlocking operation when the first continuous image is successfully acquired.

[0136] The second acquisition module 204 is used to control the image acquisition device to start acquiring the second continuous image until it receives the locking signal from the electronic lock;

[0137] The order acquisition module 205 is used to send the acquired second continuous image to the server and drive the server to send order information to the smart terminal.

[0138] Example 4

[0139] Please see Figure 11 This invention provides a machine vision-based intelligent vending machine interactive control device 400, with an application server side. The device 400 includes:

[0140] The startup command sending module 401 is used to receive a startup request from a smart terminal and send a startup command to the smart vending machine. The startup command will drive the smart vending machine to perform hardware self-test, acquire a first continuous image and a second continuous image. The smart terminal includes any one of a smartphone or a human-computer interaction device for the smart vending machine.

[0141] The second image receiving module 402 is used to receive a second continuous image from the smart vending machine, wherein the second continuous image is an image continuously acquired by the image acquisition device of the smart vending machine from the time the electronic lock is unlocked to the time it is locked.

[0142] The order generation module 403 is used to perform image recognition analysis processing on the second continuous image and generate order information;

[0143] The order sending module 404 is used to send the order information to the smart terminal.

[0144] In summary, the machine vision-based intelligent vending machine interactive control device provided in this embodiment of the invention, in response to a start command from the server, controls the intelligent vending machine to perform hardware self-test and obtain the hardware self-test results; when the hardware self-test results are normal, it controls the image acquisition device of the intelligent vending machine to acquire a first continuous image at a preset time; when the first continuous image is successfully acquired, it controls the electronic lock of the intelligent vending machine to perform an unlocking operation; it controls the image acquisition device to start acquiring a second continuous image until it receives a locking signal from the electronic lock; it sends the acquired second continuous image to the server and drives the server to send order information to the intelligent terminal, thus realizing the interactive control of the intelligent vending machine and the commodity transaction process. The method of this invention ensures the normal operation and stability of the device through hardware self-testing, and accurately identifies the commodity category and quantity and generates order information by real-time image acquisition and monitoring and sending the images to the server for image analysis, making the entire transaction process more convenient and efficient, and providing users with a good and convenient purchasing experience.

[0145] Example 5

[0146] The present invention provides an intelligent vending machine system, the system comprising an intelligent vending machine and a server, wherein the intelligent vending machine and the server are connected via wired or wireless means.

[0147] The smart vending machine includes:

[0148] The machine vision-based smart vending machine interactive control device 200 applied to the smart vending machine terminal as described in Embodiment 3 includes:

[0149] The self-test module 201 is used to respond to a start command from the server, control the hardware device of the smart vending machine to perform a hardware self-test and obtain the hardware self-test result, wherein the hardware self-test result includes normal or abnormal.

[0150] The first acquisition module 202 is used to control the image acquisition device of the smart vending machine to acquire a first continuous image at a preset time when the hardware self-test result is normal.

[0151] The unlocking module 203 is used to control the electronic lock of the smart vending machine to perform an unlocking operation when the first continuous image is successfully acquired.

[0152] The second acquisition module 204 is used to control the image acquisition device to start acquiring the second continuous image until it receives the locking signal from the electronic lock;

[0153] The order acquisition module 205 is used to send the acquired second continuous image to the server and drive the server to send order information to the smart terminal.

[0154] The server includes:

[0155] The machine vision-based intelligent vending machine interactive control device 400 applied to the server side as described in Embodiment 4 includes:

[0156] The startup command sending module 401 is used to receive a startup request from a smart terminal and send a startup command to the smart vending machine. The startup command will drive the smart vending machine to perform hardware self-test, acquire a first continuous image and a second continuous image. The smart terminal includes any one of a smartphone or a human-computer interaction device for the smart vending machine.

[0157] The second image receiving module 402 is used to receive a second continuous image from the smart vending machine, wherein the second continuous image is an image continuously acquired by the image acquisition device of the smart vending machine from the time the electronic lock is unlocked to the time it is locked.

[0158] The order generation module 403 is used to perform image recognition analysis processing on the second continuous image and generate order information;

[0159] The order sending module 404 is used to send the order information to the smart terminal.

[0160] Example 6

[0161] In addition, the machine vision-based intelligent vending machine interactive control method of this invention can be implemented by the intelligent vending machine main control system. Figure 12 A schematic diagram of the hardware structure of the intelligent vending machine master control system provided in an embodiment of the present invention is shown.

[0162] The intelligent vending machine control system may include a processor 301 and a memory 302 storing computer program instructions.

[0163] Specifically, the processor 301 may include a central processing unit (CPU), an application specific integrated circuit (ASIC), or one or more integrated circuits that can be configured to implement the embodiments of the present invention.

[0164] Memory 302 may include mass storage for data or instructions. For example, and not limitingly, memory 302 may include a hard disk drive (HDD), floppy disk drive, flash memory, optical disk, magneto-optical disk, magnetic tape, or Universal Serial Bus (USB) drive, or a combination of two or more of these. Where appropriate, memory 302 may include removable or non-removable (or fixed) media. Where appropriate, memory 302 may be internal or external to a data processing device. In a particular embodiment, memory 302 is a non-volatile solid-state memory. In a particular embodiment, memory 302 includes read-only memory (ROM). Where appropriate, the ROM may be a mask-programmed ROM, a programmable ROM (PROM), an erasable PROM (EPROM), an electrically erasable PROM (EEPROM), an electrically rewritable ROM (EAROM), or flash memory, or a combination of two or more of these.

[0165] The processor 301 reads and executes computer program instructions stored in the memory 302 to implement any of the machine vision-based smart vending machine interactive control methods in the above embodiments.

[0166] In one example, the smart vending machine's main control system may also include a communication interface 303 and a bus 310. For example, Figure 12 As shown, the processor 301, memory 302, and communication interface 303 are connected through bus 310 and complete communication with each other.

[0167] The communication interface 303 is mainly used to realize communication between various modules, devices, units and / or equipment in the embodiments of the present invention.

[0168] Bus 310 includes hardware, software, or both, that couples components of the smart vending machine's main control system together. For example, and not limitingly, bus 310 may include an Accelerated Graphics Port (AGP) or other graphics bus, an Enhanced Industry Standard Architecture (EISA) bus, a Front Side Bus (FSB), HyperTransport (HT) interconnect, an Industry Standard Architecture (ISA) bus, an Infinite Bandwidth Interconnect, a Low Pin Count (LPC) bus, a memory bus, a Microchannel Architecture (MCA) bus, a Peripheral Component Interconnect (PCI) bus, a PCI-Express (PCI-X) bus, a Serial Advanced Technology Attachment (SATA) bus, a Video Electronics Standards Association Local (VLB) bus, or other suitable buses, or combinations of two or more of these. Where appropriate, bus 310 may include one or more buses. While specific buses are described and illustrated in embodiments of the invention, the invention contemplates any suitable bus or interconnect.

[0169] It should be clarified that the present invention is not limited to the specific configurations and processes described above and shown in the figures. For the sake of brevity, detailed descriptions of known methods are omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of the present invention is not limited to the specific steps described and shown. Those skilled in the art can make various changes, modifications, and additions, or change the order of steps, after understanding the spirit of the present invention.

[0170] The functional blocks shown in the above-described structural diagram can be implemented as hardware, software, firmware, or a combination thereof. When implemented in hardware, they can be, for example, electronic circuits, application-specific integrated circuits (ASICs), appropriate firmware, plug-ins, function cards, etc. When implemented in software, the elements of this invention are programs or code segments used to perform the required tasks. The programs or code segments can be stored on a machine-readable medium or transmitted over a transmission medium or communication link via data signals carried in a carrier wave. "Machine-readable medium" can include any medium capable of storing or transmitting information. Examples of machine-readable media include electronic circuits, semiconductor memory devices, ROM, flash memory, erasable ROM (EROM), floppy disks, CD-ROMs, optical disks, hard disks, fiber optic media, radio frequency (RF) links, etc. Code segments can be downloaded via computer networks such as the Internet, intranets, etc.

[0171] It should also be noted that the exemplary embodiments mentioned in this invention describe methods or systems based on a series of steps or apparatus. However, this invention is not limited to the order of the steps described above; that is, the steps can be performed in the order mentioned in the embodiments, or in a different order, or several steps can be performed simultaneously.

[0172] The above description is merely a specific embodiment of the present invention. Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, modules, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here. It should be understood that the protection scope of the present invention is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in the present invention, and these modifications or substitutions should all be covered within the protection scope of the present invention.

Claims

1. An interactive control method for an intelligent vending machine, characterized in that, The smart vending machine includes: a main control device located at the upper right corner of the vending machine body; the main control device is equipped with a first image acquisition device and a second image acquisition device; the first image acquisition device is mounted in the main control device via a sliding groove, allowing it to slide within the groove and adjust the distance between itself and the second image acquisition device; the first image acquisition device captures images both inside and outside the smart vending machine body, while the second image acquisition device captures images outside the smart vending machine body; the interactive control method includes: The first and second image acquisition devices of the smart vending machine are controlled to acquire a first continuous image at a preset time. Once the first continuous image is successfully acquired, the electronic lock of the smart vending machine is controlled to perform an unlocking operation. The first and second image acquisition devices are controlled to start acquiring the second continuous image until a locking signal is received from the electronic lock; The acquired second continuous image is sent to the server, which then drives the server to send order information to the smart terminal.

2. The interactive control method for the intelligent vending machine according to claim 1, characterized in that, The angle between the viewing angle of the first image acquisition device of the smart vending machine and the plane where the smart vending machine door is located in the vertical direction is between 55° and 65°.

3. The interactive control method for intelligent vending machines according to claim 2, characterized in that, The angle between the viewing angle of the second image acquisition device of the smart vending machine and the horizontal plane of the smart vending machine door is 55° to 65°.

4. The interactive control method for intelligent vending machines according to any one of claims 1 to 3, characterized in that, The second image acquisition device of the smart vending machine is fixedly installed in the upper right corner of the smart vending machine door, and the position of the first image acquisition device can be adjusted by sliding according to the position, acquisition angle and acquisition range of the second image acquisition device.

5. The interactive control method for intelligent vending machines according to claim 1, characterized in that, The first image acquisition device and the second image acquisition device controlling the smart vending machine acquire a first continuous image at a preset time, including: The first image acquisition device and the second image acquisition device are controlled to simultaneously acquire the first sub-continuous image and the second sub-continuous image at a preset time. The first continuous image is obtained by merging the first sub-continuous image and the second sub-continuous image.

6. The interactive control method for the intelligent vending machine according to claim 1, characterized in that, The control of the first and second image acquisition devices to start acquiring the second continuous image until a locking signal is received from the electronic lock includes: Extract the frame image containing the smart vending machine door information from the second continuous image, and denote it as the door image; Cabinet door status information is generated in real time based on the cabinet door image, wherein the cabinet door status information includes at least one of cabinet door open and cabinet door closed. When the locking signal is received, the cabinet door status information is obtained; If the cabinet door status information indicates that the cabinet door is closed, then the image acquisition device is controlled to stop acquiring the second continuous image.

7. The interactive control method for intelligent vending machines according to claim 1, characterized in that, The control of the first and second image acquisition devices to start acquiring the second continuous image until a locking signal is received from the electronic lock includes: The first and second image acquisition devices are controlled to capture the user's movement trajectory from opening the cabinet door to taking out the goods; The product area where the user retrieved the item is located is determined based on the movement trajectory; By comparing the frame images of the product area before the user takes out the product with the frame images of the product area after the user takes out the product, the information of the product taken out by the user can be determined.

8. The interactive control method for the intelligent vending machine according to claim 1, characterized in that, Controlling the image acquisition device to acquire the second consecutive images also includes: The initial frame image is obtained by extracting several frames of images initially acquired by the first image acquisition device and the second image acquisition device. Obtain the sharpness level of the initial frame image and determine whether the sharpness level meets the preset image analysis requirements; If not, adjust the acquisition frame rate of the first image acquisition device and the second image acquisition device according to the clarity level and the preset image analysis requirements; The first few frames of images acquired by the first and second image acquisition devices after the frame rate is adjusted are denoted as the adjusted frame images. Obtain the sharpness level of the adjusted frame image and determine whether the sharpness level of the adjusted frame image meets the preset image analysis requirements; Continue until the resolution level meets the preset image analysis requirements.

9. An interactive control system for an intelligent vending machine, characterized in that, The system includes: a smart vending machine and a server communicatively connected to the smart vending machine, wherein the smart vending machine is the smart vending machine as described in any one of claims 1 to 4.

Images