Pick-and-place identification method, storage medium and unmanned retail equipment

Abstract

The invention discloses a pick-and-place identification method, a storage medium and unmanned retail equipment. The pick-and-place identification method comprises the steps that account information of a user is acquired, and under the condition that the account information of the user is valid, an electric control lock in a locked state is controlled to be in an unlocked state; identifying commodity information of commodities taken out by a user on the basis of an image in the cabinet body acquired by the camera module, and starting first timing under the condition that the feedback signal is switched from the opening and closing signal to the closing signal; in the process that the duration of the first timing reaches a preset first duration, under the condition that the commodity taken out by the user in response is not recognized all the time and the feedback signal is kept as a closed signal, performing money deduction operation on a user account; and when the duration of the first timing does not reach a preset first duration, not performing money deduction operation on a user account. The invention aims to improve the payment and settlement accuracy of the unmanned retail equipment, and further improve the use experience of a user.

Description

Technical Field

[0001] This application relates to the technical field of pick-up and drop-off identification for unmanned retail equipment, and more particularly to a pick-up and drop-off identification method, a storage medium, and an unmanned retail equipment. Background Technology

[0002] Unmanned retail machines typically have door open / close detection devices to monitor the status of the cabinet doors. These devices may include microswitches mounted on the door frame, reed switches using magnetic detection, or Hall effect sensors. Based on the detection results, the machine checks out the customer's purchases and deducts payment only after confirming the door has been closed. However, in practice, customers may partially close the door, leaving it slightly ajar. Due to factors such as the sensitivity of microswitches diminishing with prolonged use or interference from ambient magnetic fields, the detection device may send a closing signal to the machine's control system even when the door is partially closed. This results in payment being deducted before the door is fully closed. If a customer is dissatisfied and wants to cancel their purchase, this constitutes an incorrect charge, leading to a poor customer experience. Summary of the Invention

[0003] The main purpose of this application is to provide a method for identification of pickup and drop-off, a storage medium, and an unmanned retail device, which aims to improve the accuracy of payment settlement in unmanned retail devices and thus enhance the user experience.

[0004] To achieve the above objectives, this application proposes a retrieval and placement recognition method for use in unmanned retail equipment. The unmanned retail equipment includes a cabinet and a door mounted on the cabinet. The cabinet is equipped with an electronically controlled lock for locking the door. The electronically controlled lock is configured to be in an unlocked state when it receives an unlocking signal, allowing the door to be in an open / closed state, and to be triggered by the door to be in a locked state when the door is in a closed state, thereby locking the door. The cabinet contains shelves for placing goods and a camera module for acquiring images of the cabinet interior. A door open / close detection device is also provided on the cabinet. This device detects the state of the door and outputs corresponding feedback signals, including open / close signals and closed signals. The pick-up and placement identification method includes: Step S100: Obtain the user's account information. If the user's account information is valid, control the locked electric lock to unlock. Step S200: Based on the image inside the cabinet obtained by the camera module, identify the product information of the product taken out by the user; Step S300: When the feedback signal switches from the opening / closing signal to the closing signal, start the first timing. Step S400: If the user fails to release the retrieved item and the feedback signal remains closed during the first timeout period, the user's account will be deducted from the account. Step S500: If the duration of the first timeout has not reached the preset first duration, no deduction operation will be performed on the user's account.

[0005] Optionally, the camera module includes multiple static recognition cameras installed on the shelves inside the cabinet and a dynamic recognition camera installed at the top of the cabinet near the door; the step of identifying the product information of the goods taken out by the user based on the images inside the cabinet acquired by the camera module includes: Step S210: Acquire static images of goods inside the shelf based on multiple static recognition cameras; Step S220: Based on the dynamic image obtained by the dynamic recognition camera, if the user takes out the product, the product information of the product taken out by the user is identified based on the comparison of static images before and after the taking out action. The steps for when the user has not placed the item back during the first timeout period and the feedback signal remains closed include: during the first timeout period, based on the dynamic image acquired by the dynamic recognition camera, identifying that the user has not placed the item back and the feedback signal remains closed.

[0006] Optionally, the step of obtaining the user's account information and, if the user's account information is valid, controlling the locked electronic lock to unlock further includes: When the feedback signal switches from the closed signal to the open / closed signal, the motion recognition camera that is in a dormant state is brought into working state.

[0007] Optionally, the step of not deducting money from the user's account when the duration of the first timeout has not reached the preset first duration further includes: If the duration of the first timeout has not reached the preset first duration, when new user account information is obtained, the previous user account will be charged.

[0008] Optionally, after step S400, the method further includes: Step S600: If no new user account information is obtained and the user has returned the goods based on the image inside the cabinet, obtain the status information of the returned goods. Step S700: Based on the status information, determine whether to perform a refund operation.

[0009] Optionally, the step of obtaining the status information of the returned goods includes: Step S710: Based on the image inside the cabinet obtained by the camera module, if it is determined that the returned item is in an open state, set the status information of the returned item to a used state. Step S720: Determine that the returned goods are in intact packaging, and obtain the first appearance feature of the returned goods; wherein, the first appearance feature includes: trademark and product barcode; Step S730: If the trademark is inconsistent with the trademark of the previously retrieved goods or the product barcode is inconsistent with the product barcode of the previously retrieved goods, determine the status information of the returned goods and set the status information of the returned goods to an abnormal product status. Step S740: If the trademark matches the trademark of the previously retrieved product and the product barcode matches the product barcode of the previously retrieved product, obtain the second appearance feature of the returned product; wherein, the second appearance feature includes: shelf life; Step S750: If the returned product has reached its expiration date, set the status information of the returned product to expired.

[0010] Optionally, determining whether to perform a refund operation based on the status information includes: If the product's status information is either "used" or "abnormal product," a refund will not be processed. If the product's status is expired, a refund will be processed.

[0011] Optionally, the method further includes: If the product's status information indicates that it is expired or abnormal, the product will be marked as unsellable and the information of the unsellable product will be uploaded to the cloud; and if the user is detected taking out the unsellable product, the user will be notified.

[0012] This application also proposes a storage medium storing a computer program that can be executed by a processor to perform the pick-up and placement identification method described in any of the above claims. This application also proposes an unmanned retail device, which includes the storage medium described above. The unmanned retail device includes a cabinet and a door mounted on the cabinet. The cabinet is equipped with an electronic lock for locking the door. The electronic lock is configured to be in an unlocked state when it receives an unlocking signal, allowing the door to be in an open / closed state, and to be triggered by the door to be in a locked state when the door is in a closed state, thereby locking the door. The cabinet contains shelves for placing goods and a camera module for acquiring images of the cabinet interior. A door open / close detection device is also provided on the cabinet, which detects the state of the door and outputs corresponding feedback signals, including open / close signals and closed signals.

[0013] The retrieval and placement identification method of this application includes: acquiring the user's account information; if the user's account information is valid, controlling the locked electronic lock to unlock; based on the image inside the cabinet acquired by the camera module, identifying the product information of the product taken out by the user; and starting a first timer when the feedback signal switches from the open / closed signal to the closed signal; if no product is identified being placed back by the user and the feedback signal remains closed during the first timer period, a deduction operation is performed on the user's account; if the first timer period does not reach the preset first timer period, no deduction operation is performed on the user's account. Thus, in practice, when the signal from the door switch detection device changes from an open / closed signal to a closed signal, the unmanned retail device will not immediately deduct payment. Instead, it will wait for a period of time, and only if the user does not place the product back and the feedback signal remains closed during this period, will a deduction operation be performed. This prevents erroneous deductions when the user has only slightly closed the door without actually completing a purchase, thereby effectively improving the accuracy of payment settlement and the user experience of the unmanned retail device. Attached Figure Description

[0014] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0015] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0016] Figure 1 This is a schematic flowchart of an embodiment of the pick-up and place recognition method of this application; Figure 2 This is a schematic flowchart of another embodiment of the pick-up and place recognition method of this application; Figure 3 This is a schematic flowchart of another embodiment of the pick-up and place identification method of this application; Figure 4 This is a schematic flowchart of another embodiment of the pick-up and place identification method of this application.

[0017] The purpose, features, and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0018] It should be understood that the specific embodiments described herein are merely illustrative of the technical solutions of this application and are not intended to limit this application.

[0019] To better understand the technical solutions of this application, the technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of the embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0020] It should be noted that all directional indicators (such as up, down, left, right, front, back, etc.) in the embodiments of this application are only used to explain the relative positional relationships and movement of the components in a specific posture (as shown in the attached figures). If the specific posture changes, the directional indicators will also change accordingly. It should be understood that although the steps in the flowcharts of the embodiments of this application are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders.

[0021] Unmanned retail machines typically have door open / close detection devices to monitor the status of the cabinet doors. These devices may include microswitches mounted on the door frame, reed switches using magnetic detection, or Hall effect sensors. Based on the detection results, the machine checks out the customer's purchases and deducts payment only after confirming the door has been closed. However, in practice, customers may partially close the door, leaving it slightly ajar. Due to factors such as the sensitivity of microswitches diminishing with prolonged use or interference from ambient magnetic fields, the detection device may send a closing signal to the machine's control system even when the door is partially closed. This results in payment being deducted before the door is fully closed. If a customer is dissatisfied and wants to cancel their purchase, this constitutes an incorrect charge, leading to a poor customer experience.

[0022] Therefore, this application proposes a method for recognizing items and placing them, applied to unmanned retail equipment. The unmanned retail equipment includes a cabinet and a door installed on the cabinet. The cabinet is equipped with an electronic lock for locking the door. The electronic lock is configured to be in an unlocked state when it receives an unlocking signal, so that the door is in an open / closed state, and to be triggered by the door to switch to a locked state when the door is in a closed state, so as to lock the door. The cabinet is equipped with shelves for placing goods and a camera module for acquiring images inside the cabinet. The cabinet is also equipped with a door open / close detection device, which is used to detect the state of the door and output corresponding feedback signals, including open / close signals and closed signals.

[0023] The unmanned retail equipment may include a control device, which integrates a processor and storage medium. The storage medium stores a computer program, which can be executed by the processor to perform any of the following retrieval and placement recognition methods. The control device can be implemented using MCUs, DSPs (Digital Signal Processors), FPGAs (Field Programmable Gate Arrays), PLCs, or SOCs (System-on-Chips). The door switch detection device can be implemented using a microswitch mounted on the cabinet, or by combining a magnetic induction element (such as a reed switch or Hall effect sensor) mounted on the cabinet with a magnetic element mounted on the door. The camera module can be implemented using dynamic visual recognition cameras or static visual recognition cameras such as fisheye cameras. When the electric lock receives an unlock signal from the control device, it switches from the locked state to the unlocked state. At this time, the trigger on the electric lock resets. When the door is closed completely by the user, the door will trigger the trigger on the electric lock to switch from the unlocked state to the locked state, thus locking the door.

[0024] refer to Figure 1 In one embodiment of this application, the pick-up and place recognition method includes: Step S100: Obtain the user's account information. If the user's account information is valid, control the locked electric lock to unlock. Step S200: Based on the image inside the cabinet obtained by the camera module, identify the product information of the product taken out by the user; Step S300: When the feedback signal switches from the opening / closing signal to the closing signal, start the first timing. Step S400: If the user fails to release the retrieved item and the feedback signal remains closed during the first timeout period, the user's account will be deducted from the account. Step S500: If the duration of the first timeout has not reached the preset first duration, no deduction operation will be performed on the user's account.

[0025] In this embodiment, a facial recognition module, such as a camera, is also installed on the door for recognizing faces, along with an interactive interface for user interaction. When a user has a purchase request, the control device can verify the user's account information based on the facial recognition result provided by the facial recognition module, or by verifying the user's account information through QR code scanning. Once the control device determines that the user's account information is valid, it will control the electric lock to unlock, for example, by outputting an unlock signal to the electric lock. Subsequently, when the user takes goods from inside the cabinet, the system identifies the retrieved goods based on the image captured by the camera module and generates the corresponding payment.

[0026] Optionally, in one embodiment, reference is made to... Figure 2 The camera module includes multiple static recognition cameras installed on the shelves inside the cabinet and a dynamic recognition camera installed at the top of the cabinet near the door; the step of identifying the product information of the goods taken out by the user based on the images inside the cabinet acquired by the camera module includes: Step S210: Acquire static images of goods inside the shelf based on multiple static recognition cameras; Step S220: Based on the dynamic image obtained by the dynamic recognition camera, if the user takes out the product, the product information of the product taken out by the user is identified by comparing the static images before and after the taking-out action.

[0027] In this embodiment, multiple static recognition cameras can be installed along the width of the cabinet to capture static images of each item inside the cabinet. Static recognition cameras can be implemented using fisheye lenses, waterproof and fog-proof cameras, etc. Dynamic recognition cameras can be implemented using dynamic vision IP cameras, AI cameras, and starlight-level full-color cameras. The control device can operate based on the dynamic images acquired by the dynamic recognition cameras. It is understood that, to reduce energy consumption, when the control device detects valid user account information and the locked electronic lock is unlocked, if the current feedback signal switches from the closed signal to an open / close signal (because when the user can open the door to retrieve goods, the opening / closing angle of the door will necessarily cause the door switch detection output to be an open / close signal), then the dynamic recognition camera, which is in a dormant state, is activated. Furthermore, when the feedback signal switches from an open / close signal to a closed signal, the dynamic recognition camera is restored to a dormant state.

[0028] When the control device detects a user reaching into the cabinet and grabbing an item based on the dynamic image captured by the dynamic recognition camera, it will determine that the user has taken an item. It will also compare the static images before and after the grabbing to identify the specific item taken by the user and its corresponding product information.

[0029] It is understood that the control device can also identify the return action based on the dynamic recognition camera. In one embodiment, the step of not identifying the user's return of the product and keeping the feedback signal closed during the first timing period when the preset first time duration has been reached includes: during the first timing period when the preset first time duration has been reached, based on the dynamic image obtained by the dynamic recognition camera, identifying that the user has not made any action to return the product and keeping the feedback signal closed.

[0030] When the control device determines that the feedback signal from the door switch has switched from an open / close signal to a closed signal, it assumes that the user may have adjusted the door from an open / closed state to a closed state or a near-closed, slightly ajar state. To prevent incorrect deductions due to a misjudgment that the user no longer intends to purchase, the control device initiates a first timer. During the first timer's duration (pre-set by developers or managed by the unmanned retail equipment administrator), based on images of the cabinet interior captured by the aforementioned camera module, if it detects that the user has not returned previously retrieved items and the feedback signal remains closed (indicating the door has not been opened to an angle suitable for accessing items on the shelves), the control device considers the user to have completed their purchase and proceeds with the normal deduction. If the first timer has not yet reached the preset duration, the control device will not deduct payment to prevent incorrect deductions when the user has only slightly ajar the door.

[0031] In one example, assuming a preset first timeout of 15 seconds, when a user takes out item A, the control device recognizes the user's action based on the image inside the cabinet and determines the price of item A. When the control device detects a change in the feedback signal from the door open / close detection device (e.g., from a high-level signal to a low-level signal), it initiates the first timeout via internal software. If the first timeout reaches 15 seconds, and during this period, the user is not detected putting back the previously taken-out item, and the feedback signal from the door open / close detection device remains low, the control device considers the user to have completed their purchase and has no further purchase requests. In this case, it will deduct the price of item A based on the user's account information. If the first timeout has not yet reached 15 seconds, the control device will not deduct the price to prevent accidental deductions when the user has only slightly closed the door without actually completing their purchase.

[0032] The retrieval and placement identification method of this application includes: acquiring the user's account information; if the user's account information is valid, controlling the locked electronic lock to unlock; based on the image inside the cabinet acquired by the camera module, identifying the product information of the product taken out by the user; and starting a first timer when the feedback signal switches from the open / closed signal to the closed signal; if no product is identified being placed back by the user and the feedback signal remains closed during the first timer period, a deduction operation is performed on the user's account; if the first timer period does not reach the preset first timer period, no deduction operation is performed on the user's account. Thus, in practice, when the signal from the door switch detection device changes from an open / closed signal to a closed signal, the unmanned retail device will not immediately deduct payment. Instead, it will wait for a period of time, and only if the user does not place the product back and the feedback signal remains closed during this period, will a deduction operation be performed. This prevents erroneous deductions when the user has only slightly closed the door without actually completing a purchase, thereby effectively improving the accuracy of payment settlement and the user experience of the unmanned retail device.

[0033] Furthermore, in another embodiment, the step of not performing a deduction operation on the user account when the first timer duration has not reached the preset first time duration further includes: when new user account information is obtained, performing a deduction operation on the previous user account when the first timer duration has not reached the preset first time duration. In this embodiment, it can be understood that if the current control device recognizes a new user's facial image based on the camera on the door or the QR code on the interactive interface is triggered by a new user's account, it will assume that the user who previously took the goods has completed the purchase, and therefore will not wait for the first timer to reach the preset first time duration, but will directly settle the payment for the user who previously completed the purchase.

[0034] It's important to understand that in practice, some users might check the ingredient list or expiration date on a product after receiving it to determine if it's suitable for them. Especially regarding expiration date information, different products display expiration dates differently, which might require some searching. If the user's search is slow, causing the first timer in the above embodiment to exceed the preset first time, and the user discovers they've bought an expired product and wants to return it, the unmanned retail device won't recognize the returned product but will directly deduct payment based on the process described in the above embodiment. This would result in the customer incurring a loss.

[0035] Therefore, refer to Figure 3 In one embodiment of this application, after step S400, the method further includes: Step S600: If no new user account information is obtained and the user has returned the goods based on the image inside the cabinet, obtain the status information of the returned goods. Step S700: Based on the status information, determine whether to perform a refund operation.

[0036] In this embodiment, if the control device has not yet obtained new user account information, meaning no new user needs to open the door to shop via QR code or facial recognition, the control device will determine that the currently shopping user has not yet left. At this time, if the control device determines, based on the image inside the cabinet obtained by the camera module and the feedback signal from the door open / close detection device, that the currently active user has put the goods back (the feedback signal switches from a closed signal to an open / closed signal), then the control device will obtain the status information of the returned goods to determine why the user put the goods back and whether a refund is needed.

[0037] Optionally, refer to Figure 4 In one embodiment, the step of obtaining the status information of the returned goods includes: Step S710: Based on the image inside the cabinet obtained by the camera module, if it is determined that the returned product is in an open state, the status information of the returned product is set to the used state. In this embodiment, the control device will prioritize the identification of the integrity of the returned product based on the image inside the cabinet provided by the camera module. If the integrity of the returned product is abnormal, for example, if it is found that the outer packaging of the product is torn or missing when compared with the preset appearance image, the control device will assume that the user has put the used product back and set it to the used state.

[0038] Step S720: Determine that the returned goods are in intact packaging, and obtain the first appearance feature of the returned goods; wherein, the first appearance feature includes: trademark and product barcode; Step S730: If the trademark is inconsistent with the trademark of the previously retrieved goods or the barcode of the goods is inconsistent with the barcode of the previously retrieved goods, determine the status information of the returned goods and set the status information of the returned goods to an abnormal goods status. In this embodiment, based on the above process, if the outer packaging of the returned product is intact, the control device will also acquire the trademark and product barcode on the outer packaging through the camera module. It will then compare the trademark and barcode of the corresponding product with the actual returned product in its stored database. If the trademark or barcode is inconsistent with that of a previously retrieved product, the control device determines that the returned product is a counterfeit or goods not previously stored on the shelf by the same personnel. In this case, the control device will set its status information to "abnormal product status."

[0039] Step S740: If the trademark is consistent with the trademark of the previously retrieved product and the product barcode is consistent with the product barcode of the previously retrieved product, obtain the second appearance feature of the returned product; wherein, the second appearance feature includes: shelf life; Step S750: If the returned product has reached its expiration date, set the status information of the returned product to expired.

[0040] In this embodiment, if the control device determines that the packaging of the returned product is intact and that the trademark and product barcode match those of the product previously retrieved by the user, it will determine that the product currently being returned by the user is the same product previously retrieved. Then, based on information obtained from the camera module, the control device will identify the product's expiration date from its outer packaging. If the returned product has reached its expiration date, the control device will consider the product returned due to expiration and set its status to expired.

[0041] Furthermore, it is understandable that in practice, there may be situations where a user opens a product but judges it to be expired based on their sense of smell, only to discover the expiration date and return the product. Therefore, after the control device determines that the product's appearance is incomplete based on the appearance integrity judgment process described in the above embodiment, it will also check the product's trademark and barcode to ensure they match the previously retrieved product before determining the current expiration date. If it is determined that the product is not within its expiration period, the returned product will be set to an expired state to execute the subsequent refund operation.

[0042] Based on the above embodiments, in one embodiment, determining whether to perform a refund operation based on the status information includes: If the product's status information is either "used" or "abnormal product," a refund will not be processed. If the product's status is expired, a refund will be processed.

[0043] In this embodiment, if the control device determines that the product's status is either "used" or "abnormal," then it determines that the user's return is not due to a product quality issue, and therefore a refund will not be processed. If the control device determines that the product's status is "expired," it determines that the user returned the product late because they spent time reading the expiration date, and in this case, the control device will process a refund, returning the deducted amount to the user for the expired product.

[0044] Thus, through the above settings, in practical applications, if a user is mistakenly charged for returning a product after the expiration date due to reading the product's expiration information, the control device will proactively identify whether the product returned by the user is expired and refund the deducted fee to the user's account if the product is expired. This effectively improves the user experience of using unmanned retail equipment.

[0045] Meanwhile, it is understood that in one embodiment, if the control device determines, based on the process described above, that the product the user just returned is an expired product that was previously taken out, or another product that was not originally on the shelf inside the cabinet, i.e., the control device determines that the product's status information is expired or abnormal, the product will be marked as unsellable and the information of the unsellable product will be uploaded to the cloud. For example, the information of the unsellable product, such as image information, will be uploaded to the cloud via the internal communication module to remind the back-end maintenance personnel to replace it in time. Furthermore, when the control device recognizes that the user has taken out the unsellable product, it will provide a voice prompt or display relevant information on the cabinet's display screen to remind the user that the current product is unsellable and to select another product, thereby preventing the user from taking unsellable products and effectively improving the user experience of using the unmanned retail equipment.

[0046] In another embodiment, if the control device obtains the account information of a new user, the control device will determine that the previous user has indeed taken the goods and will not execute the above steps S600-S700.

[0047] This application also proposes a storage medium comprising a memory, a processor, and a pick-and-place identification method as described above, stored on the memory and operable on the processor.

[0048] It is worth noting that since the storage medium of this application includes the above-described pick-and-place identification method, the storage medium of this application also includes all embodiments of the above-described pick-and-place identification method and the effects of each embodiment, which will not be repeated here.

[0049] This application also proposes an unmanned retail device, which includes the aforementioned storage medium. The unmanned retail device includes a cabinet and a door mounted on the cabinet. The cabinet is equipped with an electronically controlled lock for locking the door. The electronically controlled lock is configured to be in an unlocked state when receiving an unlocking signal, allowing the door to be in an open / closed state, and to be triggered by the door to be in a locked state when the door is in a closed state, thereby locking the door. The cabinet contains shelves for placing goods and a camera module for acquiring images of the cabinet interior. A door open / close detection device is also provided on the cabinet, which detects the state of the door and outputs corresponding feedback signals, including open / close signals and closed signals.

[0050] It is worth noting that since the unmanned retail device of this application includes the aforementioned storage medium, the unmanned retail device of this application also includes all embodiments of the aforementioned storage medium and the effects brought about by each embodiment, which will not be repeated here.

[0051] The above description is only a part of the embodiments of this application and does not limit the patent scope of this application. All equivalent structural transformations made under the technical concept of this application and using the contents of the specification and drawings of this application, or direct / indirect applications in other related technical fields, are included in the patent protection scope of this application.

Claims

1. A method for identifying pick-up and put-down, characterized in that, This invention relates to unmanned retail equipment, which includes a cabinet and a door mounted on the cabinet. The cabinet is equipped with an electronic lock for locking the door. The electronic lock is configured to be in an unlocked state when it receives an unlocking signal, allowing the door to be in an open / closed state, and to be triggered by the door to be in a locked state when the door is in a closed state, thus locking the door. The cabinet contains shelves for placing goods and a camera module for acquiring images of the cabinet interior. A door open / close detection device is also provided on the cabinet, which detects the state of the door and outputs corresponding feedback signals, including open / close signals and closed signals. The pick-and-place identification method includes: Step S100: Obtain the user's account information. If the user's account information is valid, control the locked electric lock to unlock. Step S200: Based on the image inside the cabinet obtained by the camera module, identify the product information of the product taken out by the user; Step S300: When the feedback signal switches from the opening / closing signal to the closing signal, start the first timing. Step S400: If the user fails to release the retrieved item and the feedback signal remains closed during the first timeout period, the user's account will be deducted from the account. Step S500: If the duration of the first timeout has not reached the preset first duration, no deduction operation will be performed on the user's account.

2. The pick-up and place recognition method as described in claim 1, characterized in that, The camera module includes multiple static recognition cameras installed on the shelves inside the cabinet and a dynamic recognition camera installed at the top of the cabinet near the door; the step of identifying the product information of the goods taken out by the user based on the images inside the cabinet acquired by the camera module includes: Step S210: Acquire static images of goods inside the shelf based on multiple static recognition cameras; Step S220: Based on the dynamic image obtained by the dynamic recognition camera, if the user takes out the product, the product information of the product taken out by the user is identified based on the comparison of static images before and after the taking out action. The steps for when the user has not placed the item back during the first timeout period and the feedback signal remains closed include: during the first timeout period, based on the dynamic image acquired by the dynamic recognition camera, identifying that the user has not placed the item back and the feedback signal remains closed.

3. The pick-up and place recognition method as described in claim 2, characterized in that, The step of obtaining the user's account information and, if the user's account information is valid, controlling the locked electronic lock to unlock further includes: When the feedback signal switches from the closed signal to the open / closed signal, the motion recognition camera that is in a dormant state is brought into working state.

4. The pick-up and place recognition method as described in claim 3, characterized in that, The step of not deducting money from the user's account when the duration of the first timeout has not reached the preset first duration further includes: If the duration of the first timeout has not reached the preset first duration, when new user account information is obtained, the previous user account will be charged.

5. The pick-up and place recognition method as described in claim 4, characterized in that, After step S400, the method further includes: Step S600: If no new user account information is obtained and the user has returned the goods based on the image inside the cabinet, obtain the status information of the returned goods. Step S700: Based on the status information, determine whether to perform a refund operation.

6. The pick-up and place recognition method as described in claim 5, characterized in that, The step of obtaining the status information of the returned goods includes: Step S710: Based on the image inside the cabinet obtained by the camera module, if it is determined that the returned item is in an open state, set the status information of the returned item to a used state. Step S720: Determine that the returned goods are in intact packaging, and obtain the first appearance feature of the returned goods; wherein, the first appearance feature includes: trademark and product barcode; Step S730: If the trademark is inconsistent with the trademark of the previously retrieved goods or the barcode of the goods is inconsistent with the barcode of the previously retrieved goods, determine the status information of the returned goods and set the status information of the returned goods to an abnormal goods status. Step S740: If the trademark is consistent with the trademark of the previously retrieved product and the product barcode is consistent with the product barcode of the previously retrieved product, obtain the second appearance feature of the returned product; wherein, the second appearance feature includes: shelf life; Step S750: If the returned product has reached its expiration date, set the status information of the returned product to expired.

7. The pick-up and place recognition method as described in claim 6, characterized in that, The step of determining whether to perform a refund operation based on the status information includes: If the product's status information is either "used" or "abnormal product," a refund will not be processed. If the product's status is expired, a refund will be processed.

8. The pick-up and place recognition method as described in claim 7, characterized in that, The method further includes: If the product's status information indicates that it is expired or abnormal, the product will be marked as unsellable and the information of the unsellable product will be uploaded to the cloud; and if the user is detected taking out the unsellable product, the user will be notified.

9. A storage medium, characterized in that, The storage medium stores a computer program that can be executed by a processor to perform the pick-up and placement identification method according to any one of claims 1 to 8.

10. An unmanned retail device, characterized in that, The unmanned retail device includes the storage medium as described in claim 9; The unmanned retail device includes a cabinet and a door mounted on the cabinet. The cabinet is equipped with an electronic lock for locking the door. The electronic lock is configured to be in an unlocked state when it receives an unlocking signal, allowing the door to be in an open / closed state, and to be triggered by the door to be in a locked state when the door is in a closed state, thus locking the door. The cabinet contains shelves for placing goods and a camera module for acquiring images of the cabinet interior. A door open / close detection device is also installed on the cabinet, which detects the state of the door and outputs corresponding feedback signals, including open / close signals and closed signals.

Images