Intelligent shelf anti-theft system and anti-theft method

Abstract

The application discloses an intelligent shelf anti-theft system and an anti-theft method, and the method comprises the following steps: acquiring the appearance feature information of a user by using a shelf monitoring module in response to the sensing signal of a human body sensing module; acquiring the goods taking information by using a goods sensing module; collecting the staying information of the user by using a global monitoring module, determining the time-effect identity information of the user according to the staying information and the appearance feature information, and constructing the mapping relationship between the time-effect identity information and the goods taking information; determining the checkout completion result of the user by using a checkout cash register module; if the user has completed the checkout, determining the matching result of the checkout information and the goods taking information corresponding to the user; and if the matching result is a matching failure, generating a first alarm information. The method enables the shop owner to master the behavior trend of the user in the unmanned shop in real time, effectively prevents theft and avoids loss, and realizes reliable unmanned selling.

Description

Technical Field

[0001] This invention relates to the field of unmanned vending control technology, and in particular to an intelligent shelf anti-theft system and anti-theft method. Background Technology

[0002] With the development of mobile payment and IoT technologies, and changes in consumer habits, the retail industry has seen the emergence of unmanned sales systems for convenience stores, offering 24 / 7 operation without staff. Customers can select their desired items, check out independently, and then conveniently leave the store. Unmanned convenience stores typically offer various packaging types, including boxes, bottles, bags, and bulk items. Some bagged and boxed items have hook holes for hanging on shelves for easy customer access. However, in an unmanned environment, store owners cannot monitor real-time item usage, making it difficult to track inventory and sales. They must periodically check inventory, leading to inefficient sales management. Furthermore, owners cannot effectively prevent customers from taking items and leaving without paying, potentially resulting in losses. Summary of the Invention

[0003] This invention provides an intelligent shelf anti-theft system and method. It aims to solve the problem in existing technologies where store owners of unmanned stores cannot quickly ascertain the sales status of goods, resulting in low store management efficiency and ineffective theft prevention.

[0004] In a first aspect, embodiments of the present invention provide an intelligent shelf anti-theft system, including a shelf main module with multiple hanging racks, each rack being used to hang multiple goods; multiple goods sensing modules, each connected to one of the hanging racks in the shelf main module, for sensing the goods when they are attached to or detached from the shelf main module, and determining the goods retrieval information; a human body sensing module, located at the front end of the shelf main module, for sensing users located in front of the shelf main module; and a shelf monitoring module for further monitoring the... The system includes a main shelf module for monitoring the attachment and detachment of various goods on the main shelf module and acquiring the user's facial features; a global monitoring module for monitoring the main shelf module and the checkout status of the goods, and acquiring the user's dwell time information; a checkout / POS module for acquiring the user's checkout information; and a central processing module that is communicatively connected to the goods sensing module, the human body sensing module, the shelf monitoring module, the global monitoring module, and the checkout / POS module, for acquiring the attachment and detachment status of the goods on the main shelf module and the checkout status of the goods.

[0005] Secondly, embodiments of the present invention also provide an anti-theft method for intelligent shelves, applied to the central processing module of the intelligent shelf anti-theft system as described in the first aspect, the method comprising:

[0006] In response to the sensing signal from the human body sensing module, the shelf monitoring module is activated to record video of the main shelf module and collect the facial feature information of the user located in front of the main shelf module.

[0007] The product retrieval information of the products taken by the user in the current shelf main module is obtained through the product sensing module;

[0008] The system collects the user's dwell information through a global monitoring module, determines the user's time-sensitive identity information based on the dwell information and the user's appearance feature information, and constructs a mapping relationship between the time-sensitive identity information and the product retrieval information; wherein, the dwell information includes at least the user's dwell time and dwell identification area;

[0009] Based on the time-sensitive identity information and the goods retrieval information, the checkout and payment module determines the user's checkout completion result.

[0010] If the user's checkout completion result is "completed", then the checkout and cashier module determines the user's checkout information and determines the matching result between the checkout information and the user's corresponding goods retrieval information;

[0011] If the matching result is a failure, a first alarm message is generated;

[0012] If the matching result is successful, a purchase completion message will be generated.

[0013] Based on the anti-theft system and corresponding anti-theft method provided in the embodiments of the present invention, the intelligent shelf anti-theft system provided in the embodiments of the present invention can obtain the status of goods being taken out or returned by users in real time by setting up a goods sensing module; through the shelf monitoring module and the global monitoring module, it can obtain the appearance characteristics information of users currently selecting goods and the information of users staying in front of the main shelf module in real time, and determine the information of goods taken by users on the main shelf module by the information of staying, and bind the information of goods taken with the user's time-limited identity information, so as to realize the behavior of users in the unmanned vending store in real time, thereby ensuring the effective management of store sales by the store owner. At the same time, by comparing the user's goods taken information and payment information through the checkout and cashier module, it can effectively prevent users from leaving the unmanned vending store without paying, thereby effectively preventing theft and avoiding losses, and realizing reliable unmanned vending. Attached Figure Description

[0014] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the following description of the embodiments will be briefly introduced. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0015] Figure 1 A schematic diagram illustrating the intelligent shelf anti-theft system and its application scenarios provided in an embodiment of the present invention;

[0016] Figure 2 A schematic flowchart illustrating the anti-theft method for intelligent shelves provided in an embodiment of the present invention;

[0017] Figure 3 This is a schematic diagram of the first sub-process of the intelligent shelf anti-theft method provided in an embodiment of the present invention;

[0018] Figure 4 This is a schematic diagram of the second sub-process of the intelligent shelf anti-theft method provided in an embodiment of the present invention;

[0019] Figure 5 A schematic diagram of the third sub-process of the intelligent shelf anti-theft method provided in an embodiment of the present invention;

[0020] Figure 6 This is a schematic diagram of the fourth sub-process of the intelligent shelf anti-theft method provided in an embodiment of the present invention. Detailed Implementation

[0021] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0022] It should be understood that, when used in this specification and the appended claims, the terms "comprising" and "including" indicate the presence of the described features, integrals, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or collections thereof.

[0023] It should also be understood that the terminology used in this specification is for the purpose of describing particular embodiments only and is not intended to limit the invention. As used in this specification and the appended claims, the singular forms “a,” “an,” and “the” are intended to include the plural forms unless the context clearly indicates otherwise.

[0024] It should also be further understood that the term "and / or" as used in this specification and the appended claims refers to any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0025] Please see Figure 1 ,like Figure 1 As shown, this embodiment of the invention provides an intelligent shelf anti-theft system, which can be installed in unmanned vending stores and includes at least the following modules: a shelf main module 10, equipped with multiple hanging racks 11, each hanging rack 11 being used to hang multiple goods; multiple goods sensing modules 20, each connected to one of the hanging racks 11 in the shelf main module 10, used to sense the goods when the corresponding goods are hung into or detached from the shelf main module 10, and to determine the goods retrieval information; a human body sensing module 30, located at the front end of the shelf main module 10, used to sense users located in front of the shelf main module 10; and a shelf monitoring module 4. 0, used to further monitor the hanging or detaching status of each item on the shelf main module 10, and to obtain the user's appearance feature information; global monitoring module 50, used to monitor the shelf main module 10 and the checkout status of the items, and to obtain the user's dwell information; checkout and cashier module 60, used to obtain the user's corresponding checkout information; central processing module 70, which is communicatively connected to the item sensing module 20, the human body sensing module 30, the shelf monitoring module 40, the global monitoring module 50, and the checkout and cashier module 60, is used to obtain the hanging or detaching status of the items on the shelf main module 10 and the checkout status of the items.

[0026] In this embodiment, the central processing module 70 can be a server, intelligent processing terminal, or other module capable of acquiring and processing information. The central processing module 70 can receive various data from the goods sensing module 20, the human body sensing module 30, the shelf monitoring module 40, the global monitoring module 50, and the checkout / POS module 60, and after processing the various data, issue control commands to each module, or obtain various required information based on the various data.

[0027] The main shelf module 10 is an independent display structure for hanging goods, equipped with multiple hanging racks 11. Each item's outer packaging has a hanging hole through which the hanging rack 11 passes, allowing the goods to be hung on the rack 11. The front end of the main shelf module 10 can accommodate multiple hanging racks 11 side-by-side, and for a single hanging rack 11, all items hung on it are identical. Before hanging goods on the racks 11, the store owner or staff can pre-determine the type, model, price, and quantity of goods to be hung on each rack 11, and hang the goods on the corresponding racks 11 accordingly. To monitor the movement of goods as they are removed from or returned to the racks 11, each rack 11 can be equipped with an independent goods sensing module 20. This module can be an infrared sensor or other structure capable of sensing goods movement. The product sensing module 20 is actually installed at the head end of each rack 11. Products need to be hooked into or removed from the head end of the rack 11. Therefore, when a user or a power source takes a product off or puts a product back into the rack 11, the product sensing module 20 on the corresponding rack 11 can detect in real time whether the product has been taken or put back. When the product is stationary, no sensing is performed. Specifically, when a product moves from the head end to the tail end of the rack 11, it can be determined that the product has been put back into the rack 11; when a product moves from the tail end to the head end of the rack 11, it can be determined that the product has been removed from the rack 11.

[0028] Understandably, when store clerks put or take goods off the shelves, or when users remove or return goods from racks 11, the central processing module 70 can record the removal or return of goods based on the sensing data from the shelf sensor module, and update the storage status of the goods accordingly, so that the store owner can know the current sales status of the goods in real time. To bind the goods retrieval information with the user, a shelf monitoring module 40 and a global monitoring module 50 can be set up simultaneously. The shelf monitoring module 40 can be a camera installed on the main shelf module 10, which can record video of each rack 11 and the goods on the rack 11 at the front of the shelf monitoring module 40, and acquire facial images of users located in front of the shelf to extract the user's facial features information. To improve the efficiency and accuracy of user identification, a human body sensing module 30 can be additionally installed in front of the main shelf module 10. This human body sensing module 30 can be in the form of an infrared sensor. When the distance between the user and the corresponding main shelf module 10 is less than a preset distance threshold, the human body sensing module 30 will be triggered. At this time, the shelf monitoring module 40 will record and identify the image in front of the shelf. Simultaneously, a global monitoring module 50 can be set up at various locations within the store, and each global monitoring module 50 can acquire a global image of the store. For each global monitoring module 50, a recognition area corresponding to each main shelf module 10 can be preset, and a checkout recognition area corresponding to each checkout / POS module 60 can also be preset. When a user is located in any recognition area or checkout recognition area, the central processing module 70 can obtain the corresponding main shelf module 10 or checkout / POS module 60 through the global monitoring module 50, thereby determining the user's location and binding the product retrieval information at that location to the user, achieving effective management of both products and users.

[0029] Multiple shelf monitoring modules 40 can be configured to accommodate larger shelf main modules 10. Understandably, because the product sensing module 20 lacks identification capabilities, it cannot recognize the type of product when a user places goods back on the rack 11 and it is detected by the product sensing module 20. When a user enters an unmanned vending store equipped with the aforementioned intelligent shelf anti-theft system, they need to authorize the store to access their physical characteristics.

[0030] Corresponding to the aforementioned intelligent shelf anti-theft system, this invention also provides an intelligent shelf anti-theft method, which is applied to the central processing module of the aforementioned intelligent shelf anti-theft system. Please refer to [link to relevant documentation]. Figure 2 ,like Figure 2 As shown, the method includes the following steps:

[0031] S110. In response to the sensing signal from the human body sensing module, the shelf monitoring module is activated to record video of the main shelf module and collect the facial feature information of the user located in front of the main shelf module.

[0032] When the central processing module 70 receives a sensing signal from the human body sensor module 30, it can determine that a user is standing in front of the corresponding shelf main module 10. At this time, it can activate the shelf monitoring module 40 on the corresponding shelf main module 10 to capture images of the user. After capturing multiple facial images of the user, a preset face recognition algorithm can be used to capture the user's facial features, and a capture time can be assigned to these features to determine when the user first stood in front of the shelf main module 10. Furthermore, if the user remains in front of the shelf main module 10 for a certain period, the start and end times of that time period can be determined by the first capture time of the facial features captured by the shelf monitoring module 40 and the second capture time of the last capture of facial features.

[0033] S120. Obtain the product retrieval information of the product taken by the user in the current shelf main module through the product sensing module; wherein, the product retrieval information includes at least the shelf main module corresponding to the product and the sensing time when the product is taken.

[0034] The sensing information of the goods sensing module 20 can generate the corresponding goods retrieval information. Each goods sensing module 20 will only sense the goods of the corresponding rack 11. Therefore, the corresponding goods retrieval information can be matched one-to-one with the goods sensing module 20.

[0035] In one embodiment, please refer to Figure 3 Step S120 may specifically include the following steps S121-S123:

[0036] S121. The goods sensing module that currently senses that the goods have been taken out is identified as the first target goods sensing module, and the sensing time of the first target goods sensing module is determined.

[0037] S123. Based on the product storage information corresponding to the first target product sensing module, determine the name information and price information of the product, and update the product storage information; wherein, the product storage information includes at least the shelf main module corresponding to the product and the inventory quantity.

[0038] S123. The updated product storage information, name information, price information, sensing time, and product main module corresponding to the first target product sensing module are combined into the product retrieval information.

[0039] In this embodiment, the central processing module 70 stores the product storage information corresponding to each rack 11, and the product storage information corresponding to a single rack 11 directly corresponds to the product sensing module 20 on that rack 11. Therefore, when the first product sensing module 20 senses that a product has been taken out, the central processing module 70 can update the product storage information corresponding to the rack 11 based on the rack 11 corresponding to the first product sensing module 20. Specifically, if the first product sensing module 20 senses that a first quantity of products has been taken out of the rack 11, the inventory quantity in the product storage information corresponding to that rack 11 can be reduced and updated, and the amount of reduction in inventory quantity is the first quantity. At the same time, the product storage information also includes the product's name information, price information, etc. When the first product sensing module 20 senses that a product has been taken out, the central processing module 70 can also simultaneously obtain the product's name information and price information, and record the sensing time when the product is taken out. In subsequent processes, the sensing time can be used to bind the product retrieval information with the user. Finally, the product storage information, name information, price information, sensing time, and the product main module corresponding to the first product sensing module 20 can be combined to form the unique product retrieval information corresponding to the retrieved product, thereby determining the original storage location of the product. It is evident that while the product is being retrieved, the store owner can obtain real-time information on the product retrieval and inventory status through a terminal communicating with the central processing module 70. The central processing module 70 can also determine the quantity of currently retrieved but unpaid products for real-time anti-theft risk management.

[0040] S130. Collect the user's dwell information through the global monitoring module, determine the user's time-sensitive identity information based on the dwell information and the appearance feature information, and construct a mapping relationship between the time-sensitive identity information and the goods retrieval information; wherein, the dwell information includes at least the user's dwell time and dwell identification area.

[0041] The global monitoring module 50 can identify user images in the store panoramic image using preset recognition algorithms such as confidence recognition algorithms, and determine the current shelf main module 10 where the user is located based on the recognition area where the user image is located, that is, determine the user's dwell recognition area; at the same time, it can also determine the time the user stays in front of the current shelf main module 10, and thus determine the user's dwell time. Subsequently, the dwell time and dwell recognition area can be used together as a set of dwell information for the user to indicate that the user stayed in front of a certain shelf main module 10 for a certain period of time. Subsequently, since the product retrieval information also includes the product retrieval time, a mapping relationship can be established between the product retrieval information within the dwell time and the user to determine the user's product retrieval status in the current shelf main module 10.

[0042] In one embodiment, please refer to Figure 4 Step S130 may specifically include the following steps:

[0043] S131. The global monitoring module determines the user's dwell identification area and the user's dwell time in the dwell identification area, and uses the dwell identification area and the dwell time as the dwell information.

[0044] Because each shelf module 10 is divided into its own unique identification area in the images captured by the global monitoring module 50, when a user is located in the identification area in front of a shelf module 10, the global monitoring module 50 can determine the user's designated identification area and establish a binding relationship between the identification area and the shelf module 10. Simultaneously, it records the user's dwell time within the identification area, including the start and end times, corresponding to the user's entry and exit from the area. Subsequently, the identification area and dwell time can be used as the user's dwell information. Understandably, the global monitoring module 50 cannot confirm the user's identity, therefore it first generates dwell information for an identity to be claimed.

[0045] S132. Determine the target shelf main module corresponding to the dwell recognition area and the appearance feature information previously collected by the shelf monitoring module corresponding to the target shelf main module.

[0046] Subsequently, the central processing module 70 can obtain the previously collected facial feature information (i.e., the current facial feature information) in the area where the user is staying, and then determine the identity of the user corresponding to the facial feature information.

[0047] S133. The dwell information and the previously collected appearance feature information are determined as the user's time-sensitive identity information.

[0048] After obtaining the physical characteristics information, the stay information of the identity to be claimed can be bound to the user with the physical characteristics information to confirm the user's time-limited identity information.

[0049] S134. Map the product retrieval information of each item in the target shelf main module whose sensing time is within the dwell time to the product retrieval information corresponding to the user, so as to construct the mapping relationship between the user's time-sensitive identity information and the product retrieval information.

[0050] Subsequently, goods taken within the time period corresponding to the user's time-sensitive identity information can be identified as goods taken by that user. Then, the item take-up information within the stated time period can be mapped to the user's time-sensitive identity information, thus achieving information binding between the user and the taken goods.

[0051] In this embodiment, by determining the user's stay information and the goods taken during the stay, the goods taking information can be effectively bound to the user's time-sensitive identity information, thereby achieving effective tracking of the user and the goods taken, and playing an effective anti-theft role.

[0052] In one embodiment, the following step is included after step S134:

[0053] If the global monitoring module identifies that the user with the timed identity information is located in the dwell recognition area corresponding to another shelf main module, then the first collection time of the appearance feature information corresponding to the user by the shelf monitoring module in the other shelf main module is determined.

[0054] Determine the second acquisition time when the user leaves the identification area of ​​the other shelf main module;

[0055] If there is no other user's appearance feature information during the time period between the first collection time and the second collection time, then a mapping relationship is established between the goods retrieval information corresponding to the other shelf main module during the time period between the first collection time and the second collection time and the user's time-sensitive identity information.

[0056] In this embodiment, when the global monitoring module 50 identifies that a user is located in the dwell recognition area corresponding to another shelf main module 10, it is necessary to determine the dwell time of the user in the dwell recognition area corresponding to the shelf main module 10. At this time, the first collection time and the second collection time obtained by the shelf monitoring module 40 can be used to further determine the user's dwell time. If no other user's appearance feature information is identified between the first collection time and the second collection time, the goods taking information during the dwell time can be added to the goods taking information of the currently identified user to update the user's goods taking information. If the appearance feature information of another user exists within the time period between the first collection time and the second collection time, the product retrieval information corresponding to the products taken by each customer during the overlapping time period of their stay in front of the main shelf module 10 can be simultaneously mapped to the time-sensitive identity information of each user. When all users have completed checkout within the overlapping time period, the intersection of all product information in the checkout information of each user is confirmed. If the intersection of all product information contains product retrieval information corresponding to each product taken during the overlapping time period, then it can be determined that no user has failed to checkout. Specifically, for example, if user A and user B are simultaneously captured in their facial feature information between the first and second collection times, and there is information about them taking a product C within the overlapping time period, then this product C take-up information is mapped to both user A's and user B's time-sensitive identity information. Once it's confirmed that both user A and user B have completed checkout, the union of the product information from their checkout records can be obtained. If the union contains information about taking a product C, then it can be confirmed that no one left without checking out. If no product C take-up information exists, then tracking can be performed using the facial feature information of users A and B, thus achieving effective theft prevention.

[0057] In one embodiment, please refer to Figure 5 Step S130 may be followed by the following steps S1301-S1309:

[0058] S1301. Determine if there is a second target goods sensing module that senses the goods being returned; if yes, proceed to step S1302; if no, determine that the goods have been taken.

[0059] If a product sensor module 20 detects a product being placed, it indicates that a user has not selected the product and intends to return it to the rack 11. In this case, the product sensor module 20 can be designated as the second product sensor module 20. However, it is possible that a user may place a product on a different rack 11 than its original location. To avoid confusion in the product display, it is necessary to determine whether the product has been placed on the correct rack 11.

[0060] S1302. When the product retrieval information corresponding to the user only contains product retrieval information corresponding to the current shelf main module where the user is located, obtain the first sensing time when the second target product sensing module sensed that the product was put back, and obtain the second sensing time when the first target product sensing module sensed that the product was taken out last time.

[0061] When a user's item retrieval information only corresponds to the item retrieval information of the current shelf main module 10 where the user is located, it means that the first item sensing module 20 and the second item sensing module 20 are located on the same shelf main module 10, and the user has only taken items from the current shelf main module 10. When the shelf monitoring module 40 in the current shelf main module 10 obtains the user's appearance feature information, or when the user is identified by the global monitoring module 50 as being located in the identification area of ​​the current shelf main module 10, the central processing module 70 can automatically determine all item retrieval information corresponding to the user based on the user's time-sensitive identity information. Therefore, it is necessary to obtain the second sensing time of the first item sensing module 20 in the current shelf main module 10 that last sensed that an item was taken out, and to obtain the first sensing time of the second item sensing module 20 that sensed that an item was put back. The time interval between the first sensing time and the second sensing time is used to further determine whether the item that was put back belongs to the shelf main module 10, so as to prevent the situation where a user takes an item in another shelf main module 10 but it is not successfully recorded. The first product sensing module 20 that sensed the last time a product was taken out can also be the product sensing module 20 corresponding to the most recently taken product in the user's product retrieval information.

[0062] S1303. Determine whether the difference between the first sensing time and the second sensing time is less than a preset time difference; if yes, proceed to step S1304; if no, proceed to step S1308.

[0063] If the difference between the first sensing time and the second sensing time is greater than or equal to the preset time difference, it means that the user may have taken the product from another shelf main module 10 and placed it in the current shelf main module 10. The product taking information was not successfully recorded. At this time, it is necessary to send the first product placement error risk information to the store owner to remind the store owner that there may be a situation of product confusion between different shelf main modules 10.

[0064] S1304. Determine whether the number of appearance feature information acquired by the shelf monitoring module during the time period between the second sensing time and the first sensing time is greater than one; if yes, proceed to step S1309; if no, proceed to step S1305.

[0065] Subsequently, it is necessary to determine whether any other users appeared in front of the current shelf main module 10 between the first sensing time and the second sensing time. If different users were present, it is impossible to determine which user placed the goods back into the current shelf main module 10. The central processing module 70 needs to send a second item placement error risk information to the store owner to alert them of the potential item confusion. Alternatively, if the number of acquired facial feature information entries is greater than one, the item retrieval information corresponding to each facial feature information entry can be obtained. If only one user possesses the item retrieval information corresponding to the current shelf main module 10, then the currently returned item can be determined to be the item previously retrieved. If multiple users possess the item retrieval information corresponding to the current shelf main module 10, then the sensing time corresponding to each item retrieval information corresponding to the current shelf main module 10 is obtained, and the item retrieval information corresponding to the sensing time closest to the first sensing time is determined to be the item previously retrieved.

[0066] S1305. Determine that the goods currently being returned are the goods that were previously taken out.

[0067] If only one user is located in the current shelf main module 10 during the first and second sensing times, then it can be determined that the returned goods are the goods that were previously taken out.

[0068] S1306. Determine whether the second target goods sensing module is a different goods sensing module from the first target goods sensing module that previously sensed the goods being taken out; if yes, proceed to step S1307; if no, determine that the goods have been correctly returned.

[0069] If the second product sensor module 20 that senses the product being returned is not the same as the first product sensor module 20 that sensed the product being taken out last time, it can be determined that the product was placed on the wrong rack 11. However, if the second product sensor module 20 is the same as the first product sensor module 20, it can be determined that the user correctly placed the product back on the original rack 11.

[0070] S1307. Generate the first item placement error alarm message.

[0071] If the goods are placed incorrectly, a first goods placement error alarm message needs to be generated. This message is mainly used to remind the store owner that there are goods hanging on the wrong rack 11 on the current shelf main module 10, and the correct rack 11 corresponding to the goods is also located on the current shelf main module 10.

[0072] S1308, Generate the first item placement error risk information.

[0073] The first product placement error risk information is used to alert store owners that there may be product confusion between different shelf main modules 10.

[0074] S1309. Generate risk information for incorrect placement of the second item.

[0075] The second item placement risk information is used to alert store owners that there may be instances of mixed-up goods.

[0076] In this embodiment, by determining the difference in sensing time and the amount of appearance feature information, it is possible to effectively determine whether a product is placed on the wrong hanging rack 11 when a user takes a product from the current shelf main module 10 and then puts it back, thereby effectively managing the placement of the products. Furthermore, if multiple product retrieval records exist for the current shelf main module 10 corresponding to a user currently in front of it, a warning message can be sent to the store owner to prevent the user from hanging a product from the same shelf main module 10 on the first product sensing module 20 that was previously detected as having been taken, even if the product does not correspond to the first product sensing module 20, but rather to another product sensing module 20.

[0077] In one embodiment, please refer to Figure 6 Step S130 may further include the following steps S210-S240:

[0078] S210. Determine if there is a second target goods sensing module that senses the goods being returned; if yes, proceed to step S220; if no, determine that the goods have been taken.

[0079] If a product sensor module 20 detects a product being placed on the shelf, it indicates that a user has not selected the product and intends to return it to the rack 11. This product sensor module 20 can then be designated as the second product sensor module 20. However, it's possible that a user may take products from multiple shelf modules 10 and then place them in a different shelf module than the one originally intended for the product. To prevent this from causing confusion in the product display, it's necessary to determine whether the product has been placed in the correct shelf module.

[0080] S220. When there are multiple different shelf main modules corresponding to the goods retrieval information in the goods retrieval information corresponding to the user, obtain the goods retrieval information of each of the goods retrieved by the user in each of the shelf main modules; wherein, each shelf main module includes the current shelf main module where the user is located and the other shelf main modules where the user is not located.

[0081] When a user takes a number of items from multiple different shelf modules 10, it means that the items the user is currently returning may have originated from the current shelf module 10 or other shelf modules 10. In this case, it is necessary to obtain the shelf module 10 corresponding to each item taken from the user's item take information.

[0082] S230. Determine whether there is target product retrieval information corresponding to the first target product sensing module in the current shelf main module among the product retrieval information; if yes, proceed to step S240, otherwise proceed to step S250.

[0083] After obtaining the shelf main module 10 corresponding to each item in the user's item retrieval information, the shelf main modules 10 where the user has taken items can be compared and matched with the current shelf main module 10.

[0084] S240. When the second target goods sensing module is different from each of the first target goods sensing modules corresponding to the target goods retrieval information, a second goods placement error alarm message is generated.

[0085] If the product retrieved by the user is located in the current shelf module 10, it means the product currently being returned by the user may be located in the current shelf module 10. If the second product sensing module 20 is different from all the first product sensing modules 20 corresponding to the user, it can be determined that the returned product has been misplaced. In this case, a second product placement error alarm message is sent to alert the store owner that the product has been placed on the wrong shelf 11. To determine whether the product was originally in the current shelf module 10 or another shelf module 10, the most recent sensing time in the current shelf module 10 can be determined by obtaining the sensing times corresponding to the user's product retrieval information. The system should detect all goods before the second sensing time of the first goods sensing module 20 when the goods are taken out, and determine the corresponding shelf main module 10. If it only corresponds to the current shelf main module 10, it can be determined that the goods were originally located in the current shelf main module 10. If it corresponds to the current shelf main module 10 and several other shelf main modules 10, it can be determined that the goods may come from other shelf main modules 10. At this time, it is also necessary to send a warning message to the store owner to remind the store owner that there may be goods from other shelf main modules 10 placed in the current shelf main module 10.

[0086] S250, Generate a third item placement error alarm message.

[0087] If the current shelf module 10 is not present in any of the various shelf main modules 10 that the user has taken the goods from, then it can be directly determined that the returned goods do not belong to the current shelf main module 10. At this time, the third item placement error alarm message is used to remind the store owner that the goods have been placed on the wrong shelf main module 10.

[0088] In this embodiment, by obtaining the corresponding shelf main modules 10 for each item retrieved by the user, it is possible to effectively determine whether the user has placed items in the wrong shelf main module 10 and prompt the store owner to manage it promptly. It is understood that an image recognition algorithm for the items can also be added to directly utilize the shelf monitoring module 40 to obtain images of the items on each rack 11, and directly perform recognition and analysis on the images to determine whether different items are hung on a particular rack 11. The steps provided in this embodiment are simplified and do not rely on an image recognition algorithm for the items.

[0089] S140. Based on the time-sensitive identity information and the goods retrieval information, the checkout and payment module determines the user's checkout completion result.

[0090] The time-sensitive identity information and the goods retrieval information are mapped and matched with each other. Therefore, when a user performs a checkout operation in the checkout and cashier module 60, the checkout information generated by the checkout and cashier module 60 can be compared with the goods retrieval information to determine whether the user with the corresponding time-sensitive identity information has completed the checkout of all goods.

[0091] In one embodiment, step S140 may specifically include the following steps:

[0092] The global monitoring module determines the corresponding identifier box for the user.

[0093] If the global monitoring module obtains that the identifier box corresponding to the user is located in the checkout recognition area where the checkout cashier module is located, then it determines the first time when the user corresponding to the identifier box enters the checkout recognition area and the second time when the user leaves the checkout recognition area, and determines the checkout cashier module corresponding to the checkout recognition area.

[0094] If the checkout cashier module corresponding to the checkout recognition area contains checkout information within the time period between the first time and the second time, then the user's checkout completion result is determined to be completed.

[0095] If the checkout module corresponding to the checkout recognition area does not have checkout information during the time period between the first time and the second time, then the user's checkout completion result is determined to be incomplete.

[0096] In this embodiment, since multiple checkout / payment modules 60 are set up, users can check out at any of them. The global monitoring module 50 needs to obtain the checkout identification area to determine the correct checkout / payment module 60. Then, when the user leaves the checkout identification area, the checkout information of the corresponding checkout / payment module 60 is obtained. If checkout information exists between the first and second time periods when the user is present, it can be determined that the checkout information belongs to the corresponding user. If no checkout information exists within this time period, it can be determined that the user has not completed the checkout. If the global monitoring module 50 does not identify the user in the identification area corresponding to any shelf main module 10 or the checkout identification area corresponding to any checkout / payment module 60, it determines that the user left the store without performing any checkout operation. In this case, an alarm message needs to be sent to the store owner to notify them that the user left the store without paying.

[0097] S150. If the user's checkout completion result is "completed", then the checkout and cashier module determines the user's checkout information and determines the matching result between the checkout information and the user's corresponding goods retrieval information.

[0098] If the user has completed checkout, it is necessary to further determine whether the user has checked out all items. At this point, the checkout information needs to be matched with the user's corresponding item pickup information to determine whether the checkout information includes all item pickup information.

[0099] In one embodiment, step S150 may specifically include the following steps:

[0100] Obtain the information of each checked-out item contained in the user's checkout information;

[0101] Each of the checked-out items is matched with the item pickup information corresponding to each user.

[0102] If each of the checked-out items corresponds one-to-one with the item pickup information of the user, then the matching result is determined to be a successful match.

[0103] If any of the product retrieval information does not correspond to any of the settled product information, then the matching result is determined to be a matching failure.

[0104] In this embodiment, if each item's pickup information corresponds one-to-one with each item's checked-out information in the checkout information, the matching result is considered successful, and the user has not left the store without checking out. However, if several items' pickup information cannot be matched with any of the checked-out items, it is determined that a user has left without checking out. The checked-out item information includes the name of the checked-out item, its location on the shelf module 10, and the quantity of the same item checked out, facilitating matching with the item pickup information.

[0105] S160. If the matching result is a matching failure, a first alarm message is generated.

[0106] If the matching result is a failure, the first alarm message is used to remind the store owner that the customer has left with unpaid goods.

[0107] S170. If the matching result is a successful match, then purchase completion information is generated.

[0108] If the matching result is successful, then the purchase completion message is used to notify the store owner that the user has completed a normal purchase.

[0109] As can be seen, based on the anti-theft system and method provided in the embodiments of the present invention, the intelligent shelf anti-theft system provided in the embodiments of the present invention, by setting up a goods sensing module, can obtain in real time the situation of goods being taken out or returned by users; through the shelf monitoring module and the global monitoring module, it can obtain in real time the appearance characteristics information of users currently selecting goods and the information of users staying in front of the main shelf module, and determine the information of goods taken by users on the main shelf module through the information of staying, and bind the information of goods taken with the user's time-limited identity information, thereby enabling real-time monitoring of the user's behavior in the unmanned vending store, thereby improving the store owner's management efficiency of store sales. At the same time, by comparing the user's information of goods taken with the payment information through the checkout and cashier module, it can effectively prevent users from leaving the unmanned vending store without paying, thereby effectively preventing theft and avoiding losses.

[0110] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in the present invention, and these modifications or substitutions should all be covered within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. An intelligent shelf anti-theft system, characterized in that, include: The main module of the shelf is equipped with multiple hanging racks, each of which is used to hang multiple goods. Multiple product sensing modules are connected one-to-one to each of the hanging racks in the main shelf module. They are used to sense the product when it is hung into or detached from the main shelf module and to determine the product retrieval information. Each product sensing module is located at the head end of each hanging rack, and the product is hung into or removed from the head end of each hanging rack. A human body sensing module is installed at the front end of the main shelf module to sense users located in front of the main shelf module; The shelf monitoring module is used to further monitor the hanging or detachment of each item on the main shelf module and to obtain the user's facial features. The global monitoring module is used to monitor the main shelf module and the checkout status of the goods, and to obtain the user's dwell information; the global monitoring module is also used to determine the user's time-sensitive identity information based on the dwell information and the appearance feature information, and to construct a mapping relationship between the time-sensitive identity information and the goods retrieval information; The checkout / payment module is used to obtain the checkout information corresponding to the user; The central processing module is communicatively connected to the goods sensing module, the human body sensing module, the shelf monitoring module, the global monitoring module, and the checkout and cashier module, and is used to obtain the hanging or detaching status of the goods on the main shelf module and the checkout status of the goods. After the global monitoring module determines the user's time-sensitive identity information based on the dwell information and appearance feature information and constructs a mapping relationship between the time-sensitive identity information and the goods retrieval information, the central processing module is further configured to, if there is a second target goods sensing module that senses goods being returned, then when the goods retrieval information corresponding to the user only contains goods retrieval information corresponding to the current shelf main module where the user is located, obtain the first sensing time when the second target goods sensing module sensed the goods being returned, and obtain the first target goods sensing module that sensed the goods being taken out last time. The second sensing time of the block; if the difference between the first sensing time and the second sensing time is less than a preset time difference, then it is determined whether the number of appearance feature information acquired by the shelf monitoring module during the time period between the second sensing time and the first sensing time is greater than one; if the number of appearance feature information is equal to one, then it is determined that the goods currently being returned are the goods that were previously taken out; if the second target goods sensing module is a different goods sensing module from the first target goods sensing module that previously sensed that the goods were taken out, then a first goods placement error alarm message is generated.

2. A method for preventing theft of intelligent shelves, applied to the central processing module of the intelligent shelf anti-theft system as described in claim 1, characterized in that, The method includes: In response to the sensing signal from the human body sensing module, the shelf monitoring module is activated to record video of the main shelf module and collect the facial feature information of the user located in front of the main shelf module. The product retrieval information is obtained by the product sensing module, which acquires the product retrieval information of the product taken by the user in the current shelf main module. The product retrieval information includes at least the shelf main module corresponding to the product and the sensing time when the product is retrieved. Each product sensing module is set at the head end of each rack, and the product is either hooked into the rack or taken off from the head end of each rack. The system collects the user's dwell information through a global monitoring module, determines the user's time-sensitive identity information based on the dwell information and the user's appearance feature information, and constructs a mapping relationship between the time-sensitive identity information and the product retrieval information; wherein, the dwell information includes at least the user's dwell time and dwell identification area; Based on the time-sensitive identity information and the goods retrieval information, the checkout and payment module determines the user's checkout completion result. If the user's checkout completion result is "completed", then the checkout and cashier module determines the user's checkout information and determines the matching result between the checkout information and the user's corresponding goods retrieval information; If the matching result is a failure, a first alarm message is generated; If the matching result is a successful match, a purchase completion message will be generated; After collecting the user's dwell information through the global monitoring module, determining the user's time-sensitive identity information based on the dwell information and the appearance feature information, and constructing a mapping relationship between the time-sensitive identity information and the goods retrieval information, the method further includes: If there is a second target product sensing module that senses the product being put back, then when the product retrieval information corresponding to the user only contains product retrieval information corresponding to the current shelf main module where the user is located, the first sensing time when the second target product sensing module senses the product being put back is obtained, and the second sensing time when the first target product sensing module senses the product being taken out last time is obtained. If the difference between the first sensing time and the second sensing time is less than a preset time difference, then it is determined whether the number of appearance feature information acquired by the shelf monitoring module during the time period between the second sensing time and the first sensing time is greater than one. If the number of appearance feature information is equal to one, then the item currently being returned is determined to be the item that was previously taken out. If the second target item sensing module is a different item sensing module from the first target item sensing module that previously sensed that the item was taken out, then a first item placement error alarm message is generated.

3. The anti-theft method for intelligent shelves according to claim 2, characterized in that, The step of obtaining the product retrieval information of the product taken by the user in the current shelf main module through the product sensing module includes: The product sensing module that currently senses that the product has been taken out is identified as the first target product sensing module, and the sensing time of the first target product sensing module is determined. Based on the product storage information corresponding to the first target product sensing module, the name and price information of the product are determined, and the product storage information is updated; wherein, the product storage information includes at least the shelf main module corresponding to the product and the inventory quantity; The updated product storage information, name information, price information, sensing time, and the product main module corresponding to the first target product sensing module are combined to form the product retrieval information.

4. The anti-theft method for intelligent shelves according to claim 3, characterized in that, After collecting the user's dwell information through the global monitoring module, determining the user's time-sensitive identity information based on the dwell information and the appearance feature information, and constructing a mapping relationship between the time-sensitive identity information and the goods retrieval information, the method further includes: If there is a second target product sensing module that senses the return of goods, then when there are multiple different shelf main modules in the product retrieval information corresponding to the user, the product retrieval information of each product retrieved by the user in each of the shelf main modules is obtained; wherein, each shelf main module includes the current shelf main module where the user is located and the other shelf main modules where the user is not located. If the product retrieval information contains target product retrieval information corresponding to each of the first target product sensing modules in the current shelf main module, then when the second target product sensing module is different from the first target product sensing module corresponding to the target product retrieval information, a second product placement error alarm message is generated.

5. The anti-theft method for intelligent shelves according to claim 4, characterized in that, If a second target product sensing module is present, and the product retrieval information corresponding to multiple different shelf main modules exists in the product retrieval information for the user, after obtaining the product retrieval information of each product retrieved by the user in each of the shelf main modules, the method further includes: If the target product retrieval information corresponding to the first target product sensing module in the current shelf main module is not found in the product retrieval information, a third product placement error alarm message is generated.

6. The anti-theft method for intelligent shelves according to claim 2, characterized in that, The system collects the user's dwell time information through a global monitoring module, determines the user's time-sensitive identity information based on the dwell time information and the user's physical characteristics, and constructs a mapping relationship between the time-sensitive identity information and the product retrieval information, including: The global monitoring module determines the user's location in the designated area and the user's dwell time in that area, and uses the designated area and dwell time as the dwell information. Determine the target shelf main module corresponding to the dwell recognition area and the appearance feature information previously collected by the shelf monitoring module corresponding to the target shelf main module; The dwell information and the previously collected physical feature information are determined as the user's time-sensitive identity information; The product retrieval information of each item in the target shelf main module that is within the dwell time is mapped to the product retrieval information of the user, so as to construct the mapping relationship between the user's time-sensitive identity information and the product retrieval information.

7. The anti-theft method for intelligent shelves according to claim 6, characterized in that, After mapping the product retrieval information corresponding to the target shelf main module during the dwell time to the product retrieval information corresponding to the user, so as to construct the mapping relationship between the user's time-sensitive identity information and the product retrieval information, the method further includes: If the global monitoring module identifies that the user with the timed identity information is located in the dwell recognition area corresponding to another shelf main module, then the first collection time of the appearance feature information corresponding to the user by the shelf monitoring module in the other shelf main module is determined. The second acquisition time is determined when the user leaves the identification area of ​​the other shelf main module; If there is no other user's appearance feature information during the time period between the first collection time and the second collection time, then a mapping relationship is established between the goods retrieval information corresponding to the other shelf main module during the time period between the first collection time and the second collection time and the user's time-sensitive identity information.

8. The anti-theft method for intelligent shelves according to claim 2, characterized in that, The step of determining the user's checkout completion result through the checkout and payment module based on the time-sensitive identity information and the goods retrieval information includes: The global monitoring module determines the corresponding identifier box for the user. If the global monitoring module obtains that the identifier box corresponding to the user is located in the checkout recognition area where the checkout cashier module is located, then it determines the first time when the user corresponding to the identifier box enters the checkout recognition area and the second time when the user leaves the checkout recognition area, and determines the checkout cashier module corresponding to the checkout recognition area. If the checkout cashier module corresponding to the checkout recognition area contains checkout information within the time period between the first time and the second time, then the user's checkout completion result is determined to be completed. If the checkout module corresponding to the checkout recognition area does not have checkout information during the time period between the first time and the second time, then the user's checkout completion result is determined to be incomplete.

9. The anti-theft method for intelligent shelves according to claim 2, characterized in that, If the user's checkout completion result is "completed," then the checkout and payment module determines the user's checkout information and determines the matching result between the checkout information and the user's corresponding goods retrieval information, including: Obtain the information of each checked-out item contained in the user's checkout information; Each of the checked-out items is matched with the item pickup information corresponding to each user. If each of the checked-out items corresponds one-to-one with the item pickup information of the user, then the matching result is determined to be a successful match. If any of the product retrieval information does not correspond to any of the settled product information, then the matching result is determined to be a matching failure.

Images