A deep learning-based vending machine product identification method

Abstract

The present application relates to the technical field of control management system, and in particular to a self-service vending machine commodity identification method based on deep learning, which comprises the following steps: splitting the self-service vending machine shelf into multiple independent display areas; comparing and analyzing the multiple undetermined commodities based on the weight difference before and after the commodity replenishment and combining the warehouse commodity database to determine the multiple undetermined commodities; comparing the change area image before and after the commodity replenishment with the image data of the undetermined commodities to obtain multiple undetermined similarities; analyzing the undetermined similarities to obtain undetermined confidence values; determining the replenishment commodities based on the undetermined confidence values; replenishing the replenishment commodities to the original commodity set to obtain multiple independent full-warehouse sets. The present application can determine the commodity inventory in the self-service vending machine after the replenishment is completed by constructing the full-warehouse commodity set, so as to take the commodity inventory as the original inventory and check the commodity order purchased by the user according to the inventory change of each purchase of the user.

Description

Technical Field

[0001] The present invention relates to the technical field of control and management systems, and in particular to a method for identifying goods in self-service vending machines based on deep learning. Background Art

[0002] Self-service vending machines, which use automated technology to provide product purchasing services, are widely used in public places such as shopping malls, train stations, airports, and offices. Users can select their desired products through a touchscreen or other interactive methods, and the system automatically completes the product display, payment, and delivery process. To enhance convenience and security, modern self-service vending machines support QR code payment, credit card payment, facial recognition, product identification, and inventory management, ensuring that users can complete their purchases quickly, conveniently, and safely.

[0003] Self-service vending machines include closed vending machines and open-counter vending machines. When using the open-counter vending machine, customers can open the door to pick up the goods, which is highly selective and very convenient and fast. However, the existing open-counter vending machines generally allow customers to freely select goods, and it is difficult to intelligently generate and update the inventory of goods in the vending machine. It is also impossible to identify and monitor the process of picking up goods after the cabinet is opened, resulting in difficulty in identifying when customers take too much or take the wrong goods. Summary of the Invention

[0004] In response to the above-mentioned shortcomings of the existing technology, the present invention provides a self-service vending machine product identification method based on deep learning, which can effectively solve the problems in the existing technology that self-service vending machines are difficult to intelligently generate and update the product inventory in the vending machine, and are unable to identify and monitor the pickup process after the cabinet is opened.

[0005] To achieve the above objectives, the present invention is implemented through the following technical solutions:

[0006] The present invention provides a method for identifying goods in a self-service vending machine based on deep learning, which comprises at least the following steps:

[0007] Step 1: Split the vending machine shelves into multiple independent display areas. Based on the weight difference before and after replenishment, compare and analyze the product in the warehouse product database to determine multiple pending products. Compare the images of the changed areas before and after replenishment with the image data of the pending products to obtain multiple pending similarities. Analyze the pending similarities to obtain pending confidence values. Based on the pending confidence values, determine the supplementary products and add the supplementary products to the original product set to obtain multiple independent full warehouse sets.

[0008] Step 2: After the vending machine is replenished, it is recorded as being in a full-stock state. Each sales process after the full-stock state is recorded. In step 4, the inventory set after the sales process is completed is obtained. Based on the inventory set, the product inventory is updated after the sales process is completed. Products that do not exist in the product inventory are marked as unavailable.

[0009] Step 3: When the cabinet door is opened, obtain the target product in the product order, mark the target placement area corresponding to the target product, light up the corresponding prompt light strip of the target placement area, analyze the weight change of the target placement area, mark the target placement area as picked up, or generate an abnormal signal or abnormal behavior signal;

[0010] When all target display areas are marked as completed and the cabinet doors are closed, it is recorded as a complete sales process;

[0011] Step 4: Based on the target product set and the independent inventory set corresponding to each independent display area before the cabinet door is opened, regenerate the independent inventory set of each independent display area;

[0012] Step 5: After the vending machine is marked as full, count the number of sales processes, the time corresponding to the after-sales process, and the number of times the user scans the code, and analyze them in combination with the target product set and product inventory until the analysis generates a replenishment signal.

[0013] Furthermore, the process of determining pending products is as follows:

[0014] The independent placement areas are recorded as Part q , q = 1, 2, 3, ..., p, where p is the number of independent placement areas;

[0015] The analyzed independent placement area is recorded as the target area, a stable time threshold is set, and the value change of the weight detection device is recorded in real time. When the value of the weight detection device changes from jumping to stable and the stable time exceeds the stable time threshold, the image captured by the corresponding image acquisition device is recorded as a stable image. Where i is the record number of the stable image, Indicates the image within the target area captured by the image acquisition device before replenishing the product, and stabilizes the image The total weight of the goods in the corresponding target area is recorded as the stable weight

[0016] The most recently recorded stable image and stable weight are recorded as when When, through the formula Calculate the current change value There is a preset warehouse commodity database, which contains image data and weight data of multiple known commodities. Equivalent known commodities are recorded as pending commodities.

[0017] Furthermore, the process of determining the supplementary goods is as follows:

[0018] Acquire stable images With stable image The changing area between images Extracting the changed area image The similarity is calculated by comparing the image data of the pending product one by one, and multiple pending similarities are obtained. The pending confidence value is calculated by the pending confidence value calculation formula. The pending confidence value calculation formula is:

[0019]

[0020] represents the pending confidence value;

[0021] Xs max Indicates the maximum value among multiple pending similarities;

[0022] Represents the average value of multiple pending similarities;

[0023] Xs′ max Indicates the second largest value among multiple pending similarities;

[0024] When the pending confidence value is greater than or equal to the preset confidence threshold, select Xs max The corresponding pending product is used as the current supplementary product When the pending confidence value is less than the preset confidence threshold, an unrecognizable signal is generated, and the staff will actively enter the current supplementary product.

[0025] Furthermore, the abnormal signal generation process is as follows:

[0026] Obtain one or more target products corresponding to each target placement area to form a target product set, obtain the weight of each target product in the target product set to form a weight comparison set, calculate the difference between two consecutive weight checks in the target placement area and compare it with the elements in the weight comparison set;

[0027] When any element in the weight comparison set is equal to the difference, the element is removed from the weight comparison set and the next comparison is performed when the weight is detected to change, until the elements in the weight comparison set are cleared, the target placement area is marked as completed, and the light strip corresponding to the target placement area is turned off;

[0028] When there is no corresponding element in the difference between two adjacent weight checks during any comparison process, the target placement area is recorded as an abnormal placement area, and an abnormal signal is generated and sent to the staff's handheld terminal.

[0029] Furthermore, the abnormal behavior signal generation process is as follows:

[0030] Independent placement areas outside the target placement area are recorded as other placement areas. When the detected weight in other placement areas changes, an abnormal behavior signal is generated and sent to the staff's handheld terminal to control the self-service vending machine to issue an alarm;

[0031] Calculate the difference in total weight of goods in a vending machine The calculation formula is in Indicates the total weight of the goods before the vending machine door is opened. Indicates the total weight of the goods in the vending machine cabinet. Greater than or equal to When , an abnormal behavior signal is generated and sent to the staff's handheld terminal to control the self-service vending machine to issue an alarm, where k is the preset proportional coefficient. The total weight of the goods in the order.

[0032] Furthermore, the update process of the inventory collection and independent inventory collection is as follows:

[0033] The independent placement area of ​​the non-target placement area is recorded as other placement areas. n No abnormal signal or abnormal behavior signal is generated within the time interval, where n is the serial number of the sales process, and the independent inventory set corresponding to any independent placement area is Inventory now If the independent placement area is the target placement area, then Inventory now =Inventory′ n-1 -U D ,in:

[0034] Inventory n-1 This is the independent inventory set corresponding to the target placement area before the cabinet door is opened;

[0035] U D The target product set corresponding to the target placement area;

[0036] If the independent placement area is other placement areas, then Inventory now =Inventory′ n-1 ;

[0037] Get the independent inventory sets in all independent placement areas and find the union to get the sales process of the vending machine. n Inventory collection after closing.

[0038] Furthermore, the replenishment signal generation process is as follows:

[0039] Let the most recent sales process be Sale m , calculate Salem Sale m-1 The time difference between them is recorded as the first time parameter β time1 ;

[0040] Sale m Sale m-1 The time interval between the two is recorded as the analysis interval. The number of times the user scans the code within the analysis interval is obtained and divided by the duration of the analysis interval to obtain the first frequency parameter β 频次 ;

[0041] Get the sales processSale m The time difference from the current time is recorded as the second time parameter β time2 ,The larger the second time parameter is, the worse the sales of the vending machine is;

[0042] The time interval between the last replenishment time and the current time is recorded as the sales interval, and the number of times users scan the code within the sales interval is counted (NUM) 扫码 , through the formula Calculate the purchase ratio value β 购买比例 ;

[0043] Obtain the target product corresponding to each sales process and combine it with the inventory analysis of the self-service vending machine to calculate the hot-selling impact value β 热销 ;

[0044] By formula Calculate the replenishment judgment value α′, where μ1, μ2, and μ3 are preset weight coefficients, and the formula Calculate the remaining inventory value α″, where KC m Indicates the current inventory quantity of goods in the vending machine. KC0 indicates the inventory quantity of goods in the vending machine when the vending machine is full.

[0045] The replenishment judgment value and the remaining inventory value are normalized and compared with the preset inventory replenishment threshold value respectively. When any value is greater than or equal to the preset inventory replenishment threshold value, a replenishment signal is generated.

[0046] Furthermore, the hot-selling impact value calculation process is as follows:

[0047] Get multiple target products corresponding to each sales process and form a target product database. Count the x products that appear most frequently in the target product database and record them as hot-selling products. y , get the corresponding frequency PD y , where y = 1, 2, 3, ..., x, get the hot-selling goods Goods in the full warehouse state y Corresponding stock PC y , through the formula Calculate the hot selling impact value β 热销 .

[0048] A computer device includes a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the above method when executing the computer program.

[0049] A computer-readable storage medium stores a computer program, which implements the steps of the above method when executed by a processor.

[0050] Compared with the prior art, the technical solution provided by the present invention has the following beneficial effects:

[0051] 1. Each independent display area of ​​the present invention is equipped with an independent weight detection and image acquisition device, which can monitor the quantity and status of goods in each area in real time, ensuring accurate detection of changes in goods. Whether goods are taken away or replenished, they can be reflected immediately. When replenishing goods, the system can automatically identify the replenished goods and compare the new goods with the original goods on the shelf to confirm the type of goods replenished in each independent display area. This automated identification reduces manual intervention and improves the efficiency and accuracy of the replenishment process.

[0052] 2. The present invention continuously records the weight difference between two adjacent detections in different target placement areas and compares it with the target goods, thereby determining whether the user has taken the correct and paid goods. When the user takes a product that does not belong to the goods he has paid for, an abnormal signal is generated, and information reflecting the user's wrong taking is reported in a timely manner to facilitate remote analysis and monitoring by the staff. By analyzing and calculating and updating the total inventory set and the independent inventory set corresponding to each independent placement area, the remaining goods in the self-service vending machine can be counted in a timely manner, thereby facilitating the user to select the goods available in the self-service vending machine, thereby avoiding the user choosing sold-out goods in the self-service vending machine and affecting the shopping experience.

[0053] 3. The replenishment judgment value of the present invention is calculated based on multiple parameters (such as the first time parameter, the first frequency parameter, the purchase ratio value, the hot-selling influence value, etc.). The replenishment judgment value obtained by integrating user purchase data and product inventory data can help the system determine the actual impact of inventory reduction on sales. When the judgment value reaches the threshold, replenishment can be triggered. This method can avoid frequent replenishment while ensuring that the self-service vending machine replenishes at the most appropriate time, avoiding the impact of low inventory on sales.

[0054] 4. The automatic vending method adopted by the present invention first determines the user's order and then monitors the user's purchasing behavior, that is, a shopping method of paying first and then picking up the goods. Compared with the self-service vending machine that picks up the goods first and then checks out, it can timely discover the situation where the user takes the wrong goods or takes more than the goods when picking up the goods, reduce transaction disputes, and avoid the situation where the user does not pay after purchase. It is also convenient for inventory management. Ideally, it can update the inventory of goods in the self-service vending machine without using multi-environment recognition technology. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] To more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. Those skilled in the art can also derive other drawings based on these drawings without inventive effort.

[0056] Figure 1 It is the overall module block diagram of the present invention. DETAILED DESCRIPTION

[0057] To make the purpose, technical solutions, and advantages of the embodiments of the present invention more clear, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making any creative efforts shall fall within the scope of protection of the present invention.

[0058] The present invention will be further described below with reference to the embodiments.

[0059] See Figure 1 A method for identifying products in a self-service vending machine based on deep learning includes at least the following steps:

[0060] Step 1: Split each shelf of the vending machine into p independent display areas Part q , q = 1, 2, 3, ..., p, each independent placement area is equipped with an independent weight detection device and an image acquisition device, and the weight detection device and image acquisition device corresponding to the same independent placement area are bound to each other, respectively used to detect the real-time total weight and real-time image of the goods in the independent placement area. It should be noted that the goods in each independent placement area do not touch each other, and the real-time total weight is not affected by other independent placement areas. According to the inventory quantity in the self-service vending machine, the goods in the self-service vending machine are replenished regularly to ensure the diversity of the goods in the self-service vending machine;

[0061] During the replenishment process, the replenished products are identified and combined with the original products on the shelf to generate independent full warehouse sets in different independent placement areas. Each independent placement area is independently identified and analyzed, including:

[0062] The analyzed independent placement area is recorded as the target area, a stable time threshold is set, and the value changes of the weight detection device are recorded in real time. When the value of the weight detection device changes from jumping to stable and the stable time exceeds the stable time threshold (this means that the goods in the target area are in a stable state), the image captured by the corresponding image acquisition device is obtained and recorded as a stable image Where i is the record number of the stable image, and the record number of the stable image increases with time. Indicates the image within the target area captured by the image acquisition device before replenishing the product, and stabilizes the image The total weight of the goods in the corresponding target area is recorded as the stable weight

[0063] Furthermore, based on the stable image and stable weight Identify and analyze the corresponding supplementary products for each weight change:

[0064] The most recently recorded stable image and stable weight are recorded as when When, through the formula Calculate the current change value There is a warehouse commodity database that is preset. The warehouse commodity database contains image data and weight data of multiple known commodities (commodities that have been pre-registered). It should be noted that image data refers to character data that can be directly used in image matching algorithms to identify commodities, rather than the image of the commodity itself. The weight data is compared with the current change value. Equivalent known commodities are recorded as pending commodities;

[0065] Acquire stable images With stable image The changing area between images (The difference between the two images is usually the product added to the shelf. By comparing the images taken at different time points through the image difference method, the difference between the images can be calculated) and the image of the changed area can be extracted. The similarity is calculated by comparing the image data of the pending products one by one (the greater the similarity, the more similar the newly added product is to the pending product), and multiple pending similarities are obtained. The pending confidence value is calculated using the pending confidence value calculation formula. The pending confidence value calculation formula is:

[0066]

[0067] represents the pending confidence value;

[0068] Xs max Indicates the maximum value among multiple pending similarities;

[0069] Represents the average value of multiple pending similarities;

[0070] Xs′ max Indicates the second largest value among multiple pending similarities;

[0071] When the undetermined confidence value is greater than or equal to the preset confidence threshold (in a specific embodiment, the value is 2), select Xs max The corresponding pending product is used as the current supplementary product When the pending confidence value is less than the preset confidence threshold, an unrecognizable signal is generated, and the staff will actively enter the current supplementary product.

[0072] The pending confidence value reflects whether the pending product corresponding to the maximum value in the pending similarity (referred to as the maximum product) can be used as the product corresponding to the changing area image (referred to as the regional product). The greater the similarity between the maximum product and the regional product and the smaller the similarity with other pending products (in this case, the recognition result is more reliable because there are no other similar results and no recognition interference), the larger the pending confidence value, which means that the regional product is more likely to be identified as the maximum product, thereby improving the accuracy of the confirmation of the type of supplementary products.

[0073] Recognize images through image recognition algorithms All the products in the set form the original product set, and all the supplementary products Add to the original product collection to obtain an independent full warehouse collection in an independent display area.

[0074] It's important to note that before restocking a vending machine, staff must neatly arrange the items on the shelves to facilitate identification by the image recognition algorithm. Restocking typically involves fully stocking the shelves to facilitate user selection. By constructing a fully stocked item set, we can determine the inventory level in the vending machine after restocking. This inventory level serves as the initial inventory level, and user orders can be verified based on the inventory changes with each purchase.

[0075] Step 2: Record the state of the vending machine after each replenishment as the full warehouse state, and record each sales process after the vending machine is full warehouse state as Sale n , through step three, the sales process n Guide, monitor and record the sales process through step 4n Analyze and obtain the after-sales process n The remaining inventory items after the end are recorded as the inventory set Inventory n , Inventory n is p independent inventory sets The union of (the independent inventory set is the set of remaining inventory items in the independent display area), where n is the order of the sales process. For example, Sale1 represents the first sales process after the warehouse is full. It represents the remaining inventory set in the independent placement area with sequence number 1 after the first sales process ends. Inventory0 represents the full warehouse product set corresponding to the full warehouse state. Inventory0 is p independent full warehouse sets. The union of, for example, Indicates the independent full warehouse collection corresponding to the independent placement area with serial number 1. n After the end based on the inventory collection Inventory n Update the product inventory and mark the products that do not exist in the product inventory as unavailable.

[0076] It should be noted that the elements in any set in the present invention are not mutually different, that is, there can be multiple identical elements in the same set, and the number of times the same element appears depends on the quantity represented by the same product in the set. For example, if the number of product R in an independent display area is three, it can appear three times in the independent inventory set.

[0077] Step 3: When the cabinet door is opened, obtain the product order (selected and added by the purchasing user), record the products in the product order as target products, and obtain the independent inventory set Inventory to which each target product belongs n q The independent placement area corresponding to the independent inventory set is recorded as the target placement area. A warning light strip is set at the periphery of each independent placement area to control the warning light strip corresponding to the target placement area to light up, and each target placement area is continuously analyzed;

[0078] It should be noted that the user needs to complete the following operations before opening the cabinet door: scan the QR code on the self-service vending machine through a handheld terminal to obtain the product inventory in the self-service vending machine, select the product in the inventory and check out.

[0079] By lighting up the prompt light strip, on the one hand, the purchasing user can quickly notice the target display area, and quickly determine in which independent display area the product he wants to buy is, so that the purchasing user can quickly select the product he wants to buy. On the other hand, after the prompt light strip is lit, it can illuminate the target display area, making it easier for the purchasing user to pick up the product in a dark environment, further improving the user's purchasing experience.

[0080] Obtain one or more target commodities corresponding to each target placement area to form a target commodity set, obtain the weight of each target commodity in the target commodity set to form a weight comparison set, calculate the difference between two adjacent detection weights in the target placement area (referring to the weight displayed after the weight detection device in the target placement area detects a stable value) and compare it with the elements in the weight comparison set. When any element in the weight comparison set is equal to the difference, the element is removed from the weight comparison set and the next comparison is performed when the detection weight changes. Until the elements in the weight comparison set are cleared, the target placement area is marked as completed for pickup, and the light strip corresponding to the target placement area is turned off. When there is no corresponding element in the difference between two adjacent detection weights during any comparison process, the target placement area is recorded as an abnormal placement area, and an abnormal signal is generated and sent to the staff's handheld terminal;

[0081] When all target display areas are marked as completed and the cabinet doors are closed, it is recorded as a complete sales process.

[0082] It should be noted that by continuously recording the weight difference between two adjacent detections in different target placement areas and comparing it with the target goods, it is determined whether the user has taken the correct and paid goods. When the goods taken by the user are goods for which he has already paid, no abnormal signal will be generated. However, when the user takes goods that do not belong to the goods he has paid for, the weight value that matches the weight difference between the two adjacent detections cannot be found in the weight comparison set, thereby generating an abnormal signal and reporting the information reflecting the user's wrong taking in time for the staff to conduct remote analysis and monitoring, thereby overcoming the defect of the existing technology that it is difficult to detect the user's wrong taking behavior.

[0083] Furthermore, the independent placement area that is not the target placement area is recorded as other placement areas. When the detected weight in other placement areas changes, an abnormal behavior signal is generated and sent to the staff's handheld terminal to control the self-service vending machine to issue an alarm; the total weight difference of the goods in the self-service vending machine is calculated. The calculation formula is in Indicates the total weight of the goods before the vending machine door is opened. Indicates the total weight of the goods in the vending machine cabinet. Greater than or equal to When an abnormal behavior signal is generated and sent to the staff's handheld terminal and the self-service vending machine is controlled to issue an alarm, where k is a preset proportional coefficient (in a specific embodiment, the value is 1.2). The total weight of the goods in the order.

[0084] It should be noted that by detecting the weight in other independent display areas, users can be prevented from mistakenly taking goods in other independent display areas. By detecting the changes in the total weight of goods in the self-service vending machine, it can be discovered in time that users take more goods when purchasing, and an alarm can be issued to stop users from taking more.

[0085] Step 4: When the complete selling process is completed n No abnormal signal or abnormal behavior signal is generated within the time interval. Let the independent inventory set corresponding to any independent placement area be Inventory now If the independent placement area is the target placement area, then Inventory now =Inventory′ n-1 -U D ,in:

[0086] Inventory n-1 This is the independent inventory set corresponding to the target placement area before the cabinet door is opened;

[0087] U D is the target product set corresponding to the target placement area (assuming that the independent inventory set before the cabinet door is opened is A, and the target product set is B, then the current independent inventory set is the remaining set after removing the elements in set B from set A);

[0088] If the independent placement area is other placement areas, then Inventory now =Inventory′ n-1 ;

[0089] Get the independent inventory sets in all independent placement areas and find the union to get the sales process of the vending machine. n Inventory collection after closing.

[0090] By analyzing, calculating and updating the total inventory set and the independent inventory set corresponding to each independent placement area, the remaining goods in the self-service vending machine can be counted in a timely manner, which makes it convenient for users to select goods available in the self-service vending machine, thereby avoiding users choosing sold-out goods in the self-service vending machine and affecting the shopping experience. Furthermore, by clarifying the independent inventory set corresponding to each independent placement area, it can facilitate guidance and instructions in step three.

[0091] Step 5: After the vending machine is marked as full, it starts recording and analyzing the user's purchase data until the analysis generates a replenishment signal that is sent to the staff's handheld terminal. The staff then replenishes the goods in the vending machine to ensure that the goods in the vending machine are sufficient.

[0092] Let the most recent sales process be Sale m , calculate Sale m Sale m-1 The time difference between them is recorded as the first time parameter β time1 ,The first time parameter reflects the time difference between the two most recent ,selling processes. The larger the first time parameter is, the worse the sales of the ,self-service vending machine;

[0093] Sale m Sale m-1 The time interval between the two is recorded as the analysis interval. The number of times the user scans the code within the analysis interval is obtained and divided by the duration of the analysis interval to obtain the first frequency parameter β 频次 The first time parameter reflects the degree of attraction of the products in the self-service vending machine to users (the self-service vending machine uses a transparent cabinet door. When users see the products they like through the cabinet door, they will try to scan the code to buy them). The larger the first time parameter, the more attractive the products in the self-service vending machine are to users.

[0094] Get the sales processSale m The time difference from the current time is recorded as the second time parameter β time2 ,The larger the second time parameter is, the worse the sales of the vending machine is;

[0095] The time interval between the last replenishment time and the current time is recorded as the sales interval, and the number of times users scan the code within the sales interval is counted (NUM) 扫码 , through the formula Calculate the purchase ratio value β 购买比例 ,The purchase ratio value reflects the matching degree between the ,product inventory in the vending machine and the user demand. The larger the ,purchase ratio value is, the more the product in the vending machine matches ,the user demand.

[0096] Get multiple target products corresponding to each sales process and form a target product database. Count the x products that appear most frequently in the target product database and record them as hot-selling products. y , get the corresponding frequency PD y , where y = 1, 2, 3, ..., x, get the hot-selling goods Goods in the full warehouse state y Corresponding stock PC y , through the formula Calculate the hot selling impact value β 热销The hot-selling impact value reflects the inventory remaining of hot-selling products in the self-service vending machine. The larger the hot-selling impact value, the less the inventory remaining of hot-selling products in the self-service vending machine.

[0097] By formula Calculate the replenishment judgment value α′, where μ1, μ2, and μ3 are preset weight coefficients, and the formula Calculate the remaining inventory value α″, where KC m Indicates the current inventory quantity of goods in the vending machine. KC0 indicates the inventory quantity of goods in the vending machine when the vending machine is full.

[0098] The replenishment judgment value and the remaining inventory value are normalized and compared with the preset inventory replenishment threshold value respectively. When any value is greater than or equal to the preset inventory replenishment threshold value, a replenishment signal is generated.

[0099] It should be noted that the replenishment judgment value depends on user purchase data such as the first time parameter, the first frequency parameter, the second time parameter, the purchase ratio value, and the product inventory data of the hot-selling influence value. Changes in user purchase data directly reflect the sales situation of the vending machine, and the product inventory data will have an indirect impact on the sales of the vending machine. Therefore, the replenishment judgment value reflects the degree of impact of the reduction in product inventory in the vending machine on the sales of the vending machine. The larger the replenishment judgment value, the greater the impact of the inventory reduction on the sales. When it exceeds a certain threshold, the vending machine needs to be replenished to overcome the impact of the reduction in product inventory on sales.

[0100] The present invention also provides a computer device, comprising a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the above method when executing the computer program.

[0101] A computer-readable storage medium stores a computer program, which implements the steps of the above method when executed by a processor.

[0102] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements will not cause the essence of the corresponding technical solutions to deviate from the protection scope of the technical solutions of the various embodiments of the present invention.

Claims

1. A method for identifying products in self-service vending machines based on deep learning, comprising the following steps: Step 1: Split the vending machine shelf into multiple independent display areas, determine multiple pending products based on the weight of the products before and after replenishment, and compare the images of the changed areas before and after replenishment with the image data of the pending products to obtain multiple pending similarities, characterized in that: Analyze the pending similarity to obtain a pending confidence value, determine the supplementary products based on the pending confidence value, and add the supplementary products to the original product set to obtain multiple independent full warehouse sets; Step 2: After the vending machine is replenished, the vending machine is marked as fully stocked and the sales process is recorded. In step 4, the inventory set after the sales process is completed is obtained. Based on the inventory set, the product inventory is updated after the sales process is completed, and products that do not exist in the product inventory are marked as unavailable. Step 3: When the door is opened, obtain the target product in the product order, mark the target placement area corresponding to the target product, analyze the weight change of the target placement area, mark the target placement area as completed, or generate an abnormal signal or abnormal behavior signal; When all target display areas have been cleared and the cabinet doors are closed, it is considered a complete sales process; Step 4: Based on the target product set and the independent inventory set corresponding to each independent display area before the cabinet door is opened, regenerate the independent inventory set of each independent display area; Step 5: When the vending machine is fully stocked, the number of sales processes, the time corresponding to the after-sales process, and the number of times users scan the QR code are counted. This is combined with the target product set and product inventory for analysis until a replenishment signal is generated. The replenishment signal generation process is as follows: Assume that the most recent sales process is ,calculate and The time difference between them is recorded as the first time parameter ; Will and The time interval between the two is recorded as the analysis interval. The number of times the user scans the code within the analysis interval is obtained and divided by the duration of the analysis interval to obtain the first frequency parameter. ; Get the sales process The time difference from the current time is recorded as the second time parameter ; The time interval between the last replenishment time and the current time is recorded as the sales interval, and the number of user scans within the sales interval is counted. , through the formula Calculate the purchase ratio ; Obtain the target product corresponding to each sales process and calculate the hot-selling impact value based on the product inventory analysis in the self-service vending machine ; By formula Calculate the replenishment judgment value ,in is the preset weight coefficient, through the formula Calculate the remaining inventory value ,in Indicates the current inventory quantity of goods in the vending machine. Indicates the inventory quantity of goods in the vending machine when the warehouse is full; The replenishment judgment value and the remaining inventory value are normalized and compared with the preset inventory replenishment threshold value respectively. When any value is greater than or equal to the preset inventory replenishment threshold value, a replenishment signal is generated.

2. The method for identifying goods in a self-service vending machine based on deep learning according to claim 1, characterized in that: The specific process for determining pending products is as follows: The independent placement areas are respectively recorded as , q=1,2,3,…,p, where p is the number of independent placement areas; The analyzed independent placement area is recorded as the target area, and the value change of the weight detection device is recorded in real time. When the value of the weight detection device changes from jumping to stable and the stable time exceeds the preset stable time threshold, the image captured by the corresponding image acquisition device is recorded as a stable image. , where i is the record number of the stable image, Indicates the image within the target area captured by the image acquisition device before replenishing the product, and stabilizes the image The total weight of goods in the corresponding target area is recorded as stable weight ; The most recently recorded stable image and stable weight are recorded as ,when When, through the formula Calculate the current change value , a warehouse commodity database is preset, which contains image data and weight data of multiple known commodities, and compares the weight data with the current change value Equivalent known commodities are recorded as pending commodities.

3. The method for identifying goods in a self-service vending machine based on deep learning according to claim 2, characterized in that: The process of determining supplementary products is as follows: Acquire stable images With stable image The changing area image between , extract the changed area image The similarity is calculated by comparing the image data of the pending product one by one, and multiple pending similarities are obtained. The pending confidence value is calculated by the pending confidence value calculation formula. The pending confidence value calculation formula is: ,in: represents the pending confidence value; Indicates the maximum value among multiple pending similarities; Represents the average value of multiple pending similarities; Indicates the second largest value among multiple pending similarities; When the pending confidence value is greater than or equal to the preset confidence threshold, select The corresponding pending product is used as the current supplementary product When the pending confidence value is less than the preset confidence threshold, an unrecognizable signal is generated, and the staff actively enters the current supplementary product. .

4. The method for identifying goods in a self-service vending machine based on deep learning according to claim 1, characterized in that: The abnormal signal generation process is as follows: Obtain one or more target products corresponding to each target placement area to form a target product set, obtain the weight of each target product in the target product set to form a weight comparison set, calculate the difference between two consecutive weight checks in the target placement area and compare it with the elements in the weight comparison set; When any element in the weight comparison set is equal to the difference, the element is removed from the weight comparison set and the next comparison is performed when the weight is detected to change, until the elements in the weight comparison set are cleared, the target placement area is marked as completed, and the light strip corresponding to the target placement area is turned off; When there is no corresponding element in the difference between two adjacent weight checks during any comparison process, the target placement area is recorded as an abnormal placement area, and an abnormal signal is generated and sent to the staff's handheld terminal.

5. The method for identifying goods in a self-service vending machine based on deep learning according to claim 4, characterized in that: The abnormal behavior signal generation process is as follows: Independent placement areas outside the target placement area are recorded as other placement areas. When the detected weight in other placement areas changes, an abnormal behavior signal is generated and sent to the staff's handheld terminal to control the self-service vending machine to issue an alarm; Calculate the difference in total weight of goods in a vending machine , the calculation formula is ,in Indicates the total weight of the goods before the vending machine door is opened. Indicates the total weight of the goods in the vending machine cabinet. Greater than or equal to When an abnormal behavior signal is generated, it is sent to the staff's handheld terminal and controls the self-service vending machine to issue an alarm. is the preset scale factor, The total weight of the goods in the order.

6. The method for identifying goods in a self-service vending machine based on deep learning according to claim 1, characterized in that: The update process for inventory collections and independent inventory collections is as follows: The independent placement area of ​​non-target placement area is recorded as other placement area. No abnormal signal or abnormal behavior signal is generated within the time interval, where n is the serial number of the sales process. Let the independent inventory set corresponding to any independent placement area be , if the independent placement area is the target placement area, then ,in: This is the independent inventory set corresponding to the target placement area before the cabinet door is opened; The target product set corresponding to the target placement area; If the independent placement area is other placement areas, then ; Get the independent inventory sets in all independent placement areas and find the union to get the self-service vending machine in the vending process Inventory collection after closing.

7. The method for identifying goods in a self-service vending machine based on deep learning according to claim 1, characterized in that: The hot selling impact value calculation process is as follows: Get multiple target products corresponding to each sales process and form a target product database. Count the x products that appear most frequently in the target product database and record them as hot-selling products. , get the corresponding frequency , where y=1,2,3,…,x, get the hot-selling products in the full warehouse state Corresponding inventory , through the formula Calculate the hot selling impact value .

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images