Self-service vending machine commodity identification method based on deep learning

Abstract

The invention relates to the technical field of control management systems, in particular to a self-service vending machine commodity identification method based on deep learning, which comprises the following steps of: splitting a shelf of a self-service vending machine into a plurality of independent placement areas, comparing and analyzing to determine a plurality of to-be-determined commodities on the basis of the weight difference before and after commodity replenishment in combination with a warehouse commodity database, and comparing the change area images before and after commodity replenishment with the image data of the to-be-determined commodities to obtain a plurality of to-be-determined similarities, analyzing the to-be-determined similarities to obtain to-be-determined confidence values, determining replenishment commodities based on the to-be-determined confidence values, and replenishing the replenishment commodities to the original commodity set to obtain a plurality of independent full-warehouse sets. According to the invention, the commodity inventory in the self-service vending machine after replenishment can be determined by constructing the full-warehouse commodity set, so that the commodity inventory is taken as the original inventory, and the order of the commodities purchased by the user is checked according to the change of the inventory purchased by the user every time.

Description

Technical Field

[0001] The present invention relates to the technical field of control management systems, and in particular to a method for identifying commodities in self-service vending machines based on deep learning. Background Art

[0002] A vending machine is a device that provides commodity purchasing services through automated technology and is widely used in public places such as shopping malls, stations, airports, offices, etc. Users can select the desired commodities through a touch screen or other interactive methods, and the system automatically completes the commodity display, payment and delivery process. In order to improve convenience and safety, modern vending machines support QR code payment, credit card payment, face recognition, commodity recognition, inventory management, etc., to ensure that users can complete shopping quickly, conveniently and safely.

[0003] Self-service vending machines include closed vending machines and open cabinet vending machines. When using the open cabinet vending machines, customers can open the cabinet door to pick up the goods, which is highly selective and very convenient and fast. However, the existing open cabinet vending machines generally allow customers to freely select goods, and it is difficult to intelligently generate and update the inventory of goods in the vending machine. It is impossible to identify and monitor the process of picking up goods after opening the cabinet, which makes it difficult to identify when customers take too much or take the wrong goods. Summary of the invention

[0004] In view of the above-mentioned shortcomings of the prior art, the present invention provides a self-service vending machine product identification method based on deep learning, which can effectively solve the problems in the prior art that self-service vending machines are difficult to intelligently generate and update the product inventory in the vending machines, and are unable to identify and monitor the pickup process after opening the cabinet.

[0005] To achieve the above objectives, the present invention is implemented through the following technical solutions:

[0006] The present invention provides a method for identifying commodities in a self-service vending machine based on deep learning, which comprises at least the following steps:

[0007] Step 1: Split the vending machine shelves into multiple independent placement areas, determine multiple pending commodities based on the weight difference before and after commodity replenishment and the warehouse commodity database, compare and analyze the changed area images before and after commodity replenishment with the image data of the pending commodities to obtain multiple pending similarities, analyze the pending similarities to obtain pending confidence values, determine the supplementary commodities based on the pending confidence values, add the supplementary commodities to the original commodity set, and obtain multiple independent full warehouse sets;

[0008] Step 2: After the vending machine is replenished with goods, it is recorded as a full-warehouse state, and each sales process after the full-warehouse state is recorded. Through step 4, the inventory set after the after-sales process is completed is obtained. Based on the inventory set, the product inventory is updated after the sales process is completed, and the products that do not exist in the product inventory are marked as unavailable;

[0009] Step 3: When the cabinet door is opened, obtain the target product in the product order, mark the target placement area corresponding to the target product, light up the prompt light strip corresponding to the target placement area, analyze the weight change of the target placement area, mark the target placement area as completed, or generate an abnormal signal or abnormal behavior signal;

[0010] When all target display areas are marked as completed and the cabinet doors are closed, it is recorded as a complete sales process;

[0011] Step 4: Based on the target product set and the independent inventory set corresponding to each independent placement area before the cabinet door is opened, regenerate the independent inventory set of each independent placement area;

[0012] Step 5: After the vending machine is marked as full, count the number of sales processes, the time corresponding to the after-sales process, and the number of times the user scans the code, and analyze them in combination with the target product set and product inventory until the analysis generates a replenishment signal.

[0013] Furthermore, the process of determining pending products is as follows:

[0014] The independent placement areas are recorded as Part q , q = 1, 2, 3, ..., p, p is the number of independent placement areas;

[0015] The analyzed independent placement area is recorded as the target area, a stable time threshold is set, and the value change of the weight detection device is recorded in real time. When the value of the weight detection device changes from jumping to stable and the stable time exceeds the stable time threshold, the image captured by the corresponding image acquisition device is obtained and recorded as a stable image. Where i is the record number of the stable image, Indicates the image in the target area captured by the image acquisition device before replenishing the product, and stabilizes the image The total weight of the goods in the corresponding target area is recorded as the stable weight

[0016] The most recently recorded stable image and stable weight are recorded as when When, through the formula Calculate the current change value A warehouse commodity database is preset, which contains image data and weight data of multiple known commodities. Equivalent known commodities are recorded as pending commodities.

[0017] Furthermore, the process of determining the supplementary goods is as follows:

[0018] Get a stable image With stable image The changing area image between Extract the changed area image The similarity is calculated by comparing the image data of the pending product one by one, and multiple pending similarities are obtained. The pending confidence value is calculated by the pending confidence value calculation formula. The pending confidence value calculation formula is:

[0019]

[0020] represents the pending confidence value;

[0021] Xs max Indicates the maximum value among multiple pending similarities;

[0022] Represents the average value of multiple pending similarities;

[0023] Xs′ max Indicates the second largest value among multiple pending similarities;

[0024] When the pending confidence value is greater than or equal to the preset confidence threshold, select Xs max The corresponding pending product is used as the current supplementary product When the pending confidence value is less than the preset confidence threshold, an unrecognizable signal is generated, and the staff actively enters the current supplementary product.

[0025] Furthermore, the abnormal signal generation process is as follows:

[0026] Obtain one or more target products corresponding to each target placement area to form a target product set, obtain the weight of each target product in the target product set to form a weight comparison set, calculate the difference between two adjacent weights detected in the target placement area and compare it with the elements in the weight comparison set;

[0027] When any element in the weight comparison set is equal to the difference, the element is removed from the weight comparison set and the next comparison is performed when the detected weight changes, until the elements in the weight comparison set are cleared, the target placement area is marked as completed, and the light strip corresponding to the target placement area is turned off;

[0028] When there is no corresponding element in the difference between two adjacent detected weights during any comparison process, the target placement area is recorded as an abnormal placement area, and an abnormal signal is generated and sent to the staff's handheld terminal.

[0029] Furthermore, the abnormal behavior signal generation process is as follows:

[0030] The independent placement area that is not the target placement area is recorded as other placement areas. When the detected weight in other placement areas changes, an abnormal behavior signal is generated and sent to the handheld terminal of the staff to control the self-service vending machine to sound an alarm;

[0031] Calculate the difference in total weight of goods in the vending machine The calculation formula is in Indicates the total weight of the goods before the vending machine door is opened. Indicates the total weight of the goods in the vending machine. Greater than or equal to When an abnormal behavior signal is generated, it is sent to the handheld terminal of the staff and controls the self-service vending machine to sound an alarm, where k is the preset proportional coefficient. The total weight of the goods in the order.

[0032] Furthermore, the update process of the inventory collection and independent inventory collection is as follows:

[0033] The independent placement area of ​​the non-target placement area is recorded as other placement areas. n No abnormal signal or abnormal behavior signal is generated within the time interval, where n is the serial number of the sales process, and the independent inventory set corresponding to any independent placement area is Inventory now , if the independent placement area is the target placement area, then Inventory now =Inventory′ n-1 -U D ,in:

[0034] Inventory′ n-1 It is the independent inventory set corresponding to the target placement area before the cabinet door is opened;

[0035] U D The target product set corresponding to the target placement area;

[0036] If the independent placement area is other placement areas, then Inventory now =Inventory′ n-1 ;

[0037] Get the independent inventory sets in all independent placement areas and find the union to get the sales process of the self-service vending machine Sale n Stock collection after closing.

[0038] Furthermore, the replenishment signal generation process is as follows:

[0039] Let the most recent sales process be Sale m , calculate Salem With Sale m-1 The time difference between them is recorded as the first time parameter β time1 ;

[0040] Sale m With Sale m-1 The time interval between them is recorded as the analysis interval. The number of times the user scans the code within the analysis interval is obtained and divided by the duration of the analysis interval to obtain the first frequency parameter β 频次 ;

[0041] Get the sales processSale m The time difference from the current time is recorded as the second time parameter β time2 ,The larger the second time parameter is, the worse the sales of the vending machine is;

[0042] The time interval between the last replenishment time and the current time is recorded as the sales interval, and the number of times users scan the code within the sales interval is counted NUM 扫码 , through the formula Calculate the purchase ratio value β 购买比例 ;

[0043] Obtain the target product corresponding to each sales process and combine it with the product inventory analysis in the vending machine to calculate the hot-selling impact value β 热销 ;

[0044] By formula The replenishment judgment value α′ is calculated, where μ1, μ2, and μ3 are preset weight coefficients. Calculate the remaining inventory value α″, where KC m Indicates the current inventory quantity of goods in the vending machine, and KC0 indicates the inventory quantity of goods in the vending machine when the warehouse is full;

[0045] The replenishment judgment value and the remaining inventory value are normalized and compared with the preset inventory replenishment threshold value respectively. When any value is greater than or equal to the preset inventory replenishment threshold value, a replenishment signal is generated.

[0046] Furthermore, the hot-selling impact value calculation process is as follows:

[0047] Get multiple target products corresponding to each sales process and form a target product database. Count the x most frequently appearing products in the target product database and record them as hot-selling products. y , get the corresponding frequency PD y , where y = 1, 2, 3, ..., x, get the hot-selling goods Goods in the full warehouse state y Corresponding stock PC y , through the formula Calculate the hot selling impact value β 热销 .

[0048] A computer device comprises a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the above method when executing the computer program.

[0049] A computer-readable storage medium stores a computer program, which implements the steps of the above method when executed by a processor.

[0050] Compared with the known prior art, the technical solution provided by the present invention has the following beneficial effects:

[0051] 1. Each independent placement area of ​​the present invention is equipped with an independent weight detection and image acquisition device, which can monitor the quantity and status of the goods in each area in real time, ensure accurate detection of changes in goods, and reflect immediately whether the goods are taken away or replenished. When replenishing goods, the system can automatically identify the replenished goods and compare the new goods with the original goods on the shelf to confirm the types of goods replenished in each independent placement area. This automatic identification reduces manual intervention and improves the efficiency and accuracy of the replenishment process.

[0052] 2. The present invention continuously records the difference between two adjacent weight detections in different target placement areas and compares it with the target goods, thereby determining whether the user has taken the correct, paid goods. When the user takes a product that does not belong to the goods he has paid for, an abnormal signal is generated, and information reflecting the user's wrong taking is reported in a timely manner to facilitate remote analysis and monitoring by the staff. By analyzing and calculating and updating the total inventory set and the independent inventory sets corresponding to each independent placement area, the remaining goods in the self-service vending machine can be counted in a timely manner, thereby facilitating the user to select the goods available in the self-service vending machine, thereby avoiding the user's selection of sold-out goods in the self-service vending machine and affecting the shopping experience.

[0053] 3. The replenishment judgment value of the present invention is calculated based on multiple parameters (such as the first time parameter, the first frequency parameter, the purchase ratio value, the hot-selling influence value, etc.). The replenishment judgment value obtained by integrating the user purchase data and the commodity inventory data can help the system determine the actual impact of inventory reduction on sales. When the judgment value reaches the threshold, replenishment can be triggered. This method can avoid frequent replenishment while ensuring that the self-service vending machine replenishes at the most appropriate time to avoid the impact of low inventory on sales.

[0054] 4. The automatic vending method adopted by the present invention first determines the user's order and then monitors the user's purchasing behavior, that is, a shopping method of paying first and then picking up the goods. Compared with the self-service vending machine that picks up the goods first and then checks out, it can promptly discover the situation where the user takes the wrong goods or takes too much when picking up the goods, thereby reducing transaction disputes, avoiding the situation where the user does not pay after purchase, and facilitating inventory management. Ideally, it can update the inventory of goods in the self-service vending machine without using multi-environment recognition technology. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the prior art descriptions are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention, and for ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0056] Figure 1 It is the overall module block diagram of the present invention. DETAILED DESCRIPTION

[0057] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0058] The present invention will be further described below in conjunction with the embodiments.

[0059] See also Figure 1 , a method for identifying goods in a self-service vending machine based on deep learning, comprising at least the following steps:

[0060] Step 1: Split each shelf of the vending machine into p independent display areas Part q , q=1,2,3,…,p, each independent placement area is provided with an independent weight detection device and an image acquisition device, and the weight detection device and the image acquisition device corresponding to the same independent placement area are bound to each other, and are used to detect the real-time total weight and real-time image of the goods in the independent placement area respectively. It should be noted that the goods between each independent placement area do not touch each other, and the real-time total weight is not affected by other independent placement areas. According to the inventory quantity in the self-service vending machine, the goods in the self-service vending machine are replenished regularly to ensure the diversity of the goods in the self-service vending machine;

[0061] In the process of replenishing goods, the replenished goods are identified, and combined with the original goods on the shelf to generate independent full warehouse sets in different independent placement areas, and each independent placement area is independently identified and analyzed, including:

[0062] The analyzed independent placement area is recorded as the target area, a stable time threshold is set, and the value change of the weight detection device is recorded in real time. When the value of the weight detection device changes from jumping to stable and the stable time exceeds the stable time threshold (this means that the goods in the target area are in a stable state), the image collected by the corresponding image acquisition device is obtained and recorded as a stable image Where i is the record number of the stable image, and the record number of the stable image increases with the passage of time. Indicates the image in the target area captured by the image acquisition device before replenishing the product, and stabilizes the image The total weight of the goods in the corresponding target area is recorded as the stable weight

[0063] Furthermore, based on the stable image and stable weight Identify and analyze to determine the complementary products for each weight change:

[0064] The most recently recorded stable image and stable weight are recorded as when When, through the formula Calculate the current change value A warehouse commodity database is preset, which contains image data and weight data of multiple known commodities (commodities with pre-registered data). It should be noted that image data refers to character data that can be directly used in image matching algorithms for commodity matching and recognition, rather than the image of the commodity itself. Equivalent known commodities are recorded as pending commodities;

[0065] Get a stable image With stable image The changing area image between (i.e. the different parts between the two images, usually the goods added to the shelves. By comparing the images taken at different time points through the image difference method, the difference between the images can be calculated) and the changed area image can be extracted The similarity is calculated by comparing the image data of the pending products one by one (the greater the similarity, the more similar the newly added products are to the pending products), and multiple pending similarities are obtained. The pending confidence values ​​are calculated by the pending confidence value calculation formula. The pending confidence value calculation formula is:

[0066]

[0067] represents the pending confidence value;

[0068] Xs max Indicates the maximum value among multiple pending similarities;

[0069] Represents the average value of multiple pending similarities;

[0070] Xs′ max Indicates the second largest value among multiple pending similarities;

[0071] When the pending confidence value is greater than or equal to a preset confidence threshold (in a specific embodiment, the value is 2), select Xs max The corresponding pending product is used as the current supplementary product When the pending confidence value is less than the preset confidence threshold, an unrecognizable signal is generated, and the staff actively enters the current supplementary product.

[0072] The pending confidence value reflects whether the pending product corresponding to the maximum value in the pending similarity (recorded as the maximum product) can be used as the product corresponding to the changing area image (recorded as the regional product). The greater the similarity between the maximum product and the regional product and the smaller the similarity with other pending products (in this case, the recognition result is more reliable because there are no other similar results and no recognition interference), the larger the pending confidence value is, the more likely the regional product is to be identified as the maximum product, thereby improving the accuracy of confirming the type of supplementary products.

[0073] Recognize images through image recognition algorithms All the products in the original product set are combined into a complete set of products. Add to the original commodity collection to obtain an independent full warehouse collection in an independent display area.

[0074] It should be noted that before restocking the vending machine, the staff needs to neatly stack the goods on the shelf so that the image recognition algorithm can identify the goods on the shelf. When restocking, the goods are generally fully replenished to facilitate the user's purchase. By constructing a full warehouse product set, the inventory of goods in the vending machine after the replenishment is completed can be determined, so that the inventory of goods is used as the original inventory, and the user's purchase order is checked according to the inventory changes of each purchase.

[0075] Step 2: Record the state of the vending machine after each replenishment as the full warehouse state, and record each sales process after the vending machine is full warehouse state as Sale. n , through step three, the sales process Sale n Guide, monitor and record the sales process through step 4n Analyze and obtain the after-sales process n The remaining inventory items after the completion are recorded as the inventory collection Inventory n , Inventory n is p independent inventory sets The independent inventory set is the set of remaining inventory items in the independent display area, where n is the order of the sales process. For example, Sale1 represents the first sales process after the warehouse is full. It represents the remaining inventory set in the independent placement area with serial number 1 after the first sales process ends. Inventory0 represents the full warehouse goods set corresponding to the full warehouse state. Inventory0 is p independent full warehouse sets. The union of, for example, Indicates the independent full warehouse collection corresponding to the independent placement area with serial number 1. n After the end based on the inventory collection Inventory n Update the product inventory and mark the products that do not exist in the product inventory as unavailable for selection.

[0076] It should be noted that the elements in any set in the present invention are not mutually different, that is, there can be multiple identical elements in the same set, and the number of times the same element appears depends on the quantity of the same product represented in the set. For example, if the number of product R in an independent display area is three, it can appear three times in the independent inventory set.

[0077] Step 3: When the cabinet door is opened, obtain the product order (selected and added by the purchasing user), record the products in the product order as target products, and obtain the independent inventory set Inventory to which each target product belongs n q The independent placement area corresponding to the independent inventory set is recorded as the target placement area, and a prompt light belt is set at the periphery of each independent placement area to control the prompt light belt corresponding to the target placement area to light up, and each target placement area is continuously analyzed;

[0078] It should be noted that the user needs to complete the following operations before opening the cabinet door: scan the QR code on the self-service vending machine through a handheld terminal to obtain the product inventory in the self-service vending machine, select the product in the product inventory and check out.

[0079] By lighting up the reminder light strip, on the one hand, the purchasing user can quickly notice the target display area, and thus quickly determine in which independent display area the product he wants to buy is located, so that the purchasing user can quickly select the product he wants to buy. On the other hand, after the reminder light strip is lit, it can illuminate the target display area, making it convenient for the purchasing user to pick up the product in a dark environment, further enhancing the user's purchasing experience.

[0080] Obtain one or more target commodities corresponding to each target placement area to form a target commodity set, obtain the weight of each target commodity in the target commodity set to form a weight comparison set, calculate the difference between two adjacent detection weights in the target placement area (referring to the weight displayed after the weight detection device in the target placement area detects a stable value) and compare it with the elements in the weight comparison set. When any element in the weight comparison set is equal to the difference, remove the element from the weight comparison set and perform the next comparison when the detection weight changes, until the elements in the weight comparison set are cleared, mark the target placement area as completed, turn off the light strip corresponding to the target placement area, and when there is no corresponding element in the difference between two adjacent detection weights during any comparison process, record the target placement area as an abnormal placement area, and generate an abnormal signal to send to the staff's handheld terminal;

[0081] When all target display areas are marked as completed and the cabinet doors are closed, it is recorded as a complete sales process.

[0082] It should be noted that by continuously recording the weight difference between two adjacent detections in different target placement areas and comparing it with the target product, it is possible to determine whether the user has taken the correct, paid product. When the user takes a product that he has already paid for, no abnormal signal will be generated. However, when the user takes a product that does not belong to the products he has paid for, a weight value that matches the weight difference between the two adjacent detections cannot be found in the weight comparison set, thereby generating an abnormal signal and reporting information reflecting the user's wrong taking in a timely manner to facilitate remote analysis and monitoring by staff, thereby overcoming the defect of the prior art that it is difficult to detect the user's wrong taking behavior.

[0083] Furthermore, the independent placement area that is not the target placement area is recorded as other placement areas. When the detected weight in other placement areas changes, an abnormal behavior signal is generated and sent to the staff's handheld terminal to control the self-service vending machine to sound an alarm; the total weight difference of the goods in the self-service vending machine is calculated. The calculation formula is in Indicates the total weight of the goods before the vending machine door is opened. Indicates the total weight of the goods in the vending machine. Greater than or equal to When an abnormal behavior signal is generated, it is sent to the handheld terminal of the staff and controls the self-service vending machine to sound an alarm, where k is a preset proportional coefficient (in a specific embodiment, the value is 1.2). The total weight of the goods in the order.

[0084] It should be noted that by detecting the weight in other independent display areas, it is possible to prevent users from mistakenly taking goods in other independent display areas. By detecting the changes in the total weight of goods in the self-service vending machine, it is possible to promptly discover that users take too much goods when purchasing, and issue an alarm to stop users from taking too much.

[0085] Step 4: When the complete sales process is completed n No abnormal signal or abnormal behavior signal is generated within the time interval. Let the independent inventory set corresponding to any independent placement area be Inventory now , if the independent placement area is the target placement area, then Inventory now =Inventory′ n-1 -U D ,in:

[0086] Inventory′ n-1 It is the independent inventory set corresponding to the target placement area before the cabinet door is opened;

[0087] U D is the target product set corresponding to the target placement area (assuming that the independent inventory set before the cabinet door is opened is A, and the target product set is B, then the current independent inventory set is the remaining set after removing the elements in set B from set A);

[0088] If the independent placement area is other placement areas, then Inventory now =Inventory′ n-1 ;

[0089] Get the independent inventory sets in all independent placement areas and find the union to get the sales process of the self-service vending machine Sale n Stock collection after closing.

[0090] By analyzing, calculating and updating the total inventory set and the independent inventory sets corresponding to each independent placement area, the remaining goods in the self-service vending machine can be counted in time, which makes it convenient for users to select goods available in the self-service vending machine, thereby avoiding users choosing sold-out goods in the self-service vending machine and affecting the shopping experience. Furthermore, by clarifying the independent inventory sets corresponding to each independent placement area, it can facilitate guidance and instructions in step three.

[0091] Step 5: After the vending machine is marked as full, the user's purchase data is recorded and analyzed until the analysis generates a replenishment signal that is sent to the staff's handheld terminal. The staff replenishes the goods in the vending machine to ensure that there is sufficient stock in the vending machine.

[0092] Let the most recent sales process be Sale m , calculate Sale m With Sale m-1 The time difference between them is recorded as the first time parameter β time1 ,The first time parameter reflects the time difference between the two recent sales processes, the larger the first time parameter, the worse the sales of the self-service vending machine;

[0093] Sale m With Sale m-1 The time interval between them is recorded as the analysis interval. The number of times the user scans the code within the analysis interval is obtained and divided by the duration of the analysis interval to obtain the first frequency parameter β 频次 The first time parameter reflects the attractiveness of the products in the vending machine to users (the vending machine uses a transparent cabinet door, and when users see the desired products through the cabinet door, they will try to scan the code to buy them). The larger the first time parameter, the more attractive the products in the vending machine are to users;

[0094] Get the sales processSale m The time difference from the current time is recorded as the second time parameter β time2 ,The larger the second time parameter is, the worse the sales of the vending machine is;

[0095] The time interval between the last replenishment time and the current time is recorded as the sales interval, and the number of times users scan the code within the sales interval is counted NUM 扫码 , through the formula Calculate the purchase ratio value β 购买比例 ,The purchase ratio value reflects the matching degree between the ,product inventory in the self-service vending machine and the user demand. The larger ,purchase ratio value indicates that the products in the self-service vending machine are more ,matched with the user demand;

[0096] Get multiple target products corresponding to each sales process and form a target product database. Count the x most frequently appearing products in the target product database and record them as hot-selling products. y , get the corresponding frequency PD y , where y = 1, 2, 3, ..., x, get the hot-selling goods Goods in the full warehouse state y Corresponding stock PC y , through the formula Calculate the hot selling impact value β 热销The hot-selling impact value reflects the inventory surplus of hot-selling products in the self-service vending machine. The larger the hot-selling impact value, the less the inventory surplus of hot-selling products in the self-service vending machine.

[0097] By formula The replenishment judgment value α′ is calculated, where μ1, μ2, and μ3 are preset weight coefficients. Calculate the remaining inventory value α″, where KC m Indicates the current inventory quantity of goods in the vending machine, and KC0 indicates the inventory quantity of goods in the vending machine when the warehouse is full;

[0098] The replenishment judgment value and the remaining inventory value are normalized and compared with the preset inventory replenishment threshold value respectively. When any value is greater than or equal to the preset inventory replenishment threshold value, a replenishment signal is generated.

[0099] It should be noted that the replenishment judgment value depends on user purchase data such as the first time parameter, the first frequency parameter, the second time parameter, the purchase ratio value, and the product inventory data of the hot-selling influence value. Changes in user purchase data directly reflect the sales of the vending machine, and the product inventory data will have an indirect impact on the sales of the vending machine. Therefore, the replenishment judgment value reflects the degree of impact of the reduction in product inventory in the vending machine on the sales of the vending machine. The larger the replenishment judgment value, the greater the impact of the inventory reduction on the sales. When it exceeds a certain threshold, the vending machine needs to be replenished to overcome the impact of the reduction in product inventory on sales.

[0100] The present invention also provides a computer device, comprising a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the above method when executing the computer program.

[0101] A computer-readable storage medium stores a computer program, which implements the steps of the above method when executed by a processor.

[0102] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements will not cause the essence of the corresponding technical solutions to deviate from the protection scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for identifying goods in a self-service vending machine based on deep learning, comprising the following steps: Step 1: Split the vending machine shelf into multiple independent placement areas, determine multiple pending commodities based on the weight of the commodities before and after replenishment, and compare the images of the changed areas before and after replenishment with the image data of the pending commodities to obtain multiple pending similarities, characterized in that: Analyze the pending similarity to obtain the pending confidence value, determine the supplementary product based on the pending confidence value, add the supplementary product to the original product set, and obtain multiple independent full warehouse sets; Step 2: After the vending machine is replenished with goods, it is recorded as a full-warehouse state, and the sales process is recorded. In step 4, the inventory set after the after-sales process is completed is obtained. Based on the inventory set, the product inventory is updated after the sales process is completed, and the products that do not exist in the product inventory are marked as unavailable; Step 3: When the cabinet door is opened, obtain the target product in the product order, mark the target placement area corresponding to the target product, analyze the weight change of the target placement area, mark the target placement area as completed, or generate an abnormal signal or abnormal behavior signal; When all target display areas have been picked up and the cabinet doors are closed, it is recorded as a complete sales process; Step 4: Based on the target product set and the independent inventory set corresponding to each independent placement area before the cabinet door is opened, regenerate the independent inventory set of each independent placement area; Step 5: When the vending machine is fully stocked, count the number of sales processes, the time corresponding to the after-sales process, and the number of times the user scans the code, and analyze them in combination with the target product set and product inventory until the analysis generates a replenishment signal.

2. According to claim 1, a method for identifying goods in a self-service vending machine based on deep learning is characterized in that: The specific process of determining pending products is as follows: The independent placement areas are recorded as Part q , q = 1, 2, 3, ..., p, p is the number of independent placement areas; The analyzed independent placement area is recorded as the target area, and the value change of the weight detection device is recorded in real time. When the value of the weight detection device changes from jumping to stable and the stable time exceeds the preset stable time threshold, the image captured by the corresponding image acquisition device is obtained and recorded as a stable image. Where i is the record number of the stable image, Indicates the image in the target area captured by the image acquisition device before replenishing the product, and stabilizes the image The total weight of the goods in the corresponding target area is recorded as the stable weight The most recently recorded stable image and stable weight are recorded as when When, through the formula Calculate the current change value A warehouse commodity database is preset, which contains image data and weight data of multiple known commodities. Equivalent known commodities are recorded as pending commodities.

3. The method for identifying goods in a self-service vending machine based on deep learning according to claim 2, characterized in that: The process of determining supplementary goods is as follows: Get a stable image With stable image The changing area image between Extract the changed area image The similarity is calculated by comparing the image data of the pending product one by one, and multiple pending similarities are obtained. The pending confidence value is calculated by the pending confidence value calculation formula. The pending confidence value calculation formula is: in: represents the pending confidence value; Xs max Indicates the maximum value among multiple pending similarities; Represents the average value of multiple pending similarities; Xs′ max Indicates the second largest value among multiple pending similarities; When the pending confidence value is greater than or equal to the preset confidence threshold, select Xs max The corresponding pending product is used as the current supplementary product When the pending confidence value is less than the preset confidence threshold, an unrecognizable signal is generated, and the staff actively enters the current supplementary product.

4. According to the method of self-service vending machine commodity recognition based on deep learning in claim 1, it is characterized in that: The abnormal signal generation process is as follows: Obtain one or more target products corresponding to each target placement area to form a target product set, obtain the weight of each target product in the target product set to form a weight comparison set, calculate the difference between two adjacent weights detected in the target placement area and compare it with the elements in the weight comparison set; When any element in the weight comparison set is equal to the difference, the element is removed from the weight comparison set and the next comparison is performed when the detected weight changes, until the elements in the weight comparison set are cleared, the target placement area is marked as completed, and the light strip corresponding to the target placement area is turned off; When there is no corresponding element in the difference between two adjacent detected weights during any comparison process, the target placement area is recorded as an abnormal placement area, and an abnormal signal is generated and sent to the staff's handheld terminal.

5. The method for identifying goods in a self-service vending machine based on deep learning according to claim 4, characterized in that: The abnormal behavior signal generation process is as follows: The independent placement area that is not the target placement area is recorded as other placement areas. When the detected weight in other placement areas changes, an abnormal behavior signal is generated and sent to the handheld terminal of the staff to control the self-service vending machine to sound an alarm; Calculate the difference in total weight of goods in the vending machine The calculation formula is in Indicates the total weight of the goods before the vending machine door is opened. Indicates the total weight of the goods in the vending machine. Greater than or equal to When an abnormal behavior signal is generated, it is sent to the handheld terminal of the staff and controls the self-service vending machine to sound an alarm, where k is the preset proportional coefficient. The total weight of the goods in the order.

6. The method for identifying goods in a self-service vending machine based on deep learning according to claim 1, characterized in that: The update process for inventory collections and independent inventory collections is as follows: The independent placement area of ​​the non-target placement area is recorded as other placement areas. n No abnormal signal or abnormal behavior signal is generated within the time interval, where n is the serial number of the sales process, and the independent inventory set corresponding to any independent placement area is Inventory now , if the independent placement area is the target placement area, then Inventory now =Inventory′ n-1 -U D ,in: Inventory′ n-1 It is the independent inventory set corresponding to the target placement area before the cabinet door is opened; U D The target product set corresponding to the target placement area; If the independent placement area is other placement areas, then Inventory now =Inventory′ n-1 ; Get the independent inventory sets in all independent placement areas and find the union to get the sales process of the self-service vending machine Sale n Stock collection after closing.

7. The method for identifying goods in a self-service vending machine based on deep learning according to claim 1, characterized in that: The replenishment signal generation process is as follows: Let the most recent sales process be Sale m , calculate Sale m With Sale m-1 The time difference between them is recorded as the first time parameter β time1 ; Sale m With Sale m-1 The time interval between them is recorded as the analysis interval. The number of times the user scans the code within the analysis interval is obtained and divided by the duration of the analysis interval to obtain the first frequency parameter β 频次 ; Get the sales processSale m The time difference from the current time is recorded as the second time parameter β time2 ; The time interval between the last replenishment time and the current time is recorded as the sales interval, and the number of times users scan the code within the sales interval is counted NUM 扫码 , through the formula Calculate the purchase ratio value β 购买比例 ; Obtain the target product corresponding to each sales process and combine it with the product inventory analysis in the vending machine to calculate the hot-selling impact value β 热销 ; By formula The replenishment judgment value α′ is calculated, where μ1, μ2, and μ3 are preset weight coefficients. Calculate the remaining inventory value α″, where KC m Indicates the current inventory quantity of goods in the vending machine, and KC0 indicates the inventory quantity of goods in the vending machine when the warehouse is full; The replenishment judgment value and the remaining inventory value are normalized and compared with the preset inventory replenishment threshold value respectively. When any value is greater than or equal to the preset inventory replenishment threshold value, a replenishment signal is generated.

8. The method for identifying goods in a self-service vending machine based on deep learning according to claim 7, characterized in that: The hot-selling impact value calculation process is as follows: Get multiple target products corresponding to each sales process and form a target product database. Count the x most frequently appearing products in the target product database and record them as hot-selling products. y , get the corresponding frequency PD y , where y = 1, 2, 3, ..., x, get the hot-selling goods Goods in the full warehouse state y Corresponding stock PC y , through the formula Calculate the hot selling impact value β 热销 .

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 8 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 8 are implemented.

Images