Intelligent Identification Method, Device and Vending Machine for No Sale of Similar Commodities

Abstract

The present invention belongs to the technical field of unmanned retail, and solves the technical problem in the prior art that intelligent vending machines have highly similar commodities, which causes incorrect recognition of commodity categories by intelligent vending machines and results in poor user experience. A method, device and intelligent vending machine for intelligent identification of similar commodities in unmanned vending are provided. The method includes: confirming whether it belongs to a merchant operation or a user operation by obtaining the action type of the current operation of the intelligent vending machine; simultaneously collecting the commodity information of each basic commodity; thereby determining whether there are other commodities with high similarity to the basic commodity being operated, and then performing corresponding processing according to the action category of the current operation, including but not limited to placing commodities in different regions or not placing them during merchant operations, and determining the commodity type according to the category corresponding to the commodity position or manual review during user operations, so as to avoid misdetection caused by similar commodities, reduce the generation of abnormal orders, and improve the user experience effect.

Description

Technical Field

[0001] The present invention relates to the technical field of unmanned retail, and particularly to an intelligent identification method, device and intelligent vending machine for vending similar goods without a person. Background Art

[0002] With the continuous development of artificial intelligence technology, the sales methods in the retail industry have also undergone great changes. Among them, intelligent vending machines have been spread in various places in the city, including stations, shopping malls, tourist attractions or department stores, and various intelligent vending machines can be found. With the method of no need for a special person to watch, users can automatically place orders and check out, which greatly facilitates the shopping needs of users in special scenarios for goods.

[0003] For an intelligent vending machine, a merchant pre - places goods to be sold in the intelligent vending machine, and users can scan the code to shop through an App. Among them, the full - opening intelligent vending machine has the advantages of allowing users to select multiple items at one time and change items multiple times during one shopping process, which greatly satisfies the users' independent choices during the shopping process. Therefore, the full - opening intelligent vending machine has a good user experience effect and is deeply favored by users. Since the goods sold by intelligent vending machines are all fast - moving consumer goods such as mineral water, beverages, and coffee, merchants regularly update the goods in the intelligent vending machine, including adding existing goods and new types of goods; if the goods stocked are highly similar to the original goods in the intelligent vending machine, there will be a situation where the intelligent vending machine misidentifies similar goods when selling goods. In practice, there are often cases where goods with the same outer packaging and weight are confused, resulting in abnormal orders. At the same time, because users of full - opening intelligent vending machines can independently select goods before checking out, they often take multiple items at the same time; if the goods taken by users are different types but highly similar, the intelligent vending machine will misidentify the category of goods, resulting in abnormal orders, which not only causes losses to merchants but also leads to a poor user experience. Summary of the Invention

[0004] In view of this, embodiments of the present invention provide an intelligent identification method, device and intelligent vending machine for vending similar goods without a person, so as to solve the technical problem that existing intelligent vending machines have goods with high similarity, resulting in misidentification of the category of goods by the intelligent vending machine and poor user experience.

[0005] The technical solution adopted by the present invention is:

[0006] The present invention provides an intelligent identification method for vending similar goods without a person, and the method includes:

[0007] S1: Obtain the action category of the current operation of the vending machine and the product information of each basic product corresponding to the current operation, where the action category includes the first action category of a merchant adding products to the vending machine and the second action category of a user selecting products from the vending machine;

[0008] S2: Determine the similarity between each basic product and other products in the product area of the vending machine according to the product information of each basic product;

[0009] S3: Output the processing method of the target product among the basic products whose similarity meets the requirements according to each target similarity and the action category;

[0010] Among them, the processing method includes: the first processing method corresponding to the vending machine's stocking and the second processing method corresponding to the user's automatic shopping.

[0011] Preferably, the S2 includes:

[0012] S21: Obtain the target images of each basic product obtained simultaneously from multiple angles;

[0013] S22: Extract features from each of the target images to obtain the multi-dimensional feature vectors of the product information of each basic product;

[0014] S23: Compare the multi-dimensional feature vectors of each basic product with the multi-dimensional feature vectors of other categories of products in the vending machine to obtain the target similarity between each basic product and other categories of products in the product area of the vending machine.

[0015] Preferably, the S23 includes:

[0016] S231: Obtain multiple multi-dimensional feature vectors of each basic product;

[0017] S232: According to the multi-dimensional feature vectors of each basic product and the multi-dimensional feature vectors of other categories of products in the vending machine, use the formula Simliar(X,Y) = max(cos(x i ,y j )) to calculate the similarity between each target basic product and other categories of products;

[0018] Among them, X represents each basic product, Y represents other categories of products in the vending machine, represents the cosine distance between two vectors, x i ∈ X, x i represents the i-th feature vector of the basic product X; y j ∈ Y, y jIt represents the j-th feature vector of other category of goods Y in the intelligent vending machine; 0 ≤ i, j ≤ n, where n represents the dimension of the feature vector.

[0019] Preferably, the S21 includes:

[0020] S211: Obtain that the area where goods are placed in the intelligent vending machine is divided into multiple virtual commodity areas along the arrangement direction of the shelves of the intelligent vending machine;

[0021] S212: Obtain the basic videos within the viewing ranges collected by the cameras arranged oppositely in each commodity area;

[0022] S213: Physically splice each frame image of each of the basic videos with the frame images corresponding to the acquisition time sequence one by one to obtain each frame of the target image;

[0023] Wherein, the physical splicing means that the size of the spliced image is the sum of the sizes of all the images participating in the splicing.

[0024] Preferably, if the action category is the first action category, the S3 includes:

[0025] S310: Obtain a first similarity threshold and a second similarity threshold for the similarity of different categories of goods, wherein the first similarity threshold is less than the second similarity threshold;

[0026] S311: Compare the target similarity with the first similarity threshold and / or the second similarity threshold to obtain a first comparison result corresponding to the first action;

[0027] S312: If the first comparison result is that the target similarity is between the first similarity threshold and the second similarity threshold, output the target commodity area for placing the target commodity as the first processing method;

[0028] S313: If the first comparison result is greater than or equal to the second similarity threshold, output prohibiting the addition of the corresponding target commodity as the first processing method;

[0029] Wherein, the target commodity area is different from the commodity area where other categories of goods similar to the target commodity are located.

[0030] Preferably, if the action category is the second action category, and there are different categories of goods in the intelligent vending machine whose target similarity is greater than the first similarity threshold and less than the second similarity threshold, or greater than the second similarity threshold, the S3 includes:

[0031] S320: Obtain a third similarity threshold for the similarity of different categories of goods;

[0032] S321: If the target similarity is greater than the third similarity threshold, obtain the first position information of the commodity areas to which the target commodities belong;

[0033] S322: Output the second processing method corresponding to the position information of the target commodity according to each of the first position information;

[0034] Among them, the second processing method includes: when the first position information of each target commodity is the same, directly output the commodity category corresponding to the first position information as the first result; when the first position information of each target commodity is different, output the commodity categories of each target commodity after manual verification as the second result.

[0035] Preferably, after the S322, it further includes:

[0036] S330: Obtain the quantity and corresponding category of the target commodities included in the first result or the second result;

[0037] S331: Output the target commodity information of the remaining similar commodities in each commodity area of the vending machine according to the quantity and category of the target commodities;

[0038] Among them, the target commodity information includes the quantity, category and the position of the commodity area to which the similar commodities belong.

[0039] The present invention also provides an intelligent identification device for vending similar commodities, and the device includes:

[0040] Data acquisition module: used to obtain the action category of the current operation of the vending machine and the commodity information of each basic commodity corresponding to the current operation, where the action category includes the first action category of the merchant adding commodities to the vending machine and the second action category of the user selecting commodities from the vending machine;

[0041] Similarity calculation module: used to determine the similarity between each basic commodity and other commodities in the commodity area of the vending machine according to the commodity information of each basic commodity;

[0042] Data processing module: used to output the processing method of the target commodities among the basic commodities whose similarity meets the requirements according to each of the target similarities and the action category;

[0043] Among them, the processing method includes: the first processing method corresponding to the vending machine's replenishment and the second processing method corresponding to the user's automatic shopping.

[0044] The present invention also provides a vending machine, comprising: at least one processor, at least one memory, and computer program instructions stored in the memory, which, when executed by the processor, implement the method described in any one of the above.

[0045] The present invention also provides a medium, on which computer program instructions are stored, which, when executed by a processor, implement the method described in any one of the above.

[0046] In summary, the beneficial effects of the present invention are as follows:

[0047] An intelligent identification method, device and vending machine for unmanned vending of similar goods provided by the present invention, by obtaining the action type of the current operation of the vending machine, confirm whether it belongs to a merchant operation or a user operation; at the same time, collect the product information of each basic product; thus determine whether there are other products with high similarity to the basic product being operated, and then perform corresponding processing according to the action category of the current operation, including but not limited to placing products in different regions or not placing them during merchant operations, and determining the product type or manual review according to the category corresponding to the product position during user operations, avoiding misdetection caused by similar products, reducing the generation of abnormal orders, and improving the user experience effect. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required to be used in the embodiments of the present invention will be briefly introduced below. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained according to these drawings, and all of these are within the protection scope of the present invention.

[0049] Figure 1 It is a schematic flow chart of the intelligent identification method for unmanned vending of similar goods in Embodiment 1;

[0050] Figure 2 It is a schematic structural diagram of the vending machine in Embodiment 1;

[0051] Figure 3 It is a schematic flow chart of the product stocking process in Embodiment 1;

[0052] Figure 4 It is a schematic flow chart of the product processing method based on similarity in Embodiment 1;

[0053] Figure 5 It is a schematic flow chart of the intelligent identification method for unmanned vending of similar goods in Embodiment 2;

[0054] Figure 6 It is a schematic structural diagram of the vending machine in Embodiment 2. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0055] To make the objectives, technical solutions and advantages of the embodiments of the present invention clearer, the following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the accompanying drawings in the embodiments of the present invention. It should be noted that in this document, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. In the description of the present invention, it should be understood that the terms "center", "upper", "lower", "front", "rear", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer", etc. indicate the orientation or positional relationship based on the orientation or positional relationship shown in the drawings, and are only for the convenience of describing the present application and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be construed as a limitation on the present invention. Moreover, the term "comprising", "including" or any other variation thereof is intended to cover a non-exclusive inclusion, so that a process, method, article or device including a series of elements not only includes those elements, but also includes other elements not expressly listed, or also includes elements inherent to such process, method, article or device. Without further limitation, the elements defined by the statement "comprising..." do not exclude the presence of additional identical elements in the process, method, article or device including the said elements. If there is no conflict, the various features in the present invention and its embodiments can be combined with each other and are all within the protection scope of the present invention.

[0056] Embodiment 1

[0057] Please refer to Figure 1 , Figure 1 which is a schematic flowchart of an intelligent identification method for vending similar goods. The method includes:

[0058] S1: Obtain the action category of the current operation of the intelligent vending machine and the commodity information of each basic commodity corresponding to the current operation. Among them, the action category includes a first action category in which a merchant adds a commodity to the vending machine and a second action category in which a user selects a commodity from the vending machine;

[0059] Specifically, when the vending machine is operating normally, the operations on the vending machine include two types of actions. The first type of action is the operation of the merchant on the vending machine, which includes but is not limited to adding new products to the product area of the vending machine, replacing old products, and also includes equipment maintenance and performance detection of the vending machine, etc.; The second type of action is that the user selects products from the product area of the vending machine. For the convenience of understanding, in this article, the merchant adding new products and / or replacing old products in the product area of the vending machine is recorded as the first action category, and the user selecting products from the vending machine is recorded as the second action category; The product information includes but is not limited to: image information, operation information of merchant products or off-shelf products, and position information of products placed or taken out.

[0060] Please refer to Figure 2 , Figure 2 which is a schematic structural diagram of a fully open-door vending machine. As Figure 2 shown, the vending machine includes a cabinet body 1 and a cabinet door 2. The cabinet body 1 and the cabinet door 2 are rotationally connected. When the cabinet door 2 is in a closed state relative to the cabinet body 1, the cabinet door 2 covers all the product areas of the cabinet body 1 where products are placed, that is, it is impossible to perform operations on taking products in the cabinet. When the cabinet door 2 is opened, all the products in the cabinet body 1 are presented in front of the user. The user can choose any product during a single shopping trip, and can also choose multiple products. The user can take out the selected products and can also put back the products that need to be put back after selection. There is a shelf 11 in the cabinet body 1. The shelf 11 can be a shelf that divides the cabinet body 1 into multiple product areas 12. Among them, cameras are provided in each product area inside the cabinet body 1, so that shopping videos of users shopping from the vending machine can be obtained from multiple angles, avoiding the problem that the reliability of the shopping video collected from a single angle is not high due to occlusion problems. Figure 2 For the vending machine shown, multiple cameras are provided on both the left inner wall and the right inner wall of the vending machine, so that shopping videos can be collected from opposite perspective directions for the same product area, improving the reliability of video data.

[0061] S2: According to the product information of each basic product, determine the similarity between each basic product and other products in the product area of the vending machine;

[0062] Specifically, when the user selects products from the vending machine or the merchant puts products into the vending machine, the cameras in each product area of the vending machine obtain image information of each product of the current operation from different perspectives, and then extract the characteristic information of each product for similarity calculation to obtain the similarity between different types of products.

[0063] In one embodiment, the S2 includes:

[0064] S21: Obtain the target images of each basic product obtained simultaneously from multiple angles;

[0065] Specifically, cameras are set at different positions in the commodity area of the vending machine, so as to obtain image information of the commodity at any moment from different angles, facilitating the collection of more characteristic information of the same commodity and improving the accuracy of commodity detection.

[0066] In one embodiment, S21 includes:

[0067] S211: Divide the commodity area where the vending machine places commodities into multiple virtual commodity areas along the arrangement direction of the shelves of the vending machine;

[0068] Specifically, there are multiple layers of shelves in the vending machine. The area where the vending machine places commodities is divided into multiple commodity areas. Among them, the commodity area includes a first area corresponding to the inside of the cabinet of the vending machine and a second area outside the cabinet of the vending machine where the commodity leaves the vending machine. The area size of the second area can be freely set as needed. Each commodity area includes at least one layer of shelves. The viewing angle of the camera is set along the arrangement direction of the shelves. For example, if the shelves of the vending machine include multiple layers from top to bottom, each camera is respectively set on the left and right side walls of the vending machine, and the viewing angle of each camera is from the upper left to the lower right or from the upper right to the lower left or from top to bottom. Among them, the installation heights of the cameras set on different sides in the same commodity area are the same.

[0069] S212: Obtain the basic videos within the viewing ranges collected by the cameras arranged oppositely in each commodity area;

[0070] S213: Physically splice each frame image of each of the basic videos with the frame images corresponding to the acquisition time sequence one by one to obtain each frame of the target image;

[0071] Among them, the physical splicing means that the spliced image is the sum of the sizes of all the images participating in the splicing.

[0072] Specifically, when the user starts shopping from the vending machine or the merchant stocks the vending machine, each camera obtains the basic videos (the basic videos are the direct videos of the commodities being taken or put back, that is, the videos contain the commodities being taken or put back) of the user or the merchant taking or putting back the commodities from different angles, and physically splices each frame image of the basic videos obtained by different cameras according to the acquisition time sequence to obtain the final target video composed of spliced images; among them, the same event is the whole process of the user or the merchant's one-time shopping or stocking.

[0073] It should be noted that: physical splicing means splicing two images into one image, and the size of the spliced image is the sum of the sizes of the images participating in the splicing; at the same time, physical splicing of each frame of images of different videos means: splicing the first frame image of the first video, the first frame image of the second video... the first frame image of the Nth video, splicing the second frame image of the first video, the second frame image of the second video... the second frame image of the Nth video, and so on, splicing the nth frame image of the first video, the nth frame image of the second video... the nth frame image of the Nth video to obtain each target image.

[0074] S22: Extract features from each of the target images to obtain multi-dimensional feature vectors of the commodity information of each basic commodity;

[0075] Specifically, extract features from each basic commodity to obtain multi-dimensional feature vectors corresponding one by one to each commodity, such as 512-dimensional feature vectors and 256-dimensional feature vectors.

[0076] S23: Compare the multi-dimensional feature vectors of each basic commodity with the multi-dimensional feature vectors of other categories of commodities in the vending machine to obtain the target similarity between each basic commodity and other categories of commodities in the commodity area of the vending machine.

[0077] Specifically, compare the multi-dimensional feature vectors of each basic commodity with the multi-dimensional feature vectors of other categories of commodities to obtain the target similarities between each basic commodity and other categories of commodities; it should be noted that: when each commodity is stocked by the merchant, multi-dimensional feature vectors of each commodity are extracted and stored according to the category of the commodity, and at the same time, the quantity and placement position of each category of commodity in the vending machine are recorded in real time.

[0078] In one embodiment, the S23 includes:

[0079] S231: Obtain multiple multi-dimensional feature vectors of each basic commodity;

[0080] S232: According to the multi-dimensional feature vectors of each basic commodity and the multi-dimensional feature vectors of other categories of commodities in the vending machine, use the formula Simliar(X,Y) = max(cos(x i ,y j )) to calculate the similarity between each of the basic commodities and other categories of commodities;

[0081] Among them, X represents each basic commodity, Y represents other categories of commodities in the vending machine, represents the cosine distance between two vectors, x i ∈ X, x i represents the i-th feature vector of the basic commodity X; y j ∈ Y, y jIt represents the j-th eigenvector of other category of goods Y in the vending machine; 0 ≤ i, j ≤ n, where n represents the dimension of the eigenvector.

[0082] Specifically, each frame of the target image includes images of the same basic product taken by multiple cameras at the same moment from different angles. Therefore, the same product exists in multiple regions in each frame of the target image. Multiple multi-dimensional eigenvectors of this product are obtained from each region of each frame of the target image, and then the formula is used to perform normalization processing on the multi-dimensional basic eigenvectors of each category of goods, so as to obtain the corresponding multi-dimensional eigenvectors. Then, the matrix of multiple multi-dimensional eigenvectors of the current basic product is multiplied by the matrix of multiple multi-dimensional eigenvectors of other categories to obtain the similarities between the current basic product and other categories; specifically, the formula Simliar(X,Y) = max(cos(x i ,y j )) is used to calculate the similarities between each of the said basic products and other category of goods.

[0083] Among them, X represents each basic product, Y represents other category of goods in the vending machine, represents the cosine distance between two vectors, x i ∈X, x i represents the i-th eigenvector of the basic product X; y j ∈Y, y j represents the j-th eigenvector of other category of goods Y in the vending machine; 0 ≤ i, j ≤ n, where n represents the dimension of the eigenvector. Q k is the eigenvector corresponding to the k-th eigenvector of the basic product X after normalization processing.

[0084] S3: According to each of the said target similarities and the action category, output the processing method of the target product among the basic products whose similarities meet the requirements;

[0085] Among them, the said processing method includes: the first processing method corresponding to the stocking of the vending machine and the second processing method corresponding to the user's automatic shopping.

[0086] Specifically, according to the target similarity between the basic commodity currently operated and other types of commodities, the basic commodities corresponding to the target similarity that meet the requirements are processed to avoid false detection when the computer performs commodity detection; specifically, the basic commodities that meet the requirements are classified and processed according to the action type of the current operation, such as: if the similar commodities are in the stage of merchants adding goods to the smart vending machine, the processing method includes but is not limited to putting similar commodities into different commodity areas or preventing the addition of similar commodities to the smart vending machine. For example, the packaging similarity of the common beverages "Wanglaoji" and "Jiaduobao" is very high. When one of them is stored in the smart vending machine, it is recommended not to add the other commodity to the vending machine, or to add the two commodities to different areas of the vending machine, so as to increase the recognition of commodity identification. If the similar commodity is a commodity selected by the user, the processing method includes but is not limited to obtaining the commodity category selected by the user at the same time and the commodity area to which the commodity belongs; if the user selects multiple commodities at the same time, and there are similar commodities among the multiple commodities, manual verification is output; if the similar commodity selected by the user at one time is a commodity that is similar to the commodity that has not been selected, the commodity information recorded in the commodity area to which the commodity belongs is output.

[0087] In one embodiment, if the action category is the first action category, see Figure 3 , said S3 comprises:

[0088] S310: Obtaining a first similarity threshold and a second similarity threshold of similarities of commodities of different categories, wherein the first similarity threshold is smaller than the second similarity threshold;

[0089] Specifically, a first similarity threshold and a second similarity threshold for processing similar commodities are set for the merchant's operation of loading goods into the smart vending machine, so that differentiated processing of similar commodities can be performed.

[0090] S311: Compare the target similarity with the first similarity threshold and / or the second similarity threshold to obtain a first comparison result corresponding to the first action;

[0091] S312: if the first comparison result is that the target similarity is between the first similarity threshold and the second similarity threshold, outputting a target commodity area for placing the target commodity as the first processing method;

[0092] S313: If the first comparison result is greater than or equal to the second similarity threshold, outputting prohibition of adding corresponding target products as the first processing method;

[0093] The target commodity area is a commodity area different from other categories of commodities similar to the target commodity.

[0094] Specifically, when a merchant puts a product into the vending machine, if the product is a new category, the similarity between the product and the products already existing in the vending machine is judged. If the similarity is greater than the first similarity threshold and less than the second similarity threshold, it is considered that there may be misdetection between the new product and the existing products. Then, the new product and the similar products are stored in different product areas so as to confirm the product category by the product position. If the similarity is greater than the second similarity threshold, it is considered that there is a problem of mixed detection caused by the algorithm's inability to recognize between the new product and the existing products. Therefore, only one of these different types of products with a particularly high similarity is retained in the vending machine. By this method, the utilization rate of the vending machine can be improved.

[0095] In one embodiment, if the action category is the first action category, S3 includes:

[0096] S314: Obtain the historical stocking information and historical replacement information of various products corresponding to the target similarity;

[0097] S315: According to the historical stocking information and the historical replacement information, adjust the stock quantity and placement position of various products corresponding to the target similarity as the first processing method.

[0098] Specifically, when other products similar to the stocked product are found during stocking, at this time, obtain the historical stocking information and historical replacement information of various products (including the product categories corresponding to the currently stocked product and the product categories corresponding to other products similar to the stocked product. For example, if the stocked product is "Wanglaoji" and the similar product is "Jiaduobao", then obtain the historical stocking information and historical replacement information of "Wanglaoji" and "Jiaduobao" respectively), and then judge the degree of consumer preference for various products, so as to place the products more favored by consumers in a better viewable placement position, and provide a larger stock quantity area for placing the more favored products; among them, the historical stocking information mainly refers to the product information for replenishment when the product sales are good and the stock is insufficient; the historical replacement information mainly refers to the product information for replacing new products when the product sales are poor and the product is approaching the expiration date.

[0099] In one embodiment, if the action category is the second action category, and there are different types of products in the vending machine with a target similarity greater than the first similarity threshold and less than the second similarity threshold, or greater than the second similarity threshold, please refer to Figure 4 ,S3 includes:

[0100] S320: Obtain the third similarity threshold of the similarity of different types of products;

[0101] Specifically, when there are at least two similar products in the vending machine, a third similarity threshold for judging similar products is set. This third similarity threshold is a dynamically variable similarity value. For example, after the merchant shelves the products, the similarity between product A and product B is less than or equal to 0.8, and there is only this group of similar products in the vending machine. When the user is selecting products and the similarity between the selected target product and other types of products reaches 0.8, special processing is required; when product A or product B is sold out, the similarity detection will be cancelled at this time; when the merchant shelves the products next time, the similarity between product C and product D is 0.7, and the similarity between product E and product F is 0.8, then the third similarity threshold is less than or equal to 0.7; when product C or product D is sold out and both product E and product F are still available, the third similarity threshold is adjusted to less than or equal to 0.8.

[0102] S321: If the target similarity is greater than the third similarity threshold, obtain the first position information of the product area to which each of the target products belongs;

[0103] Specifically, when it is detected that the similarity between the product selected by the user and other types of products in the vending machine is greater than the third similarity threshold, obtain the product area to which the product selected by the user belongs, denoted as the first position information; for example: if the user only selects one product, directly determine the product information of the product according to the product area where the product is located; if the user selects multiple products and there are more than two different types of products that are similar to each other, send the video of the products selected by the user to the server for manual review; more than two different types of products that are similar to each other include, but are not limited to: multiple products taken by the user at one time or multiple products corresponding to the products taken by the user after multiple times of taking products.

[0104] S322: According to each of the first position information, output the second processing method corresponding to the position information of the target product;

[0105] Among them, the second processing method includes: when the first position information of each of the target products is the same, directly output the product category corresponding to the first position information as the first result; when the first position information of each of the target products is different, output the product categories of each of the target products after manual verification as the second result.

[0106] In an embodiment, after the S322, it further includes:

[0107] S330: Obtain the quantity and corresponding category of the target products included in the first result or the second result;

[0108] S331: According to the quantity and category of the target products, output the target product information of the remaining similar products in each product area of the vending machine;

[0109] Among them, the target commodity information includes the quantity, category, and location of the commodity area to which the similar commodities belong.

[0110] By using the intelligent identification method for vending similar commodities in this embodiment, the type of action of the current operation of the vending machine is obtained to confirm whether it is a merchant operation or a user operation; meanwhile, the commodity information of each basic commodity is collected; thereby determining whether there are other commodities with high similarity to the basic commodity being operated, and then corresponding processing is performed according to the type of action of the current operation, including but not limited to placing commodities in different areas or not placing them during merchant operations, and determining the commodity type or manual review according to the category corresponding to the commodity location during user operations, avoiding misdetection caused by similar commodities, reducing the generation of abnormal orders, and improving the user experience effect.

[0111] Embodiment 2

[0112] Based on the method of Embodiment 1, Embodiment 2 of the present invention further provides an intelligent identification device for vending similar commodities. Please refer to Figure 5 , including:

[0113] Data acquisition module: used to obtain the type of action of the current operation of the vending machine and the commodity information of each basic commodity corresponding to the current operation, where the type of action includes the first type of action for a merchant to add commodities to the vending machine and the second type of action for a user to select commodities from the vending machine;

[0114] Similarity calculation module: used to determine the similarity between each basic commodity and other commodities in the commodity area of the vending machine according to the commodity information of each basic commodity;

[0115] Data processing module: used to output the processing method of the target commodity among the basic commodities with the similarity meeting the requirements according to each target similarity and the type of action;

[0116] Among them, the processing method includes: the first processing method corresponding to the vending machine stocking and the second processing method corresponding to the user's automatic shopping.

[0117] In an embodiment, the similarity calculation module includes:

[0118] First image acquisition unit: obtaining the target images of each basic commodity simultaneously acquired from multiple angles;

[0119] First feature extraction unit: extracting features from each of the target images to obtain the multi-dimensional feature vectors of the commodity information of each basic commodity;

[0120] The first similarity comparison unit: compares the multi-dimensional feature vectors of each basic commodity with the multi-dimensional feature vectors of other categories of commodities in the vending machine to obtain the target similarity between each basic commodity and other categories of commodities in the commodity area of the vending machine.

[0121] In one embodiment, the first similarity comparison unit includes:

[0122] The first vector acquisition unit: acquires multiple multi-dimensional feature vectors of each basic commodity;

[0123] The first similarity calculation unit: according to the multi-dimensional feature vectors of each basic commodity and the multi-dimensional feature vectors of other categories of commodities in the vending machine, uses the formula Simliar(X,Y)=max(cos(x i ,y j )) to calculate the similarity between each of the basic commodities and other categories of commodities;

[0124] Among them, X represents each basic commodity, Y represents other categories of commodities in the vending machine, represents the cosine distance between two vectors, x i ∈X, x i represents the i-th feature vector of the basic commodity X; y j ∈Y, y j represents the j-th feature vector of other categories of commodities Y in the vending machine; 0 ≤ i, j ≤ n, and n represents the dimension of the feature vector.

[0125] In one embodiment, the first image acquisition unit includes:

[0126] The commodity area information acquisition unit: acquires that the commodity area where the vending machine places commodities is divided into multiple virtual commodity areas along the shelf arrangement direction of the vending machine;

[0127] The basic video acquisition unit: acquires the basic videos within the viewing ranges collected by the cameras arranged oppositely in each commodity area;

[0128] The image stitching unit: physically stitches each frame image of each of the basic videos with the frame images corresponding in the acquisition time sequence one by one to obtain each frame of the target image;

[0129] Among them, the physical stitching means that the stitched image is the sum of the sizes of all the images participating in the stitching.

[0130] In one embodiment, if the action category is the first action category, the data processing module includes:

[0131] The first similarity threshold obtaining unit: obtains the first similarity threshold and the second similarity threshold of the similarities of commodities of different categories, where the first similarity threshold is less than the second similarity threshold;

[0132] The first comparison unit: compares the target similarity with the first similarity threshold and / or the second similarity threshold to obtain the first comparison result corresponding to the first action;

[0133] The first processing unit: if the first comparison result is that the target similarity is between the first similarity threshold and the second similarity threshold, outputs the target commodity area for placing the target commodity as the first processing method;

[0134] The second processing unit: if the first comparison result is greater than or equal to the second similarity threshold, outputs prohibiting the addition of the corresponding target commodity as the first processing method;

[0135] Wherein, the target commodity area is different from the commodity areas where other category commodities similar to the target commodity are located.

[0136] In an embodiment, if the action category is the second action category, and there are different category commodities in the vending machine whose target similarities are greater than the first similarity threshold and less than the second similarity threshold, or greater than the second similarity threshold, the similarity calculation module includes:

[0137] The second similarity obtaining unit: obtains the third similarity threshold of the similarities of commodities of different categories;

[0138] The first position obtaining unit: if the target similarity is greater than the third similarity threshold, obtains the first position information of the commodity areas to which each target commodity belongs;

[0139] The third processing unit: outputs the second processing method corresponding to the position information of the target commodity according to each first position information;

[0140] Wherein, the second processing method includes: when the first position information of each target commodity is the same, directly outputs the commodity category corresponding to the first position information as the first result; when the first position information of each target commodity is different, outputs the commodity categories of each target commodity after manual verification as the second result.

[0141] In an embodiment, after the third boosting unit, there is further included:

[0142] The category information obtaining unit: obtains the quantity and corresponding categories of the target commodities included in the first result or the second result;

[0143] Commodity information output unit: Output the target commodity information of the remaining similar commodities in each commodity area of the intelligent vending machine according to the quantity and category of the target commodity;

[0144] Among them, the target commodity information includes the quantity, category, and the location of the commodity area to which the similar commodity belongs.

[0145] By using the intelligent identification device for unmanned vending of similar commodities in this embodiment, the type of action currently operated on the intelligent vending machine is obtained to confirm whether it is a merchant operation or a user operation; at the same time, the commodity information of each basic commodity is collected; thus, it is determined whether there are other commodities with high similarity to the basic commodity being operated, and then corresponding processing is performed according to the type of action currently operated, including but not limited to placing commodities in different areas or not placing them during merchant operations, and determining the commodity type according to the category recorded corresponding to the commodity location or manual review during user operations, avoiding misdetection caused by similar commodities, reducing the generation of abnormal orders, and improving the user experience effect.

[0146] Embodiment 3

[0147] The present invention provides an intelligent vending machine device and a storage medium, as Figure 6 shown, including at least one processor, at least one memory, and computer program instructions stored in the memory.

[0148] Specifically, the above-mentioned processor may include a central processing unit (CPU), or an application-specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present invention. The intelligent vending machine is provided with a cabinet door for the commodity area location that can cover all commodity areas. The cabinet door is an active cabinet door that can be opened and closed. At the same time, the intelligent vending machine also includes identification devices such as cameras, two-dimensional codes, and barcodes that facilitate shopping.

[0149] The memory may include a mass storage for data or instructions. By way of example and not limitation, the memory may include a hard disk drive (HDD), a floppy disk drive, a flash memory, an optical disk, a magneto-optical disk, a magnetic tape, or a Universal Serial Bus (USB) drive, or a combination of two or more of these. Where appropriate, the memory may include removable or non-removable (or fixed) media. Where appropriate, the memory may be internal or external to the data processing device. In a particular embodiment, the memory is a non-volatile solid-state memory. In a particular embodiment, the memory includes a read-only memory (ROM). Where appropriate, the ROM may be a mask-programmed ROM, a programmable ROM (PROM), an erasable PROM (EPROM), an electrically erasable PROM (EEPROM), an electrically rewritable ROM (EAROM), or a flash memory, or a combination of two or more of these.

[0150] The processor reads and executes the computer program instructions stored in the memory to implement any one of the order generation methods for multi-perspective image analysis and the order generation method based on image stitching and deduplication in the first embodiment of the above embodiments.

[0151] In one example, the electronic device may further include a communication interface and a bus. Among them, the processor, the memory, and the communication interface are connected through the bus and complete communication with each other.

[0152] The communication interface is mainly used to implement communication between the various modules, devices, units, and / or devices in the embodiments of the present invention.

[0153] The bus includes hardware, software, or both, and couples the components of the electronic device to each other. By way of example and not limitation, the bus may include an Accelerated Graphics Port (AGP) or other graphics bus, an Enhanced Industry Standard Architecture (EISA) bus, a Front Side Bus (FSB), a HyperTransport (HT) interconnect, an Industry Standard Architecture (ISA) bus, an InfiniBand interconnect, a Low Pin Count (LPC) bus, a memory bus, a MicroChannel Architecture (MCA) bus, a Peripheral Component Interconnect (PCI) bus, a PCI-Express (PCI-X) bus, a Serial Advanced Technology Attachment (SATA) bus, a Video Electronics Standards Association Local (VLB) bus, or other suitable buses, or a combination of two or more of these. Where appropriate, the bus may include one or more buses. Although the embodiments of the present invention describe and illustrate specific buses, the present invention contemplates any suitable bus or interconnect.

[0154] In summary, the embodiments of the present invention provide an intelligent identification method, device, intelligent vending machine, and storage medium for vending similar goods without a salesperson.

[0155] It should be clear that the present invention is not limited to the specific configurations and processes described above and shown in the figures. For the sake of brevity, detailed descriptions of known methods are omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of the present invention is not limited to the specific steps described and shown. Those skilled in the art can make various changes, modifications, and additions, or change the order between steps after understanding the spirit of the present invention.

[0156] The functional blocks shown in the above-described structural block diagrams can be implemented as hardware, software, firmware, or a combination thereof. When implemented in hardware, it can be, for example, an electronic circuit, an application-specific integrated circuit (ASIC), appropriate firmware, a plug-in, a function card, and so on. When implemented in software, the elements of the present invention are programs or code segments used to perform the required tasks. The program or code segment can be stored in a machine-readable medium or transmitted via a data signal carried in a carrier wave over a transmission medium or a communication link. A "machine-readable medium" can include any medium capable of storing or transmitting information. Examples of machine-readable media include electronic circuits, semiconductor memory devices, ROM, flash memory, erasable ROM (EROM), floppy disks, CD-ROMs, optical disks, hard disks, fiber optic media, radio frequency (RF) links, and so on. The code segment can be downloaded via a computer network such as the Internet, an intranet, and so on.

[0157] Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that: they can still modify the technical solutions recorded in the foregoing embodiments, or perform equivalent replacements on some or all of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. An intelligent identification method for vending similar goods without human intervention, characterized in that, The method includes: S1: Obtain the action category of the current operation of the vending machine and the product information of each basic product corresponding to the current operation. Among them, the action category includes the first action category of the merchant adding products to the vending machine and the second action category of the user selecting products from the vending machine; S2: Determine the similarity between each basic product and other products in the product area of the vending machine according to the product information of each basic product. S2 includes: S21: Obtain the target images of each basic product obtained simultaneously from multiple angles; S22: Extract features from each of the target images to obtain the multi-dimensional feature vectors of the product information of each basic product; S23: Compare the multi-dimensional feature vectors of each basic product with the multi-dimensional feature vectors of other categories of products in the vending machine to obtain the target similarity between each basic product and other categories of products in the product area of the vending machine; S3: Output the processing method of the target product among the basic products whose similarity meets the requirements according to each target similarity and the action category; Among them, the processing method includes: the first processing method corresponding to the vending machine stocking and the second processing method corresponding to the user's automatic shopping; If the action category is the first action category, S3 includes: S310: Obtain the first similarity threshold and the second similarity threshold of the similarity of different categories of products, where the first similarity threshold is less than the second similarity threshold; S311: Compare the target similarity with the first similarity threshold and / or the second similarity threshold to obtain the first comparison result corresponding to the first action; S312: If the first comparison result is that the target similarity is between the first similarity threshold and the second similarity threshold, output the target product area for placing the target product as the first processing method; S313: If the first comparison result is greater than or equal to the second similarity threshold, output prohibiting the addition of the corresponding target product as the first processing method; Among them, the target product area is different from the product area where other categories of products similar to the target product are located; If the action category is the second action category, and there are different categories of products in the vending machine whose target similarity is greater than the first similarity threshold and less than the second similarity threshold, or greater than the second similarity threshold, S3 includes: S320: Obtain the third similarity threshold of the similarity of different categories of products, where the third similarity threshold is a dynamically variable similarity value; S321: If the target similarity is greater than the third similarity threshold, obtain the first position information of the product area to which each target product belongs; S322: Output the second processing method corresponding to the position information of the target product according to each first position information; Among them, the second processing method includes: when the first position information of each target product is the same, directly output the product category corresponding to the first position information as the first result; when the first position information of each target product is different, output the product categories of each target product after manual verification as the second result.

2. The intelligent identification method for vending similar goods without a vendor, as claimed in claim 1, wherein The S23 includes: S231: Obtain multiple multi-dimensional basic feature vectors of each basic commodity; S232: According to the multi-dimensional feature vectors of each basic commodity and the multi-dimensional feature vectors of other categories of commodities in the vending machine, use the formula to calculate the similarity between each of the basic commodities and the other categories of commodities; Among them, represents each basic commodity, represents other categories of commodities in the vending machine, represents the cosine distance between two vectors, , represents the i-th feature vector of the basic commodity X; , represents the j-th feature vector of other categories of commodities Y in the vending machine; , and n represents the dimension of the feature vector.

3. The intelligent identification method for vending similar products without human intervention according to claim 1, wherein, The S21 includes: S211: Obtain that the commodity placement area of the vending machine is divided into multiple virtual commodity areas along the shelf arrangement direction of the vending machine; S212: Obtain the basic videos within the viewing ranges collected by the cameras arranged oppositely in each commodity area; S213: Physically splice each frame image of each of the basic videos with the frame images corresponding to the acquisition time sequence one by one to obtain each frame of the target image; Wherein, the physical splicing means that the size of the spliced image is the sum of the sizes of all the images participating in the splicing.

4. The intelligent identification method for unmanned vending of similar goods according to claim 1, wherein After the S322, it further includes: S330: Obtain the quantity and corresponding category of the target commodities included in the first result or the second result; S331: Output the target commodity information of the remaining similar commodities in each commodity area of the vending machine according to the quantity and category of the target commodities; Wherein, the target commodity information includes the quantity, category and the commodity area position to which the similar commodities belong.

5. An intelligent identification device for vending similar products without human intervention, characterized in that, The device includes: Data acquisition module: Used to obtain the action category of the current operation of the vending machine and the commodity information of each basic commodity corresponding to the current operation, wherein the action category includes the first action category of a merchant adding commodities to the vending machine and the second action category of a user selecting commodities from the vending machine; Similarity calculation module: Used to determine the similarity between each basic commodity and other commodities in the commodity area of the vending machine according to the commodity information of each basic commodity. Specifically, it is used to: obtain the target images of each basic commodity acquired simultaneously from multiple angles; perform feature extraction on each of the target images to obtain the multi-dimensional feature vectors of the commodity information of each basic commodity; compare the multi-dimensional feature vectors of each basic commodity with the multi-dimensional feature vectors of other category commodities in the vending machine to obtain the target similarity between each basic commodity and other category commodities in the commodity area of the vending machine; Data processing module: Used to output the processing method of the target commodity among the basic commodities whose similarity meets the requirements according to each of the target similarities and the action category; Wherein, the processing methods include: the first processing method corresponding to the vending machine's stocking and the second processing method corresponding to the user's automatic shopping; If the action category is the first action category, the data processing module is further used to: Obtain the first similarity threshold and the second similarity threshold of the similarities of different category commodities, wherein the first similarity threshold is less than the second similarity threshold; Compare the target similarity with the first similarity threshold and / or the second similarity threshold to obtain the first comparison result corresponding to the first action; If the first comparison result is that the target similarity is between the first similarity threshold and the second similarity threshold, then output the target commodity area for placing the target commodity as the first processing method; If the first comparison result is greater than or equal to the second similarity threshold, then output prohibiting the addition of the corresponding target commodity as the first processing method; Wherein, the target commodity area is different from the commodity area where other category commodities similar to the target commodity are located; If the action category is the second action category, and there are different types of commodities in the vending machine whose target similarity is greater than the first similarity threshold and less than the second similarity threshold, or greater than the second similarity threshold, the data processing module is further configured to: Obtain a third similarity threshold for the similarity of different types of commodities, where the third similarity threshold is a dynamically variable similarity value; If the target similarity is greater than the third similarity threshold, obtain the first position information of the commodity areas to which each of the target commodities belongs; According to each of the first position information, output the second processing method corresponding to the position information of the target commodity; Wherein, the second processing method includes: when the first position information of each of the target commodities is the same, directly output the commodity category corresponding to the first position information as the first result; when the first position information of each of the target commodities is different, output the commodity categories of each of the target commodities after manual verification as the second result.

6. An intelligent vending machine, characterized in that, including: At least one processor, at least one memory, and computer program instructions stored in the memory, which implement the method according to any one of claims 1-4 when the computer program instructions are executed by the processor.

7. A storage medium having computer program instructions stored thereon, characterized in that, When the computer program instructions are executed by the processor, the method according to any one of claims 1-4 is implemented.

Images