Vending machine and control method thereof

Abstract

The invention relates to the technical field of vending machines, solves the technical problems that a conventional detection method is inaccurate and a neural network algorithm is low in efficiency and high in cost in the prior art, and particularly relates to a vending machine and a control method thereof. A goods shelf used for storing various commodities and a visual mechanism used for detecting the remaining amount of the commodities are arranged in the shell, the visual mechanism comprises a plurality of cameras used for shooting images of the commodities on the goods shelf, and the bottom ends of the cameras are fixedly connected with a micro-processing device used for analyzing the images of the commodities. According to the method, the three-dimensional coordinates of the reference image are constructed, the features of the reference commodity can be quickly and accurately recognized, the remaining amount of the commodity is obtained according to the number of the features, the commodity category and the commodity number can be recognized, the recognition efficiency is improved, and the recognition accuracy is improved through comparison of geometric features.

Description

TECHNICAL FIELD

[0001] The present application relates to the vending machine technical field, especially to a vending machine and a control method thereof. BACKGROUND

[0002] The vending machine generally refers to a machine capable of automatically dispensing goods after money or a two-dimensional code is input, the vending machine is a common device for commercial automation, which is not limited by time and place, can save labor and facilitate transactions, is a new form of commercial retail, and is also called a 24-hour operating micro-supermarket, in the prior art, the remaining amount of goods is detected by a weight detection or infrared detection method, however, the new vending machine not only has a type of automatically pushing out goods, but also has a type of automatically taking out goods, the weight detection or infrared detection cannot accurately detect the remaining amount of different goods for the latter, the network model of deep learning is used for goods consumption identification and prediction, which is not only high in calculation amount and low in efficiency, but also high in use and maintenance cost, and seriously affects the economic benefit of the vending machine. SUMMARY

[0003] In view of the defects in the prior art, the present application provides a vending machine and a control method thereof, which solves the technical problems of inaccurate and low-efficiency and high-cost conventional detection methods, and achieves the purpose of accurate goods identification, high calculation efficiency and low cost.

[0004] To solve the above technical problems, the present application provides the following technical scheme: a vending machine comprising a shell and a management cabinet, the shell is internally provided with a goods shelf for storing a plurality of goods and a vision mechanism for detecting the remaining amount of goods.

[0005] Preferably, the vision mechanism comprises a plurality of cameras for image shooting of the goods on the goods shelf, the bottom end of the camera is fixedly connected with a micro-processing device for analyzing the image of the goods, and the micro-processing device is fixedly connected to the inner wall of the goods shelf.

[0006] Preferably, the goods shelf comprises a goods outlet box for taking goods, the top end of the goods outlet box is fixedly connected with a goods storage box for storing and displaying goods, and the rear inner wall of the goods storage box is fixedly connected with a screw push rod for pushing the goods.

[0007] Preferably, the management cabinet comprises a cabinet body and a touch display screen for interacting with the user.

[0008] The technical scheme also provides a control method applied to the vending machine, the method comprising the following steps:

[0009] S1, obtain reference images and to-be-sold images of multiple goods in the vending machine, and respectively pre-process the reference images and the to-be-sold images to obtain comparison images and to-be-tested goods images;

[0010] S2, convert the comparison images into three-dimensional images, and construct a three-dimensional coordinate system with any pixel point in the three-dimensional images as the origin to obtain a fitting surface model S of the three-dimensional images;

[0011] S3, convert the fitting surface model S into a surface parameter model P(u, v), and calculate a normal vector E b of a reference point selected on the surface parameter model P(u, v);

[0012] S4, calculate a comparison curvature Bd and a universal curvature Ps of the reference point according to the normal vector E b ;

[0013] S5, obtain a remaining amount U of the to-be-tested goods in the to-be-tested goods images according to the comparison curvature Bd and the universal curvature Ps;

[0014] S6, determine a goods replenishment threshold T based on historical data of the vending machine, and determine a replenishment situation of the to-be-tested goods according to the remaining amount U and the goods replenishment threshold T;

[0015] S7, send a replenishment reminder to the background of the vending machine according to the replenishment situation of the goods.

[0016] Preferably, in step S1, the following steps are implemented:

[0017] S11, select one of each goods in the vending machine as a reference good, and obtain a reference image of the reference good;

[0018] S12, obtain images of to-be-sold goods other than the reference good as to-be-sold images;

[0019] S13, respectively filter the reference images and the to-be-sold images through a median filter to obtain filtered images A and B;

[0020] S14, respectively perform dilation and erosion processing on the filtered images A and B to obtain comparison images and to-be-tested goods images.

[0021] Preferably, in step S2, the following steps are implemented:

[0022] S21, convert the comparison images into three-dimensional images through a depth generation method;

[0023] S22, in the three-dimensional images, take any three-dimensional pixel point as the origin, take the horizontal direction of the plane where the to-be-sold goods are located as the X-axis, take the vertical direction as the Z-axis, and take the direction perpendicular to the X-axis and the Z-axis as the Y-axis to establish a three-dimensional coordinate system;

[0024] S23、acquire three-dimensional coordinates (x d , y e , z l ) of a plurality of pixel points in the three-dimensional image, and establish a preliminary surface model S' of the commodity to be sold in the three-dimensional image, the expression of the preliminary surface model S' being:

[0025] S' = ax d 2 + by e 2 + cz l 2 + 2fx d y e + 2gy e z l + 2hx d z l

[0026] wherein S' represents the preliminary surface model, a, b, c, f, g and h respectively represent coefficients of the preliminary surface model S';

[0027] S24, optimize the preliminary surface model S' by the least square method, and obtain an optimized fitting surface model S, the expression of the fitting surface model S being:

[0028] S = Ax d 2 + By e 2 + Cz l 2 + 2Fx d y e + 2Gy e z l + 2Hx d z l

[0029] wherein S represents the preliminary surface model, A, B, C, F, G and H respectively represent coefficients of the optimized fitting surface model S.

[0030] Preferably, in step S3, the following steps are specifically implemented:

[0031] S31, convert the fitting surface model S into a surface parameter model P(u, v) by the gradient descent method, the expression being:

[0032]

[0033] wherein P(u, v) represents the surface parameter model with parameters u and v, K x (u, v), K y (u, v) and Kz (u, v) represent the function expression of x, y and z coordinates in three-dimensional space, respectively;

[0034] S32, an arbitrary three-dimensional coordinate point in the curved surface parameter model P(u, v) is selected as a reference point;

[0035] S33, the normal vector E of the reference point is calculated b , the calculation formula is:

[0036]

[0037] wherein, E b represents the normal vector of the reference point, Q u and Q v respectively represent the tangent vectors along the u and v parameter directions.

[0038] Preferably, in step S4, the specific implementation steps are as follows:

[0039] S41, the first parameters R, S and T are calculated according to the curved surface parameter model P(u, v), and the expression is:

[0040]

[0041] wherein, R, S and T respectively represent the first parameters, and respectively represent the partial derivatives along the u and v parameter directions in the curved surface parameter model P(u, v);

[0042] S42, the second parameters L, M and N are calculated according to the normal vector E b and the curved surface parameter model P(u, v), and the expression is:

[0043]

[0044] wherein, L, M and N respectively represent the second parameters;

[0045] S43, the comparison curvature Bd and the universal curvature Ps are respectively calculated according to the first parameters and the second parameters, and the calculation formula is:

[0046]

[0047] wherein, Bd represents the comparison curvature, and Ps represents the universal curvature.

[0048] Preferably, in step S5, the specific implementation steps are as follows:

[0049] S51, the image of the to-be-tested commodity is converted into a three-dimensional image by a depth generation method;

[0050] S52, obtain multiple groups of comparison curvatures Bd' and universal curvatures Ps' of multiple commodities in the image of the commodity to be tested, and select one group as a comparison group;

[0051] S53, identify the commodity according to the comparison curvatures Bd, the universal curvatures Ps of the comparison image and the comparison curvatures Bd' and the universal curvatures Ps' of the comparison group;

[0052] If Bd = Bd' and Ps = Ps', mark as the same commodity;

[0053] If Bd ≠ Bd' or Ps ≠ Ps', mark as a distinguished commodity and return to step S52;

[0054] S54, count the number of marks of the same commodity by counting method, and obtain the remaining amount U of the same commodity.

[0055] Preferably, in step S6, the following steps are implemented:

[0056] S61, obtain the consumption speed V and the replenishment time t of the commodity to be tested in the historical data;

[0057] S62, calculate the replenishment threshold T according to the consumption speed V and the replenishment time t, and the calculation formula is:

[0058] T = V × t

[0059] Wherein, T represents the replenishment threshold;

[0060] S63, obtain the interval goods amount W based on the historical data, and calculate the interval time D according to the interval goods amount W, and the calculation formula is:

[0061]

[0062] Wherein, D represents the interval time;

[0063] S64, give the replenishment situation of the commodity to be tested according to the remaining amount U of the commodity to be tested;

[0064] If U ≥ T, the replenishment situation is that the remaining amount is sufficient, waiting for the interval time D and returning to step S63;

[0065] If U < T, the replenishment situation is that the remaining amount is insufficient, and a replenishment reminder is generated and the process is ended.

[0066] By the above technical scheme, the application provides an automatic vending machine and a control method thereof, which at least have the following beneficial effects:

[0067] 1、The present application can greatly improve the accuracy of the later comparison process by preprocessing the reference image and the image to be sold, make the differences and same points of the image more obvious, not only can improve the recognition accuracy, but also can improve the recognition efficiency, save the feature recognition time.

[0068] 2、The present application can quickly and accurately identify the features of the reference goods by constructing the three-dimensional coordinates of the reference image, and quickly find the same goods in the image to be sold through the curved surface features of the goods, and obtain the remaining amount of the goods according to the number of features, which can identify the goods category and the number of goods, and the calculation amount is small, improve the recognition efficiency, and increase the accuracy of the identification through the geometric features.

[0069] 3、After identifying the remaining amount of goods of the vending machine, the present application sets the replenishment threshold by using the consumption speed and replenishment time of the goods, can accurately detect the insufficient remaining amount, and generates a replenishment reminder according to the insufficient remaining amount, so that the vending machine can replenish in time, not only solves the problem of insufficient amount, but also improves the user experience. DETAILED DESCRIPTION

[0070] The drawings described herein are used to provide further understanding of the present application, and form a part of the present application, the illustrative embodiments of the present application and the description thereof are used to explain the present application, and do not constitute improper limitation on the present application. In the drawings:

[0071] Figure 1 It is a flow chart of the control method of the vending machine of the present application;

[0072] Figure 2 It is a perspective view of the inside of the vending machine of the present application;

[0073] Figure 3 It is a partial enlarged view of the visual mechanism of the present application;

[0074] Figure 4 It is a perspective view of the outside of the vending machine of the present application.

[0075] In the figure: 1, shell;2, management cabinet;21, cabinet body;22, touch display screen;3, shelf;31, delivery box;32, storage box;33, screw push rod;4, visual mechanism;41, camera;42, micro processing device. DETAILED DESCRIPTION

[0076] In order to make the above-mentioned purposes, features and advantages of the present application more obvious and easy to understand, the present application will be further described in detail below with reference to the drawings and specific embodiments. The realization process of how to apply technical means to solve technical problems and achieve technical effects of the present application can be fully understood and implemented.

[0077] Please refer to Figure 1 - Figure 4 The embodiment provides an automatic vending machine which can accurately identify goods and has high calculation efficiency and low cost, and the automatic vending machine comprises a shell 1 and a management cabinet 2, the shell 1 is internally provided with a goods shelf 3 for storing various goods and a visual mechanism 4 for detecting the residual amount of goods.

[0078] The visual mechanism 4 comprises a plurality of cameras 41 for image shooting of the goods on the goods shelf 3, the bottom end of the camera 41 is fixedly connected with a micro processing device 42 for analyzing the image of the goods, the micro processing device 42 is fixedly connected on the inner wall of the goods shelf 3, the goods shelf 3 comprises a goods outlet box 31 for taking goods, the top end of the goods outlet box 31 is fixedly connected with a goods storage box 32 for storing and displaying goods, the rear inner wall of the goods storage box 32 is fixedly connected with a screw push rod 33 for pushing the goods to move, and the management cabinet 2 comprises a cabinet body 21 and a touch display screen 22 for interacting with a user.

[0079] The embodiment can shoot pictures or videos in the use process, image recognition and analysis of the goods are performed by using the image of each frame in the video, the goods are placed in the gap of the screw push rod 33 in the goods shelf 3 by workers, when purchase is completed, the goods fall in the goods outlet box 31, the user takes the goods from the goods outlet box 31, the micro processing device 42 detects the goods stored at intervals, and then the remaining goods are counted and a replenishment reminding is sent, the cooperation of the camera 41 and the micro processing device 42 can solve the problem of identification accuracy, the internal program improves the accurate identification of the goods, the remaining amount of goods in the vending machine is counted in real time, the travel time of workers is reduced, the work efficiency is improved, and the user experience is improved.

[0080] Embodiment two

[0081] In the embodiment, the technical problem of low efficiency and high cost of a neural network algorithm due to the inaccuracy of a conventional detection method in the prior art is solved, please refer to Figure 1 The embodiment provides an automatic vending machine control method which can accurately identify goods and has high calculation efficiency and low cost, and the method comprises the following steps.

[0082] S1, reference images and to-be-sold images of various goods in a vending machine are acquired, the reference images and the to-be-sold images are respectively preprocessed to obtain comparison images and to-be-detected goods images; the reference images are used for comparison of goods in a goods shelf, the reference images can provide a reference standard, and the problem of identification accuracy is solved, in step S1, the following specific implementation steps are implemented.

[0083] S11, select one of each commodity in the vending machine as a reference commodity, and obtain a reference image of the reference commodity; select a commodity with a relatively complete shape in the same commodity as the reference commodity, and obtain an image thereof.

[0084] S12, obtain the image of the to-be-sold commodity other than the reference commodity as a to-be-sold image; for the to-be-sold commodity inside the vending machine, the image of all to-be-sold commodities can be directly obtained, or the image of the to-be-sold commodity can be obtained layer by layer on the shelf, thereby improving the accuracy of image recognition and reducing the calculation amount, and the present application uses a layer-by-layer shooting method for recognition.

[0085] S13, filter the reference image and the to-be-sold image through a median filter to obtain a filtered image A and a filtered image B; the present application can obtain the desired result through the median filter, without the need for additional steps, and the median filter is a common filtering method, which will not be described here.

[0086] S14, dilate and erode the filtered image A and the filtered image B to obtain a comparison image and a to-be-measured commodity image; dilate and erode processing can make the contrast of the image clearer, so that the subsequent steps can obtain clearer results, and this step only needs to dilate and erode to complete the processing of the image, without the need for additional steps, and dilate and erode is a common method, which will not be described here. Through the preprocessing of the reference image and the to-be-sold image, the accuracy in the later comparison process is greatly improved, the differences and similarities of the images are more obvious, the recognition accuracy is improved, the recognition efficiency is improved, and the feature recognition time is saved.

[0087] S2, convert the comparison image into a three-dimensional image, and construct a three-dimensional coordinate system with any pixel point in the three-dimensional image as the origin to obtain a fitting surface model S of the three-dimensional image; in image recognition, the feature recognition accuracy is low due to direct recognition of two-dimensional images, and the cost of hardware devices is large using the three-dimensional point cloud method. In step S2, the specific implementation steps are as follows:

[0088] S21, convert the comparison image into a three-dimensional image by a depth generation method; the depth generation method can directly convert a two-dimensional image into a three-dimensional image, which is convenient for calculating the pixels of the three-dimensional image in the later steps. This step only needs the depth generation method, without the need for additional steps, and the depth generation method is a common method, which will not be described here.

[0089] S22, establish a three-dimensional coordinate system with any three-dimensional pixel point as the origin, with the horizontal direction of the plane where the to-be-sold commodity is located as the X axis, with the vertical direction as the Z axis, and with the direction perpendicular to the X axis and the Z axis as the Y axis;

[0090] S23. Obtain the three-dimensional coordinates (x, y) of multiple pixels in a three-dimensional image. d y e , z l And establish a preliminary surface model S' of the product to be sold in the 3D image. The expression of the preliminary surface model S' is:

[0091] S' = ax d 2 +by e 2 +cz l 2 +2fx d y e +2gy e z l +2hx d z l

[0092] Where S' represents the preliminary surface model, and a, b, c, f, g, and h represent the coefficients of the preliminary surface model S'. For the reference product, a surface is first constructed for that product. Intentional surface features are obtained through surface fitting. Since the preliminary surface model S' contains multiple parameters, at least six points need to be obtained for fitting in this step. If a parameter containing x is added to the preliminary surface model S'... d y e , z l For each item, the number of selected pixels also needs to be increased. In this invention, six items are used as an example for fitting.

[0093] S24. The initial surface model S' is optimized using the least squares method to obtain the optimized fitted surface model S. The expression for the fitted surface model S is:

[0094] S = Ax d 2 +By e 2 +Cz l 2 +2Fx d y e +2Gy e z l +2Hx d z l

[0095] Wherein, S represents the preliminary surface model, A, B, C, F, G and H represent the coefficients of the fitted surface model S after optimization respectively, after the fitted surface is obtained, since the accuracy of the fitted surface is poor, it is necessary to optimize the fitted surface, the optimization of the surface can be completed by the least square method, without additional steps, and the least square method is a common fitting surface processing method, which will not be described here, the method for obtaining the three-dimensional fitting surface through three-dimensional coordinates can not only facilitate the calculation of feature points, but also increase the accuracy of feature extraction, without the need for expensive three-dimensional point cloud acquisition equipment, high economic practicability, and cost saving.

[0096] S3, convert the fitted surface model S into a surface parameter model P(u, v), and calculate the normal vector E of the selected reference point on the surface parameter model P(u, v) b Before calculating the surface feature, the normal vector of a point on the surface needs to be calculated, in step S3, the specific implementation steps are as follows:

[0097] S31, convert the fitted surface model S into a surface parameter model P(u, v) by gradient descent method, the expression is:

[0098]

[0099] Wherein, P(u, v) represents the surface parameter model with parameters u and v, K x (u, v), K y (u, v) and K z (u, v) represent the function expressions of x, y and z coordinates in three-dimensional space respectively; before calculating the normal vector, the fitted surface model S is first converted into the parameter model P(u, v), which is more convenient and fast in calculation, and the expression is also more simple.

[0100] S32, select an arbitrary three-dimensional coordinate point as a reference point in the surface parameter model P(u, v); select a three-dimensional coordinate point on the surface as a reference point for calculation, and the calculated feature is the feature around the reference point.

[0101] S33, calculate the normal vector E b of the reference point, the calculation formula is:

[0102]

[0103] Wherein, E b represents the normal vector of the reference point, Q u and Q vrespectively represent tangent vectors along u, v parameter directions, through the calculation of the reference point normal vector, the curvature can be calculated in the subsequent steps, the reference point can be selected according to the actual situation in the calculation, in practice, multiple reference points are selected to calculate multiple sets of characteristics to increase the accuracy of the same kind of commodity identification, through the construction of the three-dimensional coordinates of the reference image, the characteristics of the reference commodity can be quickly and accurately identified, this method greatly saves the amount of calculation, not only can make the feature recognition quickly complete, improve the identification efficiency.

[0104] S4、according to the normal vector E b The comparison curvature Bd and the universal curvature Ps of the reference point are calculated, the fitting and transformation of the fitting surface model S and the calculation of the normal vector E are completed b After the calculation of the normal vector E, the characteristics of the commodity need to be further extracted, in step S4, the specific implementation steps are as follows:

[0105] S41, according to the surface parameter model P(u, v), the first parameters R, S and T are calculated, the expression is:

[0106]

[0107] Wherein, R, S and T represent the first parameters, And Respectively represent the partial derivative of the surface parameter model P(u, v) along the u, v parameter direction; according to the surface parameter model P(u, v), the first parameters R, S and T are obtained through the calculation of the parameters, which is convenient for the calculation of the subsequent steps.

[0108] S42, according to the normal vector E b And the surface parameter model P(u, v), the second parameters L, M and N are calculated, the expression is:

[0109]

[0110] Wherein, L, M and N represent the second parameters;

[0111] S43, according to the first parameter and the second parameter, the comparison curvature Bd and the universal curvature Ps are calculated, the calculation formula is:

[0112]

[0113] Wherein, Bd represents the comparative curvature, Ps represents the universal curvature, in the identification of the surface, the comparative curvature Bd and the universal curvature Ps can obviously show the characteristics of the surface, quickly distinguish the different goods from other goods, and the goods with the same surface characteristics must be the same kind of goods, and the number of the same kind of goods can also be determined according to the number of characteristics, that is, the remaining amount in the vending machine, through the surface characteristics of the goods, the same goods in the image to be sold are quickly found, and the remaining amount of the goods is obtained according to the number of characteristics, this method greatly saves the calculation amount, not only can make the feature recognition complete quickly, improve the recognition efficiency, but also increase the accuracy of identification by comparing the geometric characteristics.

[0114] S5, obtaining the remaining amount U of the goods to be tested in the image to be tested according to the comparative curvature Bd and the universal curvature Ps; after calculating the characteristics of the surface comparative curvature Bd and the universal curvature Ps, the goods need to be detected, in step S5, the specific implementation steps are as follows:

[0115] S51, converting the image of the goods to be tested into a three-dimensional image by the depth generation method; in this step, the conversion of the three-dimensional image can be completed by the depth generation method without additional steps, and the depth generation method is a common three-dimensional image conversion method, which will not be described here.

[0116] S52, obtaining a plurality of groups of comparative curvatures Bd' and universal curvatures Ps' of a plurality of goods in the image of the goods to be tested, and selecting one group as a comparison group; since there are a plurality of goods in the image of the goods to be tested, in order to identify the type of goods and the number of goods, in this step, the comparative images of a plurality of goods need to be extracted, and the comparative curvatures Bd and the universal curvatures Ps corresponding to a plurality of goods are obtained, the implementation steps are the same as steps S2-S4, then the number of characteristics is identified and obtained according to the comparative curvatures Bd and the universal curvatures Ps of different goods, in order to realize the subsequent steps.

[0117] S53, identifying the goods according to the comparative curvatures Bd, the universal curvatures Ps of the comparative images and the comparative curvatures Bd' and the universal curvatures Ps' of the comparison group;

[0118] If Bd=Bd' and Ps=Ps', it is marked as the same kind of goods;

[0119] If Bd≠Bd' or Ps≠Ps', it is marked as different goods and returns to step S52;

[0120] S54, count the number of the same goods by counting method, and obtain the remaining amount U of the same goods. After identifying the same goods, the number of the same goods can be counted by counting method. Since counting method is simple, convenient, not prone to error, and very efficient, it is a fast counting method, which will not be described here. By identifying and counting, the goods category and the number of goods can be identified, the calculation amount is small, the identification efficiency is improved, and the identification accuracy is increased by comparing the geometric features.

[0121] S6, determine the goods replenishment threshold T based on the historical data of the vending machine, and determine the replenishment situation of the to-be-tested goods according to the remaining amount U and the goods replenishment threshold T; after obtaining the remaining amount of the same goods, the threshold of the remaining amount needs to be set. In step S6, the specific implementation steps are as follows:

[0122] S61, obtain the consumption speed V and the replenishment time t of the to-be-tested goods in the historical data; the consumption speed V and the replenishment time t can know how much the goods are consumed, which is convenient for setting the threshold.

[0123] S62, calculate the replenishment threshold T according to the consumption speed V and the replenishment time t, and the calculation formula is:

[0124] T = V x t

[0125] Wherein, T represents the replenishment threshold;

[0126] S63, obtain the interval goods amount W based on the historical data, and calculate the interval time D according to the interval goods amount W, and the calculation formula is:

[0127]

[0128] Wherein, D represents the interval time;

[0129] S64, give the replenishment situation of the to-be-tested goods according to the remaining amount U of the to-be-tested goods;

[0130] If U≥T, the replenishment situation is that the remaining amount is sufficient, the interval time D is waited and returned to step S63;

[0131] If U < T, the replenishment situation is that the remaining amount is insufficient, a replenishment reminder is generated and the process is ended. This step needs to detect the replenishment of each kind of goods to ensure that all goods are in a state of having goods to buy, avoid the situation that the vending machine sells out of goods, and affect the user's use experience. After identifying the remaining amount of goods in the vending machine, the consumption speed and the replenishment time of the goods are used to set the replenishment threshold, which can accurately detect the insufficient remaining amount, generate a replenishment reminder according to the insufficient remaining amount, so that the vending machine can replenish in time, not only solve the problem of insufficient remaining amount, but also improve the user's experience.

[0132] S7, according to the replenishment of goods to the background of the vending machine to send a replenishment reminder, vending machine background after receiving the replenishment reminder, can quickly notify the staff to replenish, avoid the vending machine in the goods appear sold out state.

[0133] Those skilled in the art can understand that all or part of the steps in the above-mentioned embodiment methods can be completed by a program instructing relevant hardware, therefore, the application can adopt a completely hardware embodiment, a completely software embodiment or an embodiment combining software and hardware aspects. Moreover, the application can adopt a computer program product in the form of being implemented on one or more computer usable storage media (including but not limited to disk memory, CD-ROM, optical memory, etc.) containing computer usable program codes.

[0134] The above embodiments are used to introduce the application in detail, and the principles and embodiments of the application are described by applying specific examples. The above embodiment is only used to help understand the method and core idea of the application; meanwhile, for those skilled in the art, according to the idea of the application, the specific embodiment and application range will be changed, and the above description should not be understood as limiting the application.

Claims

1. An automatic vending machine, comprising a casing (1) and a management cabinet (2), characterized in that, The outer shell (1) is equipped with a shelf (3) for storing various goods and a vision mechanism (4) for detecting the remaining quantity of goods. The vision mechanism (4) includes multiple cameras (41) that capture images of goods on the shelf (3). The bottom of each camera (41) is fixedly connected to a microprocessor (42) for analyzing the images of the goods. The microprocessor (42) is fixedly connected to the inner wall of the shelf (3).

2. The automatic vending machine device according to claim 1, characterized in that, The shelf (3) includes a delivery box (31) for taking out goods, and a storage box (32) for storing and displaying goods is fixedly connected to the top of the delivery box (31). A spiral push rod (33) for pushing the goods is fixedly connected to the inner rear wall of the storage box (32).

3. The automatic vending machine device according to claim 1, characterized in that, The management cabinet (2) includes a cabinet body (21) and a touch screen display (22) for user interaction.

4. A control method for an automatic vending machine according to any one of claims 1-3, characterized in that, The method includes the following steps: S1. Obtain reference images and images to be sold for various products in the vending machine, and preprocess the reference images and images to be sold to obtain comparison images and images of products to be tested; S2. Convert the comparison image into a three-dimensional image, and construct a three-dimensional coordinate system with any pixel in the three-dimensional image as the origin to obtain the fitted surface model S of the three-dimensional image. S3. Transform the fitted surface model S into a surface parametric model P(u, v), and calculate the normal vector E of the selected reference point on the surface parametric model P(u, v). b ; S4, based on the normal vector E b Calculate the alignment curvature Bd and universal curvature Ps of the reference point; S5. Obtain the remaining quantity U of the product under test from the image of the product under test based on the comparison curvature Bd and the universal curvature Ps. S6. Determine the product replenishment threshold T based on the historical data of the vending machine, and determine the replenishment status of the product to be tested based on the remaining quantity U and the product replenishment threshold T. S7. Send a restocking reminder to the vending machine's backend based on the restocking status of the goods.

5. The control method according to claim 4, characterized in that, In step S1, the specific implementation steps are as follows: S11. Select one item from each type of product in the vending machine as a reference product and obtain a reference image of the reference product; S12. Obtain images of the products to be sold, excluding the reference product, as images to be sold; S13. The reference image and the image to be sold are respectively filtered by a median filter to obtain filtered image A and filtered image B; S14. Perform dilatation and erosion processing on filtered image A and filtered image B respectively to obtain comparison image and product image to be tested.

6. The control method according to claim 4, characterized in that, In step S2, the specific implementation steps are as follows: S21. Convert the compared images into three-dimensional images using a depth generation method; S22. In a three-dimensional image, a three-dimensional coordinate system is established with any three-dimensional pixel as the origin, the horizontal direction of the plane where the goods to be sold are located as the X-axis, the vertical direction as the Z-axis, and the direction that is perpendicular to both the X-axis and the Z-axis as the Y-axis. S23. Obtain the three-dimensional coordinates (x, y) of multiple pixels in a three-dimensional image. d y e , z l And establish a preliminary surface model S' of the product to be sold in the 3D image. The expression of the preliminary surface model S' is: S'=ax d 2 +by e 2 +cz l 2 +2fx d y e +2gy e z l +2hx d z l Where S' represents the preliminary surface model, and a, b, c, f, g, and h represent the coefficients of the preliminary surface model S', respectively. S24. The initial surface model S' is optimized using the least squares method to obtain the optimized fitted surface model S. The expression for the fitted surface model S is: S=Ax d 2 +By e 2 +Cz l 2 +2Fx d y e +2Gy e z l +2Hx d z l Where S represents the initial surface model, and A, B, C, F, G and H represent the coefficients of the optimized fitted surface model S.

7. The control method according to claim 6, characterized in that, In step S3, the specific implementation steps are as follows: S31. The fitted surface model S is transformed into a surface parametric model P(u, v) using the gradient descent method, with the following expression: Where P(u, v) represents the surface parametric model with parameters u and v, and K x (u, v), K y (u, v) and K z (u, v) represent the function expressions for the x, y, and z coordinates in three-dimensional space, respectively; S32. Arbitrarily select a three-dimensional coordinate point in the surface parameter model P(u, v) as a reference point; S33. Calculate the normal vector E of the reference point. b The calculation formula is: Among them, E b Q represents the normal vector of the reference point. u and Q v Let represent the tangent vectors along the directions of parameters u and v, respectively.

8. The control method according to claim 4, characterized in that, In step S4, the specific implementation steps are as follows: S41. Calculate the first parameters R, S, and T according to the surface parameter model P(u, v), and the expression is: Where R, S, and T represent the first parameter, and Let represent the partial derivatives along the u and v parameter directions in the surface parametric model P(u, v), respectively; S42, According to the normal vector E b The second parameters L, M, and N are calculated using the surface parametric model P(u, v), and the expression is as follows: where L, M, and N respectively represent the second parameters; S43. Calculate the comparison curvature Bd and the general curvature Ps respectively according to the first parameters and the second parameters, and the calculation formula is: where Bd represents the comparison curvature and Ps represents the general curvature.

9. The control method according to claim 4, characterized in that, In step S5, the specific implementation steps are as follows: S51. Convert the image of the commodity to be measured into a three-dimensional image by the depth generation method; S52. Obtain multiple groups of comparison curvatures Bd' and general curvatures Ps' of multiple commodities in the image of the commodity to be measured, and select a group as the comparison group; S53. Identify the commodity according to the comparison curvature Bd, general curvature Ps of the comparison image and the comparison curvature Bd' and general curvature Ps' of the comparison group; If Bd = Bd' and Ps = Ps', it is marked as the same commodity; If Bd ≠ Bd' or Ps ≠ Ps', it is marked as a different commodity and return to step S52; S54. Statistically count the marked quantity of the same commodity by the counting method and obtain the remaining quantity U of the same commodity.

10. The control method according to claim 4, characterized in that, In step S6, the specific implementation steps are as follows: S61. Obtain the consumption speed V and replenishment time t of the commodity to be measured from historical data; S62. Calculate the replenishment threshold T according to the consumption speed V and replenishment time t, and the calculation formula is: T = V × t where T represents the replenishment threshold; S63. Obtain the inventory gap quantity W based on historical data, and calculate the interval time D according to the inventory gap quantity W, and the calculation formula is: where D represents the interval time; S64. Give the replenishment situation of the commodity to be measured according to the remaining quantity U of the commodity to be measured; If U ≥ T, the replenishment situation is that the remaining quantity is sufficient, wait for the interval time D and return to step S63; If U < T, the replenishment situation is that the remaining quantity is insufficient, generate a replenishment reminder and end.

Images