Management methods, equipment and storage media for vending systems

Abstract

This application discloses a management method, device, and storage medium for a vending system. This application relates to the field of general control system technology. The management method of the vending system includes: determining the quantity of goods in response to the first step value of the encoder in the shelf pusher; acquiring a first user feature that triggers the encoder based on the image acquisition module of the shelf pusher; encapsulating the quantity of goods, the first user feature, and the product identifier associated with the shelf pusher into sales data, and sending the sales data to the checkout terminal; acquiring a second user feature in response to a departure target detected by the security door; and issuing an alarm message if at least one target sales data associated with the user corresponding to the second user feature has not been settled. This application can achieve the technical effect of improving the anti-theft level of supermarkets.

Description

TECHNICAL FIELD

[0001] The present application relates to the general control system technical field, and particularly relates to a management method, device and storage medium of a vending system. BACKGROUND

[0002] A supermarket settles accounts by uploading a commodity barcode by a user or scanning a commodity barcode by a salesperson. Specifically, after a customer selects a commodity, the customer scans a commodity barcode by a settlement device in the supermarket or a personal mobile terminal, so that the system can quickly identify the commodity and add the commodity to a shopping list for total price accounting. Then the customer walks out of the supermarket after settlement.

[0003] However, this way also has certain limitations, for example, the customer walks out of the supermarket without settlement, and it is impossible to accurately know how many commodities the customer has not settled. SUMMARY

[0004] The main purpose of the present application is to provide a management method, device and storage medium of a vending system, and aims to solve the technical problem that a customer walks out of a supermarket without settlement, and it is impossible to accurately know how many commodities the customer has not settled.

[0005] To achieve the above purpose, the present application provides a management method of a vending system, the vending system comprising a shelf pusher and a settlement terminal, and the management method of the vending system comprising the following steps.

[0006] Determining a commodity quantity in response to a first step value of an encoder in the shelf pusher;

[0007] Collecting a first user feature triggering the encoder based on an image collection module of the shelf pusher;

[0008] Packaging the commodity quantity, the first user feature and a commodity identifier associated with the shelf pusher into sales data, and sending the sales data to the settlement terminal;

[0009] Obtaining a second user feature in response to a leaving target detected by a safety door;

[0010] If at least one target sales data associated with the second user feature corresponds to a user who has not settled, an alarm information is sent.

[0011] In an embodiment, the step of determining the commodity quantity in response to the first step value of the encoder in the shelf pusher comprises the following steps.

[0012] Establishing a corresponding relationship between the step value of the encoder and the commodity quantity based on mechanical structure parameters of the shelf pusher, commodity placement rules and encoder accuracy;

[0013] The first step value of the encoder is received, and the corresponding relationship is used to determine the quantity of the goods, wherein, if the first step value is between two adjacent step values, the quantity of the goods is calculated by linear interpolation.

[0014] In an embodiment, the image acquisition module based on the shelf pusher acquires the first user feature triggering the encoder, including:

[0015] If the first step value is detected, a trigger signal is sent to the image acquisition module;

[0016] The image acquisition module receives the trigger signal, captures an initial image of a preset area of the shelf pusher, and determines the initial image, wherein the preset area is set to the range of the user taking the goods;

[0017] The user facial feature is extracted from the initial image as the first user feature.

[0018] In an embodiment, the vending system further includes a general image acquisition device, and after the step of extracting the user facial feature from the initial image as the first user feature, including:

[0019] If the user facial feature is missing, a first body feature and a first clothing feature are obtained;

[0020] The general image acquisition device is called to acquire a second body feature and a second clothing feature of the preset area;

[0021] The first body feature, the first clothing feature, the second body feature, and the second clothing feature are spliced into the first user feature.

[0022] In an embodiment, the step of associating the quantity of the goods, the first user feature, and the goods identifier of the shelf pusher into sales data, and sending the sales data to the settlement terminal, including:

[0023] According to a preset data format, the quantity of the goods and the goods identifier are associated as goods data;

[0024] The goods data and the first user feature are spliced into a data body of the sales data;

[0025] A data header containing data type and time stamp is added to the data body;

[0026] The encapsulated sales data is sent to the settlement terminal through a communication module.

[0027] In an embodiment, after the step of associating the commodity quantity, the first user feature, and the commodity identification of the shelf pusher, packaging the sales data, and sending the sales data to the settlement terminal, the method comprises:

[0028] In response to a triggering operation of the settlement terminal, obtaining a second user feature;

[0029] Based on at least one target sales data matched by the second user feature, generating and displaying a settlement interface for user settlement; wherein, the step of obtaining a second user feature in response to a triggering operation of the settlement terminal comprises:

[0030] If the settlement terminal detects a user triggering operation, calling a feature acquisition module of the settlement terminal;

[0031] Based on image acquisition of the settlement area by the feature acquisition module, obtaining a user image;

[0032] Based on the user image, extracting user facial feature points to generate the second user feature.

[0033] In an embodiment, after the step of extracting user facial feature points based on the user image to generate the second user feature, the method comprises:

[0034] If the corresponding face of the user image is blocked or blurred, extracting a third body feature and a third clothing feature;

[0035] Splicing the third body feature and the third clothing feature into the second user feature.

[0036] In an embodiment, after the step of generating and displaying a settlement interface for user settlement based on at least one target sales data matched by the second user feature, the method comprises:

[0037] Comparing the second user feature with the first user feature in the stored sales data one by one to determine the similarity of the second user feature to each of the sales data;

[0038] Filtering out the sales data with a similarity higher than a similarity threshold to determine as target sales data.

[0039] Summarizing the quantity and corresponding price of each type of commodity in the target sales data to obtain a summary result.

[0040] According to the summary result, generating the settlement interface according to a preset interface template;

[0041] Displaying the settlement interface on the display module of the settlement terminal for user settlement.

[0042] In addition, to achieve the above object, the application further provides a management device of a vending system, which comprises a memory, a processor and a computer program stored on the memory and executable on the processor, and the computer program is configured to implement the steps of the management method of the vending system.

[0043] In addition, to achieve the above object, the application further provides a storage medium, which is a computer readable storage medium, and a program for implementing the management method of the vending system is stored on the computer readable storage medium, and the program is executed by a processor to implement the steps of the management method of the vending system.

[0044] The application provides a management method of a vending system, which determines a number of goods in response to a first step value of an encoder in a shelf pusher, collects a first user feature triggering the encoder based on an image collection module of the shelf pusher, encapsulates the number of goods, the first user feature and a product identifier associated with the shelf pusher into sales data, and sends the sales data to a settlement terminal, acquires a second user feature in response to a triggering operation of the settlement terminal, generates and displays a settlement interface based on at least one target sales data matched by the second user feature, and settles the goods for the user. The technical problem that the barcode may be damaged or the scanning environment may be poor, which may cause the scanning to fail and affect the settlement efficiency is solved, and the technical effect of improving the payment efficiency of the supermarket user is achieved. BRIEF DESCRIPTION OF DRAWINGS

[0045] The accompanying drawings, which are incorporated into and form part of the specification, illustrate embodiments consistent with the application and, together with the specification, serve to explain the principles of the application.

[0046] In order to more clearly illustrate the technical solutions in the embodiments of the application or the prior art, the accompanying drawings needed in the embodiments or prior art description will be briefly introduced as follows. Obviously, for those skilled in the art, other drawings can also be obtained based on these drawings without creative labor.

[0047] Figure 1 A flowchart is provided for the first embodiment of the management method of the vending system of the application;

[0048] Figure 2 A flowchart is provided for the third embodiment of the management method of the vending system of the application;

[0049] Figure 3 A flowchart is provided for the sixth embodiment of the management method of the vending system of the application;

[0050] Figure 4The hardware structure schematic diagram involved in the management device of the vending system of the present application.

[0051] The object implementation, functional features and advantages of the present application will be further described with reference to the embodiments and the accompanying drawings. DETAILED DESCRIPTION

[0052] It should be understood that the specific embodiments described herein are merely intended to explain the technical solutions of the present application, and are not intended to limit the present application.

[0053] In order to better understand the technical solutions of the present application, the specific embodiments will be described in detail below with reference to the accompanying drawings and specific embodiments.

[0054] At present, when the bar code is contaminated or the scanning environment is poor, it may cause the code scanning to fail, affecting the settlement efficiency.

[0055] The main solution of the present application is: in response to the first step value of the encoder in the shelf pusher, the number of goods is determined; based on the image acquisition module of the shelf pusher, the first user feature triggering the encoder is collected; the number of goods, the first user feature and the goods identification associated with the shelf pusher are packaged into sales data, and the sales data is sent to the settlement terminal; in response to the triggering operation of the settlement terminal, the second user feature is obtained; based on at least one target sales data matched by the second user feature, a settlement interface is generated and displayed for user settlement, which realizes the technical effect of improving the payment efficiency of supermarket users.

[0056] It should be noted that the execution subject of the present embodiment can be a vending system, or a computing service device with data processing, network communication and program running functions, such as a tablet computer, a personal computer, a mobile phone, etc., or a vending system management device capable of realizing the above functions, etc. The present embodiment does not make specific limitation thereon. The present embodiment and the following embodiments will be described below taking the vending system as the execution subject.

[0057] Based on this, the present application proposes a management method of a vending system of the first embodiment, which comprises a shelf pusher and a settlement terminal, please refer to Figure 1 , the management method of the vending system comprises steps S10-S50:

[0058] Step S10, in response to the first step value of the encoder in the shelf pusher, the number of goods is determined.

[0059] In the present embodiment, the encoder is a sensor installed on the transmission mechanism of the shelf pusher, which is used to record the displacement change of the shelf pusher; the shelf pusher is a mechanical device used to automatically push goods to the pickup port, which usually contains a conveyor belt, a gear set and other components.

[0060] As an optional embodiment, the system establishes a corresponding relationship table between the encoder step value and the product quantity in advance through mechanical parameter calibration and actual test. When the user takes away the product from the shelf, the shelf pusher moves forward due to the reduction of the product, and the encoder generates a corresponding step value signal. After the system receives the signal, it first queries the corresponding relationship table. If the step value is within the preset interval, it directly matches the corresponding product quantity; if it is between two intervals, it calculates the product quantity through a linear interpolation algorithm, and checks the result for non-negative integer and upper limit of inventory.

[0061] For example, the encoder of a certain beverage shelf corresponds to the displacement of 1 bottle of beverage for every 10 step values. When the user takes away 2 bottles of beverage, the encoder generates 22 step values, and the system calculates (22-20) / (30-20) x 1+2=2.2 through linear interpolation, and after verification, takes the integer as 2 bottles. This embodiment collects the displacement change of the shelf pusher in real time through the encoder, accurately calculates the product quantity combined with the preset corresponding relationship, and provides a basic quantitative basis for the generation of subsequent sales data.

[0062] Step S20, based on the image acquisition module of the shelf pusher, acquires the first user feature triggering the encoder.

[0063] In this embodiment, the image acquisition module is a camera assembly integrated above or beside the shelf pusher, which is used to capture the image of the taking area; the user features include but are not limited to facial features, body features, clothing features, and other visual information that can be used to distinguish the identity of the user. As an optional embodiment, when the encoder detects the change of step value (i.e. the product is taken away), a trigger signal is immediately sent to the image acquisition module. The image acquisition module starts shooting within 200 milliseconds after receiving the signal, and acquires the image of the area within 1.5 meters in front of the shelf, and then pre-processes the original image by denoising and brightness adjustment, and then identifies the user subject in the image through a target detection algorithm, and extracts feature information such as facial key point coordinates and clothing color texture.

[0064] For example, when user A takes away the chips on the shelf, the encoder triggers the image acquisition module, and the camera captures the back of user A wearing a blue jacket, and the system extracts the jacket color, body proportion and other body features as the first user feature. This embodiment records the visual features of the user taking the goods in real time through the image acquisition module, establishes the association between user behavior and physical features, and provides data support for user identity matching in subsequent settlement.

[0065] Step S30, encapsulating the product quantity, the first user feature, and the product identifier associated with the shelf pusher into sales data, and sending the sales data to the settlement terminal.

[0066] In the embodiment, the commodity identifier is a code for uniquely identifying a commodity, such as information in a bar code, a two-dimensional code, or an RFID tag; and the sales data is a structured data set containing commodity information, user features, and transaction-related data. As an optional implementation, the system encapsulates the data in JSON format, integrates the number of commodities, the first user feature (such as a face feature vector), and the commodity identifier (such as a bar code string) into a data body, adds a data header containing the data type (sales data) and a timestamp (accurate to the millisecond), and sends the data to the checkout terminal through a wireless communication protocol such as Wi-Fi or Bluetooth. After sending, a timeout retransmission mechanism is started, and if no confirmation signal is received within 3 seconds, the data is re-sent, with a maximum of 3 retransmissions. For example, a user takes away 2 bottles of mineral water with a bar code of 6901234567890, and the system encapsulates the number of commodities “2”, the user feature “face feature vector V1”, and the commodity identifier “6901234567890” into JSON data, and sends the data to the checkout terminal through Wi-Fi. The embodiment ensures that the information about taking goods is synchronized to the checkout terminal in real time and accurately by standardized data encapsulation and reliable transmission mechanism, and provides a complete data source for the subsequent settlement process.

[0067] In step S40, a second user feature is acquired in response to a departure target detected by the security door.

[0068] In the embodiment, the security door is a device arranged at the exit of a supermarket and used to collect features of personnel leaving the supermarket. The departure target refers to personnel leaving the supermarket. The second user feature refers to a user feature collected by the security door for the departure target.

[0069] In step S50, if the second user feature corresponds to at least one target sales data associated with the user and not settled, an alarm information is issued.

[0070] In the embodiment, the matched first user feature is determined according to the second user feature, and the sales data associated with the first user feature is acquired. If there is unsettled sales data, the unsettled sales data is determined as the target sales data. An alarm information is output based on the target sales data to prompt the security personnel to check.

[0071] Optionally, after step S30, the method further comprises:

[0072] In step S60, a second user feature is acquired in response to a trigger operation of the checkout terminal.

[0073] In the embodiment, the checkout terminal is a device used by the user to perform a checkout operation, and usually includes a touch screen, a camera, a payment module, etc. The trigger operation includes interactive behaviors such as the user clicking a “start settlement” button, and sensing that the user stays in the settlement area for more than 5 seconds.

[0074] As an optional implementation, when the settlement terminal detects a triggering operation, the built-in camera is immediately started to collect images of the settlement area, with a resolution of 1920x1080 and a frame rate of 30 fps. The system performs face detection on the collected images, and if a clear face is detected, 68 facial key point features are extracted; if the face is obscured, clothing color, hairstyle, and other auxiliary features are extracted, and the feature data is normalized and converted into a unified dimension feature vector. For example, a user stands in front of the settlement terminal and clicks the "settlement" button, the terminal camera captures the user's front image, and the system extracts facial features such as the left eye pupil coordinates and nose bridge height to generate a second user feature vector F2. This embodiment collects the feature information of the current user in real time through the settlement terminal, establishes a real-time association between the settlement behavior and the user features, and provides a key comparison basis for subsequent sales data matching.

[0075] Step S70, based on at least one target sales data matched by the second user feature, a settlement interface is generated and displayed for the user to settle.

[0076] In this embodiment, the target sales data refers to the historical take-out records matched with the current settlement user features; and the settlement interface is an interactive interface that displays the user's purchased goods information and payment options.

[0077] As an optional implementation, the system performs cosine similarity calculation on the second user feature and the first user feature in all un-settled sales data, sets a threshold of 0.8, and filters out sales data with a similarity higher than the threshold. The matched multiple sales data are summarized, the number is merged by commodity category, the total price is calculated, and a settlement interface containing the commodity name, picture, unit price, quantity, total price, and payment method (WeChat, Alipay, cash) is generated according to a preset template and displayed on the settlement terminal screen.

[0078] For example, the similarity between the second user feature F2 of user A and the first user feature V1 collected in step S20 is 0.85, and the system matches the sales data of 2 bottles of mineral water and 1 bag of potato chips taken by user A, and generates a settlement interface with a total price of 15 yuan after summarizing. The user completes the settlement by clicking Alipay payment. This embodiment precisely associates the user's take-out records with the settlement behavior through the feature matching algorithm, automatically generates a settlement interface containing complete commodity information, realizes self-service settlement closed loop in unattended scenarios, effectively solves the problem of "difficult traceability of goods not settled" in traditional solutions, and improves the system anti-diversion capability and user settlement efficiency.

[0079] As an example of a manned supermarket. When the user enters the supermarket, the user image or clothing features of the user are obtained, and the first user features of the user are established. When the user takes the goods of the shelf propeller, the shelf propeller calls the image acquisition module to acquire the user image, and then obtains the first user features, encapsulates the number of goods and the goods identification into the sales data, and associates with the first user features. Each time the user takes the goods of the shelf propeller, it is like the above operation. When the user goes to the settlement terminal for settlement, the staff controls the settlement terminal to obtain the user image to extract the second user features. Then, according to the second user features, the target sales data is matched to generate a settlement interface for the user to pay for settlement.

[0080] Further, the staff can also control the settlement terminal to scan any one of the goods identification, and then obtain the sales data associated with the goods identification, and then obtain the first user features associated with the sales data. And obtain all the sales data associated with the first user features, that is, the target sales data. Then, according to the target sales data, a settlement interface is generated for the user to pay for settlement.

[0081] Optionally, the vending system further comprises an alarm device, after step S30, comprising: setting an image acquisition module at the supermarket security door exit, if it is detected that the user leaves from the security door, the second user features of the user are acquired. Obtain the target sales data associated with the second user features, and determine whether the target sales data is settled. If the target sales data is not settled, generate an alarm information based on the unsettled target sales data, and send it to the terminal of the staff through the alarm device to prompt the staff that there is unsettled goods. Further, the technical effect of supermarket anti-theft is realized.

[0082] Optionally, the vending system further comprises an alarm device, after step S70, comprising: obtaining the settlement information of the user, if the settlement information does not match the target sales data, generating an alarm information based on the alarm device to prompt the staff to check.

[0083] The present application responds to the first step value of the encoder in the shelf propeller to determine the number of goods; based on the image acquisition module of the shelf propeller, the first user features triggering the encoder are acquired; the number of goods, the first user features and the goods identification associated with the shelf propeller are encapsulated into sales data, and the sales data is sent to the settlement terminal; in response to the triggering operation of the settlement terminal, the second user features are acquired; based on at least one target sales data matched by the second user features, a settlement interface is generated and displayed to enable the user to settle. The technical problem that the barcode may be damaged or the scanning environment may be poor, which may cause the code scanning to fail and affect the settlement efficiency is solved, and the technical effect of improving the payment efficiency of the supermarket user is realized.

[0084] Based on embodiment one, in the second embodiment of the present application, step S10 comprises:

[0085] Step S11, based on the mechanical structure parameters of the shelf pusher, the commodity placement rules and the encoder accuracy, a corresponding relationship between the encoder step value and the commodity quantity is established.

[0086] Step S12, receiving the first step value of the encoder, determining the commodity quantity according to the corresponding relationship, wherein if the first step value is between two adjacent step values, the commodity quantity is calculated by linear interpolation.

[0087] In this embodiment, the mechanical structure parameters of the shelf pusher refer to the physical parameters inside the shelf pusher that determine its running displacement, such as the length of the conveyor belt, the diameter of the gear, the transmission ratio, etc.; the commodity placement rules refer to the way of placing commodities on the shelf, such as the number of rows and columns of commodities placed on each layer of the shelf, the spacing between commodities, etc.; the encoder accuracy refers to the minimum displacement change that the encoder can distinguish, which is usually measured by the number of step values per unit displacement.

[0088] As an optional implementation, first, the shelf pusher is disassembled and analyzed, and its mechanical structure parameters are measured and recorded, and the commodity placement rules on the shelf are counted, such as 5 columns of commodities per layer of a certain shelf, with a spacing of 5 cm. Then, according to the product specification of the encoder, its accuracy is obtained, such as 100 step values corresponding to 1 cm of displacement. Through theoretical calculation and combined with multiple actual tests, the encoder step values at different displacements and the corresponding quantities of commodities pushed are recorded to establish a corresponding relationship table. When the system receives the first step value from the encoder, it is first queried in the corresponding relationship table. If the step value matches a record in the table exactly, the corresponding commodity quantity is directly obtained. If it is between two adjacent step values, for example, the step value 1000 corresponds to 1 piece of commodity, the step value 2000 corresponds to 2 pieces of commodity, and the current step value is 1500, then the linear interpolation formula is used: (current step value-smaller step value) / (larger step value-smaller step value) x (larger step value corresponding commodity quantity-smaller step value corresponding commodity quantity)+smaller step value corresponding commodity quantity, that is, (1500-1000) / (2000-1000) x (2-1)+1=1.5, and the commodity quantity is determined according to the actual situation.

[0089] For example, in a snack shelf, the conveyor belt of the pusher advances 1 item for every 10 cm of travel. The encoder has a precision of 50 steps per cm. When the encoder generates 500 steps, it corresponds to 1 item being pushed; when it generates 1000 steps, it corresponds to 2 items being pushed. Once the system receives 750 steps from the encoder, which is between 500 and 1000, it calculates (750-500) / (1000-500) x (2-1) + 1 = 1.5, and rounds up to 1 item.

[0090] By establishing a precise correspondence and using a reasonable calculation method, the embodiment can accurately determine the number of items taken by the user, provide core basic data for accurate generation of subsequent sales data, and ensure the reliability of the item quantity statistics link in the entire vending system management method.

[0091] Based on any of the above embodiments, in Embodiment Three of the present application, referring to Figure 2 , step S20 comprises:

[0092] Step S21, if the first step value is detected, a trigger signal is sent to the image acquisition module.

[0093] In this embodiment, the first step value refers to the value generated by the encoder due to the displacement change of the shelf pusher caused by the user taking the item. The image acquisition module is usually composed of a camera and its supporting image acquisition circuit, and is used to acquire visual images.

[0094] As an optional implementation, the system polls the value of the encoder at fixed time intervals (such as every 10 milliseconds) through a special monitoring program. Once a change in the value of the encoder is found, and the change amount meets the preset minimum step value change range caused by item taking, it is immediately determined that the first step value is detected. Then, a pre-configured hardware communication interface, such as an RS485 bus, is used to send a pre-configured trigger signal code (such as binary code "10101010") to the signal receiving end of the image acquisition module through the bus.

[0095] For example, in a snack shelf of a certain unmanned supermarket, the value of the encoder is constant in the normal state. When a user takes a bag of potato chips, the encoder generates a change of 30 step values. After the system detects this change, it identifies that it meets the step value change condition caused by item taking, and sends the trigger signal code "10101010" to the image acquisition module installed on the upper side of the shelf through the RS485 bus.

[0096] This step provides a start instruction for subsequent image acquisition through precise monitoring and timely signal sending mechanism, ensures that the user's taking moment can be accurately captured, and lays a foundation for user feature extraction.

[0097] Step S22, the image acquisition module receives the trigger signal, captures the image of the preset area of the shelf pusher, and determines the initial image. The preset area is set to the range of the user taking the goods.

[0098] In this embodiment, the preset area is a preset space range around the shelf pusher, which is intended to ensure that the image of the user taking the goods can be captured. The initial image is the raw image captured by the image acquisition module after receiving the trigger signal.

[0099] As an optional implementation, the signal receiving circuit of the image acquisition module starts the internal image acquisition process as soon as it receives the trigger signal code. The camera captures the image of the preset area in the form of a hemisphere with the front end of the shelf pusher as the center and a radius of 1.5 meters according to the preset parameters, such as a resolution of 1920x1080 pixels and a frame rate of 30 frames per second. The image sensor converts the light signal into an electrical signal, which is converted into digital data and processed to generate initial image data, which is temporarily stored in the cache area of the image acquisition module.

[0100] For example, in the above-mentioned unmanned supermarket snack shelf, after the image acquisition module receives the trigger signal, the camera quickly starts and captures the preset area. In a well-lit environment, a clear image containing the upper body of the user taking the chips is successfully captured, and the image data is stored in the cache area as the initial image for subsequent processing.

[0101] This step ensures that the image acquisition module can quickly respond and accurately capture the image of the user's area at the moment the user takes the goods, obtain the raw data, and provide materials for extracting user features.

[0102] Step S23, extracting the user's facial features from the initial image as the first user feature.

[0103] In this embodiment, the user's facial features include facial contour, position and shape of facial features, facial texture, and other feature information that can be used to identify or distinguish users.

[0104] As an optional implementation, the image acquisition module transmits the initial image in the buffer area to the image processing unit at the back end. The image processing unit first scans the initial image using a cascade classifier algorithm based on Haar features to quickly locate the face region in the image. If a face is detected, the face region image is then input into a convolutional neural network (CNN) model trained with a large amount of face data. The model outputs 68 facial key point coordinates, including feature point position information of eye, nose, mouth, and other parts. At the same time, the contour curve of the face region is digitally described, and these facial feature point data and contour curve information are integrated to generate a first user feature vector and stored in the user feature database of the system.

[0105] For example, for the initial image taken by the unmanned supermarket described above, the image processing unit locates the user's face through the Haar cascade classifier, and then analyzes it through the CNN model to extract facial feature information such as the user's eye spacing, eyebrow shape, nose height, and mouth angle, generate a corresponding first user feature vector, and store it in the database to provide key data support for subsequent settlement matching and other operations.

[0106] This step uses advanced image recognition and feature extraction technology to accurately obtain user facial features from the initial image, providing core data for user behavior tracking and settlement identity matching in the entire vending system management method, and improving the safety and accuracy of the system in the unmanned vending scenario.

[0107] Optionally, after step S23, the following steps are included:

[0108] Step S24, if the user facial features are missing, obtaining first body posture features and first clothing features.

[0109] In this embodiment, user facial feature missing means that when facial features are extracted from the initial image using a predetermined algorithm, no face is detected or the key parts of the face are severely obscured or blurred, resulting in the inability to obtain valid facial feature data. The first body posture feature is a user body shape related feature extracted from the initial image, such as height ratio, limb stretching state, etc. The first clothing feature refers to the features of the clothes worn by the user obtained from the initial image, such as color, style, pattern, etc.

[0110] As an optional implementation, after the user's facial features are extracted from the initial image, the system automatically judges the effectiveness of the extraction result. If the number of facial key point coordinates returned by the facial feature extraction algorithm is less than a preset threshold (such as less than 30 key points), or the face region recognition confidence is lower than a set value (such as 0.6), it is determined that the user's facial features are missing. At this time, the initial image is analyzed again using an image recognition algorithm, the user's body contour key points are identified through a human pose estimation model, the proportions of each part of the body are calculated to obtain the first body posture feature; at the same time, the color distribution, pattern texture, etc. of the user's clothing area are analyzed using a color recognition algorithm and image segmentation technology to extract the first clothing feature.

[0111] For example, in a certain unmanned convenience store, when the user takes away the goods, the initial image is severely blocked by the user's face covered by a hat and a mask, and the Haar cascade classifier and the CNN model fail to successfully extract sufficient facial features. After the system determines that the facial features are missing, the user's body is identified to be in an upright state through a human pose estimation model, the limbs are naturally drooping, the height to shelf height ratio is about 3:4, and the first body posture feature is obtained; the color recognition and image segmentation are used to determine that the user is wearing a blue hooded sweatshirt with a white circular pattern on the front, and the first clothing feature is obtained.

[0112] This step, when the facial features are missing, supplements the user's related feature information through other feature extraction methods, to provide more dimensional data for accurately identifying the user.

[0113] Step S25, calling the general image acquisition device to collect the second body posture feature and the second clothing feature of the preset area.

[0114] In this embodiment, the general image acquisition device is a device installed in a supermarket to assist in collecting user-related features, which can supplement the information collected by the shelf pusher image acquisition module. It generally has different shooting angles, higher resolution, etc. The second body posture feature is the user's body posture feature extracted by the general image acquisition device after collecting images from the preset area. The second clothing feature is the user's clothing-related feature obtained by the general image acquisition device after collecting images.

[0115] As an optional implementation, when it is determined that the facial features of the user are missing and the first body posture and clothing features are extracted, the system sends a collection instruction to the general image collection device through a network communication protocol (such as the TCP / IP protocol). After the general image collection device receives the instruction, the shooting parameters are adjusted (such as adjusting the focal length and exposure), and the image collection is performed on the preset area of the shelf pusher. After the collection is completed, the image is transmitted back to the system. The system extracts the body posture details (such as the leg bending angle and arm swing amplitude) of the user from the newly collected image as the second body posture feature and the more detailed features of the clothes (such as the texture of the clothes, the style of the cuffs and the collars) as the second clothing feature by using an algorithm similar to that used to extract the first body posture and clothing features.

[0116] For example, in the above unmanned convenience store, the system calls the high-definition general image collection device installed in the corner of the ceiling, which adjusts the focal length to shoot the area where the goods are taken after receiving the instruction. The system analyzes that the user stands with slightly separated legs, and the arms are holding a shopping basket to obtain the second body posture feature; at the same time, the system identifies that the user's blue sweatshirt is made of cotton and has a white thread edge on the cuffs to obtain the second clothing feature.

[0117] This step uses the general image collection device to obtain more angle and detail user feature information, enriches the user feature data set, and improves the user recognition accuracy.

[0118] Step S26, the first body posture feature, the first clothing feature, the second body posture feature, and the second clothing feature are spliced into the first user feature.

[0119] In this embodiment, the splicing integrates the body posture and clothing feature data from different sources into a complete user feature set for subsequent use in user identity recognition and behavior tracing.

[0120] As an optional implementation, the system performs data fusion on the first body posture feature, the first clothing feature, the second body posture feature, and the second clothing feature obtained previously. First, the various types of feature data are standardized to unify the data format and dimension. Then, according to the preset feature splicing rule, the standardized feature data are arranged and combined in sequence to form a vector containing multi-dimensional user features, which is used as the complete first user feature and stored in the user feature database of the system.

[0121] Exemplarily, the first body shape and clothing features such as the previously obtained user height ratio, clothing color pattern, etc. and the second body shape and clothing features such as the newly acquired body posture details, clothing material texture, etc. are standardized, spliced into a vector containing multiple feature values such as [height ratio value, limb movement value, clothing color value, material texture value, …] in the order of body shape features first and clothing features second, and sorted by importance of similar features, and stored in the database.

[0122] This step integrates multi-source feature data to build a more comprehensive user feature model, providing more abundant and accurate data support for user behavior tracing and settlement identity matching in the entire vending system management method, and further improving the safety and reliability of the system in the unmanned vending scenario.

[0123] Based on any of the above embodiments, in Embodiment Four of the present application, step S30 includes:

[0124] Step S31 associates the quantity of the goods and the product identification as product data according to a preset data format.

[0125] In this embodiment, the preset data format refers to the structure form defined by the system in advance for standardizing data organization and storage, such as common JSON, XML, etc., aiming to ensure the consistency and readability of data during transmission and processing between different modules. The quantity of goods is the quantity of goods taken by the user determined by the encoder step value. The product identification is a code that can uniquely determine the information of the product category, specifications, etc., such as the numerical or character code corresponding to the bar code, two-dimensional code, or the product identification information stored in the RFID tag. The product data is a data set that integrates the quantity of goods and the product identification according to the preset format, used to describe the basic information of the goods taken by the user.

[0126] As an optional implementation, the system adopts JSON format to construct product data. When the quantity of goods and the product identification are obtained, a JSON object is created. In the object, the quantity of goods is taken as the value in a key-value pair, and the key name is “quantity”; the product identification is taken as the value in another key-value pair, and the key name is “product_id”. For example, if the quantity of goods is 3 and the product identification is “6901234567890”, the constructed JSON format product data is: {"quantity":3,"product_id":"6901234567890"}.

[0127] Exemplarily, in a certain unmanned supermarket, a user takes away 5 bottles of mineral water with a barcode of "6951234567891". After the system obtains the quantity of 5 and the product ID "6951234567891", the product data is constructed in the above JSON format, i.e., {"quantity": 5, "product_id": "6951234567891"}.

[0128] In step S32, the product data is spliced with the first user feature to form a data body of the sales data.

[0129] In the embodiment, the first user feature is obtained by the image acquisition module and is processed and extracted, and is used to describe a set of information of the user feature of taking away the product, such as a face feature vector, a body feature data, a clothing feature data, etc. The data body of the sales data is a part of the sales data containing actual business information, which is combined by the product data and the first user feature, and is used to record the product and user related information involved in a sales behavior.

[0130] As an optional implementation, the data is spliced based on the JSON format. On the basis of the product data JSON object constructed before, a key-value pair is added, the key name is "user_features", and the value is the first user feature data. If the first user feature is stored in the form of a vector, for example, the face feature vector is [0.123, 0.456, 0.789,...], the data body of the sales data JSON object after splicing is: {"quantity": 3, "product_id": "6901234567890", "user_features": [0.123, 0.456, 0.789,...]}. If the first user feature contains multiple types (such as body and clothing features), the value of "user_features" can be further constructed into a sub-JSON object to store different types of feature data.

[0131] Exemplarily, for the user who takes away 5 bottles of mineral water before, the first user feature after extraction is the face feature vector [0.234, 0.567, 0.891,...]. The system splices the product data {"quantity": 5, "product_id": "6951234567891"} constructed before with the face feature vector, and the data body of the sales data is {"quantity": 5, "product_id": "6951234567891", "user_features": [0.234, 0.567, 0.891,...]}.

[0132] The step combines commodity information and user features to form a data body containing rich sales behavior information.

[0133] In step S33, a data head containing data type and timestamp is added to the data body.

[0134] In the embodiment, the data head is a piece of data in a preset format added in front of the data body, used to describe some basic attributes of the data, facilitating correct parsing and processing of the data by the receiving end. The data type specifies the business type represented by the data, such as "sale_data" indicating sales data, so that the receiving end can adopt corresponding processing logic according to the data type. The timestamp is a numerical value recording the time of data generation, usually in milliseconds or seconds, accurate to the moment of data generation, used to mark the time sequence of sales behavior occurrence, which is helpful for subsequent data sorting, statistics and analysis.

[0135] As an optional implementation, a new JSON object is created as the data head. In the object, the "data_type" key is set with the value "sale_data" to indicate the data type; the "timestamp" key is set with the current system time in milliseconds, which is obtained by calling the system time acquisition function (such as using time.time()*1000 to obtain the current time in milliseconds in Python). Then, the data head JSON object is combined with the data body of the sales data constructed before. The data head and the data body can be serialized into strings respectively, and then spliced according to the preset protocol format (such as in the HTTP protocol, the data head is in front, the data body is behind, and a preset separator is used to separate them).

[0136] For example, assuming that the current system time in milliseconds is 1678954321000, the constructed data head JSON object is {"data_type":"sale_data","timestamp":1678954321000}. It is spliced with the data body of the sales data of the mineral water taken out before {"quantity":5,"product_id":"6951234567891","user_features":[0.234,0.567,0.891,……]} according to the preset protocol format to form the complete data structure to be sent.

[0137] This step adds a data head to the sales data, giving it key attribute information, so that the settlement terminal can accurately identify the data type and data generation time, facilitating subsequent data processing and management.

[0138] In step S34, the packaged sales data is sent to the settlement terminal through the communication module.

[0139] In the embodiment, the wireless communication module is a hardware device with wireless communication function, such as a Wi-Fi module, a Bluetooth module, a 4G / 5G communication module, etc., for realizing wireless data transmission between different devices. The settlement terminal is a device for users to perform commodity settlement operation, usually with functions of data receiving, display and payment processing, etc.

[0140] As an optional implementation, a Wi-Fi module is selected as the wireless communication module. The network parameters of the Wi-Fi module are pre-configured in the system to make it connected to the same local area network as the settlement terminal. After the packaging of the sales data (including the data header and the data body), the data is converted into a format suitable for Wi-Fi transmission (such as packaging according to the TCP / IP protocol). The packaged sales data is sent to the specified IP address and port number of the settlement terminal in the local area network through the sending function of the Wi-Fi module. The settlement terminal listens to the specified port when starting, and once the data is received, it is parsed according to the previously defined data format.

[0141] For example, in an unmanned supermarket, all devices are connected to a local area network named "Unmanned_Store_WiFi". The packaged sales data in the system is packaged according to the TCP / IP protocol and then sent to the port number "8080" of the IP address "192.168.1.100" of the settlement terminal. After the settlement terminal listens to the data of the port, it parses and obtains the commodity data, user characteristics, data type and timestamp, etc.

[0142] This step realizes the wireless transmission of sales data from the shelf pusher related system to the settlement terminal, ensures that the user's taking information can be timely and accurately conveyed to the settlement link, provides a guarantee for the user to smoothly perform the settlement operation, and perfects the information flow closed loop of the unmanned vending system.

[0143] Based on any of the above embodiments, in the fifth embodiment of the present application, step S40 comprises:

[0144] Step S41, if the settlement terminal detects a user trigger operation, a feature acquisition module of the settlement terminal is called.

[0145] In this embodiment, the settlement terminal is a device for users to perform commodity checkout operations, and is usually integrated with a display screen, a payment module, a data processing unit, and the like. The user trigger operation refers to an action performed by the user on the settlement terminal that can start the settlement process, such as clicking a “start settlement” button on the screen of the settlement terminal, placing the commodity in a preset sensing area to trigger an infrared sensing device, swiping a membership card, and the like. The feature acquisition module is a combination of hardware and software in the settlement terminal for acquiring biological features or other identifiable features of the user, and generally includes a camera, image acquisition software, feature extraction algorithms, and the like, and is mainly used for subsequent acquisition of the user image in this step.

[0146] As an optional implementation, the software system of the settlement terminal continuously monitors the user input port and various sensing devices. When the user clicks a preset “start settlement” button on the screen, the click event of the button is captured, and the system determines that the user trigger operation is detected. Subsequently, the system calls a pre-written feature acquisition module startup function to activate the hardware devices of the feature acquisition module, such as turning on the power of the camera, initializing the image acquisition software, and preparing for the next image acquisition.

[0147] For example, after the user finishes shopping in a certain unmanned convenience store, the user touches the prominent “start settlement” button on the screen. The software system of the settlement terminal instantly captures the button click event, confirms the user trigger operation, and immediately calls the feature acquisition module startup function. At this time, the indicator light of the camera is on, indicating that the feature acquisition module has been successfully called.

[0148] In step S42, the settlement area is imaged based on the feature acquisition module to obtain a user image.

[0149] In this embodiment, the settlement area refers to a preset spatial range set around the settlement terminal, within which a clear user image can be acquired. The settlement area is usually a region with a radius of 1-2 meters centered on the settlement terminal. The user image is an image containing part or all of the user's body obtained by the camera in the feature acquisition module after imaging the settlement area, and is used for subsequent extraction of the user's features.

[0150] As an optional implementation, after the feature acquisition module is called, the camera built-in the feature acquisition module works according to the pre-set parameters. The camera adjusts the focal length, aperture, exposure, and the like to adapt to the light conditions of the settlement area, and ensures that a clear image is captured. The camera performs image acquisition on the settlement area at a preset resolution (such as 1920x1080 pixels) and frame rate (such as 30 frames / second), converts the light signal into an electrical signal, and generates user image data after preliminary processing by an analog-digital conversion and image processing chip, and stores the user image data in a temporary buffer area of the settlement terminal.

[0151] Exemplarily, in the above unmanned convenience store, the camera of the checkout terminal automatically adjusts the focal length to clearly frame the user in the shooting range after the feature acquisition module is started. Under normal indoor light, the camera shoots the user image at a resolution of 1920x1080 pixels and a frame rate of 30 frames / second, obtains a clear image showing the user's front half body, and stores it in the cache area of the checkout terminal for subsequent processing.

[0152] In step S43, the user facial feature points are extracted based on the user image to generate the second user feature.

[0153] In this embodiment, the user facial feature points refer to key position points that can represent the user's facial features, such as the corners of the eyes, the centers of the pupils, the tip of the nose, the wings of the nose, the corners of the mouth, the lip peaks, etc. The second user feature is a set of data extracted from the image of the user collected at the checkout terminal, which is used to match and confirm the user's identity with the user feature before picking up the goods. In this embodiment, it is mainly generated based on the facial feature points.

[0154] As an optional implementation, the data processing unit of the checkout terminal reads the user image data from the temporary cache area. First, the user image is scanned using the Haar feature-based cascade classifier algorithm to quickly locate the face region in the image. If a face is detected, the face region image is input into a convolutional neural network (CNN) model trained with a large amount of face data. The model outputs 68 facial key point coordinates, which represent the position information of the user facial feature points. These facial feature point coordinates are sorted and standardized to form a feature vector, which is stored as the second user feature in the user feature storage area of the checkout terminal, waiting for subsequent matching with the first user feature in the sales data.

[0155] Exemplarily, for the user image shot by the above unmanned convenience store, the data processing unit of the checkout terminal locates the user face region through the Haar cascade classifier and inputs it into the CNN model. The model outputs 68 key point coordinates of the user's face, such as the left eye pupil center coordinate (x1, y1), the right corner of the mouth coordinate (x2, y2), etc. After standardization, these coordinate values are combined into a feature vector [0.12, 0.34, 0.56, …], which is stored as the second user feature for comparison with the previously recorded first user feature to confirm the user's identity and associate the sales data.

[0156] The step is used for accurately obtaining facial feature points from the user image by using advanced image recognition and feature extraction technology, generating the second user feature, providing key data for subsequent sales data matching and user identity confirmation, perfecting the association process of user shopping behavior and settlement behavior in the unmanned vending system, and improving the intelligence and security of the system.

[0157] Optionally, after step S43, the following steps are included:

[0158] In step S44, if the user image corresponds to a blocked or blurred face, third body posture features and third clothing features are extracted.

[0159] In the embodiment, the user image corresponds to a blocked or blurred face, which means that in the user image collected from the settlement area, the user's face is partially or completely blocked by a hat, a mask, a scarf or other articles, or the face image is not clear due to too dark or too bright light, camera shaking or other reasons, and the facial feature points cannot be accurately extracted. The third body posture feature is feature information related to the body shape and posture of the user extracted from the user image, for example, the standing posture of the body (upright, bent over, sideways, etc.), the stretching degree of the limbs (whether the arms are raised, whether the legs are apart, etc.), and the proportional relationship of the parts of the body (the ratio of height to shoulder width, the ratio of head to body, etc.). The third clothing feature is a feature about the clothes worn by the user obtained from the user image, including the color, style (such as T-shirt, coat, skirt, etc.), pattern (print, stripe, plaid, etc.) and material (cotton, hemp, leather, etc.) of the clothes.

[0160] As an optional implementation, after the facial feature points are extracted based on the user image, the system first performs quality evaluation on the extraction result. If the number of valid feature points returned by the facial feature point extraction algorithm is less than a preset threshold (such as less than 30), or the sharpness score of the face region is less than a set value (such as 0.5, calculated by the image sharpness algorithm), the system determines that the user image corresponds to a blocked or blurred face. At this time, the human body posture estimation algorithm is used to analyze the user image. By recognizing the skeletal joints of the human body in the image, the body posture of the user is determined, the angles and proportional relationships of the parts of the body are calculated, and the third body posture features are extracted. At the same time, the image segmentation technology is used to separate the clothes of the user from the background, and then the color recognition algorithm is used to determine the color of the clothes, the texture analysis algorithm is used to identify the pattern and material of the clothes, and the third clothing features are extracted.

[0161] Exemplarily, in a certain unmanned supermarket, a user wears a hat and a mask when checking out, and the face in the image collected by the checkout terminal is seriously occluded. After evaluating the extraction result of the facial feature points, the system determines that the face is occluded. Subsequently, the user's body is identified to be slightly sideways standing by the human body pose estimation algorithm, and the user's hands are in front of the chest, and the proportion relationship of each part of the body is calculated to obtain the third body posture feature; the user wears a black hooded sweatshirt by using image segmentation and color recognition technology, and the sweatshirt has a white letter pattern, and the material is cotton, and the third clothing feature is extracted.

[0162] In step S45, the third body posture feature and the third clothing feature are spliced into the second user feature.

[0163] In this embodiment, splicing means integrating the data of the third body posture feature and the third clothing feature to form a complete data set for representing the current checkout user feature, so as to be matched with the first user feature in the sales data and confirm the user identity subsequently.

[0164] As an optional implementation, the system performs data format uniform processing on the extracted third body posture feature and the third clothing feature. For example, the body posture in the third body posture feature is represented by a digital code (1 for standing straight, 2 for bending, 3 for sideways standing, etc.), the limb stretching degree is represented by an angle value, and the body proportion relationship is represented by a decimal number; the clothing color in the third clothing feature is represented by an RGB value, the style is represented by a pre-defined code (T-shirt is T01, coat is C01, etc.), the pattern is represented by an image feature vector, and the material is represented by a material code. Then, according to a pre-set splicing rule, the third body posture feature data and the third clothing feature data in the uniform format are arranged and combined in sequence to form a vector containing multi-dimensional user features, which is taken as a complete second user feature and stored in a user feature storage area of the checkout terminal, waiting for matching with the first user feature in the sales data.

[0165] Exemplarily, for the user with the occluded face in the above-mentioned unmanned supermarket, the third body posture feature is represented as [sideways standing code 3, arm angle value 120, body proportion decimal 0.65] after processing, and the third clothing feature is represented as [RGB value (0, 0, 0), hooded sweatshirt code C03, white letter pattern vector [0.1, 0.2, 0.3, …], and cotton material code M01]. According to the splicing rule that the body posture feature is in the front and the clothing feature is in the back, the system combines these data into a feature vector [3, 120, 0.65, (0, 0, 0), C03, [0.1, 0.2, 0.3, …], M01] as the second user feature, which is stored for comparison with the previously recorded first user feature to confirm the user identity and associate the sales data.

[0166] The embodiment extracts and integrates body features and clothing features to generate second user features when facial features cannot be effectively extracted, thereby providing a more comprehensive and reliable user identity confirmation method for the unmanned vending system in complex user scenarios, further improving the association process of user shopping behavior and settlement behavior, and improving the practicability and accuracy of the system in various situations.

[0167] Based on any of the above embodiments, in the sixth embodiment of the present application, referring to Figure 3 , step S50 comprises:

[0168] Step S51, compare the second user features with the first user features in the stored sales data one by one, and determine the similarity of the second user features to each of the sales data.

[0169] In the embodiment, the second user features are extracted from the user image collected by the settlement terminal, and are used to represent the data set of the current settlement user features, which may include facial feature points, body features, clothing features, etc. The stored sales data are generated and stored when the user takes the goods from the shelf, which includes the first user features and corresponding information such as the number of goods and the identification of goods. The first user features are obtained and extracted by the image acquisition module of the shelf pusher when the user takes the goods. The similarity is a value used to measure the matching degree between the second user features and the first user features in each sales data, which is calculated by a preset algorithm, and the higher the value, the more similar the two feature sets.

[0170] As an optional implementation, the system reads the second user feature data from the user feature storage area of the settlement terminal, and at the same time, calls all stored sales data and corresponding first user features from the sales data storage database. For the first user features in each set of sales data, the cosine similarity algorithm is used for calculation. First, the second user features and the first user features are converted into feature vectors, such as combining facial feature point coordinates, body feature values, and clothing feature codes into a multi-dimensional vector. Then, the cosine value between the two vectors is calculated, which is the similarity of the two. For example, if the second user feature vector is A=[a1,a2,a3,……], and the first user feature vector is B=[b1,b2,b3,……], then the similarity sim(A,B)=(A·B) / (||A||·||B||), where A·B is the dot product of the vectors, and ||A|| and ||B|| are the norms of vectors A and B, respectively.

[0171] For example, the second user feature vector obtained by the settlement terminal is [face feature point value, side standing code 3, arm angle value 120, body proportion decimal 0.65, RGB value (0, 0, 0), hooded sweatshirt code C03, white letter pattern vector [0.1, 0.2, 0.3, …], and cotton material code M01]. A set of first user feature vectors retrieved from the sales data storage database is [face feature point value, side standing code 3, arm angle value 110, body proportion decimal 0.63, RGB value (0, 0, 0), hooded sweatshirt code C03, white letter pattern vector [0.11, 0.21, 0.31, …], and cotton material code M01]. The similarity between the two is calculated by the cosine similarity algorithm to be 0.92.

[0172] In step S52, the sales data with a similarity higher than the similarity threshold is screened out to determine the target sales data.

[0173] In this embodiment, the similarity threshold is a pre-set value used to determine whether the similarity between the second user feature and the first user feature is high enough to determine whether the corresponding sales data is related to the current settlement user. The threshold is usually adjusted according to actual business needs and a large amount of experimental data, for example, set to 0.8. The target sales data refers to the sales data with a similarity higher than the threshold after screening, and the goods corresponding to these data are considered to be taken away by the current user and need to be settled.

[0174] As an optional implementation, the system traverses all the calculation results after completing the similarity calculation of all sales data and the second user feature. Each similarity value is compared with the pre-set similarity threshold, and if the similarity is greater than the threshold, the corresponding sales data is marked as target sales data and stored in a temporary target sales data list. For example, if the similarity threshold is set to 0.8, after calculation, the similarities of 3 groups of sales data are 0.85, 0.92, and 0.78 respectively, then the two groups of sales data with similarities of 0.85 and 0.92 will be screened out and determined as target sales data and added to the target sales data list.

[0175] In step S53, the number of each type of goods in the target sales data and the corresponding price are summarized to obtain a summary result.

[0176] In this embodiment, the target sales data contains relevant information of the goods taken away by the user, such as the number of goods and the identification of goods, etc. The price information of the goods can be associated through the identification of the goods. The summary result is a result data set obtained by adding up the number of each type of goods in all target sales data and calculating the total price of the corresponding goods, which is used to generate a settlement interface subsequently.

[0177] As an optional implementation, the system traverses the target sales data list. For each target sales data, the unit price of the commodity is queried from the commodity information database according to the commodity identifier therein. Then, the quantities of the same commodity identifier are accumulated, and the total price of the commodity is calculated (total price = commodity quantity * unit price). Finally, the quantity, unit price and total price information of all different commodities are sorted into a summary table. For example, there are two records in the target sales data list, one is that the user takes away 2 bottles of mineral water with a unit price of 3 yuan, and the commodity identifier is "P001"; the other is that the user takes away 1 bag of potato chips with a unit price of 5 yuan, and the commodity identifier is "P002". The system queries the commodity information database to obtain the unit prices of the mineral water and the potato chips, accumulates the commodity quantities to obtain the mineral water quantity as 2 and the potato chip quantity as 1, and calculates the total price of the mineral water as 6 yuan and the total price of the potato chips as 5 yuan.

[0178] In step S54, according to the summary result, the settlement interface is generated according to the preset interface template.

[0179] In the embodiment, the preset interface template is a page layout and style designed in advance for displaying settlement information, including a commodity information display area, a total price display area, a payment method selection area, etc., to provide a clear and convenient settlement operation interface for the user. The settlement interface is generated according to the summary result and the preset interface template, and is used to display the commodity information, the total price and the payment options that the user needs to settle.

[0180] As an optional implementation, the system calls a pre-stored interface template file, which can be in HTML, XML or other formats, and defines the layout structure and style of the settlement interface. The system fills the commodity information (commodity name, quantity, unit price, total price) in the summary result into the positions corresponding to the commodity information display area in the interface template; fills the total prices of all commodities into the total price display area; and adds preset payment method options, such as WeChat payment, Alipay payment, bank card payment, etc., in the payment method selection area. For example, for the above summary result, the system fills the related information of the mineral water and the potato chips into the table elements of the HTML template, fills the total price of 11 yuan into the div element of the total price display, and generates corresponding payment method buttons in the payment method selection area to generate a complete settlement interface HTML file.

[0181] In step S55, the settlement interface is displayed on the display module of the settlement terminal for the user to settle.

[0182] In this embodiment, the display module of the settlement terminal is a hardware device used by the settlement terminal to display information to the user, such as a liquid crystal display screen, a touch display screen, etc. By displaying the generated settlement interface on this module, the user can intuitively see the commodity information and payment options that he needs to settle, and perform subsequent settlement operations.

[0183] As an optional implementation, the software system of the settlement terminal reads the generated settlement interface file (such as an HTML file), parses and renders the file into a visual interface using a built-in browser engine or a graphics rendering engine. Then, the rendered settlement interface is output to the display module of the settlement terminal for display. After the user sees the settlement interface containing the commodity list, the total price, and the payment method button on the display screen of the settlement terminal, he can check the commodity information and select the appropriate payment method to perform the settlement operation. For example, on the settlement terminal of a certain unmanned convenience store, the user sees the information of the mineral water and potato chips he purchased and the total price of 11 yuan clearly displayed on the display screen, and clicks the WeChat payment button to enter the WeChat payment process to complete the settlement.

[0184] This embodiment realizes the complete process from user feature matching to settlement interface display in the unmanned vending system through a series of feature comparison, data filtering, summarizing, and interface generation and display operations, provides convenient and accurate settlement services for the user, and improves the user experience and operation efficiency of the unmanned vending system.

[0185] The application provides a vending system management device, which comprises at least one processor and a memory in communication connection with the at least one processor; the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the vending system management method in Embodiment I.

[0186] Reference will be made to the accompanying drawings Figure 4 which shows a structural schematic diagram of a vending system management device suitable for implementing the embodiments of the application. The vending system management device in the embodiments of the application can include, but is not limited to, mobile terminals such as mobile phones, notebook computers, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Descriptions), PMPs (Portable Media Players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), and the like, and fixed terminals such as digital TVs, desktop computers, and the like. Figure 4 The vending system management device shown is only an example, and should not impose any limitation on the functions and use range of the embodiments of the application.

[0187] As shown in Figure 4 The management device of the vending system can include a processing device 1001 (e.g., a central processing unit, a graphics processing unit, etc.) that can perform various appropriate actions and processes in accordance with a program stored in a read only memory (ROM) 1002 or a program loaded from a storage device 1003 into a random access memory (RAM) 1004. In the RAM 1004, various programs and data required for the operation of the management device of the vending system are also stored. The processing device 1001, the ROM 1002, and the RAM 1004 are connected to each other through a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Generally, the following systems can be connected to the I / O interface 1006: an input device 1007 including, for example, a touch screen, a touch pad, a keyboard, a mouse, an image sensor, a microphone, an accelerometer, a gyroscope, etc.; an output device 1008 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; the storage device 1003 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 1009. The communication device 1009 can allow the management device of the vending system to communicate wirelessly or by wire with other devices to exchange data. Although the management device of the vending system having various systems is shown in the figure, it should be understood that it is not required to implement or have all the systems shown. More or fewer systems can be implemented or provided instead.

[0188] In particular, the processes described above with reference to the flowcharts can be implemented as a computer software program according to embodiments of the present disclosure. For example, embodiments of the present disclosure include a computer program product comprising a computer program carried on a computer readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network by a communication device, or installed from the storage device 1003, or installed from the ROM 1002. When the computer program is executed by the processing device 1001, the above-mentioned functions defined in the methods of embodiments of the present disclosure are performed.

[0189] The management device of the vending system provided by the present disclosure adopts the vending system management method in the above embodiments, and can solve the technical problem that customers leave the supermarket without settling accounts and cannot accurately know how many goods the customers have not settled accounts for. Compared with the prior art, the management device of the vending system provided by the present disclosure has the same beneficial effects as the management device of the vending system provided by the above embodiments, and other technical features in the management device of the vending system are the same as the features disclosed in the above method, which will not be described here.

[0190] It should be understood that various parts of the present application can be realized by hardware, software, firmware or a combination thereof. In the above description of embodiments, specific features, structures, materials or characteristics can be combined in any appropriate manner in any one or more embodiments or examples.

[0191] The above description is merely illustrative of the application and is not intended to limit the scope of the application. Any variations and modifications that can be derived from the described technology without departing from the spirit and scope of the application are intended to be included within the scope of the application. Accordingly, the scope of the application should be determined by the appended claims.

[0192] The present application provides a computer readable storage medium having stored thereon computer readable program instructions (i.e., a computer program) for executing the management method of the vending system in the above-described embodiments.

[0193] The computer readable storage medium provided by the present application may, for example, be a U disk, but is not limited to an electric, magnetic, optical, electromagnetic, infrared, or semiconductor system, system, or device, or any combination thereof. More specific examples of the computer readable storage medium can include, but are not limited to, an electric connection with one or more conductive wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any appropriate combination thereof. In the present embodiment, the computer readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, system, or device. The program code contained on the computer readable storage medium can be transmitted by any appropriate medium, including but not limited to an electric wire, an optical cable, an RF (Radio Frequency), etc., or any appropriate combination thereof.

[0194] The above-described computer readable storage medium can be included in the management device of the vending system; or can exist separately without being assembled into the management device of the vending system.

[0195] The computer readable storage medium carries one or more programs, when the one or more programs are executed by the management device of the vending system, the management device of the vending system: determines the number of goods in response to a first step value of an encoder in the shelf pusher; acquires a first user feature triggering the encoder based on an image acquisition module of the shelf pusher; encapsulates the number of goods, the first user feature, and a goods identifier associated with the shelf pusher into sales data, and sends the sales data to the settlement terminal; acquires a second user feature in response to a target leaving detected by the security door; and issues an alarm information if the second user feature corresponds to at least one target sales data associated with a user and not settled.

[0196] Computer program code for carrying out operations of the present application can be written in any combination of one or more programming languages, including an object oriented programming language such as Java, Smalltalk, C++ or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code can execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the latter scenario, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection can be made to an external computer (for example, through the Internet using an Internet Service Provider).

[0197] The flow diagrams and the block diagrams in the drawings are illustrations of architectures, functionalities, and operations of possible implementations of systems, methods, and computer program products according to various embodiments of present application. In this regard, each block in the flow diagrams or block diagrams can represent a module, a procedure, or a portion of code, which comprises one or more executable instructions for implementing the specified logical functions. It should also be noted that in some alternative implementations, the functions noted in the blocks can occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently or in the reverse order, depending on the functionality involved. It is also noted that each block of the block diagrams and / or flow diagrams and combinations of blocks in the block diagrams and / or flow diagrams can be implemented by special purpose hardware-based systems which perform the specified functions or operations, or combinations of special purpose hardware and

[0198] The modules described in the embodiments of the present application can be implemented in the form of software or in the form of hardware. In some cases, the names of the modules do not limit the modules themselves.

[0199] The readable storage medium provided by the present application is a computer readable storage medium, which stores computer readable program instructions (i.e., a computer program) for executing the management method of the above-mentioned vending system, and can solve the technical problem that customers leave the supermarket without settling accounts and cannot accurately know how many goods the customers have not settled accounts. Compared with the prior art, the computer readable storage medium provided by the present application has the same beneficial effects as the management method of the vending system provided by the above-mentioned embodiments, and will not be described here.

[0200] The embodiments of the present application provide a computer program product, which includes a computer program, and the computer program is executed by a processor to implement the steps of the management method of the above-mentioned vending system.

[0201] The computer program product provided by the present application can solve the technical problem that customers leave the supermarket without settling accounts and cannot accurately know how many goods the customers have not settled accounts. Compared with the prior art, the computer program product provided by the embodiments of the present application has the same beneficial effects as the management method of the vending system provided by the above-mentioned embodiments, and will not be described here.

[0202] The above is only the preferred embodiments of the present application, and does not limit the patent scope of the present application, and any equivalent structure or equivalent process transformation using the content of the present application specification and drawings, or directly or indirectly applied to other related technical fields, are also included in the patent processing scope of the present application.

Claims

1. A management method of a vending system, characterized by, The vending system comprises a shelf pusher and a settlement terminal, and a management method of the vending system comprises: determining a number of goods in response to a first step value of an encoder in the shelf pusher; acquiring a first user feature triggering the encoder based on an image acquisition module of the shelf pusher; packaging the number of goods, the first user feature and a product identifier associated with the shelf pusher into sales data, and sending the sales data to the settlement terminal; acquiring a second user feature in response to a target leaving detected by a safety door; issuing an alarm information if at least one target sales data associated with the second user feature is not settled; the step of determining the number of goods in response to the first step value of the encoder in the shelf pusher comprises: establishing a corresponding relationship between the step value of the encoder and the number of goods based on the mechanical structure parameters of the shelf pusher, the goods placement rules and the encoder accuracy; receiving the first step value of the encoder and determining the number of goods according to the corresponding relationship, wherein if the first step value is between two adjacent step values, the number of goods is calculated by linear interpolation.

2. The method of claim 1, wherein, the step of acquiring the first user feature triggering the encoder based on the image acquisition module of the shelf pusher comprises: if the first step value is detected, a trigger signal is sent to the image acquisition module; the image acquisition module receives the trigger signal, captures an initial image of a preset area of the shelf pusher, and determines the initial image, wherein the preset area is set as the range of the user taking the goods; extracting a user facial feature from the initial image as the first user feature.

3. The method of claim 2, wherein, The vending system further comprises a general image acquisition device, and after the step of extracting the user facial feature from the initial image as the first user feature, the method comprises: if the user facial feature is missing, acquiring a first body feature and a first clothing feature; calling the general image acquisition device to acquire a second body feature and a second clothing feature of the preset area; splicing the first body feature, the first clothing feature, the second body feature and the second clothing feature into the first user feature.

4. The method of claim 1, wherein, The step of packaging the number of goods, the first user feature and the product identifier associated with the shelf pusher into sales data, and sending the sales data to the settlement terminal comprises: associating the number of goods and the product identifier as goods data according to a preset data format; splicing the goods data and the first user feature into a data body of the sales data; adding a data header containing data type and timestamp to the data body; sending the packaged sales data to the settlement terminal through a communication module.

5. The method of claim 1, wherein, After the step of packaging the number of goods, the first user feature and the product identifier associated with the shelf pusher into sales data, and sending the sales data to the settlement terminal, the method comprises: acquiring a second user feature in response to a trigger operation of the settlement terminal; Based on the second user feature matching at least one target sales data, a settlement interface is generated and displayed for user settlement; wherein, the step of acquiring the second user feature in response to the triggering operation of the settlement terminal, comprises: If the settlement terminal detects user triggering operation, the feature acquisition module of the settlement terminal is called; Based on the image acquisition of the settlement area by the feature acquisition module, the user image is obtained; Based on the user image, the user facial feature points are extracted to generate the second user feature.

6. The method of claim 5, wherein, After the step of extracting the user facial feature points based on the user image to generate the second user feature, comprising: If the corresponding face of the user image is blocked or blurred, the third body feature and the third clothing feature are extracted; The third body feature and the third clothing feature are spliced into the second user feature.

7. The method of claim 1, wherein, The step of generating and displaying the settlement interface based on the second user feature matching at least one target sales data for user settlement, comprising: The second user feature is compared with the first user feature in the stored sales data one by one to determine the similarity of the second user feature and each sales data; The sales data with a similarity higher than the similarity threshold value is screened out and determined as the target sales data; The quantity and corresponding price of each type of commodity in the target sales data are summarized to obtain the summary result; According to the summary result, the settlement interface is generated according to the preset interface template; The settlement interface is displayed on the display module of the settlement terminal for user settlement.

8. A management device of a vending system, characterized by, The management device of the vending system comprises a memory, a processor and a computer program stored on the memory and executable on the processor, and the computer program is configured to implement the steps of the vending system management method according to any one of claims 1 to 7.

9. A storage medium, characterized by The storage medium is a computer readable storage medium, and the computer readable storage medium stores a computer program, and the computer program is executed by the processor to implement the steps of the vending system management method according to any one of claims 1 to 7.

Images