Management method and equipment of vending system and storage medium

Abstract

The invention discloses a management method and device of a vending system and a storage medium, and relates to the technical field of general control systems.The management method of the vending system comprises the steps that the number of commodities is determined in response to a first stepping value of an encoder in a goods shelf propeller; based on an image acquisition module of the shelf propeller, acquiring a first user feature triggering the encoder; packaging the commodity quantity, the first user feature and a commodity identifier associated with the shelf thruster into sales data, and sending the sales data to the settlement terminal; in response to a leaving target detected by the safety door, obtaining a second user feature; and if at least one piece of target sales data associated with the user corresponding to the second user feature is not settled, sending alarm information. The technical effect of improving the anti-theft level of the supermarket can be achieved.

Description

Technical Field

[0001] This application relates to the technical field of general control systems, and particularly to a management method, device, and storage medium for a vending system. Background Art

[0002] In a supermarket, the settlement is carried out by the user independently uploading the product barcodes or by the salesperson scanning the product barcodes. Specifically, after selecting the products, the customer scans the product barcodes through the settlement device in the supermarket or a personal mobile terminal, and the system can quickly identify the products and add them to the shopping list for total price calculation. Then the customer walks out of the supermarket after settlement.

[0003] However, this method also has certain limitations. For example, if a customer walks out of the supermarket without settlement, it is impossible to accurately know how many products the customer has not settled. Summary of the Invention

[0004] The main purpose of this application is to provide a management method, device, and storage medium for a vending system, aiming to solve the technical problem that when a customer walks out of the supermarket without settlement, it is impossible to accurately know how many products the customer has not settled.

[0005] To achieve the above object, this application provides a management method for a vending system. The vending system includes a shelf pusher and a settlement terminal. The management method of the vending system includes:

[0006] Responding to the first step value of the encoder in the shelf pusher to determine the quantity of products;

[0007] Based on the image acquisition module of the shelf pusher, collecting the first user feature that triggers the encoder;

[0008] Encapsulating the quantity of products, the first user feature, and the product identifier associated with the shelf pusher into sales data, and sending the sales data to the settlement terminal;

[0009] Responding to the departure target detected by the security door to obtain the second user feature;

[0010] If at least one target sales data associated with the user corresponding to the second user feature is not settled, sending an alarm message.

[0011] In an embodiment, the step of responding to the first step value of the encoder in the shelf pusher to determine the quantity of products includes:

[0012] Based on the mechanical structure parameters of the shelf pusher, the product placement rules, and the encoder accuracy, establishing a correspondence between the encoder step value and the quantity of products;

[0013] Receive the first step value of the encoder, and determine the quantity of the commodity according to the corresponding relationship. If the first step value is between two adjacent step values, calculate the quantity of the commodity by linear interpolation.

[0014] In one embodiment, the step of collecting the first user feature that triggers the encoder by the image acquisition module based on the shelf pusher includes:

[0015] If the first step value is detected, send a trigger signal to the image acquisition module;

[0016] After receiving the trigger signal, the image acquisition module captures an image of a preset area of the shelf pusher to determine an initial image, and the preset area is set as the range of the user who takes the commodity;

[0017] Extract the user's facial feature from the initial image as the first user feature.

[0018] In one embodiment, the vending system further includes a general image acquisition device. After the step of extracting the user's facial feature from the initial image as the first user feature, it includes:

[0019] If the user's facial feature is missing, obtain the first body feature and the first clothing feature;

[0020] Call the general image acquisition device to collect the second body feature and the second clothing feature of the preset area;

[0021] Splice the first body feature, the first clothing feature, the second body feature, and the second clothing feature into the first user feature.

[0022] In one embodiment, the step of encapsulating the commodity quantity, the first user feature, and the commodity identifier associated with the shelf pusher into sales data and sending the sales data to the settlement terminal includes:

[0023] Associate the commodity quantity and the commodity identifier as commodity data according to a preset data format;

[0024] Splice the commodity data and the first user feature into the data body of the sales data;

[0025] Add a data header including the data type and the timestamp to the data body;

[0026] Send the encapsulated sales data to the settlement terminal through the communication module.

[0027] In one embodiment, after the step of encapsulating the quantity of the commodity, the first user feature, and the commodity identifier associated with the shelf pusher into sales data and sending the sales data to the settlement terminal, the following steps are included:

[0028] In response to a trigger operation of the settlement terminal, obtain a second user feature;

[0029] Generate and display a settlement interface based on at least one target sales data matched with the second user feature for the user to settle accounts; wherein, the step of obtaining the second user feature in response to the trigger operation of the settlement terminal includes:

[0030] If the settlement terminal detects a user trigger operation, call the feature collection module of the settlement terminal;

[0031] Based on the feature collection module, perform image acquisition on the settlement area to obtain a user image;

[0032] Extract user facial feature points from the user image to generate the second user feature.

[0033] In one embodiment, after the step of extracting user facial feature points from the user image to generate the second user feature, the following steps are included:

[0034] If the face corresponding to the user image is blocked or blurred, extract a third body feature and a third clothing feature;

[0035] Stitch the third body feature and the third clothing feature into the second user feature.

[0036] In one embodiment, the step of generating and displaying a settlement interface based on at least one target sales data matched with the second user feature for the user to settle accounts includes:

[0037] Compare the second user feature with the first user features in the stored sales data one by one to determine the similarity between the second user feature and each sales data;

[0038] Screen out the sales data with a similarity higher than the similarity threshold and determine them as target sales data.

[0039] Summarize the quantity and corresponding price of various commodities in the target sales data to obtain a summary result.

[0040] Generate the settlement interface according to the summary result according to a preset interface template;

[0041] Display the settlement interface on the display module of the settlement terminal for the user to settle accounts.

[0042] In addition, to achieve the above object, the present application further provides a management device for a vending system. The management device for the vending system includes: a memory, a processor, and a computer program stored on the memory and executable on the processor. The computer program is configured to implement the steps of the management method for the vending system as described above.

[0043] In addition, to achieve the above object, the present application further provides a storage medium. The storage medium is a computer-readable storage medium, and a program for implementing the management method for the vending system is stored on the computer-readable storage medium. The program for implementing the management method for the vending system is executed by a processor to implement the steps of the management method for the vending system as described above.

[0044] The present application provides a management method for a vending system. The present application determines the quantity of goods in response to the first step value of an encoder in the shelf pusher; collects the first user feature that triggers the encoder based on the image acquisition module of the shelf pusher; encapsulates the quantity of goods, the first user feature, and the product identifier associated with the shelf pusher into sales data, and sends the sales data to the settlement terminal; obtains a second user feature in response to a trigger operation of the settlement terminal; generates and displays a settlement interface based on at least one target sales data matched by the second user feature for the user to settle accounts. This solves the technical problem that barcode smudging or poor scanning environment may lead to failed barcode scanning and affect the settlement efficiency, and achieves the technical effect of improving the payment efficiency of supermarket users. Description of the Drawings

[0045] The drawings herein are incorporated into the specification and constitute a part of this specification, showing embodiments consistent with the present application, and are used together with the specification to explain the principles of the present application.

[0046] To more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, for those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0047] Figure 1 It is a schematic flowchart provided for Embodiment 1 of the management method for the vending system of the present application;

[0048] Figure 2 It is a schematic flowchart provided for Embodiment 3 of the management method for the vending system of the present application;

[0049] Figure 3 It is a schematic flowchart provided for Embodiment 6 of the management method for the vending system of the present application;

[0050] Figure 4This is a schematic diagram of the hardware structure related to the management device of the vending system of the present application.

[0051] The implementation, functional features, and advantages of the present application will be further described in conjunction with embodiments and with reference to the accompanying drawings. Specific embodiments

[0052] It should be understood that the specific embodiments described herein are only used to explain the technical solution of the present application and are not used to limit the present application.

[0053] To better understand the technical solution of the present application, the following will be described in detail in conjunction with the accompanying drawings of the specification and specific embodiments.

[0054] Currently, when the barcode is damaged or the scanning environment is poor, it may lead to a failure in barcode scanning, affecting the settlement efficiency.

[0055] The main solution of the present application is: in response to the first step value of the encoder in the shelf pusher, determine the quantity of goods; based on the image acquisition module of the shelf pusher, collect the first user feature that triggers the encoder; encapsulate the quantity of goods, the first user feature, and the product identifier associated with the shelf pusher into sales data, and send the sales data to the settlement terminal; in response to the trigger operation of the settlement terminal, obtain the second user feature; generate and display a settlement interface based on at least one target sales data matched by the second user feature for the user to settle, achieving the technical effect of improving the payment efficiency of supermarket users.

[0056] It should be noted that the execution subject of this embodiment can be a vending system, or a computing service device with data processing, network communication, and program running functions, such as a tablet computer, a personal computer, a mobile phone, etc., or a management device of a vending system capable of implementing the above functions. This embodiment does not make specific limitations in this regard. The following takes the vending system as the execution subject as an example to illustrate this embodiment and the following embodiments.

[0057] Based on this, the present application proposes a management method for the vending system of the first embodiment. The vending system includes a shelf pusher and a settlement terminal. Please refer to Figure 1 , and the management method of the vending system includes steps S10 to S50:

[0058] Step S10, in response to the first step value of the encoder in the shelf pusher, determine the quantity of goods.

[0059] In this embodiment, the encoder is a sensor installed on the transmission mechanism of the shelf pusher for recording the displacement change amount of the shelf pusher; the shelf pusher is a mechanical device for automatically pushing goods to the picking port, usually including components such as a conveyor belt and a gear set.

[0060] As an alternative implementation, the system pre - establishes a correspondence table between the encoder step value and the quantity of goods through mechanical parameter calibration and actual testing. When the user takes goods from the shelf, the shelf pusher moves forward due to the reduction of goods, and the encoder generates a corresponding step value signal. After receiving this signal, the system first queries the correspondence table. If the step value is within the preset interval, it directly matches the corresponding quantity of goods; if it is between two intervals, it calculates the quantity of goods through a linear interpolation algorithm and performs non - negative integer and inventory upper limit verification on the result.

[0061] Exemplarily, for a certain beverage shelf, every 10 rotations of the encoder's step value correspond to the displacement of 1 bottle of beverage. When the user takes away 2 bottles of beverage, the encoder generates 22 step values. The system calculates through linear interpolation: (22 - 20) / (30 - 20)×1 + 2 = 2.2, and after verification, it is rounded to 2 bottles. In this embodiment, the system collects the displacement change of the shelf pusher in real time through the encoder, and accurately calculates the quantity of goods in combination with the preset correspondence relationship, providing a basic quantitative basis for the generation of subsequent sales data.

[0062] Step S20: Based on the image acquisition module of the shelf pusher, acquire the first user feature that triggers the encoder.

[0063] In this embodiment, the image acquisition module is a camera component integrated above or on the side of the shelf pusher, which is used to capture images of the picking area; user features include but are not limited to visual information such as facial features, body postures, and clothing features that can be used to distinguish user identities. As an alternative implementation, when the encoder detects a change in the step value (i.e., goods are taken away), it immediately sends a trigger signal to the image acquisition module. The image acquisition module starts shooting within 200 milliseconds after receiving the signal, acquires images of the area within 1.5 meters in front of the shelf, then performs pre - processing such as denoising and brightness adjustment on the original images, and then identifies the user body in the images through a target detection algorithm, extracting feature information such as facial key point coordinates and clothing color textures.

[0064] Exemplarily, when user A takes away the potato chips on the shelf, the encoder triggers the image acquisition module, and the camera captures the back of user A wearing a blue coat. The system extracts visual features such as the color of his coat and height ratio as the first user feature. In this embodiment, the image acquisition module records the visual features of the picking user in real time, establishing the association between user behavior and physical features, providing data support for user identity matching during subsequent settlement.

[0065] Step S30: Package the quantity of goods, the first user feature, and the product identifier associated with the shelf pusher into sales data, and send the sales data to the settlement terminal.

[0066] In this embodiment, the product identifier is a code used to uniquely identify a product, such as the information in a barcode, QR code, or RFID tag; the sales data is a structured data set containing product information, user characteristics, and transaction-related data. As an alternative implementation, the system encapsulates the data in JSON format, integrates the quantity of products, the first user characteristic (such as a facial feature vector), and the product identifier (such as a barcode string) into a data body, adds a data header containing the data type (sales data) and a timestamp (accurate to milliseconds), and sends it to the settlement terminal via a wireless communication protocol such as Wi-Fi or Bluetooth. After sending, a timeout retransmission mechanism is started. If no confirmation signal is received within 3 seconds, it is resent, with a maximum of 3 retransmissions. Exemplarily, when a user takes away 2 bottles of mineral water with a barcode of 6901234567890, the system encapsulates the product quantity "2", the user characteristic "facial feature vector V1", and the product identifier "6901234567890" as JSON data and sends it to the settlement terminal via Wi-Fi. This embodiment ensures that the pick-up information is synchronized to the settlement terminal in real time and accurately through standardized data encapsulation and a reliable transmission mechanism, providing a complete data source for the subsequent settlement process.

[0067] Step S40: In response to the departure target detected by the security door, obtain the second user characteristic.

[0068] In this embodiment, the security door is a device installed at the supermarket exit for collecting the characteristics of people leaving the supermarket. The departure target refers to the people leaving the supermarket. The second user characteristic refers to the user characteristic collected by the security door for the departure target.

[0069] Step S50: If at least one target sales data associated with the user corresponding to the second user characteristic is not settled, send an alarm message.

[0070] In this embodiment, determine the matching first user characteristic according to the second user characteristic, and obtain the sales data associated with the first user characteristic. If there is unsettled sales data, determine the unsettled sales data as the target sales data. And output an alarm message based on the target sales data to prompt the security personnel to conduct an inspection.

[0071] Optionally, after step S30, it includes:

[0072] Step S60: In response to the trigger operation of the settlement terminal, obtain the second user characteristic.

[0073] In this embodiment, the settlement terminal is a device for the user to perform the checkout operation, usually including a touch screen, a camera, a payment module, etc.; the trigger operation includes interaction behaviors such as the user clicking the "Start Settlement" button and sensing that the user stays in the settlement area for more than 5 seconds.

[0074] As an alternative implementation, when the settlement terminal detects a trigger operation, it immediately activates the built-in camera to collect images of the settlement area with a resolution of 1920×1080 and a frame rate of 30fps. The system performs face detection on the collected images. If a clear face is detected, 68 facial key-point features are extracted; if the face is blocked, auxiliary features such as clothing color and hairstyle are extracted, and the feature data is normalized and converted into a feature vector of a unified dimension. Exemplarily, when the user stands in front of the settlement terminal and clicks the "Settlement" button, the terminal camera captures a frontal image of the user, and the system extracts facial features such as the coordinates of the left eye pupil and the height of the nose bridge of the user to generate a second user feature vector F2. In this embodiment, the feature information of the current user is collected in real time through the settlement terminal, and a real-time association between the settlement behavior and the user features is established, providing a key comparison basis for subsequent sales data matching.

[0075] Step S70, generate and display a settlement interface based on at least one target sales data matched with the second user feature for the user to settle accounts.

[0076] In this embodiment, the target sales data refers to the historical pick-up records matched with the features of the current settlement user; the settlement interface is an interactive interface that displays the information of the goods purchased by the user and payment options.

[0077] As an alternative implementation, the system calculates the cosine similarity between the second user feature and the first user feature in all unsettled sales data, sets a threshold of 0.8, and filters out the sales data with a similarity higher than the threshold. The multiple matched sales data are summarized, the quantities are merged by commodity category, the total price is calculated, and then a settlement interface containing the commodity name, picture, unit price, quantity, total price, and payment methods (WeChat, Alipay, cash) is generated according to a preset template and displayed on the screen of the settlement terminal.

[0078] Exemplarily, the similarity between the second user feature F2 of user A and the first user feature V1 collected in step S20 is 0.85. The system matches the sales data of 2 bottles of mineral water and 1 pack of potato chips taken by the user, and generates a settlement interface with a total price of 15 yuan after summarization. The user clicks Alipay to complete the settlement. In this embodiment, the pick-up records of the user are accurately associated with the settlement behavior through the feature matching algorithm, and a settlement interface containing complete commodity information is automatically generated, realizing a self-service settlement closed-loop in an unattended scenario, effectively solving the problem of "difficult traceability of unsettled goods" in the traditional solution, and improving the anti-loss ability of the system and the settlement efficiency of users.

[0079] As an example of a manned supermarket, when a user enters the supermarket, the user's image or clothing characteristics are obtained to establish the first user characteristics of the user. When the user picks up the goods on the shelf pusher, the shelf pusher calls the image acquisition module to collect the user's image, and then obtains the first user characteristics, encapsulates the quantity of goods and the goods identifier into sales data, and associates it with the first user characteristics. Every time the user picks up the goods on the shelf pusher, the operation is as described above. When the user goes to the settlement terminal for settlement, the staff operates the settlement terminal to obtain the user's image to extract the second user characteristics. Then, the target sales data is obtained by matching according to the second user characteristics, and then a settlement interface is generated for the user to make a payment settlement.

[0080] Furthermore, the staff can also operate the settlement terminal to scan any one goods identifier, then obtain the sales data associated with the goods identifier, and then obtain the first user characteristics associated with the sales data. And obtain all the sales data associated with the first user characteristics, that is, the target sales data. Then, a settlement interface is generated according to the target sales data for the user to make a payment settlement.

[0081] Optionally, the vending system further includes an alarm device. After step S30, it includes: an image acquisition module is provided at the exit of the supermarket security door. If it is detected that the user leaves through the security door, the second user characteristics of the user are collected. Obtain the target sales data associated with the second user characteristics, and determine whether the target sales data has been settled. If the target sales data has not been settled, an alarm message is generated based on the unsettled target sales data and sent to the staff's terminal through the alarm device to prompt the staff that there are unsettled goods. Thus, the technical effect of supermarket anti-theft is achieved.

[0082] Optionally, the vending system further includes an alarm device. After step S70, it includes: obtaining the settlement information of the user. If the settlement information does not match the target sales data, an alarm message is generated based on the alarm device to prompt the staff to check.

[0083] This application determines the quantity of goods in response to the first step value of the encoder in the shelf pusher; collects the first user characteristics that trigger the encoder based on the image acquisition module of the shelf pusher; encapsulates the quantity of goods, the first user characteristics, and the goods identifier associated with the shelf pusher into sales data, and sends the sales data to the settlement terminal; in response to the trigger operation of the settlement terminal, obtains the second user characteristics; generates and displays a settlement interface based on at least one target sales data matched by the second user characteristics for the user to settle. It solves the technical problem that barcode damage or poor scanning environment may cause scanning failure and affect the settlement efficiency, and achieves the technical effect of improving the payment efficiency of supermarket users.

[0084] Based on Embodiment 1, in Embodiment 2 of the present application, step S10 includes:

[0085] Step S11: Establish a correspondence between the encoder step value and the number of goods based on the mechanical structure parameters of the shelf pusher, the goods placement rules, and the encoder accuracy.

[0086] Step S12: Receive the first step value of the encoder, and determine the number of goods according to the correspondence. Wherein, if the first step value is between two adjacent step values, calculate the number of goods by linear interpolation.

[0087] In this embodiment, the mechanical structure parameters of the shelf pusher refer to the physical parameters that determine its operating displacement inside the shelf pusher, such as the conveyor belt length, gear diameter, transmission ratio, etc.; the goods placement rules are the ways in which goods are placed on the shelf, such as the number of rows and columns of goods placed on each layer of the shelf, the spacing between goods, etc.; the encoder accuracy represents the smallest displacement change that the encoder can distinguish, usually measured by the number of step values per unit displacement.

[0088] As an alternative implementation, first disassemble and analyze the shelf pusher, measure and record its mechanical structure parameters, and at the same time count the goods placement rules on the shelf, such as there are 5 columns of goods on each layer of a certain shelf and the spacing between goods is 5 cm. Then obtain its accuracy according to the product manual of the encoder, such as 100 step values corresponding to 1 cm of displacement. Through theoretical calculation and combined with multiple actual tests, record the encoder step values and the corresponding number of goods pushed at different displacements to establish a correspondence table. When the system receives the first step value transmitted by the encoder, first query in the correspondence table. If the step value exactly matches a certain record in the table, directly obtain the corresponding number of goods. If it is between two adjacent step values, for example, the step value 1000 in the record corresponds to 1 piece of goods, the step value 2000 corresponds to 2 pieces of goods, and the current step value is 1500, then use the linear interpolation formula: (current step value - smaller step value) / (larger step value - smaller step value) × (number of goods corresponding to the larger step value - number of goods corresponding to the smaller step value) + number of goods corresponding to the smaller step value, that is, (1500 - 1000) / (2000 - 1000) × (2 - 1) + 1 = 1.5, and then round up according to the actual situation to determine the number of goods.

[0089] Exemplarily, for a certain snack shelf, every time the conveyor belt of its pusher runs 10 centimeters, one item is pushed forward. The encoder accuracy is 50 step values per centimeter. After testing, when the encoder generates 500 step values, it corresponds to one item being pushed; when it generates 1000 step values, it corresponds to two items being pushed. Once, the system received 750 step values from the encoder, which is between 500 and 1000. Through linear interpolation calculation, (750 - 500) / (1000 - 500)×(2 - 1)+1 = 1.5, and after rounding, it is 1 item.

[0090] In this embodiment, by establishing an accurate correspondence relationship and combining a reasonable calculation method, the number of items taken by the user can be accurately determined, providing core basic data for the accurate generation of subsequent sales data and ensuring the reliability of the item quantity statistics link in the entire vending system management method.

[0091] Based on any of the above embodiments, in the third embodiment of the present application, referring to Figure 2 , step S20 includes:

[0092] Step S21, if the first step value is detected, a trigger signal is sent to the image acquisition module.

[0093] In this embodiment, the first step value refers to the value generated by the encoder due to the displacement change of the shelf pusher caused by the user taking an item. The image acquisition module usually consists of a camera and its supporting image acquisition circuit, and is used to acquire visual images.

[0094] As an optional implementation manner, the system polls and reads the values of the encoder at a fixed time interval (such as every 10 milliseconds) through a dedicated monitoring program. Once it is found that the value of the encoder changes, and the change amount meets the preset minimum step value change range caused by taking goods, it is immediately determined that the first step value is detected. Subsequently, using a pre-configured hardware communication interface, such as the RS485 bus, a pre-set trigger signal is encoded (such as the binary code "10101010") and sent out, and this signal is transmitted through the bus to the signal receiving end of the image acquisition module.

[0095] Exemplarily, in the snack shelf of a certain unmanned supermarket, the value of the encoder is constant under normal conditions. When a user takes a pack of potato chips, the encoder generates a change of 30 step values. After the system monitors this change, it identifies that it meets the step value change condition caused by taking goods, and sends the trigger signal code "10101010" to the image acquisition module installed diagonally above the shelf through the RS485 bus.

[0096] This step provides a start instruction for subsequent image acquisition through an accurate monitoring and timely signal sending mechanism, ensuring that the moment when the user takes goods can be accurately captured, laying a foundation for user feature extraction.

[0097] Step S22: The image acquisition module receives the trigger signal, captures an image of a preset area of the shelf pusher, and determines an initial image. The preset area is set as the range of the user who takes the commodity.

[0098] In this embodiment, the preset area is a preset spatial range demarcated in advance around the shelf pusher, aiming to ensure that the image of the user who takes the commodity can be captured. The initial image is the raw image taken by the image acquisition module without processing after receiving the trigger signal.

[0099] As an alternative implementation, once the signal receiving circuit of the image acquisition module receives the trigger signal encoding, it immediately starts the internal image acquisition process. The camera captures an image of a hemispherical preset area with a radius of 1.5 meters in front centered on the front end of the shelf pusher according to preset parameters, such as a resolution set to 1920×1080 pixels and a frame rate set to 30 frames per second. The image sensor converts the optical signal into an electrical signal, and after analog-to-digital conversion and data processing, generates initial image data and temporarily stores it in the buffer area of the image acquisition module.

[0100] Exemplarily, at the snack shelf in the above-mentioned unmanned supermarket, after the image acquisition module receives the trigger signal, the camera is quickly activated to capture an image of the preset area. In an environment with sufficient light, a clear image of the upper body of the user who takes the potato chips is successfully captured, and the image data is stored in the buffer area and becomes the initial image for subsequent processing.

[0101] This step ensures that at the moment when the user picks up the goods, the image acquisition module can quickly respond and accurately capture the image of the user's area, obtain the raw data, and provide materials for extracting user features.

[0102] Step S23: Extract the user's facial features from the initial image as the first user feature.

[0103] In this embodiment, the user's facial features include feature information such as facial contour, positions and shapes of facial features, and facial texture that can be used to identify or distinguish users.

[0104] As an alternative implementation, the image acquisition module transmits the initial image in the buffer to the image processing unit at the back end. The image processing unit first scans the initial image using the cascade classifier algorithm based on Haar features to quickly locate the face region in the image. If a face is detected, the face region image is then input into a convolutional neural network (CNN) model trained with a large amount of face data. This model outputs the coordinates of 68 facial key points, including the position information of the feature points of parts such as eyes, nose, and mouth. At the same time, the contour curve of the face region is digitally described, and these facial feature point data and contour curve information are integrated to generate a first user feature vector, which is stored in the user feature database of the system.

[0105] Exemplarily, for the initial image captured by the above-mentioned unmanned supermarket, the image processing unit locates the user's face through the Haar cascade classifier, and then analyzes it through the CNN model to extract facial feature information such as the distance between the user's eyes, the shape of the eyebrows, the height of the nose bridge, and the angle of the mouth corners, generating the corresponding first user feature vector and storing it in the database to provide key data support for subsequent settlement matching and other operations.

[0106] In this step, through advanced image recognition and feature extraction technologies, the user's facial features are accurately obtained from the initial image, providing core data for the user behavior tracing and settlement identity matching links in the entire vending system management method, and improving the security and accuracy of the system in the unmanned vending scenario.

[0107] Optionally, after step S23, it includes:

[0108] Step S24, if the user's facial features are missing, obtain the first body posture feature and the first clothing feature.

[0109] In this embodiment, the lack of user's facial features means that when extracting facial features from the initial image using a given algorithm, no face is detected or the key parts of the face are severely blocked or blurred, resulting in the inability to obtain effective facial feature data. The first body posture feature is the feature related to the user's body shape extracted from the initial image, such as the approximate proportion of height and the limb extension state. The first clothing feature refers to the features of the user's clothing obtained from the initial image, such as the color, style, and pattern of the clothes.

[0110] As an alternative implementation, after extracting the user's facial features from the initial image, the system automatically determines the validity of the extraction result. If the number of facial key point coordinates returned by the facial feature extraction algorithm is lower than a preset threshold (e.g., less than 30 key points), or the confidence level of face region recognition is lower than a set value (e.g., 0.6), it is determined that the user's facial features are missing. At this time, the initial image is analyzed again using an image recognition algorithm. Through a human pose estimation model, the key points of the user's body contour are recognized, the proportional relationship of each part of the body is calculated, and the first body posture feature is obtained. At the same time, using a color recognition algorithm and image segmentation technology, the color distribution, pattern texture, etc. of the user's clothing area are analyzed, and the first clothing feature is extracted.

[0111] Exemplarily, in a certain unmanned convenience store, when the user picks up the goods, the initial image has the user's face severely blocked by a hat and a mask, and the Haar cascade classifier and the CNN model fail to successfully extract sufficient facial features. After the system determines that the facial features are missing, it is recognized through the human pose estimation model that the user's body is in an upright state, the limbs are naturally drooping, and the ratio of height to the shelf height is about 3:4, obtaining the first body posture feature; using color recognition and image segmentation, it is determined that the user is wearing a blue hooded sweatshirt with a white circular pattern on the front of the clothes, obtaining the first clothing feature.

[0112] In this step, when the facial features are missing, other feature extraction methods are used to supplement the user's relevant feature information, providing more dimensional data for accurate user recognition.

[0113] Step S25, call the general image acquisition device to acquire the second body posture feature and the second clothing feature of the preset area.

[0114] In this embodiment, the general image acquisition device is a device installed in the supermarket for assisting in acquiring user-related features, which can supplement the deficiencies of the information acquired by the shelf pusher image acquisition module. It generally has characteristics such as different shooting angles and higher resolution. The second body posture feature is the user's body posture feature extracted from the image acquired by the general image acquisition device from the preset area. The second clothing feature is the user's clothing-related feature obtained after the general image acquisition device acquires the image.

[0115] As an optional implementation, when it is determined that the user's facial features are missing and the first posture and clothing features are extracted, the system sends an acquisition instruction to the universal image acquisition device through a network communication protocol (such as TCP / IP protocol). After receiving the instruction, the universal image acquisition device adjusts the shooting parameters (such as adjusting the focus and exposure) and acquires images of the preset area of the shelf pusher. After the acquisition is completed, the image is transmitted back to the system. The system uses an algorithm similar to that used to extract the first posture and clothing features to extract the user's body posture details (such as the leg bending angle, arm swing amplitude, etc.) from the newly acquired image as the second posture feature, as well as more detailed features of the clothing (such as the material texture of the clothing, the style of the cuffs and collar, etc.) as the second clothing feature.

[0116] For example, in the aforementioned unmanned convenience store, the system invokes a high-definition, general-purpose image acquisition device mounted in a corner of the ceiling. Upon receiving the command, the device adjusts its focus to capture the pickup area. The system analyzes the captured image to identify the user standing with legs slightly apart and a shopping basket tucked under their arms, acquiring a secondary body feature. It also identifies the user's blue sweatshirt as cotton with white ribbed cuffs, acquiring a secondary clothing feature.

[0117] This step uses a general image acquisition device to obtain user feature information from more angles and details, enrich the user feature data set, and improve user recognition accuracy.

[0118] Step S26: combining the first body feature, the first clothing feature, the second body feature, and the second clothing feature into the first user feature.

[0119] In this embodiment, stitching refers to integrating body shape and clothing feature data from different sources into a complete user feature set for subsequent use in user identification and behavior tracing.

[0120] As an optional implementation, the system fuses the previously acquired first body shape and clothing features with the newly acquired second body shape and clothing features. First, the various feature data types are standardized to unify the data format and dimension. Then, according to pre-set feature concatenation rules, the standardized feature data is sequentially arranged and combined to form a vector containing multi-dimensional user features. This vector serves as the complete first user feature and is stored in the system's user feature database.

[0121] Exemplarily, the first body and clothing features such as the previously obtained user height ratio, clothing color and pattern, etc., and the second body and clothing features such as the newly obtained body posture details, clothing material texture, etc., after being standardized, are spliced into a vector containing multiple feature values according to the rule that body features come first, clothing features come second, and similar features are sorted according to their importance, such as [height ratio value, limb movement value, clothing color value, material texture value...], and stored in the database.

[0122] This step constructs a more comprehensive user feature model by integrating multi-source feature data, providing richer and more accurate data support for the user behavior tracing and settlement identity matching links in the entire vending system management method, and further improving the security and reliability of the system in the unmanned vending scenario.

[0123] Based on any of the above embodiments, in Embodiment 4 of the present application, step S30 includes:

[0124] Step S31, associate the quantity of the commodity and the commodity identifier as commodity data according to a preset data format.

[0125] In this embodiment, the preset data format refers to the structural form predefined by the system for standardizing data organization and storage, such as common formats like JSON and XML, aiming to ensure the consistency and readability of data during transmission and processing between different modules. The quantity of the commodity is the quantity of the commodity taken by the user determined by the encoder step value. The commodity identifier is the encoding that can uniquely determine information such as the type and specification of the commodity, such as the digital or character encoding corresponding to the barcode and QR code, or the commodity identification information stored in the RFID tag. The commodity data is a data set that integrates the quantity of the commodity and the commodity identifier according to the preset format, and is used to describe the basic information of the commodity taken by the user.

[0126] As an alternative implementation, the system uses the JSON format to construct the commodity data. When the quantity of the commodity and the commodity identifier are obtained, a JSON object is created. In this object, the quantity of the commodity is used as the value in a key-value pair, and the key name is "quantity"; the commodity identifier is used as the value in another key-value pair, and the key name is "product_id". For example, if the quantity of the commodity is 3 and the commodity identifier is "6901234567890", the constructed JSON format commodity data is: {"quantity": 3, "product_id": "6901234567890"}.

[0127] Exemplarily, in a certain unmanned supermarket, a user took 5 bottles of mineral water with the barcode "6951234567891". After the system obtained the commodity quantity 5 and the commodity identifier "6951234567891", it constructed commodity data in the above JSON format, that is, {"quantity": 5, "product_id": "6951234567891"}.

[0128] Step S32: Concatenate the commodity data with the first user feature to form the data body of the sales data.

[0129] In this embodiment, the first user feature is obtained through an image acquisition module, processed and extracted, and is an information set used to describe the user features of the person who took the commodity, such as facial feature vectors, body posture feature data, clothing feature data, etc. The data body of the sales data is the part of the sales data that contains actual business information, and is composed of commodity data and the first user feature, and is used to completely record the commodity and user-related information involved in a sales behavior.

[0130] As an alternative implementation, data concatenation is also based on the JSON format. On the basis of the previously constructed commodity data JSON object, a new key-value pair is added, the key name is "user_features", and its value is the first user feature data. If the first user feature is stored in vector form, for example, the facial feature vector is [0.123, 0.456, 0.789,...], then the JSON object of the data body of the concatenated sales data is: {"quantity": 3, "product_id": "6901234567890", "user_features": [0.123, 0.456, 0.789,...]}. If the first user feature includes multiple types (such as body posture and clothing features), the value of "user_features" can be further constructed as a sub-JSON object to store different types of feature data respectively.

[0131] Exemplarily, for the user who took 5 bottles of mineral water before, the first user feature extracted is the facial feature vector [0.234, 0.567, 0.891,...]. The system concatenates the previously constructed commodity data {"quantity": 5, "product_id": "6951234567891"} with this facial feature vector to obtain the data body of the sales data as {"quantity": 5, "product_id": "6951234567891", "user_features": [0.234, 0.567, 0.891,...]}.

[0132] This step combines the commodity information with the user characteristics to form a data body containing rich sales behavior information.

[0133] Step S33: Add a data header containing the data type and timestamp to the data body.

[0134] In this embodiment, the data header is a section of data in a preset format added to the front of the data body, used to describe some basic attributes of the data, facilitating the receiving end to correctly parse and process the data. The data type clarifies the business type represented by the data. For example, "sale_data" represents sales data, so that the receiving end can adopt corresponding processing logic according to the data type. The timestamp is a numerical value recording the data generation time, usually in milliseconds or seconds, accurate to the moment when the data is generated, used to mark the time sequence of sales behavior, which helps subsequent data sorting, statistics, and analysis.

[0135] As an alternative implementation, create a new JSON object as the data header. In this object, set the "data_type" key with its value being "sale_data" to represent the data type; set the "timestamp" key with its value being the number of milliseconds of the current system time, obtained by calling the system's time acquisition function (such as using time.time() * 1000 in Python to obtain the number of milliseconds of the current time). Then, combine this data header JSON object with the data body of the previously constructed sales data. It can be achieved by serializing the data header and the data body into strings respectively, and then splicing them according to a preset protocol format (such as in the HTTP protocol, the data header is in the front, the data body is in the back, separated by a preset delimiter in the middle).

[0136] Exemplarily, assume that the number of milliseconds of the current system time is 1678954321000, and the constructed data header JSON object is {"data_type": "sale_data", "timestamp": 1678954321000}. Splice it with the data body of the sales data of taking mineral water before {"quantity": 5, "product_id": "6951234567891", "user_features": [0.234, 0.567, 0.891,...]} according to the preset protocol format to form a complete data structure to be sent.

[0137] This step endows the sales data with key attribute information by adding a data header, enabling the settlement terminal to accurately identify the data type and data generation time, facilitating subsequent data processing and management.

[0138] Step S34: Send the encapsulated sales data to the settlement terminal through the communication module.

[0139] In this embodiment, the wireless communication module is a hardware device with wireless communication functions, such as a Wi-Fi module, a Bluetooth module, a 4G / 5G communication module, etc., which is used to realize wireless data transmission between different devices. The settlement terminal is a device for users to perform commodity settlement operations, usually having functions such as data reception, display, and payment processing.

[0140] As an alternative embodiment, a Wi-Fi module is selected as the wireless communication module. The network parameters of the Wi-Fi module are pre-configured in the system so that it is connected to the same local area network as the settlement terminal. After the sales data is encapsulated (including the data header and data body), the data is converted into a format suitable for Wi-Fi transmission (such as encapsulation according to the TCP / IP protocol). Through the sending function of the Wi-Fi module, the encapsulated sales data is sent to the specified IP address and port number of the settlement terminal in the local area network. When the settlement terminal starts up, it listens on the specified port. Once the data is received, it is parsed according to the previously defined data format.

[0141] Exemplarily, in an unmanned supermarket, all devices are connected to a local area network named "Unmanned_Store_WiFi". The Wi-Fi module in the system encapsulates the sales data according to the TCP / IP protocol and sends it to the port number "8080" of the IP address "192.168.1.100" of the settlement terminal. After the settlement terminal detects the data on this port, it parses the data to obtain information such as commodity data, user characteristics, data type, and timestamp.

[0142] This step realizes the wireless transmission of sales data from the shelf pusher related system to the settlement terminal, ensures that the user's pick-up information can be conveyed to the settlement link in a timely and accurate manner, provides a guarantee for the user to perform the settlement operation smoothly, and improves the information flow closed-loop of the unmanned vending system.

[0143] Based on any of the above embodiments, in the fifth embodiment of the present application, step S40 includes:

[0144] Step S41, if the settlement terminal detects a user trigger operation, call the feature collection module of the settlement terminal.

[0145] In this embodiment, the settlement terminal is a device for users to perform the operation of settling accounts for goods, which usually integrates components such as a display screen, a payment module, and a data processing unit. The user trigger operation refers to the behavior that a user performs on the settlement terminal and can start the settlement process, such as clicking the "Start Settlement" button on the screen of the settlement terminal, placing goods in a preset induction area to trigger an infrared induction device, swiping a membership card, etc. The feature acquisition module is a combination of hardware and software in the settlement terminal for acquiring users' biometric features or other identifiable features, generally including a camera, image acquisition software, and feature extraction algorithms, etc., and is mainly used for acquiring user images in the following steps.

[0146] As an alternative implementation, the software system of the settlement terminal continuously monitors the user input ports and various induction devices. When the user clicks the preset "Start Settlement" button on the screen, the click event of the button is captured, and the system determines that the user trigger operation is detected. Subsequently, the system calls the pre-written feature acquisition module startup function internally to activate the hardware devices of the feature acquisition module, such as turning on the power of the camera, initializing the image acquisition software, and preparing for the next image acquisition.

[0147] Exemplarily, on the settlement terminal of a certain unmanned convenience store, after the user finishes shopping, the user touches the prominent "Start Settlement" button on the screen. The software system of the settlement terminal instantly captures the button click event, confirms the user trigger operation, and immediately calls the feature acquisition module startup function. At this time, the indicator light of the camera lights up, indicating that the feature acquisition module has been successfully called.

[0148] Step S42: Based on the feature acquisition module, perform image acquisition on the settlement area to obtain a user image.

[0149] In this embodiment, the settlement area refers to a preset spatial range set around the settlement terminal, within which a clear user image can be acquired. Usually, it is an area with a radius of 1 - 2 meters centered on the settlement terminal. The user image is an image containing part or all of the user's body obtained by the camera in the feature acquisition module after shooting the settlement area, and is used for subsequent extraction of user features.

[0150] As an alternative implementation, after the feature acquisition module is called, its built-in camera works according to the pre-set parameters. The camera adjusts parameters such as the focal length, aperture, and exposure to adapt to the light conditions in the settlement area and ensure that a clear image is captured. The camera performs image acquisition on the settlement area at a preset resolution (such as 1920×1080 pixels) and frame rate (such as 30 frames per second), converts the optical signal into an electrical signal, and after analog-to-digital conversion and preliminary processing by the image processing chip, generates user image data and stores it in the temporary buffer of the settlement terminal.

[0151] Exemplarily, in the above-mentioned unmanned convenience store, after the feature acquisition module of the settlement terminal is activated, the camera automatically adjusts its focal length to clearly frame the user within the settlement area in the shooting range. Under normal indoor lighting conditions, the camera captures user images at a resolution of 1920×1080 pixels and a frame rate of 30 frames per second, obtaining an image that clearly shows the front half of the user's body, and stores it in the buffer area of the settlement terminal for subsequent processing.

[0152] Step S43: Extract user facial feature points based on the user image to generate the second user feature.

[0153] In this embodiment, the user facial feature points refer to the key position points that can characterize the user's facial features, such as the corners of the eyes, the centers of the pupils, the tips of the nose, the wings of the nose, the corners of the mouth, the peaks of the lips, etc. Usually, the coordinate information of these points is identified and extracted from the user image through a preset algorithm. The second user feature is a data set extracted from the image collected by the user at the settlement terminal and used to match with the user feature during the previous pick-up to confirm the user's identity. In this implementation manner, it is mainly generated based on the facial feature points.

[0154] As an alternative implementation, the data processing unit of the settlement terminal reads the user image data from the temporary buffer area. First, use the cascade classifier algorithm based on Haar features to scan the user image to quickly locate the face area in the image. If a face is detected, the face area image is input into a convolutional neural network (CNN) model trained with a large amount of face data. This model outputs the coordinates of 68 facial key points, and these coordinates represent the position information of the user's facial feature points. Organize and standardize these facial feature point coordinates to form a feature vector, which is stored in the user feature storage area of the settlement terminal as the second user feature, waiting to be matched with the first user feature in the sales data later.

[0155] Exemplarily, for the user image captured by the above-mentioned unmanned convenience store, the data processing unit of the settlement terminal locates the user's face area through the Haar cascade classifier and inputs it into the CNN model. The model outputs the coordinates of 68 key points on the user's face, such as the left eye pupil center coordinate is (x1, y1), the right corner of the mouth coordinate is (x2, y2), etc. After standardization, these coordinate values are combined into a feature vector [0.12, 0.34, 0.56,...], which is stored as the second user feature for comparison with the previously recorded first user feature to confirm the user's identity and associate the sales data.

[0156] In this step, through advanced image recognition and feature extraction technologies, facial feature points are accurately obtained from the user image, generating the second user features, providing key data for subsequent sales data matching and user identity confirmation, improving the correlation process between user shopping behavior and settlement behavior in the vending system, and enhancing the intelligence and security of the system.

[0157] Optionally, after step S43, it includes:

[0158] Step S44, if the face in the user image is blocked or blurred, extract the third body posture feature and the third clothing feature.

[0159] In this embodiment, the face in the user image being blocked or blurred means that in the user image collected in the settlement area, part or all of the user's face is blocked by items such as hats, masks, scarves, etc., or the face image is unclear due to reasons such as too dark or too bright light, camera shake, etc., and facial feature points cannot be accurately extracted. The third body posture feature is the feature information related to the user's body shape and posture extracted from the user image, such as the standing posture of the body (upright, bent, sideways, etc.), the degree of limb extension (whether the arms are raised, whether the legs are apart, etc.), and the proportional relationship of each part of the body (the ratio of height to shoulder width, the head-to-body ratio, etc.). The third clothing feature is the feature of the user's clothing obtained from the user image, including the color of the clothing, style (such as T-shirt, coat, skirt, etc.), pattern (print, stripe, check, etc.), and the material of the clothing (cotton, linen, leather, etc.).

[0160] As an optional implementation manner, after the facial feature points are extracted based on the user image, the system first performs a quality assessment on the extraction result. If the number of valid feature points returned by the facial feature point extraction algorithm is lower than a preset threshold (such as less than 30), or the clarity score of the face area is lower than a set value (such as 0.5, calculated by the image clarity algorithm), the system determines that the face in the user image is blocked or blurred. At this time, the human body pose estimation algorithm is used to analyze the user image. By identifying the skeletal joint points of the human body in the image, the body posture of the user is determined, and the angles and proportional relationships of each part of the body are calculated, so as to extract the third body posture feature. At the same time, the image segmentation technology is used to separate the user's clothing from the background, then the color recognition algorithm is used to determine the clothing color, and the texture analysis algorithm is used to identify the clothing pattern and material, so as to extract the third clothing feature.

[0161] Exemplarily, in a certain unmanned supermarket, when a user is settling the bill, they are wearing a hat and a mask, and the face in the user image collected by the settlement terminal is severely blocked. After the system evaluates the result of facial feature point extraction, it determines that the face is blocked. Subsequently, through the human pose estimation algorithm, it identifies that the user is standing slightly sideways with their hands folded in front of their chest, calculates the proportional relationship of each part of the body, and obtains the third body posture feature; using image segmentation and color recognition technologies, it determines that the user is wearing a black hooded sweatshirt with white letter patterns and the material is cotton, and extracts the third clothing feature.

[0162] Step S45: Concatenate the third body posture feature and the third clothing feature into the second user feature.

[0163] In this embodiment, concatenation means integrating the data of the third body posture feature and the third clothing feature to form a complete data set representing the current settlement user feature, so as to match with the first user feature in the sales data and confirm the user's identity subsequently.

[0164] As an alternative implementation, the system performs unified data format processing on the extracted third body posture feature and third clothing feature. For example, the body posture in the third body posture feature is represented by digital codes (standing upright is 1, bending over is 2, sideways is 3, etc.), the degree of limb extension is represented by angle values, and the body proportional relationship is represented by decimals; the clothing color in the third clothing feature is represented by RGB values, the style is represented by predefined codes (T-shirt is T01, coat is C01, etc.), the pattern is represented by an image feature vector, and the material is represented by a material code. Then, according to the preset concatenation rule, the data of the third body posture feature and the third clothing feature in the unified format are arranged and combined in sequence to form a vector containing multi-dimensional user features as the complete second user feature, and it is stored in the user feature storage area of the settlement terminal, waiting to be matched with the first user feature in the sales data.

[0165] Exemplarily, for the user with a blocked face in the above-mentioned unmanned supermarket, the processed third body posture feature is represented as [sideways standing code 3, arm angle value 120, body proportion decimal 0.65], and the third clothing feature is represented as [RGB value (0, 0, 0), hooded sweatshirt code C03, white letter pattern vector [0.1, 0.2, 0.3,...], cotton material code M01]. The system combines these data into a feature vector [3, 120, 0.65, (0, 0, 0), C03, [0.1, 0.2, 0.3,...], M01] according to the concatenation rule with the body posture feature first and the clothing feature second, stores it as the second user feature, and uses it to compare with the previously recorded first user feature to confirm the user's identity and associate with the sales data.

[0166] In this embodiment, when facial features cannot be effectively extracted, body posture and clothing features are extracted and integrated to generate a second user feature, providing a more comprehensive and reliable user identity confirmation method for the vending system in complex user scenarios, further improving the association process between user shopping behavior and settlement behavior, and enhancing the practicability and accuracy of the system in various situations.

[0167] Based on any of the above embodiments, in the sixth embodiment of the present application, referring to Figure 3 , step S50 includes:

[0168] Step S51: Compare the second user feature with the first user features in the stored sales data one by one to determine the similarity between the second user feature and each piece of sales data.

[0169] In this embodiment, the second user feature is extracted from the user image collected by the settlement terminal and is a data set representing the current settlement user feature, which may include information such as facial feature points, body posture features, and clothing features. The stored sales data is generated and stored when the user takes goods from the shelf, and includes the first user feature and corresponding information such as the quantity of goods and the product identifier. The first user feature is the user feature obtained and extracted by the image acquisition module of the shelf pusher when the user picks up the goods. The similarity is a value used to measure the matching degree between the second user feature and the first user features in each piece of sales data, and is calculated by a preset algorithm. The higher the value, the more similar the two feature sets are.

[0170] As an alternative implementation, the system reads the second user feature data from the user feature storage area of the settlement terminal, and at the same time retrieves all the stored sales data and their corresponding first user features from the sales data storage database. For the first user feature in each group of sales data, the cosine similarity algorithm is used for calculation. First, the second user feature and the first user feature are respectively converted into feature vectors. For example, the facial feature point coordinates, body posture feature values, clothing feature codes, etc. are combined into a multi-dimensional vector. Then, the cosine value between the two vectors is calculated, and this cosine value is the similarity between them. For example, if the second user feature vector is A = [a1, a2, a3,...], and the first user feature vector is B = [b1, b2, b3,...], then the similarity sim(A, B) = (A·B) / (||A||·||B||), where A·B is the dot product of the vectors, and ||A|| and ||B|| are the norms of vectors A and B respectively.

[0171] Exemplarily, the second user feature vector obtained by the settlement terminal is [facial feature point values, side-standing code 3, arm angle value 120, body proportion decimal 0.65, RGB value (0, 0, 0), hooded sweatshirt code C03, white letter pattern vector [0.1, 0.2, 0.3,...], cotton material code M01]. A set of first user feature vectors retrieved from the sales data storage database is [facial feature point values, side-standing code 3, arm angle value 110, body proportion decimal 0.63, RGB value (0, 0, 0), hooded sweatshirt code C03, white letter pattern vector [0.11, 0.21, 0.31,...], cotton material code M01]. The similarity between the two is calculated to be 0.92 through the cosine similarity algorithm.

[0172] Step S52, screen out the sales data with the similarity higher than the similarity threshold, and determine it as the target sales data.

[0173] In this embodiment, the similarity threshold is a preset value, which is used to judge whether the similarity between the second user feature and the first user feature is high enough to determine whether the corresponding sales data is relevant to the current settlement user. This threshold is usually adjusted according to actual business requirements and a large amount of experimental data, for example, set to 0.8. The target sales data refers to the sales data with a similarity higher than the threshold to the current settlement user feature after screening. The goods corresponding to these data are considered to be the goods taken away by the current user and need to be settled.

[0174] As an optional implementation manner, after the system completes the similarity calculation of all sales data and the second user feature, it traverses all the calculation results. Compare each similarity value with the preset similarity threshold. If the similarity is greater than the threshold, mark the corresponding sales data as the target sales data and store it in a temporary target sales data list. For example, if the similarity threshold is set to 0.8, after calculation, the similarities of 3 groups of sales data are 0.85, 0.92, and 0.78 respectively. Then the two groups of sales data with similarities of 0.85 and 0.92 will be screened out, determined as the target sales data, and added to the target sales data list.

[0175] Step S53, summarize the quantities and corresponding prices of various goods in the target sales data to obtain a summary result.

[0176] In this embodiment, the target sales data contains relevant information about the goods taken away by the user, such as the quantity of goods, product identification, etc. The price information of the goods can be associated through the product identification. The summary result is a result data set obtained by accumulating the quantities of various goods in all the target sales data and calculating the total price of the corresponding goods, which is used to generate the settlement interface subsequently.

[0177] As an alternative implementation, the system traverses the target sales data list. For each piece of target sales data, according to the product identifier therein, the unit price of the product is queried from the product information database. Then, the quantities of products with the same product identifier are accumulated, and the total price of such products is calculated (total price = quantity of products × unit price). Finally, the quantity, unit price, and total price information of all different products are organized into a summary table. For example, there are two records in the target sales data list. One is that the user takes away 2 bottles of mineral water with a unit price of 3 yuan, and the product identifier is "P001"; the other is that the user takes away 1 pack of potato chips with a unit price of 5 yuan, and the product identifier is "P002". The system queries the product information database to obtain the unit prices of mineral water and potato chips. After accumulating the quantities of products, it is obtained that the quantity of mineral water is 2 and the quantity of potato chips is 1. The total price of mineral water is calculated to be 6 yuan, and the total price of potato chips is 5 yuan.

[0178] Step S54, generate the settlement interface according to the summary result and the preset interface template.

[0179] In this embodiment, the preset interface template is a pre-designed page layout and style for displaying settlement information, including a product information display area, a total price display area, a payment method selection area, etc., to provide a clear and convenient settlement operation interface for users. The settlement interface is generated according to the summary result and the preset interface template, and is a visual interface for displaying the product information, total price that the user needs to settle, and providing payment options.

[0180] As an alternative implementation, the system calls a pre-stored interface template file, which can be in formats such as HTML, XML, etc., and defines the layout structure and style of the settlement interface. The system fills the product information (product name, quantity, unit price, total price) in the summary result into the corresponding positions in the product information display area of the interface template; fills the total price of all products into the total price display area; at the same time, adds preset payment method options, such as WeChat payment, Alipay payment, bank card payment, etc., in the payment method selection area. For example, for the above summary result, the system fills the relevant information of mineral water and potato chips into the table element of the HTML template, fills the total price of 11 yuan into the div element for total price display, and generates corresponding payment method buttons in the payment method selection area to generate a complete settlement interface HTML file.

[0181] Step S55, display the settlement interface on the display module of the settlement terminal for the user to settle.

[0182] In this embodiment, the display module of the settlement terminal is a hardware device for the settlement terminal to display information to the user, such as a liquid crystal display screen, a touch display screen, etc. By displaying the generated settlement interface on this module, the user can intuitively see the commodity information and payment options that need to be settled, and perform subsequent settlement operations.

[0183] As an alternative embodiment, the software system of the settlement terminal reads the generated settlement interface file (such as an HTML file), and uses the built-in browser engine or graphics rendering engine to parse and render the file into a visual interface. Then, the rendered settlement interface is output to the display module of the settlement terminal for display. After the user sees the settlement interface including the commodity list, total price, and payment method buttons on the display screen of the settlement terminal, the user can check the commodity information and select a suitable payment method to perform the settlement operation. For example, on the settlement terminal of a certain unmanned convenience store, the user sees the information of the mineral water and potato chips purchased and the total price of 11 yuan clearly displayed on the display screen, clicks the WeChat payment button, and can enter the WeChat payment process to complete the settlement.

[0184] Through a series of operations such as feature comparison, data screening, summarization, interface generation, and display, this embodiment realizes the complete process from user feature matching to settlement interface display in the unmanned vending system, provides a convenient and accurate settlement service for users, and improves the user experience and operation efficiency of the unmanned vending system.

[0185] This application provides a management device for a vending system. The management device of the vending system includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein, the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the management method of the vending system in the first embodiment above.

[0186] Next, refer to Figure 4 , which shows a schematic structural diagram of a management device of a vending system suitable for implementing the embodiments of the present application. The management device of the vending system in the embodiments of the present application may include, but is not limited to, mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Descriptions), PMPs (Portable Media Players), in-vehicle terminals (such as in-vehicle navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. Figure 4 The management device of the vending system shown is only an example, and should not bring any limitation to the functions and usage scopes of the embodiments of the present application.

[0187] As Figure 4 shown, the management device of the vending system may include a processing device 1001 (such as a central processing unit, a graphics processing unit, etc.), which may perform various appropriate actions and processes according to a program stored in a read-only memory (ROM: Read Only Memory) 1002 or a program loaded from a storage device 1003 into a random access memory (RAM: Random Access Memory) 1004. In the RAM 1004, various programs and data required for the operation of the management device of the vending system are also stored. The processing device 1001, the ROM 1002, and the RAM 1004 are connected to each other through a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Generally, the following systems may be connected to the I / O interface 1006: an input device 1007 including, for example, a touch screen, a touch pad, a keyboard, a mouse, an image sensor, a microphone, an accelerometer, a gyroscope, etc.; an output device 1008 including, for example, a liquid crystal display (LCD: Liquid Crystal Display), a speaker, a vibrator, etc.; a storage device 1003 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 1009. The communication device 1009 may allow the management device of the vending system to communicate with other devices wirelessly or wiredly to exchange data. Although the figure shows the management device of the vending system having various systems, it should be understood that it is not required to implement or have all the systems shown. More or fewer systems may be alternatively implemented or had.

[0188] Specifically, according to the embodiments disclosed in the present application, the processes described above with reference to the flowcharts may be implemented as computer software programs. For example, the embodiments disclosed in the present application include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes program codes for performing the methods shown in the flowcharts. In such an embodiment, the computer program may be downloaded and installed from a network through the communication device, or installed from the storage device 1003, or installed from the ROM 1002. When the computer program is executed by the processing device 1001, the above functions defined in the methods of the embodiments disclosed in the present application are performed.

[0189] The management device of the vending system provided by the present application adopts the vending system management method in the above embodiments, and can solve the technical problem that customers walk out of the supermarket without settlement and it is impossible to accurately know how many goods the customers have not settled. Compared with the prior art, the beneficial effects of the management device of the vending system provided by the present application are the same as those of the management device of the vending system provided by the above embodiments, and other technical features in the management device of the vending system are the same as the features disclosed in the method of the previous embodiment, and will not be elaborated here.

[0190] It should be understood that each part disclosed in this application can be implemented by hardware, software, firmware, or a combination thereof. In the description of the above embodiments, specific features, structures, materials, or characteristics can be combined in a suitable manner in any one or more embodiments or examples.

[0191] As described above, the above is only the specific implementation manner of this application, but the protection scope of this application is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed in this application, and all should be covered by the protection scope of this application. Therefore, the protection scope of this application should be subject to the protection scope of the claims.

[0192] This application provides a computer-readable storage medium having computer-readable program instructions (i.e., computer programs) stored thereon, and the computer-readable program instructions are used to execute the management method of the vending system in the above embodiments.

[0193] The computer-readable storage medium provided by this application can be, for example, a USB flash drive, but is not limited to electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or any combination of the above. More specific examples of computer-readable storage media can include, but are not limited to: electrical connections with one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM) or flash memory, optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the above. In this embodiment, the computer-readable storage medium can be any tangible medium that contains or stores a program, and this program can be used by or combined with an instruction execution system, device, or device. The program code contained on the computer-readable storage medium can be transmitted by any appropriate medium, including but not limited to: wires, optical cables, RF (Radio Frequency), etc., or any suitable combination of the above.

[0194] The above computer-readable storage medium can be included in the management device of the vending system; it can also exist alone and not be assembled into the management device of the vending system.

[0195] The above computer-readable storage medium carries one or more programs, which, when executed by a management device of a vending system, cause the management device of the vending system to: determine the quantity of goods in response to a first step value of an encoder in the shelf pusher; collect a first user feature that triggers the encoder based on an image acquisition module of the shelf pusher; encapsulate the quantity of goods, the first user feature, and a product identifier associated with the shelf pusher into sales data, and send the sales data to the settlement terminal; obtain a second user feature in response to a departure target detected by a security door; and issue an alarm message if at least one target sales data associated with the user corresponding to the second user feature is unsettled.

[0196] Computer program code for performing the operations of this application may be written in one or more programming languages or combinations thereof. The programming languages include object-oriented programming languages such as Java, Smalltalk, and C++, and also include conventional procedural programming languages such as the "C" language or similar programming languages. The program code may execute entirely on the user's computer, partially on the user's computer, execute as a stand-alone software package, execute partially on the user's computer and partially on a remote computer, or execute entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., through the Internet using an Internet service provider).

[0197] The flowcharts and block diagrams in the accompanying drawings illustrate the possible architectures, functions, and operations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in the flowchart or block diagram may represent a module, a program segment, or a part of code that contains one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than marked in the accompanying drawings. For example, two consecutive blocks shown may actually be executed substantially in parallel, and they may sometimes be executed in the reverse order, depending on the functions involved. It should also be noted that each block in the block diagram and / or flowchart, and the combinations of blocks in the block diagram and / or flowchart, may be implemented by a dedicated hardware-based system for performing the specified functions or operations, or may be implemented by a combination of dedicated hardware and computer instructions.

[0198] The modules involved in the embodiments of the present application can be implemented in software or in hardware. Among them, the name of the module does not constitute a limitation to the unit itself in some cases.

[0199] The readable storage medium provided by the present application is a computer-readable storage medium, and the computer-readable storage medium stores computer-readable program instructions (i.e., computer programs) for executing the management method of the above-mentioned vending system, which can solve the technical problem that customers walk out of the supermarket without settlement and it is impossible to accurately know how many goods the customers have not settled. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided by the present application are the same as those of the management method of the vending system provided by the above embodiment, and will not be elaborated here.

[0200] The embodiments of the present application provide a computer program product, including a computer program, and when the computer program is executed by a processor, the steps of the management method of the vending system as described above are implemented.

[0201] The computer program product provided by the present application can solve the technical problem that customers walk out of the supermarket without settlement and it is impossible to accurately know how many goods the customers have not settled. Compared with the prior art, the beneficial effects of the computer program product provided by the embodiments of the present application are the same as those of the management method of the vending system provided by the above embodiment, and will not be elaborated here.

[0202] The above are only the preferred embodiments of the present application, and do not limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made by using the content of the specification and drawings of the present application, or directly or indirectly applied in other related technical fields, shall be included in the patent scope of the present application by the same token.

Claims

1. A management method for a vending system, characterized in that, The vending system includes a shelf pusher and a settlement terminal. The management method of the vending system includes: Responding to the first step value of the encoder in the shelf pusher to determine the quantity of goods; Based on the image acquisition module of the shelf pusher, acquiring the first user feature that triggers the encoder; Encapsulating the quantity of goods, the first user feature, and the product identifier associated with the shelf pusher into sales data, and sending the sales data to the settlement terminal; Responding to the departure target detected by the security door to obtain the second user feature; If at least one target sales data associated with the user corresponding to the second user feature is not settled, an alarm message is issued.

2. The method according to claim 1, characterized in that, The step of responding to the first step value of the encoder in the shelf pusher to determine the quantity of goods includes: Based on the mechanical structure parameters of the shelf pusher, the product placement rules, and the encoder accuracy, establishing a correspondence between the encoder step value and the quantity of goods; Receiving the first step value of the encoder, and determining the quantity of goods according to the correspondence. Among them, if the first step value is between two adjacent step values, the quantity of goods is calculated by linear interpolation.

3. The method according to claim 1, characterized in that The step of acquiring the first user feature that triggers the encoder based on the image acquisition module of the shelf pusher includes: If the first step value is detected, sending a trigger signal to the image acquisition module; After receiving the trigger signal, the image acquisition module captures an image of a preset area of the shelf pusher to determine an initial image. The preset area is set as the range of the user who takes the goods; Extracting the user's facial feature from the initial image as the first user feature.

4. The method according to claim 3, wherein The vending system further includes a general image acquisition device. After the step of extracting the user's facial feature from the initial image as the first user feature, it includes: If the user's facial feature is missing, acquiring the first body feature and the first clothing feature; Invoking the general image acquisition device to acquire the second body feature and the second clothing feature of the preset area; Splicing the first body feature, the first clothing feature, the second body feature, and the second clothing feature into the first user feature.

5. The method according to claim 1, characterized in that, The step of encapsulating the quantity of goods, the first user feature, and the product identifier associated with the shelf pusher into sales data, and sending the sales data to the settlement terminal includes: Associating the quantity of goods and the product identifier as product data according to a preset data format; Splicing the product data and the first user feature into the data body of the sales data; Adding a data header including the data type and timestamp to the data body; Sending the encapsulated sales data to the settlement terminal through the communication module.

6. The method according to claim 1, wherein, After the step of encapsulating the quantity of goods, the first user feature, and the product identifier associated with the shelf pusher into sales data, and sending the sales data to the settlement terminal, it includes: Responding to the trigger operation of the settlement terminal to obtain the second user feature; Generate and display a settlement interface for the user to settle accounts based on at least one target sales data matched with the second user feature; wherein the step of obtaining the second user feature in response to the trigger operation of the settlement terminal includes: If the settlement terminal detects a user trigger operation, call the feature acquisition module of the settlement terminal; Based on the feature acquisition module, perform image acquisition on the settlement area to obtain a user image; Extract user facial feature points from the user image to generate the second user feature.

7. The method according to claim 6, wherein After the step of extracting user facial feature points from the user image to generate the second user feature, it includes: If the face corresponding to the user image is blocked or blurred, extract the third body feature and the third clothing feature; Concatenate the third body feature and the third clothing feature as the second user feature.

8. The method according to claim 1, characterized in that, The step of generating and displaying a settlement interface for the user to settle accounts based on at least one target sales data matched with the second user feature includes: Compare the second user feature with the first user features in the stored sales data one by one to determine the similarity between the second user feature and each sales data; Screen out the sales data with the similarity higher than the similarity threshold and determine them as target sales data; Summarize the quantity and corresponding price of various commodities in the target sales data to obtain a summary result; Generate the settlement interface according to the preset interface template based on the summary result; Display the settlement interface on the display module of the settlement terminal for the user to settle accounts.

9. A management device for a vending system, characterized in that, The management device of the vending system includes: a memory, a processor, and a computer program stored on the memory and executable on the processor, and the computer program is configured to implement the steps of the management method of the vending system according to any one of claims 1 to 8.

10. A storage medium, characterized in that, The storage medium is a computer-readable storage medium, and a computer program is stored on the computer-readable storage medium, and when the computer program is executed by the processor, it implements the steps of the management method of the vending system according to any one of claims 1 to 8.

Images