A contactless payment method and device

Abstract

The present application relates to the field of unmanned vending technology, and specifically discloses a contactless payment method and device, which includes: using binocular cameras and infrared arrays arranged on the top and sides of the cabinet to capture images of multiple customers picking up goods, using posture estimation to extract key points of the hands, and using multi-target tracking to distinguish identities; using gravity sensors under the shelf layers to detect weight changes in each cargo position, and combining with visual trajectories to construct spatiotemporal associations of commodity movement, thereby achieving cross-validation of pickup behavior; using convolutional neural networks to detect abnormal behavior in multimodal data and generate risk scores; creating a virtual shopping basket containing spatiotemporal tags for each customer, and using a decision tree algorithm to determine commodity ownership; executing distributed bill generation and multi-channel deductions based on risk levels, and completing real-time settlement in multi-customer concurrent scenarios. The present application method significantly improves the accuracy of commodity ownership and payment security, and can achieve efficient and stable unmanned retail operations.

Description

Technical Field

[0001] The present application relates to the technical field of unmanned vending machines, and specifically discloses a contactless payment method and device. Background Art

[0002] Unmanned vending machines, a key form of retail automation, mostly rely on monocular or binocular cameras, weight sensors, and RFID for product identification and checkout. However, these technologies often operate independently or are simply combined. In a real-world business environment, multiple customers may simultaneously access items from the same shelf, accompanied by complex interactions such as occlusion, cross-picking, and product transfers. The system must rapidly complete customer identification, capture hand movements, detect product movement, identify unusual behavior, and generate and debit bills.

[0003] Existing solutions have difficulty in synchronously grasping customers' precise hand movements and changes in the weight of shelves in multi-customer concurrent scenarios, and are unable to reliably establish a one-to-one correspondence between goods and customers. When situations such as camera obstruction, multi-person collaboration, or malicious delivery of goods occur, there is a lack of an effective abnormal behavior quantification mechanism, making it difficult to adjust payment strategies based on risk levels, and prone to incorrect deductions, missed deductions, or disputes. Furthermore, bill generation and payment links often rely on a single channel for serial execution, lacking timely fault-tolerance compensation in the event of network fluctuations or payment failures, affecting the seamless payment experience.

[0004] Therefore, how to simultaneously improve the accuracy of product attribution, the reliability of abnormal behavior identification and the security of settlement when multiple customers are shopping concurrently remains a technical problem that needs to be urgently solved in the field of unmanned vending machines.

[0005] In view of this, the present application proposes a contactless payment method and device. Summary of the Invention

[0006] To achieve the above objectives, the present invention provides a contactless payment method and device, the specific technical solutions are as follows:

[0007] A contactless payment method, comprising:

[0008] The system uses a binocular camera and infrared sensor array to collect data on customers' picking-up behavior at unmanned vending machines. It then uses a posture estimation algorithm to extract key points of customers' hands, constructing and updating each customer's hand motion trajectory in real time, and tracking and distinguishing multiple customers simultaneously.

[0009] Gravity sensors are integrated under the shelves of unmanned vending machines to monitor the weight changes of goods in each shelf. Combined with visual perception results, a spatiotemporal correlation model of product movement is established to cross-validate product pickup behavior.

[0010] Detect abnormal behaviors in customer pickup actions based on a convolutional neural network model, assign risk scores to identified high-risk behaviors, determine the risk level, and trigger corresponding risk control and processing strategies in real time;

[0011] Design a transaction isolation mechanism based on spatiotemporal tagging, assign each customer an independent virtual shopping basket, and use a decision tree algorithm to fuse visual trajectory and gravity sensing data to determine the ownership of items when multiple customers pick up items.

[0012] Based on the commodity ownership and the user's virtual shopping cart, a distributed bill generation strategy and a multi-channel contactless payment strategy are implemented according to the risk level.

[0013] Preferably, images are collected by binocular cameras deployed on the top and sides of the unmanned vending machine, and a disparity map is calculated using a stereo matching algorithm to obtain three-dimensional depth information of the scene.

[0014] Preferably, a posture estimation algorithm based on deep learning is used to detect and track the customer's hands, construct the hand motion trajectory, and perform parallel tracking of multiple customers through a multi-target tracking framework based on Kalman filtering, while fusing the data expansion of the infrared sensor array to distinguish the customer's identity.

[0015] Preferably, independent gravity sensors are evenly installed under each shelf layer of the unmanned vending machine to continuously collect weight data of the goods, and determine whether the customer has picked up the goods based on the weight change threshold.

[0016] Preferably, a spatiotemporal correlation model of commodity movement is established in combination with visual perception results. By calculating the spatial distance between the customer's hand and the shelf and analyzing the change in hand speed, a comprehensive correlation confidence scoring mechanism is established to determine the correlation between the commodity pickup event and the customer.

[0017] Preferably, an abnormal behavior detection model based on convolutional neural networks and long short-term memory networks is constructed, multimodal data is input, and camera occlusion behavior is detected based on image brightness distribution, edge information changes, and image differences.

[0018] Preferably, a detection mechanism based on trajectory intersection analysis and the temporal relationship between hand dwell time and commodity weight change is established to identify cross-picking and commodity delivery behaviors of multiple people, and a comprehensive risk scoring mechanism is established to quantify and evaluate the risk scores of various abnormal behaviors and map them to different risk levels to trigger corresponding risk control strategies.

[0019] Preferably, an independent virtual shopping basket is created for each customer, and a product attribution determination algorithm based on a decision tree is constructed;

[0020] Using visual trajectory and gravity sensing data as decision-making features, and establishing a product ownership conflict detection and resolution mechanism when multiple customers pick up goods at the same time, products are given priority to the customer with the highest matching score, and products with unclear ownership and abnormal behavior are sent for manual review.

[0021] Preferably, a distributed bill generation strategy is implemented based on the determined commodity ownership relationship and user virtual shopping basket data, and a differentiated payment channel strategy is implemented according to the customer's risk level. At the same time, a multi-channel payment priority mechanism and a real-time notification and reconciliation mechanism for payment results are established.

[0022] A contactless payment device, used to implement the contactless payment method, characterized by comprising: a visual data acquisition module, a weight data acquisition module, a pickup anomaly detection module, a multi-user transaction separation module, and a multi-channel contactless payment module;

[0023] The visual data acquisition module uses a binocular camera and an infrared sensor array to collect data on customers' picking-up behavior in front of unmanned vending machines. It uses a posture estimation algorithm to extract key points of customers' hands, constructs and updates each customer's hand motion trajectory in real time, and tracks and distinguishes multiple customers in parallel.

[0024] The weight data acquisition module integrates gravity sensors under the shelves of the unmanned vending machine to monitor the weight changes of goods in each shelf. Combined with the visual perception results, it establishes a spatiotemporal correlation model of the movement of goods and cross-validates the product picking behavior.

[0025] The pickup anomaly detection module detects abnormal behaviors of customers when picking up their items based on a convolutional neural network model, assigns risk scores to identified high-risk behaviors, determines the risk level, and triggers corresponding risk control and processing strategies in real time;

[0026] The multi-user transaction separation module designs a transaction isolation mechanism based on spatiotemporal marking, assigns an independent virtual shopping basket to each customer, and uses a decision tree algorithm to fuse visual trajectory and gravity sensing data to determine the ownership of goods when multiple customers pick up goods;

[0027] The multi-channel contactless payment module implements a distributed bill generation strategy and a multi-channel contactless payment strategy based on the commodity ownership relationship and the user's virtual shopping basket and according to the risk level.

[0028] Beneficial effects of the present invention: This application uses stereo vision and infrared array collaborative perception to achieve millimeter-level hand key point extraction and continuous trajectory updating. Even if the hands are crossed or temporarily occluded, identity continuity can be maintained, significantly reducing the missed detection rate of picking up actions, and providing a high-precision spatiotemporal data basis for subsequent product ownership determination.

[0029] This application uses gravity sensors to capture subtle changes in the weight of each cargo location in real time, and performs bidirectional verification of time and space matching with visual trajectories to accurately distinguish between pickup and return events; multi-source cross-validation greatly improves the accuracy of commodity movement identification, providing reliable protection for inventory management and settlement accuracy.

[0030] This application uses convolutional neural networks combined with multimodal input to classify and determine abnormal actions such as occlusion and product delivery, generate quantifiable risk scores and immediately trigger graded risk control measures; without disturbing normal customers, it can promptly curb high-risk behaviors, reduce missed payments and malicious losses, and comprehensively improve the safety of ATM operations.

[0031] This application creates an independent virtual shopping basket for each customer based on transaction isolation using spatiotemporal tags. The decision tree integrates visual and weight features to automatically determine the ownership of the goods, and gives priority rankings in conflict scenarios, submitting them for manual review when necessary. This process significantly reduces incorrect charges and disputes, ensuring clear accounts and data isolation when multiple customers shop concurrently.

[0032] This application dynamically generates distributed bills based on the product attribution results and risk levels, and uses primary and backup payment channels and an asynchronous retry mechanism to complete deductions in seconds; low-risk customers are unaware of the entire process, while high-risk customers undergo secondary confirmation or manual review, effectively balancing payment convenience and security, and improving overall transaction success rate and customer satisfaction. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] Figure 1 A flow chart of a contactless payment method provided by the present invention;

[0034] Figure 2 The binocular vision and infrared fusion acquisition flow chart provided by the present invention;

[0035] Figure 3 A cross-validation flow chart for the association between gravity and visual picking provided by the present invention;

[0036] Figure 4 Flowchart of abnormal behavior detection and risk grading provided by the present invention;

[0037] Figure 5 A flowchart of the product attribution in the multi-customer virtual shopping basket provided by the present invention;

[0038] Figure 6 This is a structural diagram of a contactless payment device provided by the present invention. DETAILED DESCRIPTION

[0039] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the specific embodiments of the present invention are described in detail below with reference to the accompanying drawings.

[0040] In the following description, many specific details are set forth to facilitate a full understanding of the present invention. However, the present invention can also be implemented in other ways different from those described herein. Those skilled in the art can make similar generalizations without violating the connotation of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below.

[0041] Secondly, the term "one embodiment" or "embodiment" herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The phrase "in one embodiment" appearing in various places throughout this specification does not necessarily refer to the same embodiment, nor does it constitute a separate or selective embodiment that is mutually exclusive with other embodiments.

[0042] Example 1

[0043] Reference Figures 1 to 5 , which is the first embodiment of the present invention, as Figure 1 As shown, a contactless payment method is provided.

[0044] Step 1: Use binocular cameras and infrared sensor arrays to collect customer pickup behavior data in front of unmanned vending machines, use posture estimation algorithms to extract customer hand key points, build and update each customer's hand motion trajectory in real time, and track and distinguish multiple customers in parallel; see Figure 2 This is the binocular vision and infrared fusion acquisition flow chart for this step.

[0045] Binocular cameras are deployed on the top and sides of the unmanned vending machines to form a stereoscopic visual perception network. For example, the baseline distance of the binocular cameras is set to 120-150 mm to ensure accurate depth information within an effective pickup distance of 1-3 meters. Infrared sensor arrays are also evenly arranged along the edges of each shelf layer, with spacing between the sensors controlled at 80-100 mm, forming a dense perception grid. When a customer enters the infrared sensor's perception area, the binocular cameras capture left and right image sequences, and a stereo matching algorithm is used to calculate the disparity map to obtain three-dimensional depth information of the scene.

[0046] Based on the collected image data, a deep learning-based pose estimation algorithm is used to detect and track the customer's hands. Specifically, the MediaPipe hand detection model is used to identify the hand area in the image, and then a key point model is used to accurately locate key locations such as the palm and finger joints.

[0047] For each detected hand, extract its coordinates in 3D space ,in, Indicates the The horizontal coordinates of the hand key points, represents the vertical coordinate, Represents the depth coordinate. By matching key points between consecutive frames, the hand motion trajectory is constructed. ,in, Indicates at time The hand position, is the total number of key points in the hand motion trajectory, ( ) are the three-dimensional coordinates of the hand position. This precise keypoint localization method effectively addresses hand posture changes and partial occlusion, improving the accuracy of motion recognition when customers pick up items from unmanned vending machines.

[0048] In order to achieve parallel tracking of multiple customers, a multi-target tracking framework based on Kalman filtering is established. For each new customer, a unique identity ID is assigned and its state vector is initialized. ,in Indicates the center position of the hand, Represents the velocity component. At each time step, the client's motion state is maintained through two stages: prediction and update. The prediction stage uses the motion model ,in, is the state transition matrix, is the process noise, Represents the prediction equation, using the previous moment The state prediction at the current moment status, is the state estimate of the last time step in the Kalman filter. In the update phase, the new observation data is combined through Corrected predicted value, where is the Kalman gain, is the observation vector, is the observation matrix, To update the equation, the predicted value is corrected using the observed value.

[0049] When multiple customers appear in the pickup area at the same time, the auxiliary information of the infrared sensor array is used to enhance the ability to distinguish between customers; the occlusion events detected by the infrared sensor are aligned with the visual tracking results in time and space, and when a certain infrared sensor is triggered, the triggering time is recorded. and location , and calculate the distance between each customer's hand position at that moment ,in, Trigger time The distance between the center of the customer's hand and the triggering infrared sensor position, 、 Trigger time The 2D coordinates of the center of the hth customer's hand detected at the time are then associated with the customer with the smallest distance (less than a distance threshold) to the trigger event. By fusing visual and infrared perception information, stable identity tracking can be maintained even when the customer's hands are crossed or briefly occluded.

[0050] For example, when there are two customers A and B picking up goods at the shelf at the same time, The hand position of customer A is detected at all times mm, the hand position of customer B is mm, in seconds, the infrared sensor is at position is triggered, and the calculated mm, set the distance threshold to 80 mm; and It is less than the set threshold of 80 mm, so the trigger event is associated with customer A. and are the distances between the hand positions of customer A and customer B and the triggering position of the infrared sensor, respectively.

[0051] This step uses the above-mentioned perception method of fusing binocular vision with infrared sensing to accurately collect and track the pickup behaviors of multiple customers in front of unmanned vending machines, providing a reliable behavioral data basis for subsequent product ownership determination and contactless payment, and effectively solving the problem of tracking failure of traditional single sensors in complex scenarios.

[0052] Step 2: Integrate gravity sensors under the shelves of the unmanned vending machine to monitor the weight changes of the goods in each shelf. Combined with the visual perception results, a spatiotemporal correlation model of the goods movement is established to cross-validate the goods picking behavior; see Figure 3 , which is the cross-validation flowchart of gravity and visual picking association in this step.

[0053] Gravity sensors are evenly installed beneath each shelf in the unmanned vending machine. For example, these sensors can utilize a strain gauge structure, with a measuring range of 0-5 kg, tailored to the weight characteristics of the merchandise, and a weight resolution of 0.01 kg. Each shelf is equipped with an independent gravity sensor, connected to the shelf via an elastic support structure to ensure accurate sensing of weight changes in the merchandise at that shelf. The gravity sensors continuously collect merchandise weight data at a fixed sampling frequency, which can be set to 1-10 Hz, to meet the real-time monitoring requirements for rapid product retrieval.

[0054] When a customer picks up a product, the gravity sensor detects a weight change signal Expressed as ,in, express The weight value at the moment, Indicates the sampling time interval. When , it is determined that a product pickup event has occurred, where is the weight change threshold, weight change threshold It can be set dynamically based on a certain proportion of the lightest product weight.

[0055] Combined with the visual perception results obtained in step 1, a spatiotemporal correlation model of commodity movement is established. Detected cargo location The weight loss event, where Indicates the number of shelf layers, Indicates the cargo location number of this layer, and queries the time window before and after this moment Calculate the spatial distance between each customer's hand and the shelf ,in, For the The hand position of each customer, For cargo space The customer with the smallest distance that is less than the preset threshold is preliminarily associated with the pickup event.

[0056] To improve the accuracy of associating the weight of the goods with the customer's hand position, further analyze the customer's hand movement pattern; calculate the change in hand speed before and after picking up the goods ,in, and Respectively represent the hand speed before and after the picking action. Normal picking action is usually accompanied by a significant change in hand speed, that is, slowing down when approaching the shelf and speeding up after picking up the goods. By setting the hand speed change threshold ,when , the confidence level of the association between the customer and the pickup event is enhanced.

[0057] A spatiotemporal correlation confidence scoring mechanism is established, which comprehensively considers multiple factors such as spatial distance, temporal synchronization and movement pattern. The correlation confidence calculation formula is: ,in, 、 、 is the weight coefficient and satisfies , and is the attenuation coefficient, The moment when the customer’s hand is closest to the cargo position. is a normalized function. Through this multi-factor fusion approach, we can accurately establish the correlation between product movement and customer behavior.

[0058] For example, when the cargo space exist The weight is detected to be reduced by 200 grams in 1 second, and the visual system shows that customer A's hand is The closest distance to the cargo location is mm, speed change m / s. Setting , , , , , calculate the associated confidence At this time, combined with the weight change of the sensor and the synchronous detection of the hand movement of customer A by the visual system, it can be determined that customer A has completed the pickup behavior at this moment. exist The weight is detected to be reduced by 200 grams in seconds, and it can be determined Second Vision detected the exact moment when Customer A's hand approached the shelf, a distance of just 0.02 seconds. The hand movement closely matched the change in the weight of the product. This confirms that Customer A successfully removed the product from shelf (2,3) during that timeframe. This determination relies not only on data from a single sensor but also on the fusion of multiple sources, significantly improving the accuracy and reliability of identifying pickup behavior.

[0059] This step cross-validates product pickup behavior by bidirectionally checking gravity sensing data against visual tracking results. When the gravity sensor detects a pickup event but the visual system fails to match the hand movement, or when the visual system detects a pickup action but the gravity sensor shows no corresponding change, this is flagged as an abnormal event and triggers further verification. This dual-modality cross-validation mechanism significantly improves the accuracy and reliability of product pickup behavior recognition, providing a solid data foundation for subsequent product ownership determination and payment settlement.

[0060] Step 3: Detect abnormal behaviors of customers picking up goods based on the convolutional neural network model, assign risk scores to identified high-risk behaviors, determine the risk level, and trigger corresponding risk control and processing strategies in real time; see Figure 4 , which is the abnormal behavior detection and risk classification flow chart of this step.

[0061] A convolutional neural network-based abnormal behavior detection model was constructed, using a ResNet-50 network as the backbone to extract spatial features from video frames and combining it with a long short-term memory (LSTM) model to capture temporal dynamic information. The multimodal data acquired in steps 1 and 2 was preprocessed, including video frame sequences, hand trajectory data, and gravity sensor signals, and then fed into the abnormal behavior detection model. For each behavior sequence within a time window, a multidimensional feature vector was extracted, including spatial position, motion speed, acceleration, and gravity change.

[0062] In order to detect and identify abnormal behaviors of customers who may block the camera when purchasing at unmanned vending machines, we analyze the brightness distribution and edge information changes of the images collected by the binocular camera. We calculate the image difference between consecutive frames. ,in, and Represents the height and width of the image respectively, express The image at the moment When If the duration exceeds the preset threshold, it is determined to be camera blocking behavior, where: is the difference threshold. Combined with the average brightness of the image ,when The confidence of occlusion judgment is enhanced when the image drops sharply. The detection method based on image statistical features can quickly identify malicious occlusion behavior and trigger the early warning mechanism in time.

[0063] For complex scenarios where multiple people are picking up goods at the same time, a detection mechanism based on trajectory intersection analysis is established. The minimum distance between different customers' hand trajectories is calculated. ,in, and Respectively represent customers A and B at time The hand position, is the observation time window. When , it is determined that a track crossing occurs, where is the cross distance threshold, the cross distance threshold The setting of needs to consider the range of the customer's hand movement, the camera's depth perception error and the complexity of the scene. Set to half the hand's range of motion minus the perception error; the recommended value is 50-150 mm. In crowded scenes, the threshold can be appropriately increased to reduce missed detections, while in single-person or low-risk scenes, the threshold can be reduced to improve detection accuracy. The threshold can also be dynamically optimized based on actual data statistics.

[0064] Further analyze the change in movement direction before and after the hand trajectory crosses, and calculate the angle of the direction vector ,in, and are the velocity vectors of the two customers respectively. When it is close to 180 degrees, it indicates that the two customers' hand movements are in opposite directions, and there may be commodity transfer behavior.

[0065] By analyzing the temporal relationship between the hand's dwell time and the change in the weight of the goods, it is possible to detect whether goods are being passed between customers. Define the dwell time of the hand in a specific area. ,in, and Represents the moment when the hand enters and leaves the area. When two customers’ hands are detected to be in the non-shelf area at the same time and hour, is the residence time threshold, the residence time threshold The threshold is set based on the average dwell time of normal pickup behavior. By analyzing historical data to calculate the distribution of hand dwell time in non-shelf areas, the 95th percentile of the normal dwell time is usually taken as the initial threshold, for example, 2-5 seconds, and dynamically adjusted based on the complexity of the scene. In crowded scenes with many people, the threshold can be lowered to improve detection sensitivity, while in single-person scenes, the threshold can be appropriately increased to reduce misjudgments. Combined with gravity sensor data, it is analyzed whether there is any abnormal transfer of product weight. If no corresponding shelf weight change is detected within the time period, it is determined that a product transfer has occurred between customers A and B.

[0066] Establish a comprehensive risk scoring mechanism to quantitatively assess the risk scores of various abnormal behaviors detected. The calculation formula is ,in, is the total number of abnormal behavior types, For the The weight coefficient of abnormal behavior, The frequency of such behavior. The severity coefficient is assigned to the behavior of blocking the camera, followed by the delivery of goods, and the severity of multiple people picking up goods is relatively low. , which is mapped to risk levels: When the risk is low, When the risk is medium, High risk, among which and The risk level threshold is set based on a comprehensive assessment of historical data statistics and scenario requirements. First, by analyzing the risk score distribution of various abnormal behaviors, the upper limit of the score for low-risk behaviors is determined. , such as covering 90% of normal customer behaviors and the lower limit of scores for high-risk behaviors , such as covering 95% of malicious behaviors. The medium risk range is set between and The threshold needs to be dynamically adjusted based on actual operating scenarios to balance the sensitivity and false alarm rate of detection, ensuring both normal customer experience and timely detection of high-risk behaviors.

[0067] For example, a customer detects a cross-picking behavior of multiple people within a 10-second observation window ( ), 2 suspected commodity delivery behaviors ( ), unobstructed camera behavior ( ). Calculate risk score If set , , the customer is marked as medium risk level.

[0068] Based on the identified risk level, corresponding risk control strategies are triggered in real time. For low-risk customers, the normal contactless payment process is maintained; for medium-risk customers, the sampling frequency and analysis depth of subsequent behavior monitoring are enhanced, and a secondary confirmation mechanism is added to the payment process; for high-risk customers who have been banned, the manual review process is triggered, and a secondary confirmation mechanism is added to the payment process.

[0069] This step integrates multimodal data such as visual images, hand trajectories, and gravity sensors to construct an abnormal behavior detection model based on convolutional neural networks and time series analysis. This model accurately identifies behaviors such as camera obstruction, product delivery, and cross-collection by multiple people. It also incorporates a risk scoring mechanism and dynamic control strategies. Through this graded response mechanism, while ensuring a normal customer shopping experience, it effectively prevents economic losses caused by malicious behavior, thereby improving the operational safety and reliability of unmanned vending machines.

[0070] Step 4: Design a transaction isolation mechanism based on spatiotemporal labeling, assign an independent virtual shopping basket to each customer, and use a decision tree algorithm to fuse visual trajectory and gravity sensing data to determine the ownership of goods when multiple customers pick up goods; see Figure 5 , which is the flowchart of multi-customer virtual shopping basket product attribution in this step.

[0071] Create an independent virtual shopping basket for each customer who enters the sensing area of ​​the unmanned vending machine and assign a unique time and space tag. The time and space tag adopts a composite structure. ,in, It is the customer’s unique identification ID. Enter a timestamp for the client, To enter the position coordinates, This is a hash value of the customer's behavior history. By strongly binding the customer's identity to their shopping behavior, this ensures that in scenarios where multiple customers are picking up their goods concurrently, each customer's transaction data is isolated from each other, avoiding confusion over the ownership of goods.

[0072] A product ownership determination algorithm based on a decision tree is constructed, using the visual trajectory data obtained in step 1 and the gravity sensing data in step 2 as decision feature inputs. The root node of the decision tree first determines whether there is a gravity change event. If so, it enters the product pickup determination branch. In this branch, key decision features extracted include: the minimum distance between the hand and the shelf , the time the hand stays near the cargo area , the time difference between the moment of gravity change and the moment of hand arrival , smoothness of hand movement trajectory The smoothness of the hand motion trajectory is obtained by calculating the curvature change of adjacent hand motion trajectory points. , For the The curvature of the trajectory points, is the total number of trajectory points.

[0073] The branch conditions of the decision tree are adaptively determined based on the training data. Typical decision rules include: and and When , the goods are attributed to the corresponding customers, among which, 、 、 is the threshold parameter obtained through training. For each leaf node of the decision tree, calculate its attribution confidence ,in, The number of samples correctly classified for this leaf node, is the total number of samples that reach the leaf node. When the confidence level is lower than the preset threshold, the product is marked as uncertain.

[0074] In complex scenarios where multiple customers are picking up their goods at the same time, a product ownership conflict detection and resolution mechanism is established; when multiple customers' decision paths all point to the same product, a comprehensive matching score is calculated for each customer. ,in, 、 、 、 is a weight parameter and satisfies , The maximum allowable time difference is 0. Products are preferentially assigned to the customer with the highest matching score, while the information of the second-best customer is recorded for subsequent verification. A comprehensive scoring method based on multi-dimensional features can accurately determine product ownership in most cases, reducing the occurrence of ownership conflicts.

[0075] When an item's ownership is determined to be unclear or abnormal behavior is detected, the system automatically packages relevant video clips, sensor data, and decision tree path information and sends them to the backend manual review platform. Manual reviewers replay multi-angle videos to determine the item's true ownership. To improve manual review efficiency, key time points and suspicious behavior clips are automatically annotated. The results of manual verification are fed back into the decision tree model and used as incremental learning samples to optimize the accuracy of subsequent automatic judgments.

[0076] For example, when customer A and customer B's hands are at the same location Nearby activities, gravity sensor in Seconds to detect a 200g weight loss. Minimum distance of customer A’s hand mm, dwell time Seconds, time difference seconds; the corresponding value for client B is mm, Second, sec. Set , , , , Seconds, calculated , Therefore, the product belongs to customer A, but due to is also high, marking the event as a mild risk event.

[0077] This step achieves accurate product allocation in scenarios where multiple customers are picking up goods concurrently through a transaction isolation mechanism based on spatiotemporal marking and an intelligent product ownership determination algorithm. At the same time, through risk grading and human-machine collaboration, it ensures transaction accuracy and fairness in complex and abnormal situations, laying a solid foundation for the reliable implementation of contactless payment.

[0078] Step 5: Based on the product ownership and the user's virtual shopping cart, implement a distributed bill generation strategy and a multi-channel contactless payment strategy according to the risk level.

[0079] Based on the commodity ownership relationship and user virtual shopping cart data determined in step 4, a distributed bill generation strategy is implemented. When a customer leaves the unmanned vending machine sensing area, the bill generation process is triggered to read all the commodity information in the customer's virtual shopping cart, including commodity code, quantity, unit price and other data. The bill amount is calculated using the formula ,in, is the total bill amount, is the number of product types in the shopping basket, For the The unit price of the commodity, is the corresponding purchase quantity, The discount rate for the product. To ensure bill accuracy in multi-customer concurrent scenarios, a distributed lock mechanism is used to ensure that each customer's bill generation process is independent of each other, avoiding data contention and calculation errors.

[0080] Implement differentiated payment channel strategies based on the customer's risk level. For low-risk customers, directly enable the pre-bound password-free payment channel and complete automatic deductions by calling the payment interface. For medium-risk customers, add a bill confirmation step before the password-free payment, push the generated bill information to the customer's mobile device, and set a confirmation timeout. If the customer does not raise any objection within the specified time, the payment will be automatically completed. For high-risk customers, manually verify the goods purchased by the customer and then push the generated bill information to the customer's mobile device, setting a confirmation timeout. If the customer does not raise any objection within the specified time, the payment is automatically completed. This tiered payment strategy ensures a convenient experience for most normal customers while effectively controlling the risk of abnormal transactions.

[0081] Establish a multi-channel payment priority mechanism to ensure the reliability and success rate of payment. Configure the main payment channel and backup payment channel for each customer. When the main payment channel encounters network anomalies, insufficient balance or other failures, it will automatically switch to the backup channel for payment retry. Payment retry uses an exponential backoff algorithm, and the retry interval is ,in, is the initial retry interval, The current number of retries. Set the maximum number of retries When the maximum number of retries is reached and the transaction is still unsuccessful, the transaction is marked as pending and the customer is reminded to complete the payment manually via SMS or app notification.

[0082] To improve the real-time nature of payments, an asynchronous processing architecture is employed to separate bill generation and payment execution. Once bill generation is complete, a confirmation of purchase completion is immediately returned to the customer, and the payment task is added to a message queue for asynchronous processing. The payment task contains information such as the customer ID, bill number, amount, and timestamp. A dedicated payment processing process retrieves the task from the queue and performs the actual deduction. This asynchronous processing approach mitigates the impact of payment network latency on the customer experience, while the message queue's persistence mechanism ensures that payment tasks are not lost.

[0083] Establish a real-time payment result notification and reconciliation mechanism. After the customer successfully pays, the payment voucher is sent to the customer through the push service, including transaction serial number, product details, payment amount, payment time, etc. At the same time, the payment result is written into the distributed ledger, recording the complete transaction chain information, including product identification process, risk assessment results, payment channel selection and other data. Regularly execute the automatic reconciliation process, compare the system billing records with the actual deduction records of the payment channel, and calculate the reconciliation difference rate ,in, is the recorded amount receivable, is the actual deduction amount of the payment channel; When the warning threshold is exceeded, the manual verification process is triggered; the setting of the warning threshold is based on the statistical distribution of the historical reconciliation difference rate and the error range in actual operation; by analyzing the reconciliation difference rate of normal transactions, the upper limit value covering 95%-99% of normal transactions is set as the warning threshold; when the reconciliation difference rate When this threshold is exceeded, a manual verification process is triggered to ensure that abnormal transactions are handled in a timely manner while avoiding excessive false alarms that affect system efficiency.

[0084] For example, a low-risk customer purchased 3 items with unit prices of Yuan, Yuan, Yuan, quantity is 1 piece, no discount ( ). Calculate the bill amount Yuan. Due to the customer risk score Less than the risk level threshold , complete the automatic deduction directly through the pre-bound payment channel. If the first payment fails due to network problems, set seconds, the second retry will be Seconds later.

[0085] This step achieves efficient settlement in multi-customer concurrent shopping scenarios by implementing distributed bill generation and multi-channel contactless payment strategies based on risk levels. It ensures payment security while maintaining the convenience of contactless payment to the greatest extent. As a contactless payment method, it can significantly improve the overall operational efficiency and customer satisfaction of unmanned vending machines.

[0086] Example 2

[0087] Reference Figure 6 , which is the second embodiment of the present invention, provides a contactless payment device.

[0088] The system includes: a visual data acquisition module, a weight data acquisition module, a pickup anomaly detection module, a multi-user transaction separation module and a multi-channel contactless payment module.

[0089] The visual data acquisition module collects customer pickup behavior data in front of unmanned vending machines through a binocular camera and an infrared sensor array, uses a posture estimation algorithm to extract key points of customers' hands, constructs and updates the hand motion trajectory of each customer in real time, and tracks and distinguishes the identities of multiple customers in parallel.

[0090] The weight data acquisition module integrates a gravity sensor under the shelf layer of the unmanned vending machine to monitor the weight changes of the goods in each shelf. Combined with the visual perception results, it establishes a spatiotemporal correlation model of the movement of goods and cross-validates the goods picking behavior.

[0091] The pickup anomaly detection module detects abnormal behaviors in customer pickup actions based on a convolutional neural network model, assigns risk scores to identified high-risk behaviors, determines the risk level, and triggers corresponding risk control and processing strategies in real time.

[0092] The multi-user transaction separation module designs a transaction isolation mechanism based on spatiotemporal marking, allocates an independent virtual shopping basket to each customer, and determines the ownership of goods when multiple customers pick up goods by fusing visual trajectory and gravity sensing data based on a decision tree algorithm.

[0093] The multi-channel contactless payment module implements a distributed bill generation strategy and a multi-channel contactless payment strategy based on the multi-modal commodity ownership relationship determination and the user's virtual shopping cart according to the risk level, thereby realizing concurrent settlement and real-time deduction for multiple customers.

[0094] Example 3

[0095] In the embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are merely schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some communication interface, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0096] The embodiments of the present invention are described above in conjunction with the accompanying drawings, but the present invention is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of the present invention, ordinary technicians in this field can also change, modify, replace and modify the above-mentioned embodiments without departing from the scope of protection of the purpose of the present invention and the claims, and all of these are protected by the present invention.

Claims

1. A contactless payment method, characterized in that: include: The system uses a binocular camera and infrared sensor array to collect data on customers' picking-up behavior at unmanned vending machines. It then uses a posture estimation algorithm to extract key points of customers' hands, constructing and updating each customer's hand motion trajectory in real time, and tracking and distinguishing multiple customers simultaneously. Gravity sensors are integrated under the shelves of unmanned vending machines to monitor the weight changes of goods in each shelf. Combined with visual perception results, a spatiotemporal correlation model of product movement is established to cross-validate product pickup behavior. Detect abnormal behaviors in customers' pickup actions based on a convolutional neural network model and assign risk scores to identified high-risk behaviors; Establish a detection mechanism based on trajectory intersection analysis and the temporal relationship between hand dwell time and product weight changes to identify cross-picking and product transfer behaviors by multiple people. Establish a comprehensive risk scoring mechanism to quantify and assess the risk scores of various abnormal behaviors and map them to different risk levels. Once the risk level is determined, the corresponding risk control and handling strategies are triggered in real time. Design a transaction isolation mechanism based on spatiotemporal tagging, assign each customer an independent virtual shopping basket, and use a decision tree algorithm to fuse visual trajectory and gravity sensing data to determine the ownership of items when multiple customers pick up items. Based on the commodity ownership and the user's virtual shopping cart, a distributed bill generation strategy and a multi-channel contactless payment strategy are implemented according to the risk level.

2. A contactless payment method according to claim 1, characterized in that: Images are collected by binocular cameras deployed on the top and sides of the unmanned vending machine, and the disparity map is calculated using a stereo matching algorithm to obtain the three-dimensional depth information of the scene.

3. A contactless payment method according to claim 2, characterized in that: A deep learning-based posture estimation algorithm is used to detect and track customer hands, construct hand motion trajectories, and track multiple customers in parallel through a multi-target tracking framework based on Kalman filtering. At the same time, the data expansion of the infrared sensor array is integrated to distinguish customer identities.

4. A contactless payment method according to claim 3, characterized in that: Independent gravity sensors are evenly installed under each shelf layer of the unmanned vending machine to continuously collect the weight data of the goods and determine whether the customer has picked up the goods based on the weight change threshold.

5. A contactless payment method according to claim 4, characterized in that: Combined with the visual perception results, a spatiotemporal correlation model of product movement is established. By calculating the spatial distance between the customer's hand and the shelf and analyzing the changes in hand speed, a comprehensive correlation confidence scoring mechanism is established to determine the correlation between the product pickup event and the customer.

6. A contactless payment method according to claim 5, characterized in that: An abnormal behavior detection model based on convolutional neural networks and long short-term memory networks is constructed. Multimodal data is input and camera occlusion behavior is detected based on image brightness distribution, edge information changes, and image differences.

7. A contactless payment method according to claim 6, characterized in that: Create an independent virtual shopping basket for each customer and build a product ownership determination algorithm based on a decision tree; Using visual trajectory and gravity sensing data as decision-making features, and establishing a product ownership conflict detection and resolution mechanism when multiple customers pick up goods at the same time, products are given priority to the customer with the highest matching score, and products with unclear ownership and abnormal behavior are sent for manual review.

8. A contactless payment method according to claim 7, characterized in that: Based on the determined commodity ownership relationship and user virtual shopping basket data, a distributed bill generation strategy is implemented, and differentiated payment channel strategies are implemented according to the customer's risk level. At the same time, a multi-channel payment priority mechanism and a real-time notification and reconciliation mechanism for payment results are established.

9. A contactless payment device, used to implement the contactless payment method according to any one of claims 1 to 8, characterized in that: include: Visual data acquisition module, weight data acquisition module, pickup anomaly detection module, multi-user transaction separation module, and multi-channel contactless payment module; The visual data acquisition module uses a binocular camera and an infrared sensor array to collect data on customers' picking-up behavior in front of unmanned vending machines. It uses a posture estimation algorithm to extract key points of customers' hands, constructs and updates each customer's hand motion trajectory in real time, and tracks and distinguishes multiple customers in parallel. The weight data acquisition module integrates gravity sensors under the shelves of the unmanned vending machine to monitor the weight changes of goods in each shelf. Combined with the visual perception results, it establishes a spatiotemporal correlation model of the movement of goods and cross-validates the product picking behavior. The pickup anomaly detection module detects abnormal behaviors of customers when picking up their items based on a convolutional neural network model, assigns risk scores to identified high-risk behaviors, determines the risk level, and triggers corresponding risk control and processing strategies in real time; The multi-user transaction separation module designs a transaction isolation mechanism based on spatiotemporal marking, assigns an independent virtual shopping basket to each customer, and uses a decision tree algorithm to fuse visual trajectory and gravity sensing data to determine the ownership of goods when multiple customers pick up goods; The multi-channel contactless payment module implements a distributed bill generation strategy and a multi-channel contactless payment strategy based on the commodity ownership relationship and the user's virtual shopping basket and according to the risk level.

Images