Settlement method and system of weighing and settlement all-in-one machine

Through the face recognition and human body binding scheme based on RGB images and depth images, the problem of cumbersome tray binding steps and the risk of mis-tracking in the self-service meal pickup system is solved, and efficient and accurate customer identity recognition and meal pickup price calculation are achieved. , reducing merchant costs and misidentification risks.

CN119027117AActive Publication Date: 2024-11-26GUANGZHOU PAIKEPUSHI INFORMATION TECH CO LTD
View PDF 4 Cites 0 Cited by

Patent Information

Application Number
CN202411131886.3
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-16
Publication Date
2024-11-26
Estimated Expiration
2044-08-16

AI Technical Summary

Technical Problem

In the existing self-service meal pickup system, the tray binding steps are cumbersome and costly, and the multi-target tracking algorithm is prone to misidentification, resulting in incorrect identity identification and improper deductions.

Method used

Using face recognition and human body binding scheme based on RGB images and depth images, the face frame and human body frame are extracted, and the living body detection and target tracking are performed to replace the traditional pallet binding process. , and set the threshold value by setting the inter-frame depth distance to reduce the risk of false tracking.

Benefits of technology

No need for customers to bind pallets, which reduces merchant costs and the complexity of dining processes, improves customer dining efficiency, reduces the risk of mis-tracking, and improves the stability and reliability of the system.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119027117A_ABST
    Figure CN119027117A_ABST
Patent Text Reader

Abstract

The invention discloses a settlement method and system of a weighing and settlement all-in-one machine, and relates to the technical field of food sales. The method comprises the following steps: acquiring an RGB image and a depth image of a target customer; extracting a face frame and a human body frame of the target customer in the RGB image; acquiring center coordinate information of the face frame according to the RGB image; acquiring depth information of the human body frame according to the depth image; when the to-be-recognized condition is met, human face sub-images in the RGB image are intercepted, human face recognition is carried out, and corresponding ID information is obtained; binding the human body frame and the ID information to perform target tracking on the human body frame, unbinding when an unbinding condition is satisfied, and generating a meal taking price corresponding to the ID information; the unbinding condition comprises that human body frame tracking is lost or the inter-frame depth distance between the human body frame of the current frame and the human body frame of the previous frame exceeds a preset threshold value; according to the invention, the tray binding step is reduced, the merchant cost is saved, the customer dining efficiency is improved, and the error tracking risk can be reduced.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of food sales, and in particular to a settlement method and system of an all-in-one weighing and settlement machine. Background Art

[0002] With the rapid advancement of science and technology and the increasingly accelerated pace of life in modern society, self-service systems have been widely infiltrated into all walks of life, especially in the catering industry. The application of self-service meal pickup and settlement systems has become an important trend to promote service efficiency and optimize customer consumption experience. In the current self-service meal pickup system, the weighing and settlement integrated machine is the core terminal device, and its application is particularly common. When using it, the user must first scan the code to bind the personal payment method with the tray equipped with PFID or one-dimensional QR code, and then place the tray in the meal pickup identification area. The device determines the entire process from the start to the end of meal pickup by identifying the tray information. During the meal pickup process, the system will calculate the weight of the meal and generate the corresponding fee. After the meal is picked up, the system will automatically complete the settlement of the meal fee through the payment deduction platform pre-authorized by the user, realizing the automation and convenience of the entire process.

[0003] The core of this self-service meal pickup system is how to bind the meal pickup price to the person who picks up the meal. The main modes on the market are: 1. Embed RFID chips on the trays, and the users scan the code to pick up the trays, bind the trays to the users, and then bind the users to the meal pickup price through the device identifying the tray chip; 2. Print one-dimensional and two-dimensional codes on the trays for scanning and identification to replace RFID chips, reducing the cost of trays; 3. Use multiple surveillance cameras to identify and track the current food pickup personnel; this solution replaces the original method of binding the meal pickup price through the tray as a medium through face recognition, and determines the start and end of meal pickup through pedestrian tracking.

[0004] However, the above modes all have their shortcomings: in the first mode, a large number of trays with chips need to be purchased, which is costly, and diners are required to scan the code in advance to bind the trays, which increases the steps of dining; in the second mode, a large number of trays also need to be purchased. Although the cost of chips is replaced by printing one-dimensional and two-dimensional trays, they are easily damaged during the disinfection and cleaning process, and the dining steps of diners are increased; in the third mode, the use of multi-channel camera personnel tracking reduces the steps required to bind trays, but there is a problem of high cost, because one more surveillance camera needs to be purchased for each or two scales, and a private server is required for every few scales; the installation of surveillance cameras has high requirements on the on-site environment, is difficult to construct, and has low flexibility.

[0005] At the same time, in scenes involving uniforms, if only pure RGB images are relied upon for target tracking, there will be certain risks, especially in crowded places such as cafeterias; due to the consistency of uniforms, when multiple people wearing the same uniform move in the picture, the target tracking algorithm may mistakenly "inherit" the target frame of a pedestrian to another person wearing similar clothes. This misidentification is particularly serious in payment scenes and may lead to incorrect identification and improper deductions. Although lowering the matching threshold of the target frame seems to solve some of the mistracking problems, in fact, this approach will make the tracking conditions for the same person more stringent, thereby reducing the accuracy and stability of tracking. Summary of the invention

[0006] In order to solve the problems existing in the prior art, the present invention provides a settlement method and system of a weighing and settlement integrated machine, which can reduce the tray binding steps, save merchant costs, improve customer dining efficiency, and reduce the risk of mistracking.

[0007] To achieve the above object, the present invention provides a settlement method of a weighing and settlement integrated machine, comprising:

[0008] Obtain the RGB image and depth image of the target customer;

[0009] Extracting the face frame and body frame of the target customer in the RGB image;

[0010] According to the RGB image, obtain the center coordinate information of the face frame, and according to the depth image, obtain the depth information of the human body frame;

[0011] When the conditions to be recognized are met, a face sub-image in the RGB image is intercepted; the conditions to be recognized include: determining that the face frame is located at the target recognition position according to the center coordinate information, determining that the target customer is a living person based on the face frame, and determining that the corresponding human body frame is located in a preset area according to the depth information;

[0012] Performing face recognition based on the face sub-image to obtain corresponding ID information;

[0013] Binding the human body frame and the ID information to perform target tracking on the human body frame;

[0014] When the unbinding condition is met, the binding between the human body frame and the ID information is released, and a meal pickup price corresponding to the ID information is generated; the unbinding condition includes: the inter-frame depth distance between the human body frame of the current frame and the human body frame of the previous frame exceeds a preset threshold.

[0015] Optionally, extracting a face frame and a body frame of the target customer in the RGB image includes:

[0016] Using the pose model of YOLO v8 to detect the human skeleton points of the target customer in the RGB image;

[0017] The human skeleton points are converted into a human body frame and a human face frame.

[0018] Optionally, before acquiring the depth information of the human body frame according to the depth image, the method further includes:

[0019] Collect the point set X on the x-axis of the RGB image RGB , and the point set X on the x-axis of the depth image TOF ;

[0020] Use the mapping function F to map the point set X RGB and point set X TOF To collaborate; the expression of the mapping function F is: F = [(X RGB , 1) T (X RGB , 1)] -1 (X RGB , 1) T X TOF ;

[0021] Where T represents the transpose of the matrix.

[0022] Optionally, acquiring depth information of the human body frame according to the depth image includes:

[0023] Acquire TOF data corresponding to the depth image;

[0024] The left boundary L of the human body frame RGB and the right boundary R RGB The median value D mapped on the TOF data is used as the depth information; the expression of the median value D is:

[0025] D=Median(d([x i , 1]·F)), i=[L RGB , L RGB +1, ..., R RGb -1, R RGR ];

[0026] Where i represents the x-axis component set from the left boundary to the right boundary of the human body frame; x i represents the element in the x-component vector of the RGB image coordinates; F represents the mapping function between the RGB camera and the TOF camera about the X-axis coordinates; d(·) represents the depth information output by the tof camera; Median() represents median filtering.

[0027] Optionally, the expression of the preset threshold is:

[0028]

[0029] Where, t represents the preset threshold; D diff Indicates the empirical value of the depth distance between frames; D max Indicates D diff Weighted maximum effective distance; D min Indicates D diff Weighted minimum effective distance; D oli Indicates the depth distance of the human body frame in the previous frame.

[0030] Optionally, generating a meal pickup price corresponding to the ID information includes:

[0031] The meal price corresponding to the ID information is generated according to the first meal plate weight of the weighing area obtained initially and the second meal plate weight of the weighing area obtained last during the binding of the human body frame and the ID information.

[0032] Optionally, obtain the RGB image and depth image of the target customer, including:

[0033] Use the RGB camera to shoot at a horizontal angle to obtain the RGB image of the target customer;

[0034] The TOF camera is used to shoot downward at a preset inclination angle to obtain a depth image of the target customer; the RGB camera and the TOF camera are integrated on the weighing and settlement integrated machine.

[0035] Optionally, obtain the RGB image and depth image of the target customer, including:

[0036] Acquire RGB images of target customers by using multiple RGB cameras set at different locations;

[0037] Based on the RGB image, a depth image of the target customer is obtained through camera calibration and depth calculation.

[0038] The present invention also provides a settlement system of a weighing and settlement integrated machine, comprising:

[0039] Image acquisition module, used to acquire RGB images and depth images of target customers;

[0040] Image processing module for:

[0041] Extracting the face frame and body frame of the target customer in the RGB image;

[0042] According to the RGB image, obtain the center coordinate information of the face frame, and according to the depth image, obtain the depth information of the human body frame;

[0043] Face recognition module for:

[0044] When the conditions to be recognized are met, a face sub-image in the RGB image is intercepted; the conditions to be recognized include: determining that the face frame is located at the target recognition position according to the center coordinate information, determining that the target customer is a living person based on the face frame, and determining that the corresponding human body frame is located in a preset area according to the depth information;

[0045] Performing face recognition based on the face sub-image to obtain corresponding ID information;

[0046] Tracking pricing module for:

[0047] Binding the human body frame and the ID information to perform target tracking on the human body frame;

[0048] When the unbinding condition is met, the binding between the human body frame and the ID information is released, and a meal pickup price corresponding to the ID information is generated; the unbinding condition includes: the inter-frame depth distance between the human body frame of the current frame and the human body frame of the previous frame exceeds a preset threshold.

[0049] According to the specific embodiments provided by the present invention, the present invention discloses the following technical effects:

[0050] The settlement method of the weighing and settlement integrated machine provided by the present invention does not require the customer to bind a tray, and uses a face recognition and body binding solution based on RGB images and depth images, which replaces the previous dining steps that require pre-binding of plates, reduces the merchant's costs while reducing the dining process of diners; in terms of obtaining the RGB image and depth image of the target customer, it is only necessary to install the corresponding RGB camera and TOF camera, and the deployment is relatively simple without the need for additional construction; when performing multi-target tracking, the RGB image and the depth image are combined, which effectively reduces the risk of mistracking that may occur when relying solely on RGB images for visual tracking. Compared with the existing multi-target pedestrian tracking algorithm, it can reduce the situation where human body tracking is "inherited" without changing the tracking effect of the same person. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] The above and other objects, features and advantages of the present invention will become more apparent through a more detailed description of exemplary embodiments of the present invention in conjunction with the accompanying drawings, wherein like reference numerals generally represent like components throughout the exemplary embodiments of the present invention.

[0052] Figure 1 A schematic diagram of a method flow of a settlement method of a weighing and settlement integrated machine according to an embodiment of the present invention;

[0053] Figure 2A schematic diagram of a target customer tracking process according to an embodiment of the present invention;

[0054] Figure 3 The present invention is a schematic diagram of the module structure of the settlement system of the integrated weighing and settlement machine shown in an embodiment of the present invention. DETAILED DESCRIPTION

[0055] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0056] The purpose of the present invention is to provide a settlement method and system of a weighing and settlement integrated machine, which can reduce the tray binding steps, save merchant costs and improve customer dining efficiency, while also reducing the risk of mistracking.

[0057] See also Figure 1 , Figure 1 The above-mentioned method for the settlement of the weighing and settlement machine includes:

[0058] Step 101: Obtain an RGB image and a depth image of a target customer.

[0059] In the application, the RGB camera can be used to shoot horizontally to obtain the RGB image of the target customer; and the TOF camera can be used to shoot downward at a preset angle to obtain the depth image of the target customer. The RGB camera and the TOF camera are integrated on the weighing and settlement machine. The TOF camera can be an 8*8 pixel TOF camera.

[0060] Among them, the RGB camera shoots at a horizontal angle, focusing on capturing the facial information and body movements of the target customers; while the TOF camera shoots downward at a certain inclination angle (preset inclination), so that it can adapt to diners of different heights and ensure that the distance dynamics of each target customer can be accurately captured.

[0061] Compared with the previous solution of installing multiple surveillance cameras, the present invention only adds a TOF camera and an RGB camera for position binding, which reduces the hardware cost and deployment difficulty, is more economical and easy to implement.

[0062] In the application, the RGB image of the target customer can be obtained by using multiple RGB cameras set at different positions; based on the RGB image, the depth image of the target customer can be obtained through camera calibration and depth calculation.

[0063] Among them, multiple RGB cameras, such as two RGB cameras, are respectively set at different positions, so that through camera calibration and depth calculation, the position of the target customer can be determined and the corresponding depth image can be obtained.

[0064] In addition, the same effect can be achieved by setting up an RGBD camera. The RGBD camera can obtain both RGB images and depth images; the depth information of the RGBD camera can be obtained by a TOF camera, other structured light cameras, or binocular RGB cameras, and the one-to-one pixel calibration between sensors is done.

[0065] Step 102: extracting the face frame and body frame of the target customer in the RGB image.

[0066] In the application, deep learning models (such as YOLO, SSD, etc.) can be used to process RGB images to extract the face frame and body frame of the target customer.

[0067] In one embodiment, the above step 102 includes:

[0068] Use the pose model of YOLO v8 to detect the human skeleton points of the target customer in the RGB image;

[0069] Convert human skeleton points into human body frames and face frames.

[0070] Step 103: Obtain the center coordinate information of the face frame according to the RGB image, and obtain the depth information of the body frame according to the depth image.

[0071] In the application, the real-time location of the target customer can be known based on the depth image, and then by fusing the depth image and the RGB image, the location of the target customer's face frame on the RGB image, that is, the center coordinate information, can be obtained. Similarly, the distance between the target customer's body frame and the TOF camera, that is, the depth information, can also be obtained.

[0072] In one embodiment, in order to ensure that the information between the RGB camera and the TOF camera can be integrated, before step 103, the following operations may be performed:

[0073] Collect the point set X on the x-axis of the RGB image RGB , and the point set X on the x-axis of the depth image TOF ;

[0074] Using the mapping function F to the point set X RGB and point set X TOF To collaborate; the expression of the mapping function F is: F = [(X RGB , 1) T (X RGB , 1)] -1(X RGB , 1) T X TOF Among them, collaboration is to establish point set X RGB and point set X TOF The mapping relationship between them.

[0075] In one embodiment, in the above step 103, obtaining the depth information of the human body frame according to the depth image specifically includes:

[0076] Obtaining TOF data corresponding to the depth image; wherein the TOF data includes distance information between each pixel in the human body frame and the TOF camera;

[0077] The left boundary L of the human body box RGB and the right boundary R RGB The median D mapped on the TOF data is used as the depth information; the expression of the median D is:

[0078] D=Median(d([x i , 1]·F)), i=[L RGB , L RGB +1, ..., R RGB -1, R RGB ];

[0079] Where i represents the x-axis component set from the left boundary to the right boundary of the human body frame; x i represents the element in the x-component vector of the RGB image coordinates; F represents the mapping function between the RGB camera and the TOF camera about the X-axis coordinates; d(·) represents the depth information output by the tof camera; Median() represents median filtering.

[0080] Step 104: When the conditions to be recognized are met, a face sub-image in the RGB image is intercepted.

[0081] Among them, the conditions to be identified include: determining that the face frame is located at the target identification position according to the center coordinate information, determining that the target customer is alive based on the face frame, and determining that the corresponding human body frame is located in a preset area according to the depth information.

[0082] The target recognition position refers to the position interval of the face frame in the RGB image, so as to facilitate face recognition. In the application, when the detected face frame meets the preset conditions (the recognition position meets the requirements, the liveness detection passes), and the corresponding human frame meets the specific distance range (D min , D max ), a face sub-image of the target customer can be captured from the RGB image. It can be understood that the preset area corresponds to D min To D maxFor example, the preset area can be set to be an area within a certain distance from the TOF camera, such as 0.1 meter to 1 meter.

[0083] When performing liveness detection, it is possible to determine whether the face frame is alive based on the detailed features in the RGB image (such as skin texture, blinking movements, etc.); further, liveness detection can be performed in conjunction with an IR camera (infrared camera), using the principle that screens (such as mobile phone displays) cannot form images under IR, and that photo-type paper does not match the imaging texture. This can effectively deal with the situation of using other people's photos for face recognition and prevent misidentification.

[0084] Step 105: Perform face recognition based on the face sub-image to obtain corresponding ID information.

[0085] In the application, the captured face sub-image can be input into the face recognition system, and the corresponding customer ID information can be obtained by comparing it with the pre-stored face information in the database.

[0086] Step 106: Bind the human frame and ID information to perform target tracking on the human frame.

[0087] In the application, the acquired ID information is bound to the previously detected human frame to achieve continuous tracking of the target customer while ensuring that the currently tracked individual has the correct identity.

[0088] Step 107: When the unbinding condition is met, the binding between the human frame and the ID information is released, and a meal pickup price corresponding to the ID information is generated.

[0089] The unbinding condition includes: the inter-frame depth distance between the human body frame of the current frame and the human body frame of the previous frame exceeds a preset threshold.

[0090] See also Figure 2 , when the diners (target customers) are taking the dishes, the currently bound human body can be tracked throughout the process, and the target can be continuously locked until the tracking signal is interrupted, and then the binding is released. Considering that the uniform uniform may cause confusion in the tracking process, the present invention sets an empirical value D on the inter-frame depth distance diff To make adaptive judgment; in the application, the depth distance D of the human body frame in the previous frame can be used old Generate the inter-frame distance threshold T (preset threshold) required for the current frame.

[0091] Exemplarily, the expression of the preset threshold value T may be:

[0092]

[0093] Where, t represents the preset threshold; D diff Indicates the empirical value of the depth distance between frames; Dmax Indicates D diff Weighted maximum effective distance; D min Indicates D diff Weighted minimum effective distance; D old Indicates the depth distance of the human body frame in the previous frame.

[0094] Among them, the inter-frame depth distance refers to the difference between the first distance between the human body frame and the TOF camera in the previous frame and the second distance between the human body frame and the TOF camera in the next frame in two adjacent frames of images; among them, the distance between the human body frame and the TOF camera can be expressed by the above median D.

[0095] This means that no matter whether the target is temporarily lost for some reason during the target tracking process, as long as the inter-frame depth distance of the tracked target exceeds the preset threshold, that is: DD old >t; it will give up marking the object, actively end the current meal collection process, and generate the meal collection price; thus reducing the situation of improper deductions due to tracking errors. This design can improve the stability and reliability of the self-service weighing and settlement machine application and provide a guarantee for the dining experience of diners.

[0096] In one embodiment, generating a meal pickup price corresponding to the ID information includes:

[0097] The meal pickup price corresponding to the ID information is generated based on the first meal plate weight of the weighing area initially obtained during the binding of the human body frame and the ID information, and the second meal plate weight of the weighing area finally obtained.

[0098] The settlement method provided by the present invention can be applied to the canteen food weighing and settlement integrated machine; the previous canteen weighing and settlement integrated machine system often needs to bind the tray to complete the process of serving food and calculating the price when in use, while the present invention uses face recognition and body tracking technology to completely replace the cumbersome process of tray binding; this not only significantly reduces the hardware cost of the canteen, reduces the investment and maintenance costs of the equipment, but also simplifies the operation steps of the diners and improves the dining experience. In addition, by using the depth image of the TOF camera, the risk of mistracking that may occur when relying solely on RGB images for visual tracking is also effectively reduced.

[0099] Corresponding to the aforementioned application function realization method embodiment, the present invention also provides a settlement system of a weighing and settlement integrated machine and corresponding embodiments.

[0100] See also Figure 3 , Figure 3 The above-mentioned settlement system includes:

[0101] An image acquisition module 31 is used to acquire an RGB image and a depth image of a target customer;

[0102] The image processing module 32 is used for:

[0103] Extract the face frame and body frame of the target customer in the RGB image;

[0104] According to the RGB image, the center coordinate information of the face frame is obtained, and according to the depth image, the depth information of the human body frame is obtained;

[0105] The face recognition module 33 is used to:

[0106] When the conditions to be recognized are met, a face sub-image in the RGB image is intercepted; the conditions to be recognized include: determining that the face frame is located at the target recognition position according to the center coordinate information, determining that the target customer is alive based on the face frame, and determining that the corresponding human body frame is located in a preset area according to the depth information;

[0107] Perform face recognition based on face sub-graphs to obtain corresponding ID information;

[0108] The tracking and pricing module 34 is used to:

[0109] Bind the human frame and ID information to track the human frame;

[0110] When the unbinding conditions are met, the binding between the human body frame and the ID information is released, and a meal pickup price corresponding to the ID information is generated; the unbinding conditions include: the inter-frame depth distance between the human body frame of the current frame and the human body frame of the previous frame exceeds a preset threshold.

[0111] Regarding the system in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated again here.

[0112] The embodiments of the present invention have been described above, and the above description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Many modifications and changes will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The selection of terms used herein is intended to best explain the principles of the embodiments, practical applications, or improvements to the technology in the market, or to enable other persons of ordinary skill in the art to understand the embodiments disclosed herein.

Claims

1. A settlement method for a weighing and settlement integrated machine, characterized in that: include: Obtain the RGB image and depth image of the target customer; Extracting the face frame and body frame of the target customer in the RGB image; According to the RGB image, obtain the center coordinate information of the face frame, and according to the depth image, obtain the depth information of the human body frame; When the conditions to be recognized are met, a face sub-image in the RGB image is intercepted; the conditions to be recognized include: determining that the face frame is located at the target recognition position according to the center coordinate information, determining that the target customer is a living person based on the face frame, and determining that the corresponding human body frame is located in a preset area according to the depth information; Performing face recognition based on the face sub-image to obtain corresponding ID information; Binding the human body frame and the ID information to perform target tracking on the human body frame; When the unbinding condition is met, the binding between the human body frame and the ID information is released, and a meal pickup price corresponding to the ID information is generated; the unbinding condition includes: the inter-frame depth distance between the human body frame of the current frame and the human body frame of the previous frame exceeds a preset threshold.

2. The settlement method of the weighing and settlement integrated machine according to claim 1, characterized in that: Extracting the face frame and body frame of the target customer in the RGB image includes: Using the pose model of YOLO v8 to detect the human skeleton points of the target customer in the RGB image; The human skeleton points are converted into a human body frame and a human face frame.

3. The settlement method of the weighing and settlement integrated machine according to claim 1, characterized in that: Before acquiring the depth information of the human body frame according to the depth image, the method further includes: Collect the point set X on the x-axis of the RGB image RGB , and the point set X on the x-axis of the depth image TOF ; Use the mapping function F to map the point set X RGB and point set X TOF To collaborate; the expression of the mapping function F is: F = [(X RGB , 1) T (X RGB , 1)] -1 (X RGB , 1) T X TOF .

4. The settlement method of the weighing and settlement integrated machine according to claim 3, characterized in that: Acquiring depth information of the human body frame according to the depth image includes: Acquire TOF data corresponding to the depth image; The left boundary L of the human body frame RGB and right boundary X RGB The median value D mapped on the TOF data is used as the depth information; the expression of the median value D is: D=Median(s([x i ,1]·F)),i=[L RGB ,L RGB +1,...,R RGB -1,R RGB ]; Where i represents the x-axis component set from the left boundary to the right boundary of the human body frame; x i represents the element in the x-component vector of the RGB image coordinates; F represents the mapping function between the RGB camera and the TOF camera about the x-axis coordinates; d(·) represents the depth information output by the TOF camera; Median() represents median filtering.

5. The settlement method of the weighing and settlement integrated machine according to claim 1, characterized in that: The expression of the preset threshold is: Where, t represents the preset threshold; D diff Indicates the empirical value of the depth distance between frames; D max Indicates D diff Weighted maximum effective distance; D min Indicates D diff Weighted minimum effective distance; D old Indicates the depth distance of the human body frame in the previous frame.

6. The settlement method of the weighing and settlement integrated machine according to claim 1, characterized in that: Generate a meal pickup price corresponding to the ID information, including: The meal price corresponding to the ID information is generated according to the first meal plate weight of the weighing area obtained initially and the second meal plate weight of the weighing area obtained last during the binding of the human body frame and the ID information.

7. The settlement method of the weighing and settlement integrated machine according to claim 1, characterized in that: Get the target customer's RGB image and depth image, including: Use the RGB camera to shoot at a horizontal angle to obtain the RGB image of the target customer; The TOF camera is used to shoot downward at a preset inclination angle to obtain a depth image of the target customer; the RGB camera and the TOF camera are integrated on the weighing and settlement integrated machine.

8. The settlement method of the weighing and settlement integrated machine according to claim 1, characterized in that: Get the target customer's RGB image and depth image, including: Acquire RGB images of target customers by using multiple RGB cameras set at different locations; Based on the RGB image, a depth image of the target customer is obtained through camera calibration and depth calculation.

9. A settlement system of a weighing and settlement integrated machine, characterized in that: include: Image acquisition module, used to acquire RGB images and depth images of target customers; Image processing module for: Extracting the face frame and body frame of the target customer in the RGB image; According to the RGB image, obtain the center coordinate information of the face frame, and according to the depth image, obtain the depth information of the human body frame; Face recognition module for: When the conditions to be recognized are met, a face sub-image in the RGB image is intercepted; the conditions to be recognized include: determining that the face frame is located at the target recognition position according to the center coordinate information, determining that the target customer is a living person based on the face frame, and determining that the corresponding human body frame is located in a preset area according to the depth information; Performing face recognition based on the face sub-image to obtain corresponding ID information; Tracking pricing module for: Binding the human body frame and the ID information to perform target tracking on the human body frame; When the unbinding condition is met, the binding between the human body frame and the ID information is released, and a meal pickup price corresponding to the ID information is generated; the unbinding condition includes: the inter-frame depth distance between the human body frame of the current frame and the human body frame of the previous frame exceeds a preset threshold.

Citation Information

Patent Citations

  • Intelligent unmanned supermarket control system

    CN111127174A

  • Digital canteen non-inductive payment method

    CN112183306A

  • Human face and human body identity association method and system, medium and equipment

    CN116935460A

  • Human body and face feature linear coordinate matching algorithm based on human body bone joint feature points

    CN117496207A