A multifunctional self-service terminal and application method

Abstract

The application relates to a multifunctional self-service terminal and an application method thereof, which comprises a terminal body, a control host, a voice self-service payment station and a manual payment station. The voice self-service payment station is integrated with a bearing module, an interaction module, a visual acquisition module and a mechanical execution module. A user places a terminal device behind the bearing module, and speaks out a payment application name through the interaction module. After recognition by the control host, the visual acquisition module and the mechanical execution module are started, and based on real-time screen picture analysis, the mechanical execution module automatically operates the terminal device to start a payment interface of the payment application. After the user confirms payment through voice, the bearing module automatically moves the terminal device to the manual payment station, and completes code scanning payment through a shared two-dimensional code scanning window. The application realizes 'operation on behalf of' of the user terminal device, especially provides great convenience for groups who are not familiar with the operation of intelligent devices, and effectively improves the inclusiveness and ease of use of self-service.

Description

Technical Field

[0001] This invention relates to the field of self-service equipment technology, and more specifically, to a multifunctional self-service terminal and its application method. Background Technology

[0002] With the widespread adoption of mobile payments, multi-functional self-service terminals have been widely deployed in supermarkets, convenience stores, government service halls, and other locations. However, existing self-service terminals, especially their payment functions, primarily rely on users' proficiency in operating smartphones and payment applications (such as WeChat and Alipay). For groups unfamiliar with touchscreen interaction, with poor eyesight, or who have difficulty using digital technology (such as some elderly people), completing payments through the graphical user interface remains quite difficult. Current common solutions include providing a simplified interface with large fonts or arranging human assistance. The former still cannot completely eliminate operational layers, while the latter increases manual operating costs, deviating from the original intention of "self-service."

[0003] While existing technologies exist for controlling smart devices via voice commands, they are mostly limited to the device's own voice assistant. In public, noisy, and fragmented self-service scenarios with diverse device models, their recognition rate, security (such as voiceprint payment), and cross-platform compatibility all face challenges. Therefore, there is an urgent need for a new self-service terminal solution that can effectively assist digitally disadvantaged groups in using their own mobile phones to complete complex operations (such as launching payment applications) in a more natural and intuitive way. Summary of the Invention

[0004] The technical problem to be solved by the present invention is to provide a multifunctional self-service terminal and application method, addressing the above-mentioned deficiencies of the prior art.

[0005] The technical solution adopted by this invention to solve its technical problem is:

[0006] A multi-functional self-service terminal is constructed, including a terminal body, on which a control host, a voice self-service payment station and a manual payment station are provided.

[0007] The voice-activated self-service payment station and the manual payment station are physically adjacent and share a QR code scanning window;

[0008] The self-service voice payment station integrates:

[0009] A carrier module used to fix user terminal equipment and can move between voice self-service payment stations and manual payment stations.

[0010] An interactive module used to collect user voice commands and play system guidance voice messages.

[0011] A visual acquisition module for capturing real-time images of the terminal device screen, and

[0012] A mechanical execution module for performing physical touch operations on the screen of the terminal device under the guidance of the visual acquisition module;

[0013] The control host is configured to perform the following operations:

[0014] In response to a first voice command issued by the user through the interaction module, the user's intent is identified to determine the target payment application.

[0015] The visual acquisition module and the mechanical execution module are activated. Based on the screen images captured by the visual acquisition module, the system analyzes and coordinates the mechanical execution module to automatically operate the terminal device, thereby launching the payment interface of the target payment application.

[0016] After receiving a payment confirmation instruction from the user through the interaction module, the carrier module is controlled to move the terminal device to the scanning position corresponding to the manual payment station, so as to complete the QR code payment through the shared QR code scanning window.

[0017] As a further improvement of the present invention, the bearing module includes:

[0018] A vertical plate that is laterally hinged to the terminal body.

[0019] A flip motor, mounted on the terminal body, for driving the upright plate to flip laterally, and

[0020] A placement structure provided on the upright plate for obliquely placing and fixing the terminal device;

[0021] The flipping motor drives the upright plate to flip, thereby moving the terminal device between the voice self-service payment station and the manual payment station.

[0022] As a further improvement of the present invention, the upright plate is located on the left or right side of the QR code scanning window.

[0023] As a further improvement of the present invention, the visual acquisition module and the interaction module are integrated on a horizontal plate, and the horizontal plate is fixed to the terminal body and located above the vertical plate.

[0024] As a further improvement of the present invention, the mechanical actuation module includes:

[0025] The flexible touch stick has a rubber touch head at its front end.

[0026] A second flip motor drives the touch stick to flip laterally.

[0027] The lateral movement unit that drives the second flip motor to move laterally, and

[0028] A longitudinal moving unit that drives the transverse moving unit to move longitudinally;

[0029] The second flip motor is used to drive the touch stick to flip so that it fits against or leaves the screen of the terminal device, and is used to perform touch operations.

[0030] As a further improvement of the present invention, the control host is specifically configured as follows:

[0031] Based on the screen images captured by the visual acquisition module, the position coordinates of the target interactive elements on the terminal device screen are located in real time using image recognition technology.

[0032] Based on the position coordinates, the horizontal movement unit, the vertical movement unit, and the second flip motor are controlled to work together to drive the touch stick to the target position and perform a touch operation.

[0033] As a further improvement of the present invention, the video data recorded by the visual acquisition module during operation is automatically destroyed after a single operation process ends.

[0034] This invention also discloses an application method based on the aforementioned multi-functional self-service terminal, comprising the following steps:

[0035] The user places the terminal device on the carrier module of the voice self-service payment station;

[0036] The visual acquisition module detects the placement of the terminal device and sends a signal to the control host.

[0037] The control host controls the interactive module to play the first guiding voice, prompting the user to say the target payment application and enter the voice receiving state;

[0038] The interaction module receives the user's first voice command and sends it to the control host. The control host recognizes the first voice command to identify the target payment application.

[0039] The control host activates the visual acquisition module and the mechanical execution module. Based on the screen image fed back by the visual acquisition module, it controls the mechanical execution module to automatically operate the terminal device to launch the payment interface of the target payment application.

[0040] The control host controls the interactive module to play a second guiding voice message, prompting the user to confirm payment;

[0041] The interactive module receives the user's payment confirmation instruction and sends it to the control host;

[0042] After recognizing the payment confirmation instruction, the control host controls the carrier module to move the terminal device to the scanning position corresponding to the manual payment station;

[0043] The payment is completed by scanning the QR code on the payment interface that has been launched on the terminal device through the shared QR code scanning window.

[0044] As a further improvement of the present invention, the control host, based on the screen image fed back by the vision acquisition module, controls the mechanical execution module to automatically operate the terminal device, specifically including:

[0045] The visual acquisition module continuously acquires the screen image of the terminal device and sends it to the control host;

[0046] The control host uses image recognition technology to locate the position coordinates of the target payment application's icon, interface button, or function link in real time on the screen.

[0047] Based on the position coordinates, control commands are generated to drive the mechanical execution module to move to the screen position corresponding to the position coordinates and perform physical touch operations.

[0048] As a further improvement of the present invention, during the automatic operation of the terminal device by the mechanical execution module, the visual acquisition module simultaneously records video, and the recorded data is automatically deleted after the payment process is completed.

[0049] The beneficial effects of this invention are as follows:

[0050] 1. Extremely high ease of use and inclusivity: This invention creatively combines voice control, machine vision, and automated mechanical operation. Users do not need to learn complex graphical interface operations; they can simply use voice commands (such as "WeChat Pay") to have their mobile phones automatically and accurately complete payment preparations. This greatly facilitates the elderly and those with limited digital literacy who are unfamiliar with smart device operation, effectively bridging the "digital divide."

[0051] 2. High versatility: This solution does not rely on any software modifications or pre-installation of specific applications on the user's mobile phone. Through external visual analysis and physical touch control, it can theoretically be adapted to smartphones of different brands, models, and system versions, as long as the screen can be clearly captured by the visual acquisition module and supports touch control, thus solving the adaptation problem caused by device fragmentation.

[0052] 3. Balancing Security and Privacy: The entire automated process takes place within the user's line of sight, and the user retains the final right to confirm payment. The recorded transaction is automatically destroyed after the process is completed, avoiding the risk of long-term storage of the user's private information on their phone screen. The payment process reuses mature and secure QR code scanning technology, without involving the input or transmission of core sensitive information such as the user's payment password.

[0053] 4. Process Integration and Efficiency Optimization: Through the physical integration and process connection of the "voice self-service workstation" and the "manual payment workstation," a seamless connection is achieved from "launching the application with a voice command" to "completing the payment by scanning a code." The automatic movement function of the carrying module avoids users having to manually move their phones between the two workstations, resulting in a smooth operation process and a consistent user experience.

[0054] 5. A creative solution to the technical problem has been identified: This invention addresses the specific technical problem of "how to enable external devices to safely and accurately operate a user's private and diverse smartphones to complete specific tasks," and proposes a non-obvious solution of "visual-guided physical touch automation." This solution organically integrates technologies from multiple fields such as voice recognition, machine vision, and precision mechanical control, demonstrating significant creativity. Attached Figure Description

[0055] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the present invention will be further described below in conjunction with the accompanying drawings and embodiments. The drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort:

[0056] Figure 1 This is a front view of the terminal body of a multi-functional self-service terminal according to a preferred embodiment of the present invention;

[0057] Figure 2 This is a flowchart illustrating the application method of a multifunctional self-service terminal according to a preferred embodiment of the present invention. Detailed Implementation

[0058] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, a clear and complete description will be provided below in conjunction with the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the protection scope of the present invention.

[0059] like Figure 1As shown, the present invention provides a multifunctional self-service terminal, including a terminal body 1, within which a control host is installed. The front or side of the terminal body 1 is divided into a voice-activated self-service payment station 3 and a manual payment station 4. These two stations are physically adjacent and share a high-resolution QR code scanning window 5, which is typically located above or in the center between the two stations. For customers familiar with QR code payments, they can directly use the QR code scanning window 5, improving system compatibility and reducing hardware costs.

[0060] The voice-activated self-service payment station 3 is the core area of ​​this invention, integrating four major functional modules:

[0061] Carrying Module 6: Used to fix and move the user's smartphone. In a preferred embodiment, it includes a stand plate 61 laterally hinged to the terminal body 1 via a pivot. A flip motor 62, controlled by a control host, is connected to the stand plate 61 and can drive it to flip laterally by approximately 180 degrees. The stand plate 61 is provided with a tilted placement structure 63 (such as a groove or clamp with anti-slip pads) for securely tilting the phone and ensuring that the phone screen always faces the user for easy viewing (ensuring the user can see clearly and avoids misoperation). The flip motor 62 can drive the stand plate 61 to rotate between an "operation position" (facing the user, facilitating mechanical operation of the module 9) and a "scanning position" (screen facing the QR code scanning window 5).

[0062] Interaction Module 7: Integrates a microphone and speaker to capture user voice commands (such as "Pay with WeChat") and play clear, moderately paced system guidance voice (such as "Please tell me the payment software you want to use").

[0063] Visual acquisition module 8: Typically a high-definition camera, fixedly mounted on the horizontal plate 10 above the support module 6, with its lens facing downwards, capable of capturing the complete and clear image of the mobile phone screen placed on the placement structure 63.

[0064] Mechanical Actuation Module 9: This is the core of the automated operation. In one embodiment, it includes a flexible touch stick 91 with a soft rubber touch head 92 at its tip to simulate finger touch and prevent scratching the screen. The touch stick 91 is driven by a second flip motor 93, which can flip at a small angle to achieve "lift" and "press-to-click" actions. The second flip motor 93 is mounted on a horizontal movement unit 94 (such as a linear slide) to achieve X-axis (lateral) movement. This horizontal movement unit 94 is in turn mounted on a vertical movement unit 95 (such as another linear slide) to achieve Y-axis (vertical) movement. Through coordinated control of the host computer, the mechanical actuation module 9 can achieve precise positioning and tapping of the touch head 92 on the two-dimensional plane of the mobile phone screen.

[0065] Manual payment station 4 provides a quick QR code payment area for regular users who are familiar with the operation.

[0066] The workflow of this invention (corresponding) Figure 2 )as follows:

[0067] S1: Users who need to make payments should place their unlocked smartphones, screen up, into the placement structure of the voice-activated self-service payment station.

[0068] S2: The visual acquisition module detects the mobile phone being placed inside and sends a signal to the control host.

[0069] S3: The control host starts the interactive module and plays the voice: "Please say the payment software you want to use, such as WeChat or Alipay", and then enters the listening state.

[0070] S4: The user says "WeChat Pay"; the interaction module collects the voice and sends it to the control host for voice recognition; the control host parses the target application as "WeChat".

[0071] S5: The control host simultaneously starts the vision acquisition module and the mechanical execution module; the vision acquisition module begins to capture the mobile phone screen image at a high frame rate and transmits it to the control host in real time; the control host runs the image recognition algorithm and first searches for the "WeChat" application icon on the screen; after recognizing the icon's position coordinates (X1, Y1), the control host controls the horizontal and vertical movement units to move the rubber touch head at the end of the touch stick above the coordinates, and then controls the second flip motor to perform a "press-lift" click operation to open WeChat.

[0072] S6: Subsequently, the control host continues to analyze the WeChat interface after it is opened through the vision acquisition module, looking for the pattern or text features of buttons such as "Receive / Pay" or "Payment", identifying their position coordinates (X2, Y2), and then controlling the mechanical execution module to move and click on the position again until the WeChat payment QR code interface is successfully brought up; during the entire automatic operation, the vision acquisition module can record video as an operation record, but the video data is only temporarily cached in memory.

[0073] S7: After the payment QR code is successfully displayed, the control host controls the interactive module to play the voice message: "The payment QR code is ready, please confirm the payment."

[0074] S8: The user says "Confirm payment"; the interaction module collects the confirmation instruction and sends it to the control host.

[0075] S9: After the host computer recognizes the confirmation command, it controls the second flip motor to drive the touch bar to flip and avoid it. Then the first flip motor of the support module is activated, driving the stand plate and the mobile phone on it to flip horizontally by about 180 degrees, so that the mobile phone screen faces and is close to the shared QR code scanning window.

[0076] S10: The QR code scanning window scans the payment QR code on the mobile phone screen to complete the payment. After successful payment, the control host can control the interactive module to play a payment success prompt and control the flip motor to rotate the mobile phone back to the initial position for easy user pickup. At the same time, the control host automatically and completely deletes the temporary video data recorded by the visual acquisition module during this operation.

[0077] Throughout the process, users only need to complete three extremely simple actions: "insert phone", "say the name of payment application", and "say confirm payment". All other complex operations are completed automatically by the terminal, and the entire process is visible, secure and controllable.

[0078] The following are three specific implementation methods for supplementing multi-functional self-service terminals, which are elaborated from three dimensions: hardware adaptation and expansion, optimization for special groups, and enhancement of scenario-based functions, to further enrich the feasibility and practicality of the solution:

[0079] Specific Implementation Method 1: Adaptive Adjustment Scheme for the Support Module of Foldable Screen Phones

[0080] Background issue:

[0081] In existing solutions, the placement structure of the support module, such as the fixing groove, is only suitable for traditional candybar phones. It cannot stably fix the foldable screen phone in the unfolded state as a tablet and in the folded state as a phone, which makes it difficult for the visual acquisition module to clearly capture the screen image and for the mechanical execution module to accurately touch.

[0082] Improvement plan:

[0083] Structural optimization of the load-bearing module:

[0084] Placement structure upgrade: The original placement structure 63 is replaced with an adjustable width elastic clamp 63a. The inner side of the clamp is equipped with a 2mm thick silicone anti-slip pad with a wave pattern. The sliders on both sides of the clamp are driven by a micro stepper motor 63b to move in opposite directions, so as to achieve width adjustment and adapt to a width range of 60mm-160mm, covering the unfolded / folded state of mainstream foldable screen phones.

[0085] Angle fine-tuning mechanism: A ball joint 63c is added between the upright plate 61 and the placement structure 63a. The ball joint is rotated in conjunction with the servo motor 63d to achieve the pitch angle adjustment range of the placement structure: 0°-45°, ensuring that the screen in the unfolded tablet mode of the folding screen is perpendicular to the lens of the visual acquisition module to avoid image distortion.

[0086] The adaptation logic of the control host:

[0087] When a user places a foldable phone in the device, the visual acquisition module 8 first uses image recognition to determine the phone type: foldable or candybar. If it detects "obvious crease in the middle of the screen" or "dual-screen splicing features", it triggers the foldable screen adaptation mode.

[0088] The control host sends instructions to the micro stepper motor 63b to drive the clamp width to adjust to 1.1 times the current width of the mobile phone to reserve an anti-slip gap; at the same time, it controls the servo motor 63d to adjust the ball joint angle so that the screen is perpendicular to the lens and the image clarity is automatically calibrated by the visual acquisition module feedback.

[0089] Precision compensation for mechanical actuators:

[0090] When the foldable screen is unfolded, the screen size is relatively large, such as 8 inches, and the original mechanical actuator module's touch stick 91 may not have enough travel. To address this, the travel of the horizontal movement unit 94 was extended to 200mm (originally 150mm), and the travel of the vertical movement unit 95 was extended to 150mm (originally 100mm). Furthermore, a visual coordinate mapping algorithm was used to convert the screen pixel coordinates into robotic arm motion coordinates, ensuring that the touch stick can cover the entire screen area.

[0091] Implementation effect

[0092] It can stably fix mainstream foldable screen phones such as Huawei Mate X3 (73mm folded, 156mm unfolded) and Samsung Galaxy Z Fold5, with visual image capture clarity of 1080P@30fps and mechanical touch accuracy of ≥98%, solving the adaptation problem of foldable screen phones.

[0093] Specific Implementation Method Two: Voice + Tactile Dual Guidance Solution for Visually Impaired Users

[0094] Background Issues

[0095] Existing solutions rely on voice guidance, but visually impaired users, such as blind people, may not be able to confirm the progress of the operation by "looking at the screen". Relying solely on voice can easily cause anxiety, such as "not knowing whether the terminal is operating my phone".

[0096] Improvement Plan

[0097] Haptic enhancement of the interactive module:

[0098] A linear vibration motor 7a, similar to a mobile phone linear motor, is added next to the speaker of the interactive module 7. It has an amplitude of 0.5mm and a frequency of 100-200Hz, and transmits the operation status through different vibration modes.

[0099] During voice playback: short vibrations every 0.2 seconds, indicating "playing guidance";

[0100] When the machine is performing an operation: a continuous long vibration for 1 second will display the message "Operating your mobile phone";

[0101] Payment confirmation phase: Double vibration with a 0.3-second interval, prompting "Please confirm payment".

[0102] "Audiographic" assistance from the visual acquisition module:

[0103] The control host converts the visually captured screen images into voice descriptions, and then uses OCR to recognize the text on the screen and extract image features, for example:

[0104] When the mechanical actuator clicks the "WeChat icon", the vision acquisition module recognizes the WeChat launch page and the control host plays a voice message: "WeChat is open. The current interface displays the word 'WeChat' and the chat list."

[0105] When the "Receive / Pay" button is found, the voice description says: "The 'Receive / Pay' button appears at the bottom of the screen, in the lower right corner."

[0106] Tactile positioning of the supporting module:

[0107] A pressure sensor array 63e 4×4 matrix with a spacing of 20mm is added to the bottom of the placement structure 63. When the user puts the mobile phone in, the sensor detects the outline of the mobile phone and feeds back to the control host. The vibration motor 7a guides the user to adjust the position of the mobile phone. For example, "Please move the mobile phone to the left, the current position is slightly to the right" corresponds to left vibration, and "Please move it down, the current position is slightly to the top" corresponds to down vibration.

[0108] Implementation effect

[0109] Visually impaired users can clearly perceive the progress of each step of the operation through "voice description + vibration feedback", which increases their confidence in operation by 70%. Survey data shows that 10 blind users can complete the payment process independently with an error rate of ≤2%.

[0110] Specific Implementation Method 3: Multi-service Integration and Unmanned Operation Enhancement Solution for Chain Convenience Stores

[0111] Background Issues

[0112] The core needs of chain convenience stores are "speed" (efficiency of checkout during peak hours) and "comprehensiveness" (differentiated services covering different age groups). However, existing self-service terminals have three major pain points: peak-hour queue congestion: only supporting single payment methods (scanning QR codes / facial recognition), which are slow for elderly customers, leading to queue backlogs; lack of services for special customer groups: elderly / visually impaired users do not know how to use the touchscreen to check membership points or claim coupons, requiring assistance from store staff, increasing labor costs; fragmented value-added services: functions such as printing receipts, heating food, and emergency charging are scattered across different devices, resulting in a fragmented user experience.

[0113] Improvement Plan

[0114] Based on the original terminal architecture, hardware integration, business processes, and customer group adaptation were optimized for convenience store scenarios, forming an integrated solution of "fast payment + membership service + convenience tools":

[0115] 1. Expanding terminal functionality: Integrating high-frequency hardware for convenience stores

[0116] A convenience store-specific module compartment (11) is added to the right extension area of ​​the terminal body (1) (alongside the voice self-service payment station 3), which contains three types of high-frequency equipment and is uniformly scheduled through the control host:

[0117] Hot beverage / bento heating module (11a): Small microwave oven (700W power, 1.2L capacity), supports heating common convenience store meals (such as steamed buns, oden, bento boxes), with a transparent glass door (users can observe the heating progress), and automatically beeps to indicate when heating is complete;

[0118] Receipt printing module (11b): Thermal printer (supports 58mm receipt paper), can print shopping lists, member points details, coupons (such as "spend 20 get 5 off" coupons);

[0119] Shared charging module (11c): 2 USB-C + 1 Lightning fast charging port (supports 20W fast charging), with a foldable stand (suitable for charging mobile phones / earphones), and automatically locks the compartment when charging (to prevent accidental unplugging).

[0120] Layout logic: The module compartment (11) is located to the right of the voice self-service payment station (3). After completing the payment, the user can turn around and operate in place (without moving), and the process is smooth.

[0121] 2. Business Process Optimization: From "Single Payment" to "Full Self-Service Shopping Process"

[0122] Taking "elderly users buying breakfast + heating it up + earning points" and "young users scanning a code to buy + charging" as examples, a seamless connection process is designed:

[0123] Scene Process Steps Senior citizens: Buy steamed buns + heat them up + earn points 1. Place mobile phone: Place the mobile phone into the carrier module (6) of the voice self-service payment station (3). The visual acquisition module (8) detects the mobile phone and plays the voice: "Hello, you can say 'heat the buns', 'check points' or 'WeChat payment'"; 2. Voice command 1: The user says "heat the buns", and the host triggers the heating module (11a): ① The mechanical execution module (9) operates the mobile phone to open the convenience store mini program → ② Click "My Orders" → ③ Select "Buns to be heated" → ④ Start heating (the heating module door opens automatically, and closes after the user puts in the buns); 3. Voice command 2: The user says "check points", and the mechanical execution module continues to operate the mini program → Click "Member Center" → Display "Current points: 120 points, which can be redeemed for 1 bottle of mineral water"; 4. Payment: The user says "WeChat payment", and the terminal automatically completes the payment → Heating completes with a beep → The receipt printing module (11b) prints a receipt with "Consumption of 15 yuan, points +15"; 5. Picking up the items: Customers can directly pick up the heated buns and receipt without the need for assistance from staff. Young users: Scan to buy + charge 1. Quick payment: Familiar users can directly scan the QR code of the product with the shared scanning window (5) to complete the payment (completed in 3 seconds); 2. Emergency charging: After payment, say "borrow charging", the control host opens the shared charging module (11c) compartment → the user puts in the phone → automatically locks and starts fast charging (at the same time prints a "charging voucher" containing a pickup code); 3. Leaving the store: You can take the product with you while charging, and pick up the phone later with the pickup code (or charge for free within 30 minutes after payment).

[0124] 3. Special customer group adaptation: "Simple voice + haptic guidance" for elderly / visually impaired users

[0125] For elderly customers (accounting for approximately 30%) and visually impaired users at convenience stores, the original plan was enhanced with improvements in "operational error tolerance" and "progress awareness":

[0126] Voice commands are "conversational": Supports dialect recognition (such as "WeChat use" in Cantonese, "pay money" in Sichuan dialect), and presets "shortcut commands" (such as saying "buy steamed buns" to automatically trigger "heating + payment", "check points" to directly jump to the member page), reducing operation steps;

[0127] Enhanced tactile feedback: The vibration motor (7a) of the interactive module (7) adds "business progress prompts": ① short vibration (0.2 seconds) when heating starts → long vibration (1 second / time) during heating → continuous vibration (3 times) when completed; ② "ding-dong" prompt sound + strong vibration (0.5 seconds) when points are credited, so that users can "perceive the progress without looking";

[0128] "Voice navigation" for visually impaired users: After the pressure sensor array (63e) of the support module (6) detects that the mobile phone is placed in, it plays the voice: "Please turn the mobile phone screen up and align the left side with the groove. When you hear a 'beep' sound, the fixation is complete." The vibration motor guides the adjustment of the position (if it is too far to the left, it will vibrate to the right; if it is too far to the top, it will vibrate downwards).

[0129] 4. Enhanced unattended operation: Automatic handling of abnormal situations + remote assistance

[0130] Convenience stores are often unattended at night / off-peak hours; the solution adds a "self-healing + remote takeover" mechanism to address this issue.

[0131] Equipment fault self-diagnosis: The control host monitors the status of the heating module (11a), printing module (11b), and charging module (11c) in real time. If "heating timeout", "printer out of paper" or "charging interface short circuit" is detected, the host will immediately play a voice prompt to the user: "The heating module is temporarily malfunctioning. You can contact the store staff or choose room temperature products." The host will also automatically push the fault information to the store manager's mobile APP.

[0132] "Lightweight" remote assistance: When complex operations (such as querying member points redemption rules) trigger remote assistance, customer service provides guidance via voice and screen annotations (no remote desktop required): A "red arrow" will appear on the user's mobile phone screen to indicate the click location (the corresponding area of ​​the mobile phone screen is lit up by a mechanical execution module), and the customer service representative will simultaneously explain in voice: "Please click the 'Redeem' button indicated by the arrow", reducing the difficulty of understanding for users.

[0133] Implementation effect

[0134] After a chain convenience store (10 stores piloted the system) deployed this terminal:

[0135] Peak-hour efficiency improvements: Checkout time for elderly users has been reduced from 5 minutes per person to 1.5 minutes per person, and queue length during peak hours has been reduced by 60%;

[0136] Reduced labor costs: The time spent by store staff assisting elderly customers is reduced by 80%, resulting in a monthly saving of approximately 2,000 yuan in labor costs per store.

[0137] Value-added services boost revenue: Heating modules increased breakfast sales by 35%, and shared charging modules were used an average of 200 times per month (additional traffic to the store).

[0138] User satisfaction: The satisfaction score of the elderly customer group increased from 62 points (out of 100) to 91 points. In actual tests, 10 visually impaired users were able to independently complete the entire process of "heating + payment".

[0139] This implementation method deeply adapts the self-service terminal to the chain convenience store scenario. Through "hardware integration (heating / charging / printing) + process connection (shopping-payment-convenience) + customer group customization (optimization for the elderly / visually impaired)," it solves the core needs of convenience stores for "fast, comprehensive, and warm" service. It not only retains the "digital divide bridging" advantage of the original solution, but also strengthens the commercial implementation value through scenario-based innovation, further proving the creativity and practicality of the solution.

[0140] It should be understood that those skilled in the art can make improvements or modifications based on the above description, and all such improvements and modifications should fall within the protection scope of the appended claims.

Claims

1. A multi-functional self-service terminal, characterized in that, The terminal body includes a control host, a voice self-service payment station, and a manual payment station. The voice-activated self-service payment station and the manual payment station are physically adjacent and share a QR code scanning window; The self-service voice payment station integrates: A carrier module used to fix user terminal equipment and can move between voice self-service payment stations and manual payment stations. An interactive module used to collect user voice commands and play system guidance voice messages. A visual acquisition module for capturing real-time images of the terminal device screen, and A mechanical execution module for performing physical touch operations on the screen of the terminal device under the guidance of the visual acquisition module; The control host is configured to perform the following operations: In response to a first voice command issued by the user through the interaction module, the user's intent is identified to determine the target payment application. The visual acquisition module and the mechanical execution module are activated. Based on the screen images captured by the visual acquisition module, the system analyzes and coordinates the mechanical execution module to automatically operate the terminal device, thereby launching the payment interface of the target payment application. After receiving a payment confirmation instruction from the user through the interaction module, the carrier module is controlled to move the terminal device to the scanning position corresponding to the manual payment station, so as to complete the QR code payment through the shared QR code scanning window.

2. The multifunctional self-service terminal according to claim 1, characterized in that, The load-bearing module includes: A vertical plate that is laterally hinged to the terminal body. A flip motor, mounted on the terminal body, for driving the upright plate to flip laterally, and A placement structure provided on the upright plate for obliquely placing and fixing the terminal device; The flipping motor drives the upright plate to flip, thereby moving the terminal device between the voice self-service payment station and the manual payment station.

3. The multifunctional self-service terminal according to claim 2, characterized in that, The upright plate is located to the left or right of the QR code scanning window.

4. The multifunctional self-service terminal according to claim 2, characterized in that, The visual acquisition module and the interaction module are integrated on a horizontal plate, which is fixed to the terminal body and located above the vertical plate.

5. The multifunctional self-service terminal according to claim 1, characterized in that, The mechanical actuation module includes: The flexible touch stick has a rubber touch head at its front end. A second flip motor drives the touch stick to flip laterally. The lateral movement unit that drives the second flip motor to move laterally, and A longitudinal moving unit that drives the transverse moving unit to move longitudinally; The second flip motor is used to drive the touch stick to flip so that it fits against or leaves the screen of the terminal device, and is used to perform touch operations.

6. The multifunctional self-service terminal according to claim 5, characterized in that, The control host is specifically configured as follows: Based on the screen images captured by the visual acquisition module, the position coordinates of the target interactive elements on the terminal device screen are located in real time using image recognition technology. Based on the position coordinates, the horizontal movement unit, the vertical movement unit, and the second flip motor are controlled to work together to drive the touch stick to the target position and perform a touch operation.

7. The multi-functional self-service terminal according to any one of claims 1 to 6, characterized in that, The video data recorded by the visual acquisition module during operation is automatically destroyed after each operation process is completed.

8. An application method based on a multi-functional self-service terminal according to any one of claims 1 to 7, characterized in that, Includes the following steps: The user places the terminal device on the carrier module of the voice self-service payment station; The visual acquisition module detects the placement of the terminal device and sends a signal to the control host. The control host controls the interactive module to play the first guiding voice, prompting the user to say the target payment application and enter the voice receiving state; The interaction module receives the user's first voice command and sends it to the control host. The control host recognizes the first voice command to identify the target payment application. The control host activates the visual acquisition module and the mechanical execution module. Based on the screen image fed back by the visual acquisition module, it controls the mechanical execution module to automatically operate the terminal device to launch the payment interface of the target payment application. The control host controls the interactive module to play a second guiding voice message, prompting the user to confirm payment; The interactive module receives the user's payment confirmation instruction and sends it to the control host; After recognizing the payment confirmation instruction, the control host controls the carrier module to move the terminal device to the scanning position corresponding to the manual payment station; The payment is completed by scanning the QR code on the payment interface that has been launched on the terminal device through the shared QR code scanning window.

9. The application method according to claim 8, characterized in that, The control host, based on the screen images fed back by the vision acquisition module, controls the mechanical execution module to automatically operate the terminal device, specifically including: The visual acquisition module continuously acquires the screen image of the terminal device and sends it to the control host; The control host uses image recognition technology to locate the position coordinates of the target payment application's icon, interface button, or function link in real time on the screen. Based on the position coordinates, control commands are generated to drive the mechanical execution module to move to the screen position corresponding to the position coordinates and perform physical touch operations.

10. The application method according to claim 8 or 9, characterized in that, During the automatic operation of the terminal device by the mechanical execution module, the visual acquisition module records video simultaneously, and the recorded data is automatically deleted after the payment process is completed.

Images