A self-service system, data processing method, terminal and storage medium for adding AI functions

Abstract

The present invention discloses a self-service system with enhanced AI capabilities, a data processing method, a terminal, and a storage medium. The self-service system with enhanced AI capabilities includes: an original self-service module, an AI front-end module, an APP add-on module, and an AI server module. The original self-service module is connected to the APP add-on module, and the AI front-end module, the APP add-on module, and the AI server module are interconnected. Without changing the application operation interface of ordinary self-service devices, and without modifying the hardware or software of the self-service devices that have been deployed and used, the present invention can perform natural interactions with the devices through voice and vision, complete the device interaction actions that were originally performed by users through visual understanding and screen clicks, enable the self-service devices to understand user questions and intentions, complete the consultations or business handling required by users, achieve the effect similar to that of a manual window service, and save the cost of equipment replacement.

Description

Technical Field

[0001] The present invention relates to the technical field of data processing, and particularly to a self-service system with enhanced AI capabilities, a data processing method, a terminal, and a computer-readable storage medium. Background Art

[0002] Public self-service devices refer to machine devices placed in public places for citizens to handle various services independently. These devices usually integrate multiple functions such as identity recognition, information query, and service handling, aiming to relieve the queuing pressure at the manual service windows and improve service efficiency. However, when existing self-service devices are in use, users need to flip through the menu functions on the terminal by themselves to search. For users who are not familiar with the function names on the self-service devices, it is very difficult to find the corresponding menu and click to handle according to their own problems, and they need the assistance of staff or volunteers.

[0003] With the emergence of large language models in AI (Artificial Intelligence), AI has been able to complete semantic understanding of conversations with people. Empowering self-service devices with AI makes it a reality for users to ask questions and for AI to complete the operations of self-service devices. However, when adding AI capabilities to current self-service devices, it usually requires changing the application operation interface of ordinary self-service devices, modifying the hardware or software of the self-service devices that have been deployed and used, and the process of AI empowerment is complicated, increasing the equipment replacement cost.

[0004] Therefore, the existing technology still needs to be improved and developed. Summary of the Invention

[0005] The main purpose of the present invention is to provide a self-service system with enhanced AI capabilities, a data processing method, a terminal, and a computer-readable storage medium, aiming to solve the problem that when adding AI capabilities to current self-service devices in the existing technology, it usually requires changing the application operation interface of ordinary self-service devices, modifying the hardware or software of the self-service devices that have been deployed and used, resulting in a complicated process of AI empowerment and a high equipment replacement cost.

[0006] To achieve the above object, the present invention provides a self-service system with enhanced AI capabilities, and the self-service system with enhanced AI capabilities includes:

[0007] An original self-service module, an AI front-end module, an APP external module, and an AI server module, where the original self-service module is connected to the APP external module, and the AI front-end module, the APP external module, and the AI server module are connected to each other;

[0008] The original self-service module is used to display a self-service interface on the screen and execute self-service functions;

[0009] The AI front-end module is used to detect the user's wake-up signal and send the wake-up signal to the APP plug-in module;

[0010] The APP plug-in module is used to enter the working state after receiving the wake-up signal;

[0011] The AI front-end module is also used to, after detecting the wake-up signal, execute corresponding actions according to a pre-configured welcome script, collect the user's input content, and send the input content to the AI server module;

[0012] The APP plug-in module is also used to obtain the screen content of the self-service interface and send the packaged screen content to the AI server module;

[0013] The AI server module is used to process the input content and the screen content, generate a response result, generate a script file according to the response result, and send the script file to the AI front-end module;

[0014] The AI front-end module is also used to receive the script file and send the script file to the corresponding units of the APP plug-in module and the AI front-end module respectively to automatically execute the self-service function.

[0015] Optionally, for the self-service system with enhanced AI function, the AI front-end module includes a MONITOR unit, an AWU unit, an SCU unit, a VCU unit, an SPU unit, a DPPU unit, and a DTD unit;

[0016] The MONITOR unit is used to coordinate and manage the communication between the AI front-end module, the APP plug-in module, and the AI server module;

[0017] The AWU unit is used to detect the user's wake-up signal and send the wake-up signal to the APP plug-in module;

[0018] The SCU unit is used to collect the user's voice input and send the voice input to the AI server module after detecting the wake-up signal;

[0019] The VCU unit is used to collect the user's video input and send the video input to the AI server module after detecting the wake-up signal;

[0020] The SPU unit is used to play the corresponding voice content according to the voice playback instruction in the script file;

[0021] The DPPU unit is used to execute the corresponding action according to the digital human action instruction in the script file;

[0022] The DTD unit is used to play the corresponding text content according to the dialogue text display instruction in the script file.

[0023] Optionally, in the self-service system with enhanced AI function, the APP plug-in module includes: an SRU unit and an ICSU unit;

[0024] The SRU unit is used to identify and obtain the screen content of the self-service interface;

[0025] The ICSU unit is used to simulate keyboard input operations or screen click operations after receiving the script file.

[0026] Optionally, in the self-service system with enhanced AI function, the AI server module includes: an STT unit, a VIA unit, an SCR unit, a KAG unit, and a SCRIPT GENERATOR unit;

[0027] The STT unit is used to receive the voice input in the input content and convert the voice input into text information;

[0028] The VIA unit is used to receive the video input in the input content, identify and analyze the video input, and obtain the user's facial information;

[0029] The SCR unit is used to receive the screen content of the self-service interface and analyze the screen content through screen OCR technology to obtain the user input box position and input requirements;

[0030] The KAG unit is used to receive the text information, the facial information, the user input box position, and the input requirements, and input the text information, the facial information, the user input box position, and the input requirements into a pre-trained knowledge model to generate a response result;

[0031] The SCRIPT GENERATOR unit is used to receive the response result and generate a script file according to the response result.

[0032] Optionally, in the self-service system with enhanced AI function, the AI front-end module runs on an AI box independent of the original self-service module;

[0033] The APP plug-in module runs on the original self-service module;

[0034] The AI server module runs on the AI box, an independent AI server, or a cloud AI server.

[0035] In addition, to achieve the above object, the present invention also provides a data processing method for a self-service system with enhanced AI capabilities, wherein the data processing method includes:

[0036] After the AI front-end module detects a wake-up signal from the user, it sends the wake-up signal to the APP add-on module, and the APP add-on module enters the working state after receiving the wake-up signal;

[0037] The AI front-end module performs corresponding actions according to a pre-configured welcome script, collects the input content of the user, and sends the input content to the AI server module;

[0038] The APP add-on module obtains the screen content of the self-service interface, and sends the screen content after packaging to the AI server module;

[0039] The AI server module processes the input content and the screen content, generates a response result, generates a script file according to the response result, and sends the script file to the AI front-end module;

[0040] The AI front-end module receives the script file, and sends the script file to the corresponding units of the APP add-on module and the AI front-end module respectively to automatically execute the self-service functions of the original self-service module.

[0041] Optionally, in the data processing method for the self-service system with enhanced AI capabilities, before the step of, after the AI front-end module detects a wake-up signal from the user, it sends the wake-up signal to the APP add-on module, further includes:

[0042] After the original self-service module enters the working state, initialize the SCU unit and the VCU unit of the AI front-end module;

[0043] Detect the wake-up signal of the user through the microphone of the SCU unit and the camera of the VCU unit.

[0044] Optionally, in the data processing method for the self-service system with enhanced AI capabilities, the script file includes: voice playback instructions, digital human action instructions, dialogue text display instructions, and APP operation instructions;

[0045] The step of sending the script file to the corresponding units of the APP add-on module and the AI front-end module respectively to automatically execute the self-service functions of the original self-service module specifically includes:

[0046] Send the voice playback instruction to the SPU unit of the AI front-end module, and the SPU unit plays the corresponding voice content according to the voice playback instruction;

[0047] Send the digital human action instruction to the DPPU unit of the AI front-end module, and the DPPU unit executes the corresponding action according to the digital human action instruction;

[0048] Send the dialogue text display instruction to the DTD unit of the AI front-end module, and the DTD unit plays the corresponding text content according to the dialogue text display instruction;

[0049] Send the APP operation instruction to the ICSU unit of the APP external module, and the ICSU unit simulates keyboard input operations or screen click operations according to the APP operation instruction.

[0050] In addition, to achieve the above object, the present invention also provides a terminal, wherein the terminal includes: a memory, a processor, and a data processing program of a self-service system with enhanced AI functions stored on the memory and executable on the processor. When the data processing program of the self-service system with enhanced AI functions is executed by the processor, the steps of the data processing method of the self-service system with enhanced AI functions as described above are implemented.

[0051] In addition, to achieve the above object, the present invention also provides a computer-readable storage medium, wherein the computer-readable storage medium stores a data processing program of a self-service system with enhanced AI functions. When the data processing program of the self-service system with enhanced AI functions is executed by a processor, the steps of the data processing method of the self-service system with enhanced AI functions as described above are implemented.

[0052] The present invention discloses a self-service system with enhanced AI functions, a data processing method, a terminal, and a storage medium. The self-service system with enhanced AI functions includes: an original self-service module, an AI front-end module, an APP external module, and an AI server module. The original self-service module is connected to the APP external module, and the AI front-end module, the APP external module, and the AI server module are connected to each other. The present invention can, without changing the application operation interface of ordinary self-service devices, and without modifying the hardware or software of the self-service devices that have been deployed and used, perform natural interactions with the devices through voice and vision, complete the device interaction actions that were originally performed by users through visual understanding and screen clicks, enable the self-service devices to understand user questions and intentions, complete the consultations or business handling required by users, achieve the effect similar to that of a manual window service, and save the cost of equipment replacement. Description of the Drawings

[0053] Figure 1 It is the overall architecture diagram of the self-service system with AI function added in the present invention;

[0054] Figure 2 It is the flowchart of the preferred embodiment of the data processing method of the self-service system with AI function added in the present invention;

[0055] Figure 3 It is the structure diagram of the preferred embodiment of the terminal in the present invention. Detailed implementation manners

[0056] To make the purpose, technical solutions and advantages of the present invention clearer and more definite, the following further describes the present invention in detail with reference to the accompanying drawings and by way of examples. It should be understood that the specific examples described herein are only used to explain the present invention and are not used to limit the present invention.

[0057] To solve the problems in the prior art, this embodiment provides a self-service system with AI function added, as Figure 1 shown, the original self-service module, AI front-end module, APP plug-in module and AI server module.

[0058] Among them, the original self-service module is connected to the APP plug-in module, and the AI front-end module, the APP plug-in module and the AI server module are connected to each other.

[0059] The original self-service module is used to display the self-service interface on the screen and execute the self-service function;

[0060] The AI front-end module is used to detect the user's wake-up signal and send the wake-up signal to the APP plug-in module;

[0061] The APP plug-in module is used to enter the working state after receiving the wake-up signal;

[0062] The AI front-end module is also used to execute corresponding actions according to the pre-configured welcome script after detecting the wake-up signal, collect the user's input content, and send the input content to the AI server module;

[0063] The APP plug-in module is also used to obtain the screen content of the self-service interface and send the screen content after packaging to the AI server module;

[0064] The AI server module is used to process the input content and the screen content, generate a response result, generate a script file according to the response result, and send the script file to the AI front-end module;

[0065] The AI front-end module is also used to receive the script file and send the script file to the corresponding units of the APP plug-in module and the AI front-end module respectively to automatically execute the self-service function.

[0066] It can be understood that the AI front-end module, the AI plug-in module, and the AI server module together form an AI assistant. The AI front-end module runs on an AI box independent of the original self-service module; the APP plug-in module runs on the original self-service module; the AI server module runs on the AI box, an independent AI server, or a cloud AI server. Among them, the AI box can be an independent computer device or an intelligent terminal device such as a user's mobile phone. The AI assistant ensures that without changing the application operation interface of the ordinary self-service device, without modifying the hardware or software of the self-service device that has been deployed and used, it can naturally interact with the device through voice and vision, complete the device interaction actions that the original user performed through visual understanding and screen clicking, enable the self-service device to understand the user's questions and intentions, and complete the consultation or business handling required by the user.

[0067] Furthermore, the present invention also provides a data processing unit included in each module for performing corresponding data collection and processing operations.

[0068] Specifically, the AI front-end module includes a MONITOR unit (monitor unit), an AWU unit (Activation Wake-Up Unit), an SCU unit (Speech Collection Unit), a VCU unit (Video Collection Unit), an SPU unit (Speech Playback Unit), a DPPU unit (Digital Person Playback Unit), and a DTD unit (Dialog Text Display).

[0069] Among them, the MONITOR unit is used to coordinate and manage the communication between the AI front-end module, the APP plug-in module, and the AI server module; the AWU unit is used to detect the user's wake-up signal and send the wake-up signal to the APP plug-in module; the SCU unit is used to collect the user's voice input after detecting the wake-up signal and send the voice input to the AI server module; the VCU unit is used to collect the user's video input after detecting the wake-up signal and send the video input to the AI server module; the SPU unit is used to play the corresponding voice content according to the voice playback instruction in the script file; the DPPU unit is used to execute the corresponding action according to the digital human action instruction in the script file; the DTD unit is used to play the corresponding text content according to the dialogue text display instruction in the script file.

[0070] Further, the APP plug-in module includes: an SRU unit (Screen Recognition Unit) and an ICSU unit (Input and Click Simulation Unit).

[0071] Among them, the SRU unit is used to identify and obtain the screen content of the self-service interface; the ICSU unit is used to simulate keyboard input operations or screen click operations after receiving the script file.

[0072] Furthermore, the AI server module includes: an STT unit (Speech to Text), a VIA unit (Video Identification and Analysis), an SCR unit (Screen content recognition), a KAG unit (Knowledge Augmented Generation), and a SCRIPT GENERATOR unit.

[0073] Among them, the STT unit is used to receive the voice input in the input content and convert the voice input into text information; the VIA unit is used to receive the video input in the input content, identify and analyze the video input, and obtain the user's facial information; the SCR unit is used to receive the screen content of the self-service interface and analyze the screen content through screen OCR (Optical Character Recognition) technology to obtain the user input box position and input requirements; the KAG unit is used to receive the text information, the facial information, the user input box position and the input requirements, and input the text information, the facial information, the user input box position and the input requirements into a pre-trained knowledge model to generate a response result; the SCRIPTGENERATOR unit is used to receive the response result and generate a script file according to the response result.

[0074] Furthermore, the original self-service module is an existing self-service device, including hardware, system and application software. The original self-service module includes an APP unit, which is a general term for application systems running on existing self-service devices and is an application program or multiple application programs. It can be understood that, among them, the modules of the system of the present invention can be connected in the forms of physical wiring, program communication, network communication, etc., and the units can be connected in the forms of program communication, network communication, etc.

[0075] As can be seen from the above, the present invention ensures that without changing the application operation interface of ordinary self-service devices, without modifying the hardware or software of the self-service devices that have been deployed and used, natural interaction with the devices can be achieved through voice and vision, and the device interaction actions that the original users performed through visual understanding and screen clicking can be completed.

[0076] Based on the self-service system with added AI function described in the above embodiments, the present invention also provides a data processing method for the self-service system with added AI function, specifically as Figure 2 shown, the data processing method of the self-service system with added AI function includes the following steps:

[0077] Step S10: When the AI front-end module detects the user's wake-up signal, send the wake-up signal to the APP plug-in module, and the APP plug-in module enters the working state after receiving the wake-up signal.

[0078] Before the step of when the AI front-end module detects the user's wake-up signal and sends the wake-up signal to the APP plug-in module, it further includes:

[0079] After the original self-service module enters the working state, initialize the SCU unit and the VCU unit of the AI front-end module;

[0080] Detect the wake-up signal of the user through the microphone of the SCU unit and the camera of the VCU unit.

[0081] It can be understood that the detection method of user approach can be a single method or a combination of infrared detection, radio wave detection, voice detection or video detection, and the intention to interact with the self-service device (the detection method of the intention to interact with the AI assistant is that the user presses the start chat button or the duration of the person arrival signal reaches the preset value, the voice persists to the preset value or the video captures the preset value, one or more combinations).

[0082] Further, when it is detected that the user approaches the device, trigger the wake-up signal: when the AWU unit detects that someone stays in front of the self-service device or talks to the device (through the microphone or camera), the AWU unit will immediately wake up the MONITOR unit of the APP plug-in module. After receiving the wake-up signal, the MONITOR unit enters the working state. At the same time, it is also possible to choose to perform AI front-end initialization at this time.

[0083] Step S20: The AI front-end module performs corresponding actions according to the pre-configured welcome script, collects the input content of the user, and sends the input content to the AI server module.

[0084] After the AI front-end module receives the wake-up signal, the AI front-end module performs corresponding actions according to the pre-configured welcome script and plays a welcome message (for example: "Welcome to use the self-service. How can I help you?"). At the same time, the MONITOR unit notifies the DPPU unit of the AI front-end module to perform welcome actions (for example: the digital human smiles and waves, etc.).

[0085] Further, the AI front-end module also wakes up the SCU unit and the VCU unit of the AI front-end module. The SCU unit and the VCU unit are always ready to record the voice and video of the user (collect the input content of the user). Among them, the SCU unit starts to record a voice input of the user. The duration can be set according to the demand (for example, 5 seconds) or start recording automatically when the user starts speaking, and the recording ends when the duration of the user's speech stillness reaches the set time, such as 1 second. The VCU unit can optionally record a video input of the user at the same time (such as face recognition or other video analysis), and the duration needs to be the same as that of the SCU unit.

[0086] Further, directly pass the input content to the AI server module for subsequent intelligent processing (or send the input content to the MONITOR unit of the APP plug-in module, and then the MONITOR unit packages and sends it to the AI server module).

[0087] Step S30: The APP plug-in module obtains the screen content of the self-service interface, packages the screen content, and sends it to the AI server module.

[0088] Specifically, the MONITOR unit of the APP plug-in module receives the input content. At the same time, the SRU unit of the APP plug-in module intercepts the screen content of the current self-service interface to obtain the currently displayed screen content (such as menus, options, etc.). Finally, the screen content is packaged and sent to the AI server module (or the input screen content can also be directly passed to the AI server module).

[0089] Step S40: The AI server module processes the input content and the screen content, generates a response result, generates a script file according to the response result, and sends the script file to the AI front-end module.

[0090] Specifically, after receiving the data, the AI server module starts the following processing steps for its underlying units: First, the STT unit converts the audio into text and extracts the user's voice commands. Then, the VIA unit identifies and analyzes the video to extract information such as the user's facial expressions and gestures. Next, the SCR unit analyzes the screenshot content through technologies such as screen OCR to extract the position of the user input box in the screenshot image or information, input requirements (such as phone numbers or addresses, etc., as well as click positions and corresponding information, such as confirm or cancel). Finally, the KAG unit generates a corresponding response result based on the pre-trained knowledge model, combined with the user's voice commands, video analysis results, and the current screen content.

[0091] Further, the Script Generator unit creates a script file containing the following content according to the generated response result: Voice playback instructions, used to tell the SPU unit the voice content to be played. Dialogue text display instructions, used to tell the DTD unit the text content to be displayed. Digital human action instructions, used to tell the DPPU unit the actions to be performed (such as nodding, smiling, pointing in a certain direction, etc.). APP operation instructions, used to tell the ICSU unit the page URL (Uniform Resource Locator) to be opened, simulated keyboard input, or the order of screen click positions (such as clicking a certain button, entering a password, etc.).

[0092] Step S50: The AI front-end module receives the script file and sends the script file to the corresponding units of the APP plug-in module and the AI front-end module respectively to automatically execute the self-service function of the original self-service module.

[0093] In this embodiment, the script file includes: voice playback instructions, digital human action instructions, dialogue text display instructions, and APP operation instructions.

[0094] The sending the script file to the corresponding units of the APP plug-in module and the AI front-end module to automatically execute the self-service function of the original self-service module specifically includes:

[0095] Sending the voice playback instructions to the SPU unit of the AI front-end module, and the SPU unit plays the corresponding voice content according to the voice playback instructions;

[0096] Sending the digital human action instructions to the DPPU unit of the AI front-end module, and the DPPU unit executes the corresponding actions according to the digital human action instructions;

[0097] Sending the dialogue text display instructions to the DTD unit of the AI front-end module, and the DTD unit plays the corresponding text content according to the dialogue text display instructions;

[0098] Sending the APP operation instructions to the ICSU unit of the APP plug-in module, and the ICSU unit simulates keyboard input operations or screen click operations according to the APP operation instructions.

[0099] It can be understood that after automatically executing the self-service function of the original self-service module according to the script file, if the user's requirements are not completed, the above process will be repeated to continue the interaction until the user completes all operations or leaves the device. When the user completes all operations or leaves the device, the AWU unit detects that the user has left the device, and the MONITOR unit will notify the SPU unit and the DPPU unit to end the current session and restore the system to the initial state, waiting for the next user wake-up.

[0100] Furthermore, in another embodiment, the units of the self-service system with enhanced AI capabilities communicate by emulating the IP communication of the current network: The unit ID is analogous to the IP for accessing the Internet on a computer and is a coordinate identifier pointing to the communication object upon which communication between two units depends. The prerequisite for a unit to send a directed notification to another unit is that the communication packet includes its own unit ID and the ID of the other unit. For example, if Unit A wants to obtain its own unit ID and the ID of another unit, it can read them from its own unit's preset file when powered on. If it cannot be read, it needs to send a broadcast notification to all units, requesting a reply from the units that know. All units can receive this broadcast notification. For example, after receiving it, the AI server module will send a reply notification, which Unit A can save in the preset file for future use. That is to say, in this embodiment, each unit has a unique ID in the system as the unit ID, and each unit can obtain its own unit ID or the ID of other units. The way to obtain the unit ID can be to read the unit ID or query the unit ID.

[0101] Among them, a notification is a process of information exchange between units. Each piece of information in a unit communication is called a notification and consists of parts such as the source unit, target unit, source unit type, information type, chat ID, session ID, and information body. The source unit refers to the unit ID of the communication initiator, the target unit refers to the unit ID of the communication recipient, the source unit type is the unit type of the communication initiator unit, the information type is predefined by the system for the target unit to understand the received information classification, and the information body is a byte stream generated by the source unit according to a set of information rules for the communication between the source unit and the target unit predefined by the system.

[0102] When reading the unit ID, it is necessary to find the unit ID from the unit preset file according to the unit type. Each unit has a unit preset file, which can be a part of the software code or an independent file. The unit preset file also stores its own unit type, the other unit types of the digital human where this unit is located, and the unique identifier of the hardware associated with these units when the unit is installed. The unique hardware identifier refers to the unique identifier of the AI box or self-service device where the unit is located in the system.

[0103] Furthermore, one AI assistant consists of one or more digital humans and one AI server module, and one digital human consists of one AI front-end module and one AI plug-in module. Each module consists of one or more units. Among them, the units of one AI front-end module can be stored and run on one AI box or multiple AI boxes, and one AI box can be an independent computer device, a user's mobile phone, or the original self-service device. The units of one AI plug-in module are stored and run on the self-service device module. The units of one AI server module can be stored and run on one or more AI boxes, an independent server, or a cloud server.

[0104] Each unit of the digital human has an executable action table, which is stored in the unit preset file by the system administrator when each unit is installed, and a copy is also stored in the SCRIPT GENERATOR unit of the AI server, or is sent to the SCRIPT GENERATOR unit by the unit notification during the initialization of each unit of the digital human and saved in the SCRIPT GENERATOR unit. The storage form of the SCRIPT GENERATOR unit is unit ID + executable action table. Among them, a script file consists of one or more instruction sets, an instruction set consists of 1 instruction header and an instruction body, the instruction body consists of 1 or more instructions, the instruction header represents the target unit, and the instruction represents one or more actions to be performed by the target unit.

[0105] At the beginning of each chat, the AI server module will assign a system-unique ID as the identifier of the chat message, called the chat ID. At the beginning of each round of conversation, MONITOR will assign a unique ID under this chat ID as the identifier of the conversation information, called the conversation ID. Generally, the conversation ID can start from 0 and increment by 1 each time.

[0106] As an example, the following is the process for a unit to obtain the unit ID of a specified type of unit (it can obtain its own unit ID or the unit ID of other units):

[0107] Step 1: Read the unit ID from the unit preset file according to the unit type. If it fails, execute Step 2.

[0108] Step 2: The unit sends a notification to query the unit ID. If it can read its own unit ID, the original unit ID is its own unit ID; otherwise, the source unit ID in the notification is empty; read its own unit type in the preset file as the source unit type; the information type is to query the unit ID; the information body contains the hardware unique identifier and the unit type of the unit ID to be queried; if it can read the unit ID of MONITOR, the target unit ID of the notification is the MONITOR unit ID; otherwise, if it can read the unit ID of the AI server module, the target unit ID of the notification is the AI server module unit ID; otherwise, the target unit ID of the notification is empty.

[0109] Step 3: The unit checks the notification received with the information type of querying the unit ID reply, and saves the unit type, unit ID, and unit-associated hardware unique identifier in the information body to the preset file.

[0110] When a user operates on a self-service device without AI function, the user needs to flip through the pages according to the menu function of the terminal by himself / herself, and it is very difficult to find the corresponding menu to click and handle according to his / her own questions. There is a need for volunteers or volunteers to assist. For this problem, the self-service system with enhanced AI function designed by the present invention does not need to eliminate the existing public self-service devices, does not require the participation of the original self-service device manufacturer, reduces the cost of self-service device transformation, and saves a large amount of social costs. At the same time, the implementation of the present invention can empower a large number of existing self-service device manufacturers. After they purchase the implementation products of the present invention as accessories for the original self-service devices, they can become manufacturers capable of producing AI self-service devices.

[0111] Further, as Figure 3 shown, based on the above self-service system with enhanced AI function and the data processing method of the self-service system with enhanced AI function, the present invention also correspondingly provides a terminal, which includes a processor 10, a memory 20 and a display 30. Figure 3 Only some components of the terminal are shown, but it should be understood that it is not required to implement all the shown components, and more or fewer components can be implemented alternatively.

[0112] The memory 20 may be an internal storage unit of the terminal in some embodiments, such as the hard disk or memory of the terminal. The memory 20 may also be an external storage device of the terminal in other embodiments, such as a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) card, a Flash Card, etc. equipped on the terminal. Further, the memory 20 may also include both the internal storage unit and the external storage device of the terminal. The memory 20 is used to store application software installed on the terminal and various types of data, such as the program code for installing the terminal. The memory 20 may also be used to temporarily store data that has been output or will be output. In one embodiment, a data processing program 40 of the self-service system with enhanced AI function is stored on the memory 20, and the data processing program 40 of the self-service system with enhanced AI function can be executed by the processor 10, so as to implement the data processing method of the self-service system with enhanced AI function in the present application.

[0113] The processor 10 may be a central processing unit (CPU), a microprocessor or other data processing chips in some embodiments, and is used to run the program code stored in the memory 20 or process data, such as executing the data processing method of the self-service system with enhanced AI function.

[0114] In some embodiments, the display 30 may be an LED display, a liquid crystal display, a touch liquid crystal display, an OLED (Organic Light-Emitting Diode) touch device, etc. The display 30 is used to display information of the terminal and to display a visual user interface. Components 10-30 of the terminal communicate with each other via a system bus.

[0115] In one embodiment, when the processor 10 executes the data processing program 40 of the self-service system with enhanced AI capabilities in the memory 20, the following steps are implemented:

[0116] When the AI front-end module detects a wake-up signal from the user, it sends the wake-up signal to the APP plug-in module, and the APP plug-in module enters the working state after receiving the wake-up signal;

[0117] The AI front-end module performs corresponding actions according to a pre-configured welcome script, collects the input content of the user, and sends the input content to the AI server module;

[0118] The APP plug-in module obtains the screen content of the self-service interface, and sends the packaged screen content to the AI server module;

[0119] The AI server module processes the input content and the screen content, generates a response result, generates a script file according to the response result, and sends the script file to the AI front-end module;

[0120] The AI front-end module receives the script file, and sends the script file to the corresponding units of the APP plug-in module and the AI front-end module respectively to automatically execute the self-service functions of the original self-service module.

[0121] Among them, before the step of when the AI front-end module detects a wake-up signal from the user and sends the wake-up signal to the APP plug-in module, it further includes:

[0122] After the original self-service module enters the working state, initialize the SCU unit and the VCU unit of the AI front-end module;

[0123] Detect the wake-up signal of the user through the microphone of the SCU unit and the camera of the VCU unit.

[0124] Among them, the script file includes: voice playback instructions, digital human action instructions, dialogue text display instructions, and APP operation instructions;

[0125] Sending the script file to the corresponding units of the APP plug-in module and the AI front-end module respectively to automatically execute the self-service function of the original self-service module specifically includes:

[0126] Sending the voice playback instruction to the SPU unit of the AI front-end module, and the SPU unit plays the corresponding voice content according to the voice playback instruction;

[0127] Sending the digital human action instruction to the DPPU unit of the AI front-end module, and the DPPU unit executes the corresponding action according to the digital human action instruction;

[0128] Sending the dialogue text display instruction to the DTD unit of the AI front-end module, and the DTD unit plays the corresponding text content according to the dialogue text display instruction;

[0129] Sending the APP operation instruction to the ICSU unit of the APP plug-in module, and the ICSU unit simulates keyboard input operations or screen click operations according to the APP operation instruction.

[0130] The present invention also provides a computer-readable storage medium, wherein the computer-readable storage medium stores a data processing program of a self-service system with an increased AI function, and when the data processing program of the self-service system with an increased AI function is executed by a processor, the steps of the data processing method of the self-service system with an increased AI function as described above are implemented.

[0131] In summary, the present invention provides a self-service system with an increased AI function, a data processing method, a terminal, and a storage medium. The self-service system with an increased AI function includes: an original self-service module, an AI front-end module, an APP plug-in module, and an AI server module. The original self-service module is connected to the APP plug-in module, and the AI front-end module, the APP plug-in module, and the AI server module are connected to each other. Without changing the application operation interface of ordinary self-service devices, the present invention can, without modifying the hardware or software of the self-service devices that have been deployed and used, perform natural interactions with the devices through voice and vision, complete the device interaction actions that the original users performed through visual understanding and screen clicks, enable the self-service devices to understand the user's questions and intentions, complete the consultation or business handling required by the users, achieve the effect similar to that of a manual window service, and save the cost of equipment replacement.

[0132] It should be noted that in this text, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, such that a process, method, article or terminal comprising a series of elements not only includes those elements, but also includes other elements not expressly listed, or further includes elements inherent to such process, method, article or terminal. Without further limitation, an element defined by the statement "comprising one..." does not exclude the existence of additional identical elements in the process, method, article or terminal comprising such element.

[0133] Of course, those of ordinary skill in the art can understand that all or part of the processes of implementing the above-described embodiments of the method can be completed by instructing relevant hardware (such as a processor, a controller, etc.) through a computer program. The program can be stored in a computer-readable storage medium readable by a computer. When the program is executed, it can include the processes of the above-described method embodiments. The computer-readable storage medium can be a memory, a magnetic disk, an optical disk, etc.

[0134] It should be understood that the application of the present invention is not limited to the above examples. For those of ordinary skill in the art, improvements or transformations can be made according to the above description. All such improvements and transformations should fall within the protection scope of the appended claims of the present invention.

Claims

1. A self-service system with AI function added, characterized in that: The self-service system with added AI function includes: an original self-service module, an AI front-end module, an APP plug-in module and an AI server module, wherein the original self-service module is connected to the APP plug-in module, and the AI ​​front-end module, the APP plug-in module and the AI ​​server module are connected to each other; The original self-service module is used to display the self-service interface on the screen and perform the self-service function; The AI ​​front-end module is used to detect a user's wake-up signal and send the wake-up signal to the APP plug-in module; The APP plug-in module is used to enter a working state after receiving the wake-up signal; The AI ​​front-end module is also used to perform corresponding actions according to a pre-configured welcome script after detecting the wake-up signal, collect user input content, and send the input content to the AI ​​server module; The APP plug-in module is also used to obtain the screen content of the self-service interface, and package the screen content and send it to the AI ​​server module; The AI ​​server module is used to process the input content and the screen content, generate a response result, generate a script file according to the response result, and send the script file to the AI ​​front-end module; The AI ​​front-end module is also used to receive the script file, and send the script file to the corresponding units of the APP plug-in module and the AI ​​front-end module to automatically execute the self-service function; The AI ​​front-end module includes a MONITOR unit, an AWU unit, an SCU unit, a VCU unit, an SPU unit, a DPPU unit and a DTD unit; The MONITOR unit is used to coordinate and manage the communication between the AI ​​front-end module, the APP plug-in module and the AI ​​server module; The AWU unit is used to detect a user's wake-up signal and send the wake-up signal to the APP plug-in module; The SCU unit is used to collect the user's voice input after detecting the wake-up signal, and send the voice input to the AI ​​server module; The VCU unit is used to collect the user's video input after detecting the wake-up signal, and send the video input to the AI ​​server module; The SPU unit is used to play the corresponding voice content according to the voice playing instruction in the script file; The DPPU unit is used to execute corresponding actions according to the digital human action instructions in the script file; The DTD unit is used to play the corresponding text content according to the dialogue text display instruction in the script file; The AI ​​server module includes: an STT unit, a VIA unit, an SCR unit, a KAG unit and a SCRIPT GENERATOR unit; The STT unit is used to receive the voice input in the input content and convert the voice input into text information; The VIA unit is used to receive the video input in the input content, and identify and analyze the video input to obtain the user's facial information; The SCR unit is used to receive the screen content of the self-service interface, and analyze the screen content through screen OCR technology to obtain the user input box position and input requirements; The KAG unit is used to receive the text information, the facial information, the user input box position and the input requirement, and input the text information, the facial information, the user input box position and the input requirement into a pre-trained knowledge model to generate a response result; The SCRIPT GENERATOR unit is used to receive the response result and generate a script file according to the response result; The AI ​​front-end module runs on an AI box that is independent of the original self-service module; The APP plug-in module runs on the original self-service module; The AI ​​server module runs on the AI ​​box, independent AI server or cloud AI server; The AI ​​front-end module, the APP plug-in module and the AI ​​server module together constitute an AI assistant, which interacts naturally with the device through voice and vision without changing the application operation interface of the ordinary self-service device.

2. The self-service system with added AI function according to claim 1, characterized in that: The APP plug-in module includes: an SRU unit and an ICSU unit; The SRU unit is used to identify and obtain the screen content of the self-service interface; The ICSU unit is used to simulate keyboard input operation or screen click operation after receiving the script file.

3. A data processing method for a self-service system with an AI function added according to any one of claims 1 to 2, characterized in that: The data processing method comprises: When the AI ​​front-end module detects the user's wake-up signal, it sends the wake-up signal to the APP plug-in module, and the APP plug-in module enters the working state after receiving the wake-up signal; The AI ​​front-end module executes corresponding actions according to the pre-configured welcome script, collects the user's input content, and sends the input content to the AI ​​server module; The APP plug-in module obtains the screen content of the self-service interface, and packages the screen content and sends it to the AI ​​server module; The AI ​​server module processes the input content and the screen content, generates a response result, generates a script file according to the response result, and sends the script file to the AI ​​front-end module; The AI ​​front-end module receives the script file, and sends the script file to the corresponding units of the APP plug-in module and the AI ​​front-end module to automatically execute the self-service function of the original self-service module.

4. The data processing method of the self-service system with AI function added according to claim 3 is characterized in that: When the AI ​​front-end module detects the user's wake-up signal, the wake-up signal is sent to the APP plug-in module, and the above also includes: When the original self-service module enters the working state, the SCU unit and the VCU unit of the AI ​​front-end module are initialized; The user's wake-up signal is detected by the microphone of the SCU unit and the camera of the VCU unit.

5. The data processing method of the self-service system with added AI function according to claim 3, characterized in that: The script file includes: voice playback instructions, digital human action instructions, dialogue text display instructions and APP operation instructions; The sending of the script file to the corresponding units of the APP plug-in module and the AI ​​front-end module to automatically execute the self-service function of the original self-service module specifically includes: Sending the voice playing instruction to the SPU unit of the AI ​​front-end module, and the SPU unit plays the corresponding voice content according to the voice playing instruction; Sending the digital human action instruction to the DPPU unit of the AI ​​front-end module, and the DPPU unit performs corresponding actions according to the digital human action instruction; Sending the dialogue text display instruction to the DTD unit of the AI ​​front-end module, and the DTD unit plays the corresponding text content according to the dialogue text display instruction; The APP operation instruction is sent to the ICSU unit of the APP plug-in module, and the ICSU unit simulates a keyboard input operation or a screen click operation according to the APP operation instruction.

6. A terminal, characterized in that: The terminal includes: a memory, a processor, and a data processing program for the self-service system with added AI function stored in the memory and executable on the processor. When the data processing program for the self-service system with added AI function is executed by the processor, the steps of the data processing method for the self-service system with added AI function as described in any one of claims 3 to 5 are implemented.

7. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a data processing program for a self-service system with an AI function, and when the data processing program for the self-service system with an AI function is executed by a processor, the steps of the data processing method for a self-service system with an AI function are implemented as described in any one of claims 3 to 5.

Images