Self-service system added with AI function, data processing method, terminal and storage medium

Abstract

The invention discloses a self-service system added with an AI function, a data processing method, a terminal and a storage medium, the self-service system added with the AI function comprises an original self-service module, an AI front-end module, an APP plug-in module and an AI server-side module, the original self-service module is connected with the APP plug-in module, and the AI front-end module is connected with the AI server-side module. The AI front end module, the APP plug-in module and the AI server side module are connected with one another. Under the condition of not changing an application operation interface of a common self-service device, hardware or software of the self-service device which is already deployed and used does not need to be modified, natural interaction with the device can be carried out through voice and vision, original device interaction actions of a user through visual understanding and screen clicking are completed, and the user experience is improved. The self-service equipment can understand problems and intentions of the user, consultation or business handling needed by the user is completed, the effect similar to manual window service is achieved, and the equipment updating cost is saved.

Description

Technical Field

[0001] The present invention relates to the field of data processing technology, and in particular to a self-service system with AI function, a data processing method, a terminal and a computer-readable storage medium. Background Art

[0002] Public self-service equipment refers to machines and equipment placed in public places for citizens to handle various types of business by themselves. These devices usually integrate multiple functions such as identity recognition, information query, and business handling, aiming to reduce the queue pressure at the manual window and improve service efficiency. However, when using existing self-service equipment, users need to turn pages according to the menu functions of the terminal. For users who are not familiar with the function names on the self-service equipment, it is difficult to find the corresponding menu according to their own problems and click to handle them. They need staff or volunteers to assist.

[0003] With the emergence of big-model AI (Artificial Intelligence), AI can already understand the semantics of conversations with people. By empowering self-service devices with AI, it has become a reality that users can ask questions and AI can complete the operation of self-service devices. However, when adding AI functions to current self-service devices, it is usually necessary to change the application operation interface of ordinary self-service devices and modify the hardware or software of self-service devices that have been deployed. The AI ​​empowerment process is complicated and increases the cost of equipment replacement.

[0004] Therefore, the prior art still needs to be improved and developed. Summary of the invention

[0005] The main purpose of the present invention is to provide a self-service system, data processing method, terminal and computer-readable storage medium with added AI functions, aiming to solve the problem in the prior art that when adding AI functions to current self-service devices, it is usually necessary to change the application operation interface of ordinary self-service devices and modify the hardware or software of self-service devices that have been deployed and used, resulting in complicated AI empowerment process and high equipment replacement cost.

[0006] To achieve the above object, the present invention provides a self-service system with an AI function added, the self-service system with an AI function added comprising: The original self-service module, the AI ​​front-end module, the APP plug-in module and the AI ​​server module, the original self-service module is connected to the APP plug-in module, and the AI ​​front-end module, the APP plug-in module and the AI ​​server module are connected to each other; The original self-service module is used to display the self-service interface on the screen and perform the self-service function; The AI ​​front-end module is used to detect a user's wake-up signal and send the wake-up signal to the APP plug-in module; The APP plug-in module is used to enter a working state after receiving the wake-up signal; The AI ​​front-end module is also used to perform corresponding actions according to a pre-configured welcome script after detecting the wake-up signal, collect user input content, and send the input content to the AI ​​server module; The APP plug-in module is also used to obtain the screen content of the self-service interface, and package the screen content and send it to the AI ​​server module; The AI ​​server module is used to process the input content and the screen content, generate a response result, generate a script file according to the response result, and send the script file to the AI ​​front-end module; The AI ​​front-end module is also used to receive the script file and send the script file to the corresponding units of the APP plug-in module and the AI ​​front-end module to automatically execute the self-service function.

[0007] Optionally, in the self-service system with added AI function, the AI ​​front-end module includes a MONITOR unit, an AWU unit, an SCU unit, a VCU unit, an SPU unit, a DPPU unit and a DTD unit; The MONITOR unit is used to coordinate and manage the communication between the AI ​​front-end module, the APP plug-in module and the AI ​​server module; The AWU unit is used to detect a user's wake-up signal and send the wake-up signal to the APP plug-in module; The SCU unit is used to collect the user's voice input after detecting the wake-up signal, and send the voice input to the AI ​​server module; The VCU unit is used to collect the user's video input after detecting the wake-up signal, and send the video input to the AI ​​server module; The SPU unit is used to play the corresponding voice content according to the voice playing instruction in the script file; The DPPU unit is used to execute corresponding actions according to the digital human action instructions in the script file; The DTD unit is used to play the corresponding text content according to the dialogue text display instruction in the script file.

[0008] Optionally, in the self-service system with added AI function, the APP plug-in module includes: an SRU unit and an ICSU unit; The SRU unit is used to identify and obtain the screen content of the self-service interface; The ICSU unit is used to simulate keyboard input operation or screen click operation after receiving the script file.

[0009] Optionally, in the self-service system with added AI function, the AI ​​server module includes: an STT unit, a VIA unit, an SCR unit, a KAG unit and a SCRIPT GENERATOR unit; The STT unit is used to receive the voice input in the input content and convert the voice input into text information; The VIA unit is used to receive the video input in the input content, and identify and analyze the video input to obtain the user's facial information; The SCR unit is used to receive the screen content of the self-service interface, and analyze the screen content through screen OCR technology to obtain the user input box position and input requirements; The KAG unit is used to receive the text information, the facial information, the user input box position and the input requirement, and input the text information, the facial information, the user input box position and the input requirement into a pre-trained knowledge model to generate a response result; The SCRIPT GENERATOR unit is used to receive the response result and generate a script file according to the response result.

[0010] Optionally, in the self-service system with added AI function, the AI ​​front-end module runs on an AI box independent of the original self-service module; The APP plug-in module runs on the original self-service module; The AI ​​server module runs on the AI ​​box, an independent AI server or a cloud AI server.

[0011] In addition, to achieve the above-mentioned purpose, the present invention also provides a data processing method for a self-service system with an AI function added, wherein the data processing method comprises: When the AI ​​front-end module detects the user's wake-up signal, it sends the wake-up signal to the APP plug-in module, and the APP plug-in module enters the working state after receiving the wake-up signal; The AI ​​front-end module executes corresponding actions according to the pre-configured welcome script, collects the user's input content, and sends the input content to the AI ​​server module; The APP plug-in module obtains the screen content of the self-service interface, and packages the screen content and sends it to the AI ​​server module; The AI ​​server module processes the input content and the screen content, generates a response result, generates a script file according to the response result, and sends the script file to the AI ​​front-end module; The AI ​​front-end module receives the script file, and sends the script file to the corresponding units of the APP plug-in module and the AI ​​front-end module to automatically execute the self-service function of the original self-service module.

[0012] Optionally, the data processing method of the self-service system with added AI function, wherein when the AI ​​front-end module detects the user's wake-up signal, the wake-up signal is sent to the APP plug-in module, and the method also includes: When the original self-service module enters the working state, the SCU unit and the VCU unit of the AI ​​front-end module are initialized; The user's wake-up signal is detected by the microphone of the SCU unit and the camera of the VCU unit.

[0013] Optionally, in the data processing method of the self-service system with added AI function, the script file includes: voice playback instructions, digital human action instructions, dialogue text display instructions and APP operation instructions; The sending of the script file to the units corresponding to the APP plug-in module and the AI ​​front-end module to automatically execute the self-service function of the original self-service module specifically includes: Sending the voice playing instruction to the SPU unit of the AI ​​front-end module, and the SPU unit plays the corresponding voice content according to the voice playing instruction; Sending the digital human action instruction to the DPPU unit of the AI ​​front-end module, and the DPPU unit performs corresponding actions according to the digital human action instruction; Sending the dialogue text display instruction to the DTD unit of the AI ​​front-end module, and the DTD unit plays the corresponding text content according to the dialogue text display instruction; The APP operation instruction is sent to the ICSU unit of the APP plug-in module, and the ICSU unit simulates a keyboard input operation or a screen click operation according to the APP operation instruction.

[0014] In addition, to achieve the above-mentioned purpose, the present invention also provides a terminal, wherein the terminal includes: a memory, a processor, and a data processing program for the self-service system with added AI function stored on the memory and executable on the processor, wherein the data processing program for the self-service system with added AI function, when executed by the processor, implements the steps of the data processing method for the self-service system with added AI function as described above.

[0015] In addition, to achieve the above-mentioned purpose, the present invention also provides a computer-readable storage medium, wherein the computer-readable storage medium stores a data processing program for a self-service system with an added AI function, and when the data processing program for the self-service system with an added AI function is executed by a processor, the steps of the data processing method for the self-service system with an added AI function as described above are implemented.

[0016] The present invention discloses a self-service system with AI function, a data processing method, a terminal and a storage medium, wherein the self-service system with AI function includes: an original self-service module, an AI front-end module, an APP plug-in module and an AI server module, wherein the original self-service module is connected to the APP plug-in module, and the AI ​​front-end module, the APP plug-in module and the AI ​​server module are interconnected. The present invention can interact with the device naturally through voice and vision without changing the application operation interface of the ordinary self-service device and modifying the hardware or software of the self-service device that has been deployed and used, and complete the device interaction actions that the original user understood through vision and clicked on the screen, so that the self-service device can understand the user's questions and intentions, complete the consultation or business handling required by the user, and achieve an effect similar to that of manual window service, saving the cost of equipment replacement. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 This is the overall architecture diagram of the self-service system with AI function added in the present invention; Figure 2 It is a flow chart of a preferred embodiment of a data processing method of a self-service system with an AI function added in the present invention; Figure 3 It is a structural diagram of a preferred embodiment of the terminal of the present invention. DETAILED DESCRIPTION

[0018] In order to make the purpose, technical solution and advantages of the present invention clearer and more specific, the present invention is further described in detail below with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0019] To solve the problems in the prior art, this embodiment provides a self-service system with an AI function, such as Figure 1As shown, the original self-service module, AI front-end module, APP plug-in module and AI server module.

[0020] Among them, the original self-service module is connected to the APP plug-in module, and the AI ​​front-end module, the APP plug-in module and the AI ​​server module are interconnected.

[0021] The original self-service module is used to display the self-service interface on the screen and perform the self-service function; The AI ​​front-end module is used to detect a user's wake-up signal and send the wake-up signal to the APP plug-in module; The APP plug-in module is used to enter a working state after receiving the wake-up signal; The AI ​​front-end module is also used to perform corresponding actions according to a pre-configured welcome script after detecting the wake-up signal, collect user input content, and send the input content to the AI ​​server module; The APP plug-in module is also used to obtain the screen content of the self-service interface, and package the screen content and send it to the AI ​​server module; The AI ​​server module is used to process the input content and the screen content, generate a response result, generate a script file according to the response result, and send the script file to the AI ​​front-end module; The AI ​​front-end module is also used to receive the script file and send the script file to the corresponding units of the APP plug-in module and the AI ​​front-end module to automatically execute the self-service function.

[0022] It is understandable that the AI ​​front-end module, the AI ​​plug-in module and the AI ​​server module together constitute an AI assistant, the AI ​​front-end module runs on an AI box independent of the original self-service module; the APP plug-in module runs on the original self-service module; the AI ​​server module runs on the AI ​​box, an independent AI server or a cloud AI server. Among them, the AI ​​box can be an independent computer device or a smart terminal device such as a user's mobile phone. The AI ​​assistant ensures that it can interact naturally with the device through voice and vision without changing the application operation interface of ordinary self-service equipment and without modifying the hardware or software of the self-service equipment that has been deployed and used, and completes the original user's device interaction actions through visual understanding and screen clicking, so that the self-service equipment can understand the user's problems and intentions, and complete the consultation or business processing required by the user.

[0023] Furthermore, the present invention also provides a data processing unit included in each module, which is used to perform corresponding data collection and processing operations.

[0024] Specifically, the AI ​​front-end module includes a MONITOR unit (monitor unit), an AWU unit (Activation Wake-Up Unit), an SCU unit (Speech Collection Unit), a VCU unit (Video Collection Unit), an SPU unit (Speech Playback Unit), a DPPU unit (Digital Person Playback Unit) and a DTD unit (Dialog Text Display Unit).

[0025] Among them, the MONITOR unit is used to coordinate and manage the communication between the AI ​​front-end module, the APP plug-in module and the AI ​​server module; the AWU unit is used to detect the user's wake-up signal and send the wake-up signal to the APP plug-in module; the SCU unit is used to collect the user's voice input after detecting the wake-up signal, and send the voice input to the AI ​​server module; the VCU unit is used to collect the user's video input after detecting the wake-up signal, and send the video input to the AI ​​server module; the SPU unit is used to play the corresponding voice content according to the voice playback instruction in the script file; the DPPU unit is used to perform the corresponding action according to the digital human action instruction in the script file; the DTD unit is used to play the corresponding text content according to the dialogue text display instruction in the script file.

[0026] Furthermore, the APP plug-in module includes: an SRU unit (Screen Recognition Unit) and an ICSU unit (Input and Click Simulation Unit).

[0027] The SRU unit is used to identify and obtain the screen content of the self-service interface; and the ICSU unit is used to simulate keyboard input operations or screen click operations after receiving the script file.

[0028] Furthermore, the AI ​​server module includes: STT unit (Speech to Text), VIA unit (Video Identification and Analysis), SCR unit (Screen content recognition), KAG unit (Knowledge Augmented Generation) and SCRIPT GENERATOR unit.

[0029] Among them, the STT unit is used to receive the voice input in the input content and convert the voice input into text information; the VIA unit is used to receive the video input in the input content, and recognize and analyze the video input to obtain the user's facial information; the SCR unit is used to receive the screen content of the self-service interface, and analyze the screen content through the screen OCR (Optical Character Recognition) technology to obtain the user input box position and input requirements; the KAG unit is used to receive the text information, the facial information, the user input box position and the input requirements, and input the text information, the facial information, the user input box position and the input requirements into the pre-trained knowledge model to generate a response result; the SCRIPTGENERATOR unit is used to receive the response result and generate a script file according to the response result.

[0030] Furthermore, the original self-service module is an existing self-service device, including hardware, system and application software. The original self-service module includes an APP unit, and the APP unit is a general term for an application system running on an existing self-service device, which is an application or multiple applications. It can be understood that the modules of the system of the present invention can be connected in the form of physical connection, program communication, network communication, etc., and the units can be connected in the form of program communication, network communication, etc.

[0031] As can be seen from the above, the present invention ensures that, without changing the application operation interface of the ordinary self-service device, and without modifying the hardware or software of the self-service device that has been deployed and used, it is possible to interact with the device naturally through voice and vision, thereby completing the device interaction actions that the user originally performed through visual understanding and screen clicking.

[0032] Based on the self-service system with AI function added in the above embodiment, the present invention also provides a data processing method for the self-service system with AI function added, specifically as follows: Figure 2As shown in , the data processing method of the self-service system with added AI function includes the following steps: Step S10: When the AI ​​front-end module detects the user's wake-up signal, it sends the wake-up signal to the APP plug-in module, and the APP plug-in module enters a working state after receiving the wake-up signal.

[0033] When the AI ​​front-end module detects the user's wake-up signal, the wake-up signal is sent to the APP plug-in module, and the above also includes: When the original self-service module enters the working state, the SCU unit and the VCU unit of the AI ​​front-end module are initialized; The user's wake-up signal is detected by the microphone of the SCU unit and the camera of the VCU unit.

[0034] It is understandable that the user approach detection method can be a single method or a combination of infrared detection, radio wave detection, voice detection or video detection, and the intention is to interact with the self-service device (the intention detection method for interacting with the AI ​​assistant is one or more combinations of the user pressing the start chat button or the duration of the human signal reaches a preset value, the voice persists for a preset value, or the video captures a preset value).

[0035] Furthermore, when a user is detected approaching the device, a wake-up signal is triggered: when the AWU unit detects that someone is staying in front of the self-service device or speaking to the device (through a microphone or camera), the AWU unit will immediately wake up the MONITOR unit of the APP plug-in module. After receiving the wake-up signal, the MONITOR unit enters the working state. At the same time, you can also choose to perform AI front-end initialization at this time.

[0036] Step S20: The AI ​​front-end module executes corresponding actions according to the pre-configured welcome script, collects the user's input content, and sends the input content to the AI ​​server module.

[0037] When the AI ​​front-end module receives the wake-up signal, it performs corresponding actions according to the pre-configured welcome script and plays a welcome speech (for example, "Welcome to the self-service, how can I help you?"). At the same time, the MONITOR unit notifies the DPPU unit of the AI ​​front-end module to perform a welcome action (for example, the digital human smiles and waves, etc.).

[0038] Furthermore, the AI ​​front-end module will also wake up the SCU unit and VCU unit of the AI ​​front-end module, and the SCU unit and VCU unit are always ready to record the user's voice and video (collect the user's input content). Among them, the SCU unit starts recording a segment of the user's voice input, and the duration can be set according to demand (for example, 5 seconds) or automatically recognize that the user is speaking and start recording, and the recording ends when the user's speaking is silent for a set time, such as 1 second. The VCU unit can also optionally record a segment of the user's video input (such as facial recognition or other video analysis), and the duration must be consistent with the SCU unit.

[0039] Furthermore, the input content is directly passed to the AI ​​server module for subsequent intelligent processing (the input content can also be sent to the MONITOR unit of the APP plug-in module, and then packaged by the MONITOR unit and sent to the AI ​​server module).

[0040] Step S30: The APP plug-in module obtains the screen content of the self-service interface, and packages the screen content and sends it to the AI ​​server module.

[0041] Specifically, the MONITOR unit of the APP plug-in module receives the input content. At the same time, the SRU unit of the APP plug-in module will capture the screen content of the current self-service interface, obtain the currently displayed screen content (such as menus, options, etc.), and finally, package the screen content and send it to the AI ​​server module (the input screen content can also be directly passed to the AI ​​server module).

[0042] Step S40: The AI ​​server module processes the input content and the screen content to generate a response result, generates a script file according to the response result, and sends the script file to the AI ​​front-end module.

[0043] Specifically, after receiving the data, the AI ​​server module initiates the following processing steps for the units under it: first, the STT unit converts the audio into text and extracts the user's voice commands; then, the VIA unit recognizes and analyzes the video to extract the user's facial expressions, gestures and other information; next, the SCR unit analyzes the screenshot content through screen OCR and other technologies to extract the user input box location and input requirements (such as phone number or address, etc., as well as the click location and the corresponding information of the click, such as confirmation or cancellation) of the screenshot image or information; finally, the KAG unit will generate a corresponding response result based on the pre-trained knowledge model, combined with the user's voice commands, video analysis results and current screen content. Furthermore, the Script Generator unit creates a script file containing the following contents according to the generated response result: Voice playback instructions, used to tell the SPU unit the voice content to be played. Dialogue text display instructions, used to tell the DTD unit the text content to be displayed. Digital human action instructions, used to tell the DPPU unit the action to be performed (such as nodding, smiling, pointing in a certain direction, etc.). APP operation instructions, used to tell the ICSU unit the URL (Uniform Resource Locator) of the page to be opened, the simulated keyboard input or the screen click position order (such as clicking a button, entering a password, etc.).

[0044] Step S50: The AI ​​front-end module receives the script file, and sends the script file to the corresponding units of the APP plug-in module and the AI ​​front-end module to automatically execute the self-service function of the original self-service module.

[0045] In this embodiment, the script file includes: voice playback instructions, digital human action instructions, dialogue text display instructions and APP operation instructions.

[0046] The sending of the script file to the corresponding units of the APP plug-in module and the AI ​​front-end module to automatically execute the self-service function of the original self-service module specifically includes: Sending the voice playing instruction to the SPU unit of the AI ​​front-end module, and the SPU unit plays the corresponding voice content according to the voice playing instruction; Sending the digital human action instruction to the DPPU unit of the AI ​​front-end module, and the DPPU unit performs corresponding actions according to the digital human action instruction; Sending the dialogue text display instruction to the DTD unit of the AI ​​front-end module, and the DTD unit plays the corresponding text content according to the dialogue text display instruction; The APP operation instruction is sent to the ICSU unit of the APP plug-in module, and the ICSU unit simulates a keyboard input operation or a screen click operation according to the APP operation instruction.

[0047] It is understandable that after the self-service function of the original self-service module is automatically executed according to the script file, if the user's needs are not completed, the above process will be repeated to continue the interaction until the user completes all operations or leaves the device. When the user completes all operations or leaves the device, the AWU unit detects that the user has left the device, and the MONITOR unit notifies the SPU unit and the DPPU unit to end the current session and restore the system to the initial state, waiting for the next user wake-up.

[0048] Furthermore, in another embodiment, the units of the self-service system with AI functions are communicated by imitating the IP communication of the current network: the unit ID is analogous to the IP of the computer Internet access, and the communication between the two units depends on a coordinate identifier pointing to the communication object. The premise for a unit to send a notification to another unit is to include its own unit ID and the other unit ID in the communication package. For example, if unit A wants to obtain its own unit ID and another unit ID, it can read it from its own unit preset file when it is powered on. If it cannot be read, it will send a broadcast notification to all units and request the known units to reply. All units can receive this broadcast notification. For example, the AI ​​server module will reply with a notification after receiving it, and the A unit can save it in the preset file for next use after receiving it. That is to say, in this embodiment, each unit has a unique ID identifier in the system as the unit ID, and each unit can obtain the ID of its own unit or other units. The unit ID can be obtained by reading the unit ID or querying the unit ID.

[0049] Among them, notification is the information exchange process between units. Each unit communication information is called a notification, which consists of source unit, target unit, source unit type, information type, chat ID and session ID and information body. The source unit refers to the unit ID that initiates the communication, the target unit refers to the unit ID to which the communication is sent, the source unit type is the unit type of the communication initiating unit, the information type is the system predefined classification of information for the target unit to understand, and the information body is the byte stream generated by the source unit according to a set of information rules predefined by the system between the source unit and the target unit.

[0050] When reading the unit ID, you need to find the unit ID from the unit preset file by unit type. Each unit has a unit preset file, which can be part of the software code or an independent file. The unit preset file also saves the unit type when the unit is installed, the other unit types of the digital human in which the unit is located, and the unique identifier of the hardware associated with these units. The hardware unique identifier refers to the unique identifier of the AI ​​box or self-service device where the unit is located in the system.

[0051] Furthermore, an AI assistant is composed of one or more digital humans and an AI server module, and a digital human is composed of an AI front-end module and an AI plug-in module. Each module is composed of one or more units. Among them, each unit of an AI front-end module can be stored and run in an AI box or multiple AI boxes respectively, and an AI box can be an independent computer device, a user's mobile phone or an original self-service device. Each unit of an AI plug-in module is stored and run on a self-service device module. Each unit of an AI server module can be stored and run on one or more AI boxes or independent servers or cloud servers.

[0052] Each unit of the digital human has an executable action table, which is stored in the unit preset file by the system administrator when each unit is installed, and also stored in the SCRIPT GENERATOR unit on the AI ​​server, or sent to the SCRIPT GENERATOR unit by the unit notification when each unit of the digital human is initialized and saved in the SCRIPTGENERATOR unit. The storage form of the SCRIPT GENERATOR unit is unit ID + executable action table. Among them, a script file consists of one or more instruction sets, an instruction set consists of an instruction header and an instruction body, and the instruction body consists of one or more instructions. The instruction header represents the target unit, and the instruction represents one or more actions to be performed by the target unit.

[0053] At the beginning of each chat, the AI ​​server module will assign a system-unique ID as the identifier of the chat information, called the chat ID. At the beginning of each round of conversation, the MONITOR will assign a unique ID under the chat ID as the identifier of the conversation information, called the session ID. Generally, the session ID starts from 0 and increases by 1 each time.

[0054] As an example, the following is a process for a unit to obtain the unit ID of a specified type of unit (it can obtain its own unit ID or other unit IDs): Step 1: Read the unit ID from the unit preconfiguration file according to the unit type. If it fails, go to step 2.

[0055] Step 2: The unit sends a notification to query the unit ID. If the unit ID can be read, the original unit ID is its own unit ID, otherwise the source unit ID in the notification is empty; read its own unit type in the preset file as the source unit type; the information type is to query the unit ID; the information body contains the hardware unique identifier and the unit type of the unit ID to be queried; if the unit ID of the MONITOR can be read, the target unit ID of the notification is the MONITOR unit ID, otherwise if the unit ID of the AI ​​server module can be read, the target unit ID of the notification is the AI ​​server module unit ID, otherwise the target unit ID of the notification is empty.

[0056] Step 3: The unit checks the received notification whose information type is a query unit ID reply, and saves the unit type, unit ID, and unit-associated hardware unique identifier in the information body to a preset file.

[0057] When users operate on self-service devices without AI functions, they need to turn pages according to the menu functions of the terminal. It is difficult for users to find the corresponding menu and click to handle their own problems. They need volunteers to assist. The self-service system with AI functions designed by the present invention does not need to eliminate the public self-service equipment in use, and does not require the participation of the original self-service equipment manufacturers, which reduces the cost of self-service equipment transformation and saves a lot of social costs. At the same time, the implementation of the present invention can empower a large number of existing self-service equipment manufacturers, so that they can become manufacturers that can produce AI self-service equipment after purchasing the implementation products of the present invention as accessories for the original self-service equipment.

[0058] Furthermore, if Figure 3 As shown, based on the above-mentioned self-service system with added AI function and the data processing method of the self-service system with added AI function, the present invention also provides a terminal accordingly, and the terminal includes a processor 10, a memory 20 and a display 30. Figure 3 Only some components of the terminal are shown, but it should be understood that it is not required to implement all of the components shown, and more or fewer components may be implemented instead.

[0059] In some embodiments, the memory 20 may be an internal storage unit of the terminal, such as a hard disk or memory of the terminal. In other embodiments, the memory 20 may also be an external storage device of the terminal, such as a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (SecureDigital, SD) card, a flash card (Flash Card), etc. equipped on the terminal. Further, the memory 20 may also include both an internal storage unit of the terminal and an external storage device. The memory 20 is used to store application software and various types of data installed on the terminal, such as the program code of the installation terminal. The memory 20 may also be used to temporarily store data that has been output or is to be output. In one embodiment, a data processing program 40 of a self-service system with an added AI function is stored on the memory 20, and the data processing program 40 of the self-service system with an added AI function can be executed by the processor 10, thereby realizing the data processing method of the self-service system with an added AI function in the present application.

[0060] In some embodiments, the processor 10 may be a central processing unit (CPU), a microprocessor or other data processing chip, used to run the program code stored in the memory 20 or process data, such as executing the data processing method of the self-service system with added AI function.

[0061] In some embodiments, the display 30 may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, an OLED (Organic Light-Emitting Diode) touch device, etc. The display 30 is used to display information on the terminal and to display a visual user interface. The components 10-30 of the terminal communicate with each other via a system bus.

[0062] In one embodiment, when the processor 10 executes the data processing program 40 of the self-service system with AI function added in the memory 20, the following steps are implemented: When the AI ​​front-end module detects the user's wake-up signal, it sends the wake-up signal to the APP plug-in module, and the APP plug-in module enters the working state after receiving the wake-up signal; The AI ​​front-end module executes corresponding actions according to the pre-configured welcome script, collects the user's input content, and sends the input content to the AI ​​server module; The APP plug-in module obtains the screen content of the self-service interface, and packages the screen content and sends it to the AI ​​server module; The AI ​​server module processes the input content and the screen content, generates a response result, generates a script file according to the response result, and sends the script file to the AI ​​front-end module; The AI ​​front-end module receives the script file, and sends the script file to the corresponding units of the APP plug-in module and the AI ​​front-end module to automatically execute the self-service function of the original self-service module.

[0063] Wherein, when the AI ​​front-end module detects the user's wake-up signal, the wake-up signal is sent to the APP plug-in module, and the above also includes: When the original self-service module enters the working state, the SCU unit and the VCU unit of the AI ​​front-end module are initialized; The user's wake-up signal is detected by the microphone of the SCU unit and the camera of the VCU unit.

[0064] The script file includes: voice playback instructions, digital human action instructions, dialogue text display instructions and APP operation instructions; The sending of the script file to the units corresponding to the APP plug-in module and the AI ​​front-end module to automatically execute the self-service function of the original self-service module specifically includes: Sending the voice playing instruction to the SPU unit of the AI ​​front-end module, and the SPU unit plays the corresponding voice content according to the voice playing instruction; Sending the digital human action instruction to the DPPU unit of the AI ​​front-end module, and the DPPU unit performs corresponding actions according to the digital human action instruction; Sending the dialogue text display instruction to the DTD unit of the AI ​​front-end module, and the DTD unit plays the corresponding text content according to the dialogue text display instruction; The APP operation instruction is sent to the ICSU unit of the APP plug-in module, and the ICSU unit simulates a keyboard input operation or a screen click operation according to the APP operation instruction.

[0065] The present invention also provides a computer-readable storage medium, wherein the computer-readable storage medium stores a data processing program for a self-service system with an added AI function, and when the data processing program for the self-service system with an added AI function is executed by a processor, the steps of the data processing method for the self-service system with an added AI function as described above are implemented.

[0066] In summary, the present invention provides a self-service system with AI functions, a data processing method, a terminal and a storage medium, wherein the self-service system with AI functions includes: an original self-service module, an AI front-end module, an APP plug-in module and an AI server module, wherein the original self-service module is connected to the APP plug-in module, and the AI ​​front-end module, the APP plug-in module and the AI ​​server module are interconnected. The present invention can interact naturally with the device through voice and vision without changing the application operation interface of ordinary self-service equipment and modifying the hardware or software of the self-service equipment that has been deployed and used, and complete the device interaction actions that the original user understood through vision and clicked on the screen, so that the self-service equipment can understand the user's problems and intentions, complete the consultation or business processing required by the user, and achieve an effect similar to that of manual window service, saving the cost of equipment replacement.

[0067] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or terminal including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or terminal. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the presence of other identical elements in the process, method, article or terminal including the element.

[0068] Of course, those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing related hardware (such as a processor, a controller, etc.) through a computer program, and the program can be stored in a computer-readable storage medium that can be read by a computer, and the program can include the processes of the above-mentioned method embodiments when executed. The computer-readable storage medium can be a memory, a disk, an optical disk, etc.

[0069] It should be understood that the application of the present invention is not limited to the above examples. For ordinary technicians in this field, improvements or changes can be made based on the above description. All these improvements and changes should fall within the scope of protection of the claims attached to the present invention.

Claims

1. A self-service system with AI function added, characterized in that: The self-service system with added AI function includes: an original self-service module, an AI front-end module, an APP plug-in module and an AI server module, wherein the original self-service module is connected to the APP plug-in module, and the AI ​​front-end module, the APP plug-in module and the AI ​​server module are connected to each other; The original self-service module is used to display the self-service interface on the screen and perform the self-service function; The AI ​​front-end module is used to detect a user's wake-up signal and send the wake-up signal to the APP plug-in module; The APP plug-in module is used to enter a working state after receiving the wake-up signal; The AI ​​front-end module is also used to perform corresponding actions according to a pre-configured welcome script after detecting the wake-up signal, collect user input content, and send the input content to the AI ​​server module; The APP plug-in module is also used to obtain the screen content of the self-service interface, and package the screen content and send it to the AI ​​server module; The AI ​​server module is used to process the input content and the screen content, generate a response result, generate a script file according to the response result, and send the script file to the AI ​​front-end module; The AI ​​front-end module is also used to receive the script file and send the script file to the corresponding units of the APP plug-in module and the AI ​​front-end module to automatically execute the self-service function.

2. The self-service system with added AI function according to claim 1, characterized in that: The AI ​​front-end module includes a MONITOR unit, an AWU unit, an SCU unit, a VCU unit, an SPU unit, a DPPU unit and a DTD unit; The MONITOR unit is used to coordinate and manage the communication between the AI ​​front-end module, the APP plug-in module and the AI ​​server module; The AWU unit is used to detect a user's wake-up signal and send the wake-up signal to the APP plug-in module; The SCU unit is used to collect the user's voice input after detecting the wake-up signal, and send the voice input to the AI ​​server module; The VCU unit is used to collect the user's video input after detecting the wake-up signal, and send the video input to the AI ​​server module; The SPU unit is used to play the corresponding voice content according to the voice playing instruction in the script file; The DPPU unit is used to execute corresponding actions according to the digital human action instructions in the script file; The DTD unit is used to play the corresponding text content according to the dialogue text display instruction in the script file.

3. The self-service system with added AI function according to claim 1, characterized in that: The APP plug-in module includes: an SRU unit and an ICSU unit; The SRU unit is used to identify and obtain the screen content of the self-service interface; The ICSU unit is used to simulate keyboard input operation or screen click operation after receiving the script file.

4. The self-service system with added AI function according to claim 1, characterized in that: The AI ​​server module includes: an STT unit, a VIA unit, an SCR unit, a KAG unit and a SCRIPT GENERATOR unit; The STT unit is used to receive the voice input in the input content and convert the voice input into text information; The VIA unit is used to receive the video input in the input content, and identify and analyze the video input to obtain the user's facial information; The SCR unit is used to receive the screen content of the self-service interface, and analyze the screen content through screen OCR technology to obtain the user input box position and input requirements; The KAG unit is used to receive the text information, the facial information, the user input box position and the input requirement, and input the text information, the facial information, the user input box position and the input requirement into a pre-trained knowledge model to generate a response result; The SCRIPT GENERATOR unit is used to receive the response result and generate a script file according to the response result.

5. The self-service system with added AI function according to claim 1, characterized in that: The AI ​​front-end module runs on an AI box that is independent of the original self-service module; The APP plug-in module runs on the original self-service module; The AI ​​server module runs on the AI ​​box, an independent AI server or a cloud AI server.

6. A data processing method for a self-service system with an AI function added according to any one of claims 1 to 5, characterized in that: The data processing method comprises: When the AI ​​front-end module detects the user's wake-up signal, it sends the wake-up signal to the APP plug-in module, and the APP plug-in module enters the working state after receiving the wake-up signal; The AI ​​front-end module executes corresponding actions according to the pre-configured welcome script, collects the user's input content, and sends the input content to the AI ​​server module; The APP plug-in module obtains the screen content of the self-service interface, and packages the screen content and sends it to the AI ​​server module; The AI ​​server module processes the input content and the screen content, generates a response result, generates a script file according to the response result, and sends the script file to the AI ​​front-end module; The AI ​​front-end module receives the script file, and sends the script file to the corresponding units of the APP plug-in module and the AI ​​front-end module to automatically execute the self-service function of the original self-service module.

7. The data processing method of the self-service system with AI function added according to claim 6, characterized in that: When the AI ​​front-end module detects the user's wake-up signal, the wake-up signal is sent to the APP plug-in module, and the above also includes: When the original self-service module enters the working state, the SCU unit and the VCU unit of the AI ​​front-end module are initialized; The user's wake-up signal is detected by the microphone of the SCU unit and the camera of the VCU unit.

8. The data processing method of the self-service system with AI function added according to claim 6, characterized in that: The script file includes: voice playback instructions, digital human action instructions, dialogue text display instructions and APP operation instructions; The sending of the script file to the units corresponding to the APP plug-in module and the AI ​​front-end module to automatically execute the self-service function of the original self-service module specifically includes: Sending the voice playing instruction to the SPU unit of the AI ​​front-end module, and the SPU unit plays the corresponding voice content according to the voice playing instruction; Sending the digital human action instruction to the DPPU unit of the AI ​​front-end module, and the DPPU unit performs corresponding actions according to the digital human action instruction; Sending the dialogue text display instruction to the DTD unit of the AI ​​front-end module, and the DTD unit plays the corresponding text content according to the dialogue text display instruction; The APP operation instruction is sent to the ICSU unit of the APP plug-in module, and the ICSU unit simulates a keyboard input operation or a screen click operation according to the APP operation instruction.

9. A terminal, characterized in that: The terminal includes: a memory, a processor, and a data processing program for the self-service system with added AI function stored in the memory and executable on the processor. When the data processing program for the self-service system with added AI function is executed by the processor, the steps of the data processing method for the self-service system with added AI function as described in any one of claims 6 to 8 are implemented.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a data processing program for a self-service system with an AI function, and when the data processing program for the self-service system with an AI function is executed by a processor, the steps of the data processing method for a self-service system with an AI function are implemented as described in any one of claims 6 to 8.

Images