Question and answer method and device based on terminal intelligent assistant and electronic equipment

Abstract

The invention discloses a question and answer method and device based on a terminal intelligent assistant and electronic equipment, and relates to the field of financial science and tech, and the method comprises the steps: receiving a target instruction inputted by a user side, the target instruction being an instruction generated after a display page of a financial intelligent assistant running in the user side receives the operation of a user, the instruction content of the target instruction at least comprises a question asked by a user and / or a concerned financial product type and a specified financial service request; and analyzing the target instruction by adopting a deep learning model, extracting instruction content and a user operation intention, generating a reply instruction answer according to the user historical preference information, the instruction content and the user operation intention, and sending the reply instruction answer to the user side. The technical problem of poor user service experience caused by the fact that intelligent assistants in financial institution APPs cannot accurately understand user intentions in related technologies is solved.

Description

Technical Field

[0001] The present invention relates to the field of financial technology or other related fields, and in particular, to a question-answering method and device based on a terminal intelligent assistant, and an electronic device. Background Art

[0002] At a time when digitalization and intelligence are accelerating, smart assistants in various financial institution apps installed on mobile terminals have become the main tools for people's daily financial transactions and banking services. Smart assistants perform intelligent recommendations, smart searches, and direct service based on user habits and scenarios, providing users with a quick entry point for operations.

[0003] However, in the prior art, the intelligent assistant function in the APP of financial institutions still has many limitations. These limitations restrict the more efficient and intelligent interaction between users and APPs. For example, the current intelligent assistant can only complete some simple operations based on the input text or voice, the response is not accurate, and it cannot integrate the user's historical data to bring customers a greater intelligent and convenient experience, and the convenience brought to user operations is limited. At the same time, the intelligent assistants in the prior art mainly rely on simple keyword matching and preset response rules, and have limited ability to handle instructions or questions input by users. Especially when it comes to complex banking operations, intelligent assistants often cannot accurately understand user intentions, resulting in a poor service experience.

[0004] To address the above-mentioned problems, no effective solution has been proposed yet. Summary of the invention

[0005] The embodiments of the present invention provide a question-answering method and device based on a terminal intelligent assistant, and an electronic device, so as to at least solve the technical problem in the related art that the intelligent assistant in the APP of a financial institution cannot accurately understand the user's intention, resulting in a poor user service experience.

[0006] In order to achieve the above-mentioned purpose, according to one aspect of the present application, a question-and-answer method based on a terminal intelligent assistant is provided, comprising: receiving a target instruction input by a user terminal, wherein the target instruction is an instruction generated by a display page of a financial intelligent assistant running in the user terminal after receiving a user operation, and the instruction content of the target instruction includes at least: the user asks a question and / or pays attention to a type of financial product, and specifies a financial business request; using a deep learning model to parse the target instruction, extract the instruction content and the user's operation intention, wherein the deep learning model is a pre-trained model for parsing the user's question instruction and the corresponding user intention; generating a reply instruction answer based on the user's historical preference information, the instruction content and the user's operation intention, and sending the reply instruction answer to the user terminal.

[0007] Optionally, the step of using a deep learning model to parse the target instruction and extract the instruction content and the user's operation intention includes: when the target instruction is a text instruction, calling a preset natural language processing model as the deep learning model; using the preset natural language model to parse the context of the target instruction to obtain a question-and-answer context; parsing the target instruction based on the question-and-answer context, and preprocessing the instruction text obtained by the analysis to obtain an instruction statement; extracting synonyms, polyphones and sentence structures of the instruction statement, identifying the preprocessed instruction text based on the text extraction result, and obtaining the instruction type and instruction content; locating the user's operation intention based on the context statement, the instruction type and the instruction content.

[0008] Optionally, the step of generating a reply instruction answer based on the user's historical preference information, the instruction content and the user's operation intention includes: when the target instruction is a text instruction, determining the user's investment preference type based on the user's historical preference information, the instruction content and the user's operation intention; based on the user's investment preference type, using a collaborative filtering algorithm to select a proposed recommended financial product that meets the instruction content and the user's operation intention; generating the reply instruction answer based on the proposed recommended financial product, the instruction type and the question and answer context.

[0009] Optionally, the step of using a deep learning model to parse the target instruction and extract the instruction content and the user's operation intention includes: when the target instruction is a voice instruction, calling a preset voice recognition model as the deep learning model; using the preset voice recognition model to analyze the surrounding environment of the target instruction to obtain a voice instruction environment, wherein the voice instruction environment includes: user tone type, user intonation type, user voice volume and surrounding environment sound; based on the voice instruction environment, identifying the user's voice emotional state and environmental risk level; when the user's voice emotional state and environmental risk level indicate that the current user's voice operation is safe, identifying the target instruction to obtain the instruction content and the user's operation intention.

[0010] Optionally, the step of generating a reply instruction answer based on the user's historical preference information, the instruction content and the user's operation intention includes: when the target instruction is a voice instruction, determining the instruction type based on the instruction content and the user's operation intention; when the instruction type is a financial business operation, based on the user's historical preference information, the instruction content and the user's operation intention, executing the target instruction according to the instruction content and generating a reply instruction answer, wherein the financial business operation includes at least one of the following: transfer, account checking, asset income and expenditure report; or, when the instruction type is a financial institution branch-related business, generating a reply instruction answer based on the user's historical preference information, the instruction content and the user's operation intention, wherein the financial institution branch-related business includes at least one of the following: branch recommendation, branch navigation, and branch queuing.

[0011] Optionally, the step of using a deep learning model to parse the target instruction and extract the instruction content and the user's operation intention includes: when the target instruction is a picture instruction, using the called optical character recognition model as the deep learning model; preprocessing the picture in the target instruction, and parsing the preprocessed target instruction to obtain an instruction scene, wherein the preprocessing strategy includes: denoising, binarization, and tilt correction; based on the instruction scene, using the optical character recognition model to parse the picture instruction to obtain a character feature set; comparing the character feature set with a pre-trained character library to obtain instruction statement information; parsing the instruction statement information to obtain the instruction content and the user's operation intention.

[0012] Optionally, the step of generating a reply instruction answer based on the user's historical preference information, the instruction content and the user's operation intention includes: when the target instruction is a picture instruction, determining the target financial business to be executed based on the instruction content and the user's operation intention; based on the target financial business, the instruction content and the user's operation intention, executing the target instruction in accordance with the instruction content, and generating a reply instruction answer.

[0013] According to another aspect of an embodiment of the present invention, a question-and-answer device based on a terminal intelligent assistant is also provided, including: an instruction receiving unit, used to receive a target instruction input by a user terminal, wherein the target instruction is an instruction generated by a display page of a financial intelligent assistant running in the user terminal after receiving a user operation, and the instruction content of the target instruction includes at least: the user asks a question and / or pays attention to a type of financial product, and specifies a financial business request; an instruction parsing unit, used to use a deep learning model to parse the target instruction, extract the instruction content and the user's operation intention, wherein the deep learning model is a pre-trained model for parsing the user's question instruction and the corresponding user intention; an assistant reply answer unit, used to generate a reply instruction answer based on the user's historical preference information, the instruction content and the user's operation intention, and send the reply instruction answer to the user terminal.

[0014] Optionally, the instruction parsing unit includes: a first model determination module, which is used to use the called preset natural language processing model as the deep learning model when the target instruction is a text instruction; a context parsing module, which is used to use the preset natural language model to parse the context of the target instruction to obtain a question and answer context; an instruction text preprocessing module, which is used to parse the target instruction based on the question and answer context, and preprocess the instruction text obtained by the parsing to obtain an instruction statement; an instruction statement extraction module, which is used to extract synonyms, polyphones and sentence structures of the instruction statement, and identify the preprocessed instruction text based on the text extraction result to obtain the instruction type and instruction content; a first operation intention locating module, which is used to locate the user operation intention based on the context statement, the instruction type and the instruction content.

[0015] Optionally, the assistant reply answer unit includes: an investment preference type determination module, which is used to determine the user's investment preference type based on the user's historical preference information, the instruction content and the user's operation intention when the target instruction is a text instruction; a product selection module, which is used to select the proposed recommended financial products that are consistent with the instruction content and the user's operation intention based on the user's investment preference type using a collaborative filtering algorithm; and a first reply answer generation module, which is used to generate the reply instruction answer based on the proposed recommended financial product, the instruction type and the question and answer context.

[0016] Optionally, the instruction parsing unit includes: a second model determination module, which is used to use the called preset speech recognition model as the deep learning model when the target instruction is a voice instruction; a speech environment analysis module, which is used to use the preset speech recognition model to analyze the surrounding environment of the target instruction to obtain a voice instruction environment, wherein the voice instruction environment includes: user tone type, user intonation type, user voice volume and surrounding environment sound; a speech emotion recognition module, which is used to identify the user voice emotion state and environmental risk level based on the voice instruction environment; a speech instruction recognition module, which is used to identify the target instruction and obtain the instruction content and the user operation intention when the user voice emotion state and environmental risk level indicate that the current user voice operation is safe.

[0017] Optionally, the assistant reply answer unit includes: an instruction type determination module, which is used to determine the instruction type based on the instruction content and the user's operation intention when the target instruction is a voice instruction; a second reply answer generation module, which is used to execute the target instruction according to the instruction content and generate a reply instruction answer based on the user's historical preference information, the instruction content and the user's operation intention when the instruction type is a financial business operation, wherein the financial business operation includes at least one of the following: transfer, account checking, asset income and expenditure report; or, a third reply answer generation module, which is used to generate a reply instruction answer based on the user's historical preference information, the instruction content and the user's operation intention when the instruction type is a financial institution branch-related business, wherein the financial institution branch-related business includes at least one of the following: branch recommendation, branch navigation, and branch queuing.

[0018] Optionally, the instruction parsing unit includes: a third model determination module, which is used to use the called optical character recognition model as the deep learning model when the target instruction is a picture instruction; an image preprocessing module, which is used to preprocess the image in the target instruction and parse the preprocessed target instruction to obtain an instruction scene, wherein the preprocessing strategy includes: denoising, binarization, and tilt correction; an image parsing module, which is used to parse the image instruction based on the instruction scene using the optical character recognition model to obtain a character feature set; a character comparison module, which is used to compare the character feature set with a pre-trained character library to obtain instruction statement information; a second operation intention positioning module, which is used to parse the instruction statement information to obtain the instruction content and the user operation intention.

[0019] Optionally, the assistant reply answer unit also includes: a financial business determination module, which is used to determine the target financial business to be executed based on the instruction content and the user's operation intention when the target instruction is a picture instruction; an operation intention generation module, which is used to execute the target instruction according to the instruction content based on the target financial business, the instruction content and the user's operation intention, and generate a reply instruction answer.

[0020] According to another aspect of an embodiment of the present invention, a computer-readable storage medium is also provided, wherein the computer-readable storage medium includes a stored computer program, wherein when the computer program is running, the device where the computer-readable storage medium is located is controlled to execute any one of the above-mentioned question-and-answer methods based on terminal intelligent assistants.

[0021] According to another aspect of an embodiment of the present invention, there is also provided an electronic device, comprising one or more processors and a memory, wherein the memory is used to store one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement any one of the above-mentioned question-and-answer methods based on a terminal intelligent assistant.

[0022] According to another aspect of an embodiment of the present invention, a computer program product is further provided, including a computer program, which, when executed by a processor, implements the steps of any one of the above-mentioned question-and-answer methods based on a terminal intelligent assistant.

[0023] In the present disclosure, a target instruction input by a user terminal is received, and a deep learning model is used to parse the target instruction, extract the instruction content and the user's operation intention. The deep learning model is a pre-trained model for parsing the user's question instructions and the corresponding model of the user's intention. According to the user's historical preference information, the instruction content and the user's operation intention, a reply instruction answer is generated, and the reply instruction answer is sent to the user terminal. The target instruction is an instruction generated by the display page of the financial intelligent assistant running in the user terminal after receiving the user's operation, and the instruction content of the target instruction includes at least: the user asks questions and / or pays attention to the type of financial products, and specifies financial business requests.

[0024] Based on the above-mentioned public content, the intelligent assistant in the financial institution APP can accurately parse the operation instructions sent by the user, extract the instruction content and the user's operation intention, and thus provide personalized financial product and service recommendations, reduce the user's search and operation time in the APP, and improve the user service experience, thereby solving the technical problem in the related technology that the intelligent assistant in the financial institution APP cannot accurately understand the user's intention, resulting in a poor user service experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] The drawings described herein are used to provide a further understanding of the present invention and constitute a part of this application. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings:

[0026] Figure 1 A hardware structure block diagram of a computer terminal (or mobile device) for implementing a question-answering method based on a terminal intelligent assistant is shown;

[0027] Figure 2 is a flow chart of an optional question-answering method based on a terminal intelligent assistant according to an embodiment of the present invention;

[0028] Figure 3 is a schematic diagram of an optional question-answering device based on a terminal intelligent assistant according to an embodiment of the present invention;

[0029] Figure 4 It is a structural block diagram of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0030] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0031] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0032] To facilitate those skilled in the art to understand the present invention, some terms or nouns involved in the embodiments of the present invention are explained below:

[0033] Smart assistants are intelligent service modules integrated into financial institution apps (e.g., mobile banking apps). In the present invention, smart assistants can respond to user text, voice, and image input, understand user needs, and link various functions of the bank app to operate. Smart assistants provide personalized services such as accurate product recommendations, fast transfer operations, risk prevention and control, branch recommendations, and navigation services by learning and analyzing user historical data, aiming to improve the convenience and efficiency of user operations while protecting user funds and privacy security. The smart assistant in the present invention can also support multimodal input of users and apply deep learning technology to provide a smarter and safer service experience.

[0034] BERT model, Birectional Encoder Representations from Transformers, is a deep learning model used to process natural language processing (NLP) tasks such as text understanding, semantic parsing, and question answering. In the present invention, it can be based on the Transformer architecture to parse the text instructions issued by the user through the intelligent assistant. It can identify the keywords and phrases in the instructions, understand the user's intentions, and dynamically adjust the semantic representation through the context even in the case of synonyms, polyphones or complex sentences, so as to achieve more accurate parsing.

[0035] OCR text recognition, Optical Character Recognition, is a tool that converts text content in a picture into an editable and searchable text format. In the present invention, the intelligent assistant uses OCR technology to process the pictures sent by the user, and first pre-processes the pictures, such as denoising, binarization, and tilt correction, to improve the accuracy of recognition. Then, through text segmentation and feature extraction, it is compared with the trained character library to identify the text information in the picture. OCR technology enables the intelligent assistant to understand the scene and text in the picture, thereby analyzing the user's intention and making corresponding operation suggestions or executing commands, expanding the interaction mode of the intelligent assistant, enabling it to handle more complex and diverse user needs.

[0036] It should be noted that the question-and-answer method based on the terminal intelligent assistant and the device thereof in the present disclosure can be used in the field of financial technology. When the intelligent assistant based on the financial institution APP can parse the user's instructions and provide better business services / product services, it can also be used in any field other than the field of financial technology. When the intelligent assistant based on the financial institution APP can parse the user's instructions and provide better business services / product services, the present disclosure does not limit the application field of the question-and-answer method based on the terminal intelligent assistant and the device thereof.

[0037] It should be noted that the information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) collected by this disclosure are information and data authorized by the user or fully authorized by all parties, and the collection, storage, use, processing, transmission, provision, disclosure and application of relevant data are in compliance with the relevant laws, regulations and standards of the relevant regions, necessary confidentiality measures are taken, and public order and good customs are not violated, and corresponding operation portals are provided for users to choose to authorize or refuse. For example, an interface is set up between this system and relevant users or organizations. Before obtaining relevant information, it is necessary to send an acquisition request to the aforementioned user or organization through the interface, and obtain relevant information after receiving the consent information fed back by the aforementioned user or organization.

[0038] It should be noted that in the present disclosure, when collecting and analyzing customer information, corresponding operation entrances are provided for users to choose to agree or reject the automated decision results; if the user chooses to reject, the expert decision-making process is entered.

[0039] The following embodiments of the present invention can be applied to various systems / applications / devices based on question-answering of terminal intelligent assistants. The present invention provides a full range of intelligent banking services based on the intelligent assistant of the financial institution APP, covering a variety of application scenarios from daily user inquiries, financial management recommendations, transfer operations, abnormal risk prevention and control to branch service recommendations. Through the intelligent assistant, the user's instructions are accurately understood, and personalized bank product and service recommendations are provided, which reduces the time for users to search and operate in the APP, and realizes the convenience and intelligence of operation. At the same time, the risk prevention and control mechanism built into the intelligent assistant, such as account anomaly identification and large-amount transfer confirmation, can effectively protect the user's funds, prevent fraud and abnormal transactions, and can also automatically perform common banking operations, such as transfers, inquiries, etc., while integrating map navigation and queuing systems to simplify user operation processes and improve efficiency.

[0040] It should be noted that the intelligent assistant provided by the present invention supports multiple input methods such as text, voice, and pictures, which improves the convenience of interaction between users and the assistant, especially the improvement of voice and dialect recognition capabilities, so that the intelligent assistant can serve a wider user group.

[0041] The present invention is described in detail below in conjunction with various embodiments.

[0042] Embodiment 1

[0043] According to an embodiment of the present invention, an embodiment of a question-answering method based on a terminal intelligent assistant is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0044] The question-and-answer method embodiment based on the terminal intelligent assistant provided in the first embodiment of the present application can be executed in a mobile terminal, a computer terminal or a similar computing device. Figure 1 The hardware structure block diagram of a computer terminal (or mobile device) for implementing a question-answering method based on a terminal intelligent assistant is shown. Figure 1 As shown, the computer terminal 10 (or mobile device) may include one or more ( Figure 1 The computer 102 includes a processor 102 (the processor 102 may include but is not limited to a microprocessor MCU (Microcontroller Unit) or a programmable logic device FPGA (Field Programmable Gate Array)), a memory 104 for storing data, and a transmission device 106 for communication functions. In addition, it may also include: a display, an input / output interface (I / O interface), a Universal Serial Bus (USB) port (which may be included as one of the ports of the BUS), a network interface, a power supply and / or a camera. Those skilled in the art will understand that Figure 1 The structure shown is only for illustration and does not limit the structure of the above electronic device. Figure 1 More or fewer components as shown, or with Figure 1 Different configurations shown.

[0045] It should be noted that the one or more processors 102 and / or other data processing circuits described above may generally be referred to herein as "data processing circuits". The data processing circuits may be embodied in whole or in part as software, hardware, firmware, or any other combination thereof. In addition, the data processing circuit may be a single independent processing module, or may be incorporated in whole or in part into any of the other components in the computer terminal 10 (or mobile device). As described in the embodiments of the present application, the data processing circuit acts as a processor control (e.g., selection of a variable resistor terminal path connected to an interface).

[0046] The memory 104 can be used to store software programs and modules of application software, such as the program instructions / data storage device corresponding to the question-answering method based on the terminal intelligent assistant in the embodiment of the present application. The processor 102 executes various functional applications and data processing by running the software programs and modules stored in the memory 104, that is, realizing the above-mentioned question-answering method based on the terminal intelligent assistant. The memory 104 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include a memory remotely arranged relative to the processor 102, and these remote memories may be connected to the computer terminal 10 via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and a combination thereof.

[0047] The transmission device 106 is used to receive or send data via a network. The specific example of the above network may include a wireless network provided by a communication provider of the computer terminal 10. In one example, the transmission device 106 includes a network adapter (Network Interface Controller, NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device 106 can be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0048] The display may be, for example, a touch screen liquid crystal display (LCD), which may enable a user to interact with a user interface of the computer terminal 10 (or mobile device).

[0049] It should be noted that the application scenario of the embodiment of the present invention can be a financial product recommendation scenario. For example, when a user asks "recommend a short-term stable financial product" in the smart assistant, the smart assistant extracts the user's key needs through deep learning natural language processing technology, such as the BERT model, and recommends matching financial products based on the user's historical investment preferences using a collaborative filtering algorithm to enhance the user's investment experience. It can also be applied to transfer operation scenarios, such as when a user transfers money through voice or text commands, such as "help me transfer 100 yuan to Zhang San". The smart assistant can not only accurately parse instructions, but also confirm the payee information by searching historical transfer records, and automatically start risk prevention and control mechanisms, such as the payee account status check, to ensure transaction security. It can also be applied to the branch service recommendation scenario, such as when a user asks "recommend the nearest branch", the smart assistant recommends the nearest branch with less queues based on the user's location, combined with map data and the real-time queue situation of each branch, and provides navigation services, while automatically helping users queue online to reduce waiting time. Furthermore, this embodiment can also be applied to personalized service scenarios. For example, the intelligent assistant can provide warm personalized services based on the user's operating habits, historical data and real-time input, such as adjusting the reply strategy according to the user's emotions and tone to enhance the user experience.

[0050] Under the above operating environment, this application provides Figure 2 The question-answering method based on the terminal intelligent assistant is shown. Figure 2 is a flow chart of an optional question-answering method based on a terminal intelligent assistant according to an embodiment of the present invention, such as Figure 2 As shown, the method comprises the following steps:

[0051] Step S201, receiving a target instruction input by a user terminal.

[0052] Among them, the target instruction is the instruction generated by the display page of the financial intelligent assistant running in the user terminal after receiving the user's operation. The instruction content of the target instruction at least includes: the user asks questions and / or pays attention to the type of financial products, and specifies financial business requests.

[0053] The smart assistant first needs to receive the target command sent by the user in the financial institution APP (this embodiment is illustrated by the mobile banking APP) through various input methods (such as text, voice, pictures, etc.). The target command can be a question asked by the user about bank products or services, such as "recommend a short-term stable financial product", or a specific financial service request that the user wants to execute, such as "help me transfer 100 yuan to Zhang San". The display page of the smart assistant will monitor the user's input operations in real time and convert these operations into a command format that the system can understand.

[0054] In step S201, the intelligent assistant can receive and process a variety of instructions from the mobile banking APP user end, which are generated when the user interacts with the intelligent assistant. The target instruction is not limited to text input, but can also be in the form of voice, pictures, etc., such as questions asked by users: users may have questions about various bank products, service policies, operating procedures, etc., and convey them to the intelligent assistant in the form of questions. For example, asking "How to check my account balance?" or "What is the risk level of short-term financial products?" These questions may be simple or complex, and the intelligent assistant needs to accurately understand and provide detailed answers. Or, pay attention to the type of financial products: users may be interested in specific types of financial products, such as deposits, loans, financial products, insurance products, etc. In this case, the target instruction will directly or indirectly reflect the user's curiosity or demand for a certain product, such as "I want to know the latest high-yield financial products", and the intelligent assistant needs to identify the product type and provide relevant information. It can also be a specified financial service request: such as a user performing specific financial service operations through an intelligent assistant, such as transfer, account checking, account opening, loan application, etc. The instruction content will clearly specify the business type and specific requirements, such as "help me transfer 500 yuan to Li Si". After receiving such instructions, the intelligent assistant needs to complete the business operation and feedback the results.

[0055] It should be noted that in order to adapt to the needs and habits of different users, the intelligent assistant provided in this embodiment is designed with a variety of input methods: for example, text input, the user can directly enter text in the dialog box of the intelligent assistant, such as search requirements, problem descriptions, business requests, etc. It can also be voice input: the user can issue instructions by voice, and the voice recognition model of the intelligent assistant can recognize the user's voice information, including the content of the voice instructions, the user's tone, intonation, etc., and further analyze the user's emotional state and environmental risks to ensure safety before executing the instructions. Or picture input: the user can also send pictures to the intelligent assistant, such as bank documents, product brochures, etc. The intelligent assistant recognizes the text information in the picture through preprocessing and OCR technology, and understands the business needs or information query requests behind the picture.

[0056] When processing target instructions, the smart assistant will call the corresponding models and algorithms for analysis based on the instruction type and content. For example, for instructions involving product recommendations, the smart assistant will combine user historical preference information and use recommendation technologies such as collaborative filtering algorithms to provide personalized financial product recommendations. For instructions to execute specific business, the smart assistant will link with the backend system to ensure the security and accuracy of the operation, and may also conduct risk assessments, such as secondary confirmation before large transfers.

[0057] Step S202: Use a deep learning model to parse the target instruction and extract the instruction content and the user's operation intention, wherein the deep learning model is a pre-trained model for parsing the user's question instruction and the corresponding user intention.

[0058] After receiving the target instruction, the intelligent assistant will call the deep learning model to parse the instruction. Here, the deep learning model can be selected according to the format of the direct instruction, such as selecting the BERT natural language processing model. Through the bidirectional Transformer architecture, it can understand the complex semantics and contextual information of the instruction, and extract the core content of the user's question and potential business needs. For example, in the instruction "Help me transfer 100 yuan to Zhang San", the BERT model can identify key information and operation intentions such as "transfer", "100 yuan" and "give Zhang San", and provide accurate parsing results for the execution of subsequent steps.

[0059] Step S203, generating a reply instruction answer according to the user's historical preference information, instruction content and user operation intention, and sending the reply instruction answer to the user terminal.

[0060] After obtaining the parsed results of the target instruction, the intelligent assistant will further consider the user's personal historical preference information, including but not limited to the user's past transaction records, browsing data, search history, and possible savings and financial management preferences. Combining the user's historical preferences and the content of the current instruction, the intelligent assistant can generate a reply instruction answer that better meets the user's needs. For example, when recommending financial products, it will consider the user's past investment type and risk preference to provide personalized suggestions. For specific financial business requests, the intelligent assistant can also optimize the execution process based on the user's operation intention and historical data. For example, in a transfer operation, the recipient can be quickly located through historical preference information to reduce user input. Finally, the intelligent assistant sends the generated reply instruction answer back to the user end and presents it to the user in an intuitive and quick way, realizing intelligent response and operation execution to user requests and improving user experience.

[0061] Through the above steps, the target instruction input by the user terminal can be received, the target instruction can be parsed by a deep learning model, the instruction content and the user's operation intention can be extracted, the deep learning model is a pre-trained model for parsing the user's question instruction and the corresponding model of the user's intention, and the reply instruction answer is generated according to the user's historical preference information, the instruction content and the user's operation intention, and the reply instruction answer is sent to the user terminal, wherein the target instruction is the instruction generated by the display page of the financial intelligent assistant running in the user terminal after receiving the user's operation, and the instruction content of the target instruction at least includes: the user asks questions and / or pays attention to the type of financial products, and specifies the financial business request. In this embodiment, the operation instruction sent by the user can be accurately parsed by the intelligent assistant in the financial institution APP, the instruction content and the user's operation intention can be extracted, so as to provide personalized financial product and service recommendations, reduce the time for users to search and operate in the APP, and improve the user service experience, thereby solving the technical problem that the intelligent assistant in the financial institution APP in the related technology cannot accurately understand the user's intention, resulting in poor user service experience.

[0062] The embodiments of the present invention are described in detail below in combination with the above-mentioned implementation steps.

[0063] Optionally, the step of using a deep learning model to parse the target instruction and extract the instruction content and the user's operation intention includes: when the target instruction is a text instruction, calling a preset natural language processing model as a deep learning model; using the preset natural language model to parse the context of the target instruction to obtain a question-and-answer context; parsing the target instruction based on the question-and-answer context, and preprocessing the instruction text obtained by the analysis to obtain an instruction statement; extracting synonyms, polyphones and sentence structures of the instruction statement, identifying the preprocessed instruction text based on the text extraction result, and obtaining the instruction type and instruction content; locating the user's operation intention based on the context sentence, instruction type and instruction content.

[0064] When the target instruction is entered in text form (for example, "Recommend me a short-term and stable financial product"), the intelligent assistant first calls the preset natural language processing (NLP) model, such as the BERT model, which parses the context of the instruction and identifies the background information and context of the instruction, such as the meaning of "short-term" and "stable" in the recommendation of financial products, so as to more accurately understand the user's needs. After that, the instruction text obtained by the analysis is further preprocessed, including text cleaning and formatting, to remove irrelevant information, such as punctuation and stop words, to ensure the accuracy of subsequent analysis.

[0065] It should be noted that after obtaining the instruction statement, the present embodiment also needs to extract text features, such as extracting synonyms, polyphones, and sentence structures in the instruction text through NLP technology, which helps to understand the multiple meanings of the instruction, such as "Zhang San" may represent different users in different contexts, or "transfer" may involve multiple operation methods. Based on the extracted text features, the intelligent assistant can identify the type of instruction (such as query, transfer, recommendation, etc.) and specific content (such as transfer amount, product type, etc.), which is the basis for performing correct operations.

[0066] Finally, by integrating the context, command type and content, the intelligent assistant can locate the user's specific operational intention, such as whether the user needs to be recommended a financial product, what features the financial product should have, etc., to prepare for the subsequent provision of precise services.

[0067] Next, the intelligent assistant will generate a reply instruction answer that best meets the user's needs based on an in-depth understanding of the user's instructions and the user's historical preference information. Optionally, the step of generating a reply instruction answer based on the user's historical preference information, instruction content, and user operation intention includes: when the target instruction is a text instruction, determining the user's investment preference type based on the user's historical preference information, instruction content, and user operation intention; based on the user's investment preference type, using a collaborative filtering algorithm to select a proposed recommended financial product that meets the instruction content and the user's operation intention; generating a reply instruction answer based on the proposed recommended financial product, instruction type, and question and answer context.

[0068] The smart assistant will analyze the user's historical transaction records, browsing preferences, search records and other data to determine the user's investment preference type, such as preference for short-term, stable investment products, and then use the collaborative filtering algorithm to select financial products. It should be noted here that collaborative filtering is a recommendation algorithm based on user behavior or user similarity. The smart assistant uses the collaborative filtering algorithm and combines the user's investment preference type to select the product that best meets the user's current needs from a large number of financial products, thereby ensuring the accuracy and personalization of the recommendation. Once the financial product to be recommended is determined, the smart assistant will generate a detailed reply instruction answer based on product information, instruction type (such as "recommendation") and context. This answer not only includes product recommendations, but may also include product introductions, investment advice, risk warnings and other information to fully respond to user needs. Finally, the smart assistant displays this reply instruction answer in a user-friendly way on the interface of the mobile banking APP, and the user can immediately see the recommendation results and perform subsequent operations or feedback. The smart assistant will also record user feedback to further optimize its recommendation model to make future recommendations more accurate.

[0069] In the case where the target instruction is a voice instruction, the present invention uses a preset voice recognition model in a deep learning model for analysis to understand the user's specific needs and operation intentions. Optionally, the step of using a deep learning model to parse the target instruction and extract the instruction content and the user's operation intention includes: in the case where the target instruction is a voice instruction, the called preset voice recognition model is used as a deep learning model; the preset voice recognition model is used to analyze the surrounding environment of the target instruction to obtain a voice instruction environment, wherein the voice instruction environment includes: user tone type, user intonation type, user voice volume, and surrounding environmental sounds; based on the voice instruction environment, the user's voice emotional state and environmental risk are identified; when the user's voice emotional state and environmental risk indicate that the current user's voice operation is safe, the target instruction is identified to obtain the instruction content and the user's operation intention.

[0070] The voice recognition model is called to analyze the surrounding environment when the user issues a command, including identifying the tone type, intonation type, voice volume, and background sounds in the environment, to help the smart assistant understand the user's emotional state and physical environment, and judge the safety of the operation. Then, based on the results of the voice environment analysis, the smart assistant can identify the user's emotional state (such as calm, nervous, excited, etc.) and the environmental risk level of the operation (such as whether the user is in a noisy public place, whether he is being monitored, etc.), which is an important basis for the smart assistant to decide whether to execute the command, ensuring that financial operations are carried out under safe conditions and protecting user privacy and financial security.

[0071] When the evaluation results indicate that the current voice operation is safe, the intelligent assistant will recognize the voice command, convert it into understandable text content, and then extract the core content of the command and the user's specific operation intention through natural language processing technology. For commands involving synonyms, polyphones, or ambiguous expressions, the intelligent assistant will accurately judge the user's intention based on contextual understanding, such as identifying the recipient's name and transfer amount in a transfer instruction. Finally, the intelligent assistant further analyzes the content of the instruction to determine the type of instruction, such as whether it is a transfer, an account check, or an asset income and expenditure report request.

[0072] Optionally, the step of generating a reply instruction answer based on the user's historical preference information, instruction content and user's operation intention includes: when the target instruction is a voice instruction, determining the instruction type based on the instruction content and the user's operation intention; when the instruction type is a financial business operation, executing the target instruction according to the instruction content based on the user's historical preference information, instruction content and user's operation intention, and generating a reply instruction answer, wherein the financial business operation includes at least one of the following: transfer, account checking, asset income and expenditure report; or, when the instruction type is a financial institution branch-related business, generating a reply instruction answer based on the user's historical preference information, instruction content and user's operation intention, wherein the financial institution branch-related business includes at least one of the following: branch recommendation, branch navigation, branch queuing and calling.

[0073] When the target command is a voice command, it is necessary to confirm that the type of the voice command is a financial business operation (such as transfer, account check, asset income and expenditure report) or a financial institution branch-related business (such as branch recommendation, navigation, queuing and calling). Then different operations are performed according to the command type. For example, when the command type is a financial business operation, the intelligent assistant will automatically perform the corresponding financial transaction based on the user's historical preference information, the specific content of the command (such as the transfer amount, the payee) and the user's intention. For example, when a user requests a transfer, the intelligent assistant will automatically check whether the payee's account is abnormal, execute the transfer operation after confirming that the amount is correct, and generate feedback information on the completion or abnormality of the operation. If the command is related to a bank branch, the intelligent assistant will provide branch recommendation services (such as the nearest branch with the least queue), navigation to the recommended branch path and automatic queuing services based on user preferences, location information and operation intentions, reducing user waiting time and improving offline service experience.

[0074] It should be noted that before executing any instruction, the smart assistant will conduct a risk assessment to ensure the security of the operating environment. For sensitive operations, such as large transfers, the smart assistant will initiate additional security verification steps, such as secondary confirmation and password verification, to ensure the safety of user funds. The smart assistant will generate corresponding reply instructions based on the results of the instruction execution, including whether the operation is successful, transaction details, recommended information, etc., and feedback to the user in a user-friendly manner to ensure that the user can clearly understand the operation results.

[0075] When the target instruction is in the form of an image, the intelligent assistant in this embodiment uses an optical character recognition (OCR) model to parse the text information in the image, thereby understanding the user's intentions and needs. Optionally, the step of using a deep learning model to parse the target instruction and extract the instruction content and the user's operation intention includes: when the target instruction is an image instruction, the called optical character recognition model is used as a deep learning model; the image in the target instruction is preprocessed, and the preprocessed target instruction is parsed to obtain an instruction scene, wherein the preprocessing strategy includes: denoising, binarization, and tilt correction; based on the instruction scene, the optical character recognition model is used to parse the image instruction to obtain a character feature set; the character feature set is compared with the pre-trained character library to obtain instruction statement information; the instruction statement information is parsed to obtain the instruction content and the user's operation intention.

[0076] After receiving the image command, the smart assistant first needs to preprocess the image to improve the recognition accuracy of the OCR model. Preprocessing strategies may include denoising, binarization, and tilt correction, which help remove interference factors in the image and make the text clearer for subsequent recognition. After preprocessing, the smart assistant begins to parse the content of the image and identify the scene information, which may involve financial-related scenes such as bank receipts, product brochures, and identity documents. Scene analysis helps to locate the context of text information in the image and further improve recognition accuracy.

[0077] Based on the instruction scenario, the smart assistant uses the OCR model to recognize the text in the picture, extract the character features of the text, such as font, size, position and other information, and form a character feature set. The smart assistant compares the obtained character feature set with the pre-trained character library, identifies the specific character content, and finally converts it into understandable instruction statement information. The character library is trained based on a large amount of sample data and can recognize text in various fonts and writing styles to ensure the accuracy and robustness of recognition. The smart assistant further parses the instruction statement information, understands the meaning of the instruction through the deep learning model, extracts the core content of the instruction, such as product name, transaction amount, operation requirements, etc., and locates the user's specific operation intention, such as querying product details, executing transactions, etc.

[0078] In the case where the target instruction is a picture instruction, the intelligent assistant generates and executes the corresponding financial business operation based on the instruction content and user operation intention obtained by parsing, combined with the user's historical preference information, and finally feeds back the result to the user. Optionally, the step of generating a reply instruction answer based on the user's historical preference information, instruction content and user operation intention includes: in the case where the target instruction is a picture instruction, determining the target financial business to be executed based on the instruction content and user operation intention; based on the target financial business, instruction content and user operation intention, executing the target instruction according to the instruction content, and generating a reply instruction answer.

[0079] The smart assistant analyzes the content of the instruction and the user's intention to determine the specific financial business the user wants to perform, such as financial product inquiry, transfer operation, account information confirmation, etc. Then, based on the determined financial business type, the smart assistant links the backend system of the mobile banking APP and automatically performs the corresponding financial operation according to the instruction content. For example, if the user sends a picture containing the name of a financial product, the smart assistant will automatically query the detailed information of the product, including risk level, expected return, etc., and make recommendations based on user preferences.

[0080] It should be noted that after the command is executed, the intelligent assistant will generate a reply command answer containing the operation results, such as detailed information on product recommendations, notifications on whether the transfer is successful or not, confirmation results of account information, etc. These answers will be presented to users in an intuitive and easy-to-understand way to enhance the user experience.

[0081] When processing image instructions, the smart assistant will also conduct security checks and risk assessments to ensure the legality and security of the transaction, such as checking whether the recipient's account in the image is abnormal, whether the operation amount exceeds the user's daily transaction range, etc. If necessary, it will trigger the security verification process to protect the user's funds.

[0082] The intelligent assistant in the financial institution's APP can better manage and integrate all available functions in the mobile terminal, train models based on user historical data, and complete user recommendations and other instructions in the most efficient and accurate manner, thereby improving user experience and efficiency.

[0083] Another optional specific implementation is described in detail below.

[0084] In daily life, mobile terminals (such as mobile phones and tablets) have become indispensable electronic devices for users, and online banking services are becoming more and more common. In daily use, some queries are inconvenient or you can't find where to query / set. Each function has its own entrance, and you can only operate it manually after entering it separately. The cumbersome operation and the inability to free your hands have become the pain points of most users. For the elderly and others who are not familiar with the functions, the operation is more difficult.

[0085] The embodiment of the present invention is based on the recommendation algorithm, image recognition, natural language processing, machine learning and other technologies in the field of artificial intelligence. According to the text / picture / voice and other instructions issued by the user through the intelligent assistant, the mobile phone APP function is automatically linked to complete the operation. Recommended operations can be performed according to the intelligent assistant, using the user data stored in the mobile phone as the data source, and the model is trained according to the user's instructions to complete the instructions or recommended tasks. Through this intelligent assistant, all functions that can be used in the mobile banking APP can be completed, freeing your hands while improving user experience and efficiency, and lowering the threshold for customer use.

[0086] The smart assistant of the mobile banking APP can effectively utilize the user's historical operations and business data, making the user's operation of the mobile banking APP smarter and faster. The smart assistant page supports sending text / language / pictures, and the smart assistant will give corresponding operations based on the analyzed content. For example:

[0087] (1) Enter the command in the smart assistant: Recommend me a short-term and stable financial product. This enables accurate recommendations of bank products.

[0088] The smart assistant receives the command, recognizes the text through voice, and the BERT model, a deep learning algorithm in natural language processing, parses the text, extracts keywords and phrases, etc., splits it into task instructions and sends it to the corresponding function for processing. The investment type and preference are confirmed based on the user's historical investment and browsing data, and the user-based collaborative filtering algorithm is used to recommend financial products that meet the user's investment habits and instruction requirements from a large number of financial products, and returns them to the page for display, realizing intelligent and accurate recommendations. If there are synonyms or polyphones in the input language, the BERT model, as a feature extractor, can understand the context and dynamically adjust the semantic representation based on the context environment to make the interpretation more accurate.

[0089] (2) Send a picture to the smart assistant to analyze the user's intention in the picture and make corresponding instructions.

[0090] The smart assistant scans the image and pre-processes the image using OCR text recognition technology in deep learning (here, pre-processing technology may include denoising, binarization, tilt correction and other operations) to improve the accuracy of subsequent recognition. Then, the text area in the image is extracted and the characters are segmented using text segmentation technology. The features of each character are then extracted and compared with the pre-trained character library. Finally, the content of each character is identified, and corresponding instructions are given based on the scene, text and other information in the identified image. If the instruction is unclear, the page returns all the predicted related operations for the user to choose.

[0091] (3) Enter text / voice in the smart assistant: Help me transfer 100 yuan to Zhang San. The smart assistant can automatically transfer money and prevent risks.

[0092] After the smart assistant parses the instruction, it will search for the user named "Zhang San" in the transfer history according to the user name, and pop up a pop-up window for the customer to confirm. After confirmation, the transfer will be completed quickly. If no historical transfer record can be found, a transfer pop-up window will pop up, and the recipient name will be filled in as "Zhang San" by default, and the transfer amount will be 100 yuan, reducing user operations.

[0093] If the transfer amount is large, such as exceeding the amount of historical daily transfers or suddenly reaching the bank limit, it can be identified as a suspected large / abnormal transfer. At this time, the smart assistant will automatically monitor and identify risks based on the amount, remind the user to confirm the transfer amount and recipient information, and simultaneously verify whether the recipient account is abnormal. If an abnormal account is identified, the transfer function will be automatically closed and the transfer function will be restored after a period of time.

[0094] Identification of abnormal accounts: An abnormal account database is created based on abnormal accounts in the bank's transaction account data. Based on the transaction data of each bank, accounts such as accounts that have been temporarily frozen due to too many incorrect password inputs, dormant accounts that have not been used for a long time, and accounts with frequent large transactions are dynamically updated to the abnormal account database. If the receiving account hits the data in the abnormal account database, it is determined to be an abnormal account.

[0095] If the user enters the transfer instruction by voice, the voice assistant will determine whether the user is in a normal state by analyzing the user's tone, intonation, volume, ambient sound, etc., automatically identify risks, and provide users with a safer payment environment and protection.

[0096] (4) Input text / voice into the smart assistant: recommend the nearest outlets. The smart assistant can achieve accurate positioning, link with map navigation, and link with offline quick service.

[0097] The smart assistant will locate the current location based on the parsed instructions, link the map app to find the nearest outlet, recommend the outlet that can be reached the fastest based on the degree of congestion, and start navigation through the map. Based on the estimated journey time and the current queue situation at the outlet, it will automatically help users to queue up and reduce the waiting time for users to handle business at the outlet. It will collect and monitor the queuing and calling systems of each outlet, collect real-time data on number collection and calling for business handling, and update the queue situation of each outlet in real time.

[0098] Smart assistants can achieve increasingly precise and intelligent speech recognition through machine learning and deep learning technologies. With long-term / frequent interactions between users and smart assistants, smart assistants can increasingly understand users' personal preferences and needs, thereby providing more accurate recommendations and services.

[0099] Interaction with smart assistants is not limited to simple questions and answers. Through interaction with smart assistants, a database is established as source data based on user data, including historical data such as customer bank transactions / operations / browsing / clicks / language / tone / emotions, voice data in various languages / dialects, and bank transaction risk cases. The data is pre-processed and de-noised, and then segmented and tagged with parts of speech. The text is parsed to count the amount of operation and number of operations corresponding to each action, and the voice or text in various languages / dialects is converted into data annotation areas as training data sets. At the same time, the BERT model is used to train the model, and the model is continuously optimized through user use. The smart assistant can better understand user needs, understand user intentions based on language, tone, emotion, and expression, and make more appropriate responses, thereby improving customer experience and user stickiness.

[0100] Through the above implementation methods, the intelligent assistant supports multiple input contents such as text, voice, and pictures, supports the recognition of dialects in different regions, and supports use by different users, bringing convenience to users.

[0101] The following is a detailed description in conjunction with another embodiment.

[0102] Embodiment 2

[0103] A question-and-answer device based on a terminal intelligent assistant provided in this embodiment includes multiple implementation units, each of which corresponds to each implementation step in the above-mentioned embodiment 1. Its specific implementation methods and beneficial effects can refer to the above-mentioned method embodiments and will not be repeated here.

[0104] Figure 3 is a schematic diagram of an optional question-answering device based on a terminal intelligent assistant according to an embodiment of the present invention, such as Figure 3 As shown, the question-answering device based on the terminal intelligent assistant may include: an instruction receiving unit 31, an instruction parsing unit 32, and an assistant reply answer unit 33.

[0105] Among them, the instruction receiving unit 31 is used to receive the target instruction input by the user terminal, wherein the target instruction is the instruction generated by the display page of the financial intelligent assistant running in the user terminal after receiving the user's operation, and the instruction content of the target instruction at least includes: the user asks questions and / or pays attention to the type of financial products, and specifies financial business requests.

[0106] The instruction parsing unit 32 is used to parse the target instruction using a deep learning model to extract the instruction content and the user's operation intention, wherein the deep learning model is a pre-trained model for parsing the user's question instructions and the corresponding user intention.

[0107] The assistant reply answer unit 33 is used to generate a reply instruction answer based on the user's historical preference information, instruction content and user operation intention, and send the reply instruction answer to the user terminal.

[0108] The above-mentioned question-answering device based on the terminal intelligent assistant can receive the target instruction input by the user terminal through the instruction receiving unit 31, wherein the target instruction is the instruction generated by the display page of the financial intelligent assistant running in the user terminal after receiving the user's operation, and the instruction content of the target instruction at least includes: the user asks questions and / or pays attention to the type of financial products, specifies financial business requests, and uses the instruction parsing unit 32 to parse the target instruction using a deep learning model to extract the instruction content and the user's operation intention, wherein the deep learning model is a pre-trained model for parsing the user's question instruction and the corresponding model of the user's intention, and generates a reply instruction answer according to the user's historical preference information, the instruction content and the user's operation intention through the assistant reply answer unit 33, and sends the reply instruction answer to the user terminal. In this embodiment, the intelligent assistant in the financial institution APP can accurately parse the operation instruction sent by the user, extract the instruction content and the user's operation intention, so as to provide personalized financial product and service recommendations, reduce the time for users to search and operate in the APP, and improve the user service experience, thereby solving the technical problem that the intelligent assistant in the financial institution APP in the related technology cannot accurately understand the user's intention, resulting in poor user service experience.

[0109] Optionally, the instruction parsing unit includes: a first model determination module, which is used to call a preset natural language processing model as a deep learning model when the target instruction is a text instruction; a context parsing module, which is used to use a preset natural language model to parse the context of the target instruction to obtain a question and answer context; an instruction text preprocessing module, which is used to parse the target instruction based on the question and answer context, and preprocess the instruction text obtained by the parsing to obtain an instruction statement; an instruction statement extraction module, which is used to extract synonyms, polyphones and sentence structures of the instruction statement, identify the preprocessed instruction text based on the text extraction result, and obtain the instruction type and instruction content; a first operation intention locating module, which is used to locate the user's operation intention based on the context statement, instruction type and instruction content.

[0110] Optionally, the assistant reply answer unit includes: an investment preference type determination module, which is used to determine the user's investment preference type based on the user's historical preference information, instruction content and user operation intention when the target instruction is a text instruction; a product selection module, which is used to select the proposed recommended financial products that are in line with the instruction content and the user's operation intention based on the user's investment preference type and adopt a collaborative filtering algorithm; a first reply answer generation module, which is used to generate a reply instruction answer based on the proposed recommended financial product, instruction type and question and answer context.

[0111] Optionally, the instruction parsing unit includes: a second model determination module, which is used to use the called preset speech recognition model as a deep learning model when the target instruction is a voice instruction; a speech environment analysis module, which is used to use the preset speech recognition model to analyze the surrounding environment of the target instruction to obtain the voice instruction environment, wherein the voice instruction environment includes: user voice type, user voice intonation type, user voice volume and surrounding environment sound; a speech emotion recognition module, which is used to identify the user's voice emotion state and environmental risk level based on the voice instruction environment; a voice instruction recognition module, which is used to identify the target instruction and obtain the instruction content and user operation intention when the user's voice emotion state and environmental risk level indicate that the current user's voice operation is safe.

[0112] Optionally, the assistant reply answer unit includes: an instruction type determination module, which is used to determine the instruction type based on the instruction content and the user's operation intention when the target instruction is a voice instruction; a second reply answer generation module, which is used to execute the target instruction according to the instruction content based on the user's historical preference information, the instruction content and the user's operation intention when the instruction type is a financial business operation, and generate a reply instruction answer, wherein the financial business operation includes at least one of the following: transfer, account checking, asset income and expenditure report; or, a third reply answer generation module, which is used to generate a reply instruction answer based on the user's historical preference information, the instruction content and the user's operation intention when the instruction type is a financial institution branch-related business, wherein the financial institution branch-related business includes at least one of the following: branch recommendation, branch navigation, and branch queuing.

[0113] Optionally, the instruction parsing unit includes: a third model determination module, which is used to use the called optical character recognition model as a deep learning model when the target instruction is a picture instruction; an image preprocessing module, which is used to preprocess the image in the target instruction and parse the preprocessed target instruction to obtain an instruction scene, wherein the preprocessing strategy includes: denoising, binarization, and tilt correction; an image parsing module, which is used to parse the image instruction based on the instruction scene using an optical character recognition model to obtain a character feature set; a character comparison module, which is used to compare the character feature set with a pre-trained character library to obtain instruction statement information; a second operation intention positioning module, which is used to parse the instruction statement information to obtain the instruction content and the user's operation intention.

[0114] Optionally, the assistant reply answer unit also includes: a financial business determination module, which is used to determine the target financial business to be executed based on the instruction content and the user's operation intention when the target instruction is a picture instruction; an operation intention generation module, which is used to execute the target instruction according to the instruction content based on the target financial business, instruction content and user operation intention, and generate a reply instruction answer.

[0115] The above-mentioned question-and-answer device based on terminal intelligent assistant can also include a processor and a memory. The above-mentioned instruction receiving unit 31, instruction parsing unit 32, assistant reply answer unit 33, etc. are all stored in the memory as program units, and the processor executes the above-mentioned program units stored in the memory to realize the corresponding functions.

[0116] The processor includes a kernel, which retrieves the corresponding program unit from the memory. One or more kernels can be set, and the kernel parameters can be adjusted to implement financial product recommendations / provide financial services based on the smart assistant in the financial institution APP.

[0117] The above-mentioned memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash memory (fl ash RAM), and the memory includes at least one storage chip.

[0118] Embodiment 3

[0119] An embodiment of the present application may provide an electronic device, Figure 4 is a structural block diagram of an electronic device according to an embodiment of the present application. Figure 4 As shown, the electronic device may include: one or more ( Figure 4 (only one is shown) processor 402, memory 404, storage controller, and peripheral interface, wherein the peripheral interface is connected to the radio frequency module, audio module and display.

[0120] Among them, the memory can be used to store software programs and modules, such as the program instructions / modules corresponding to the question-and-answer method and device based on the terminal intelligent assistant in the embodiment of the present application. The processor executes various functional applications and data processing by running the software programs and modules stored in the memory, that is, realizing the above-mentioned question-and-answer method based on the terminal intelligent assistant. The memory may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory may further include a memory remotely arranged relative to the processor, and these remote memories can be connected to the terminal via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and a combination thereof.

[0121] The processor can call the information and application stored in the memory through the transmission device to perform the following steps: receiving a target instruction input by the user terminal, wherein the target instruction is an instruction generated by the display page of the financial intelligent assistant running in the user terminal after receiving the user's operation, and the instruction content of the target instruction at least includes: the user asks questions and / or pays attention to the type of financial products, and specifies financial business requests; using a deep learning model to parse the target instruction, extract the instruction content and the user's operation intention, wherein the deep learning model is a pre-trained model for parsing the user's question instructions and the corresponding user intention; generating a reply instruction answer based on the user's historical preference information, the instruction content and the user's operation intention, and sending the reply instruction answer to the user terminal.

[0122] The processor can also call the information and application programs stored in the memory through the transmission device to perform the following steps: when the target instruction is a text instruction, the preset natural language processing model is called as a deep learning model; the context of the target instruction is parsed using the preset natural language model to obtain a question-and-answer context; the target instruction is parsed based on the question-and-answer context, and the instruction text obtained by the parsing is preprocessed to obtain an instruction statement; synonyms, polyphones and sentence structures of the instruction statement are extracted, and the preprocessed instruction text is identified based on the text extraction results to obtain the instruction type and instruction content; based on the context sentence, instruction type and instruction content, the user's operation intention is located.

[0123] The processor can also call the information and application programs stored in the memory through the transmission device to perform the following steps: when the target instruction is a text instruction, determine the user's investment preference type based on the user's historical preference information, instruction content and user operation intention; based on the user's investment preference type, use a collaborative filtering algorithm to select the proposed recommended financial products that meet the instruction content and user operation intention; based on the proposed recommended financial products, instruction type and question and answer context, generate a reply instruction answer.

[0124] The processor can also call the information and application programs stored in the memory through the transmission device to perform the following steps: when the target instruction is a voice instruction, the called preset voice recognition model is used as a deep learning model; the preset voice recognition model is used to analyze the surrounding environment of the target instruction to obtain the voice instruction environment, wherein the voice instruction environment includes: user tone type, user intonation type, user voice volume and surrounding environment sound; based on the voice instruction environment, the user voice emotional state and environmental risk level are identified; when the user voice emotional state and environmental risk level indicate that the current user voice operation is safe, the target instruction is identified to obtain the instruction content and the user operation intention.

[0125] The processor can also call the information and application programs stored in the memory through the transmission device to perform the following steps: when the target instruction is a voice instruction, determine the instruction type based on the instruction content and the user's operation intention; when the instruction type is a financial business operation, based on the user's historical preference information, the instruction content and the user's operation intention, execute the target instruction according to the instruction content and generate a reply instruction answer, wherein the financial business operation includes at least one of the following: transfer, account checking, asset income and expenditure report; or, when the instruction type is a financial institution branch-related business, generate a reply instruction answer based on the user's historical preference information, the instruction content and the user's operation intention, wherein the financial institution branch-related business includes at least one of the following: branch recommendation, branch navigation, branch queuing and calling.

[0126] The processor can also call the information and application programs stored in the memory through the transmission device to perform the following steps: when the target instruction is a picture instruction, the called optical character recognition model is used as a deep learning model; the picture in the target instruction is preprocessed, and the preprocessed target instruction is parsed to obtain an instruction scene, wherein the preprocessing strategy includes: denoising, binarization, and tilt correction; based on the instruction scene, the optical character recognition model is used to parse the picture instruction to obtain a character feature set; the character feature set is compared with the pre-trained character library to obtain instruction statement information; the instruction statement information is parsed to obtain the instruction content and the user's operation intention.

[0127] The processor can also call the information and applications stored in the memory through the transmission device to perform the following steps: when the target instruction is a picture instruction, determine the target financial business to be executed based on the instruction content and the user's operation intention; based on the target financial business, instruction content and user operation intention, execute the target instruction according to the instruction content and generate a reply instruction answer.

[0128] It can be understood by those skilled in the art that Figure 4 The structure shown is for illustration only, and the electronic device may also be a terminal device such as a smart phone, a tablet computer, a PDA, a mobile Internet device (Mobile Internet Devices, MID), a PAD, etc. Figure 4 The structure of the electronic device is not limited. Figure 4 More or fewer components (such as network interfaces, display devices, etc.) shown in, or having Figure 4 Different configurations shown.

[0129] A person of ordinary skill in the art can understand that all or part of the steps in the various terminal intelligent assistant-based question-and-answer methods in the above-mentioned embodiments can be completed by instructing the hardware related to the terminal device through a program, and the program can be stored in a computer-readable storage medium, and the storage medium may include: a flash drive, a read-only memory (ROM), a random access memory (RAM), a disk or an optical disk, etc.

[0130] Embodiment 4

[0131] The embodiment of the present application further provides a storage medium. Optionally, in this embodiment, the storage medium can be used to store the program code executed by the question-answering method based on the terminal intelligent assistant provided in the first embodiment.

[0132] According to another aspect of an embodiment of the present invention, a computer-readable storage medium is also provided, the computer-readable storage medium including a stored computer program, wherein when the computer program is running, the device where the computer-readable storage medium is located is controlled to execute any one of the terminal intelligent assistant question-and-answer methods in the above-mentioned embodiment 1.

[0133] Optionally, in this embodiment, the above storage medium may be located in any computer terminal in a computer terminal group in a computer network, or in any mobile terminal in a mobile terminal group.

[0134] The present application also provides a computer program product, including a computer program, which, when executed by a processor, implements the steps of the question-and-answer method based on a terminal intelligent assistant described in each embodiment of the present application.

[0135] The present application also provides a computer program product, including a non-volatile computer-readable storage medium, wherein the non-volatile computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the question-and-answer method based on the terminal intelligent assistant described in each embodiment of the present application are implemented.

[0136] The serial numbers of the above embodiments of the present invention are only for description and do not represent the advantages or disadvantages of the embodiments.

[0137] In the above embodiments of the present invention, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0138] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only schematic. For example, the division of the units can be a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0139] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0140] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0141] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, magnetic disk or optical disk and other media that can store program codes.

[0142] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principle of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A question-answering method based on a terminal intelligent assistant, characterized in that: include: Receive a target instruction input by the user terminal, wherein the target instruction is an instruction generated by the display page of the financial intelligent assistant running in the user terminal after receiving the user's operation, and the instruction content of the target instruction at least includes: the user asks questions and / or pays attention to the type of financial products, and specifies financial business requests; A deep learning model is used to parse the target instruction and extract the instruction content and the user's operation intention, wherein the deep learning model is a pre-trained model for parsing the user's question instruction and the corresponding user intention; A reply instruction answer is generated according to the user's historical preference information, the instruction content and the user's operation intention, and the reply instruction answer is sent to the user terminal.

2. The question-answering method according to claim 1, characterized in that: The step of using a deep learning model to parse the target instruction and extract the instruction content and the user's operation intention includes: When the target instruction is a text instruction, the preset natural language processing model is called as the deep learning model; The preset natural language model is used to parse the context of the target instruction to obtain a question-answering context; Parsing the target instruction based on the question-answer context, and preprocessing the instruction text obtained by the parsing to obtain an instruction statement; Extracting synonyms, polyphones and sentence structures of the instruction sentence, identifying the preprocessed instruction text based on the text extraction result, and obtaining the instruction type and instruction content; The user operation intention is located based on the context statement, the instruction type and the instruction content.

3. The question-answering method according to claim 2, characterized in that: The step of generating a reply instruction answer according to the user's historical preference information, the instruction content and the user's operation intention includes: In the case where the target instruction is a text instruction, determining the user investment preference type based on the user historical preference information, the instruction content and the user operation intention; Based on the user's investment preference type, a collaborative filtering algorithm is used to select the recommended financial products that meet the instruction content and the user's operation intention; The reply instruction answer is generated based on the proposed recommended financial product, the instruction type and the question and answer context.

4. The question-answering method according to claim 1, characterized in that: The step of using a deep learning model to parse the target instruction and extract the instruction content and the user's operation intention includes: When the target instruction is a voice instruction, calling a preset voice recognition model as the deep learning model; The preset speech recognition model is used to analyze the surrounding environment of the target instruction to obtain a speech instruction environment, wherein the speech instruction environment includes: user tone type, user intonation type, user voice volume and surrounding environment sound; Based on the voice command environment, identifying the user's voice emotional state and the environmental risk level; When the user voice emotional state and the environmental risk level indicate that the current user voice operation is safe, the target instruction is identified to obtain the instruction content and the user operation intention.

5. The question-answering method according to claim 4, characterized in that: The step of generating a reply instruction answer according to the user's historical preference information, the instruction content and the user's operation intention includes: In the case where the target instruction is a voice instruction, determining the instruction type based on the instruction content and the user operation intention; In the case where the instruction type is a financial business operation, based on the user's historical preference information, the instruction content and the user's operation intention, the target instruction is executed according to the instruction content, and a reply instruction answer is generated, wherein the financial business operation includes at least one of the following: transfer, account checking, asset income and expenditure report; or In the case where the instruction type is a financial institution branch-related service, a reply instruction answer is generated based on the user's historical preference information, the instruction content and the user's operation intention, wherein the financial institution branch-related service includes at least one of the following: branch recommendation, branch navigation, and branch queuing and calling.

6. The question-answering method according to claim 1, characterized in that: The step of using a deep learning model to parse the target instruction and extract the instruction content and the user's operation intention includes: When the target instruction is a picture instruction, the called optical character recognition model is used as the deep learning model; Preprocessing the image in the target instruction, and parsing the preprocessed target instruction to obtain an instruction scene, wherein the preprocessing strategy includes: denoising processing, binarization processing, and tilt correction processing; Based on the instruction scenario, the optical character recognition model is used to parse the image instruction to obtain a character feature set; Comparing the character feature set with a pre-trained character library to obtain instruction statement information; The instruction statement information is parsed to obtain the instruction content and the user operation intention.

7. The question-answering method according to claim 6, characterized in that: The step of generating a reply instruction answer according to the user's historical preference information, the instruction content and the user's operation intention includes: In the case where the target instruction is a picture instruction, determining a target financial business to be executed based on the instruction content and the user operation intention; Based on the target financial business, the instruction content and the user's operation intention, the target instruction is executed according to the instruction content, and a reply instruction answer is generated.

8. A question-answering device based on a terminal intelligent assistant, characterized in that: include: An instruction receiving unit is used to receive a target instruction input by a user terminal, wherein the target instruction is an instruction generated by a display page of a financial intelligent assistant running in the user terminal after receiving an operation by a user, and the instruction content of the target instruction at least includes: a question asked by the user and / or a type of financial product concerned, and a specified financial business request; An instruction parsing unit, used to parse the target instruction using a deep learning model to extract instruction content and user operation intention, wherein the deep learning model is a pre-trained model for parsing user question instructions and user intentions; The assistant reply answer unit is used to generate a reply instruction answer based on the user's historical preference information, the instruction content and the user's operation intention, and send the reply instruction answer to the user terminal.

9. An electronic device, characterized in that: It includes one or more processors and a memory, wherein the memory is used to store one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement the question-answering method based on the terminal intelligent assistant as described in any one of claims 1 to 7.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the question-answering method based on a terminal intelligent assistant as described in any one of claims 1 to 7 are implemented.

Images